Integration Broker

UserName not defined in database

If you get this message with the web services listening connectors e.g. SOAP or REST:

UserName not defined in database. (158,55)

Then check the Default User ID set for the ANONYMOUS Node (used by IB for web services) and make sure that it is a valid username and that the user account is active (e.g. not locked out).

Web Service Target Locations

Web Service Target Locations are defined under:

PeopleTools > Integration Broker > Configuration > Service Configuration

Click on the "Setup Target Locations" hyperlink:

ib-configuration-setup-target-locations.png

On the target locations page, set the relevant values as suggested e.g.

SOAP/XML web services:

REST web services

External System Contact Error

While configuring an integration gateway in integration broker I kept getting the following error when attempting to load the connectors for the gateway (which involves contacting the gateway through the gateway URL):

Integration Gateway - External System Contact Error (158,10721)

external-system-contact-error.png

Using the DNS entry, the gateway connectors would not load, however changing the URL to use the IP address worked, e.g:

Gateway URL = http://123.123.123.123:8000/PSIGW/PeopleSoftListeningConnector

So this indicated an issue with the DNS entry or something in that space. Further checking using the nslookup command in Windows/Linux confirmed the issue, and after making relevant changes the DNS entry resolved as expected. A final step that was required was to restart the web server after making the DNS changes.

Not Authorized to Invoke Service Operation

While testing a web service (via a CI), the following message came back (this is using soapUI):

<IBResponse type="error">
  <DefaultTitle>Integration Broker Response</DefaultTitle>
  <StatusCode>20</StatusCode>
  <MessageID>536</MessageID>
  <DefaultMessage>User USER not authorized to invoke Service Operation EXAMPLE. (158,536)</DefaultMessage>
  <MessageParameters>
    <Parameter>USER</Parameter>
    <Parameter>EXAMPLE</Parameter>
  </MessageParameters>
</IBResponse>

This relates to service operation security. Specifically, the user in question does not have security to the relevant service operation.

Here's how to check/update the service operation security:

PeopleTools > Integration Broker > Integration Setup > Service Operations

Error in service HomepageT

While working on some Integration Broker configuration on an environment, I started getting a number of errors, and could no longer get past the login screen. On the PIA, the site just kept displaying the following:

an-error-has-occurred.png

Not very helpful! The application server logs (APPSRV_MMDD.LOG) was slighty more informative, giving the following messages:

Error in service HomepageT, CREF with URL can not be found: 
http://webserver.com:8000/psp/ps/EMPLOYEE/HRMS/?tab=DEFAULT
Failed to execute HomepageTemplate request  
Service HomepageT failed

Turns out, that the HRMS node had been made inactive. This was the node being used by the EMPLOYEE portal for HRMS/Campus Solutions, so it needed to be active. You can check the nodes being used by looking at the PORTAL_HOSTNODENAM field in PSPRDMDEFN.

Updating the node to active, by setting ACTIVE_NODE = 1 in PSMSGNODEDEFN for the HRMS node, stopping the application server, physically deleting the cache files, and restarting the application server resolved the issue.

Viewing an existing WSDL

If you've been creating web services in PeopleSoft, there will most likely be at time where you will need to significantly modify or remove a web service. Fortunately, PeopleSoft have provided a ways to manage web services (WSDLs).

To view the current WSDLs:

PeopleTools > Integration Broker > Provide Web Service

Search for your existing service (just hit search if you don't know what it is called). Select the appropriate service, press Next.

view-wsdl-search.png

Select the operations and press Next on step 2. Step 3 lets you view the WSDL. Press Finish on Step 4 to re-export the WSDL to the repository and get the URL to view it:

wsdl-repository-info.png

As you can see WSDL follow the following format which you might be able to use as a shortcut to find it:

http(s)://<machine>:<port>/PSIGW/PeopleSoftListeningConnector/<SERVICE_NAME>.<VERSION>.wsdl

Deleting a WSDL

To delete an existing WSDL, navigate to:

PeopleTools > Integration Broker > Service Utilities > Service Administration

The first page (tab) allows for you to search for and delete the WSDL associated with a service (which was been provided as a web service).

delete-wsdl.png

Note in PeopleTools 8.5x, there is also a hyperlink to Clear WSDL Export Status which is used to find and resolve inconsistencies with WSDL exported to the WSDL repository. It is essentially a tool to find and fix service operations without an associated WSDL which have been provided as web services.

Message Referenced in Runtime Tables

Once a message has been published in Integration broker, you can no longer update it. The fields will be disabled and the following warning will appears in the status:

Message cannot be changed. Message referenced in runtime tables.

What if you didn't get the message and schema quite right? How do you update your message?

Behind the scenes the logic that disables the message is in the function checkruntime which lives in the record PeopleCode PSSERVICESWRK.SAVE_PB.FieldFormula. The checkruntime function basically checks four tables, and if any return a row for the service operation associated with your message, the message is disabled. The four tables are:

What you'll need to do is delete any transactions associated with your message through the service operations monitor.

PeopleTools > Integration Broker > Service Operations Monitor > Monitoring
NOTE: that you'll need to open up the messages for the appropriate service & service type.

Alternatively, if you have a lot of transactions, you could clear out the four tables through the database.

Also if you see this with a message used by a web service, you'll need to check the synchronous services area of the service operations monitor. This can be viewed from PS_AMM_SYNCLIST which is a view to PSIBLOGHDR.

Exclude PeopleSoft Authentication Token

Integration broker in PeopleTools 8.5x includes a new setting in the service configuration to exclude the PeopleSoft authentication token on outbound requests to other PeopleSoft nodes. In plain english, this means you can configure service operations so that they don't require the same user/operator to exist between two PeopleSoft systems, for example between PeopleSoft HRMS and PeopleSoft Financials.

Previously, this was a requirement, otherwise the integration would fail with a message like this:

User {username} not authorized to invoke service operation {service operation}

The exclude PeopleSoft authentication token setting is available from the following navigation:

PeopleTools > Integration Broker > Configuration > Service Configuration > Exclude PSFT AUth Token Tab

From the screen, search for a relevant service or service operation (e.g. VENDOR_SYNC) and check the Exclude Token to exclude authentication and save.

exclude-psft-auth-token.png

NOTE: this is just a search page, so the item won't appear in the service operations table next time you open the page. However, you can confirm the settings have been saved by searching for all services/service operations with the Exclude PSFT Auth Token flag set (check box).

Deleting Duplicate Routings

Duplicate routings can cause problems with Integration Broker messaging. Fortunately PeopleSoft has provided (as of PeopleTools 8.48 anyway) functionality to search for and delete duplicate routings.

This is available under in the Routings tab under:

PeopleTools > Integration Broker > Service Utilities > Service Administration

delete-duplicate-routings.png

The search will find any duplicate routings and allow you to select and delete them.

Behinds the scenes PeopleCode searches the PSIBRTNGDEFN Integration Broker Routing Definition for any cases where more than one active routing is associated with the same service operation for any local to local routings, any to local routings and point to point routings.

I'm not convinced the code being used is perform the search for duplicate routings is 100% correct. I also used the following query to identify potential duplicate routings and found some cases that weren't being picked up so it might be worth double checking with this query as well.
select 
    IB_OPERATIONNAME, 
    VERSIONNAME, 
    SENDERNODENAME, 
    RECEIVERNODENAME,     
    count(distinct ROUTINGDEFNNAME)
from 
    PSIBRTNGDEFN
where 
    EFF_STATUS = 'A'
group by 
    IB_OPERATIONNAME, 
    VERSIONNAME, 
    SENDERNODENAME, 
    RECEIVERNODENAME
having count(distinct ROUTINGDEFNNAME) > 1;

PSHttpFactory init or setCertificate fails

During IB configuration while loading gateway connectors the following error messages kept appearing:

HttpTargetConnector:PSHttpFactory init or setCertificate fails (158,99999)

This is to do with two entries in the integrationGateway.properties configuration file which you can access through the Advanced Properties Page link under:

PeopleTools > Integration Broker > Configuration > Gateways > {Open Gateway} > Gateway Setup Properties

The two entries are:

The secureFileKeystorePath needs to point to the correct location of the PeopleSoft Keystore (pskey) which for example might live under $PS_HOME/webserv/<domain./piaconfig/keystore/pskey but this will vary depending on your web server and PeopleTools version. Just make sure the path is correct.

The secureFileKeyStorePasswd has a catch to it:

To encrypt the password (only in PeopleTools 8.5x), use the password encryption utility at the bottom of the gateway properties page, and paste the encrypted value in the field.

If you are setting up integration between two systems, and one is on PeopleTools 8.5x or above and one is on PeopleTools 8.4x, then you need to make sure you have set up the 1integrationGateway.properties` file correctly:

So, with such a set up, you will have a combination of encrypted and unencrypted passwords that need to match up. If you have any of this wrong, you'll get the same message above when you try to ping nodes between the two systems. That will tell you which side you have wrong.

Restarting PUBSUB

For integration broker configuration changes, it is possible just to restart the PUBSUB group of processes on the application server (rather than the entire application server).

Here's how:

In the tuxedo command line, use the help or h command to show a list of commands. The two you'll need to know are boot or b and shutdown or stop.

First you'll need to shutdown all processes in the PUBSUB group:

> shutdown -g PUBSUB

Shutting down server processes ...

        Server Id = 300 Group Id = PUBSUB Machine = server.domain.com:   shutdown succeeded
        Server Id = 301 Group Id = PUBSUB Machine = server.domain.com:   shutdown succeeded
        Server Id = 200 Group Id = PUBSUB Machine = server.domain.com:   shutdown succeeded
        Server Id = 201 Group Id = PUBSUB Machine = server.domain.com:   shutdown succeeded
        Server Id = 100 Group Id = PUBSUB Machine = server.domain.com:   shutdown succeeded
        Server Id = 101 Group Id = PUBSUB Machine = server.domain.com:   shutdown succeeded
6 processes stopped.

Then boot up the PUBSUB group again:

> boot -g PUBSUB

INFO: Oracle Tuxedo, Version 10.3.0.0, 64-bit, Patch Level (none)

Booting server processes ...

exec PSBRKHND -o ./LOGS/stdout -e ./LOGS/stderr -s PSBRKHND_dflt:BrkProcess -- -D DBNAME -S PSBRKHND_dflt :
        process id=18265 ... Started.
exec PSBRKDSP -o ./LOGS/stdout -e ./LOGS/stderr -s PSBRKDSP_dflt:Dispatch -- -D DBNAME -S PSBRKDSP_dflt :
        process id=18268 ... Started.
exec PSPUBHND -o ./LOGS/stdout -e ./LOGS/stderr -s PSPUBHND_dflt:PubConProcess -- -D DBNAME -S PSPUBHND_dflt :
        process id=18271 ... Started.
exec PSPUBDSP -o ./LOGS/stdout -e ./LOGS/stderr -s PSPUBDSP_dflt:Dispatch -- -D DBNAME -S PSPUBDSP_dflt :
        process id=18275 ... Started.
exec PSSUBHND -o ./LOGS/stdout -e ./LOGS/stderr -s PSSUBHND_dflt:SubConProcess -- -D DBNAME -S PSSUBHND_dflt :
        process id=18287 ... Started.
exec PSSUBDSP -o ./LOGS/stdout -e ./LOGS/stderr -s PSSUBDSP_dflt:Dispatch -- -D DBNAME -S PSSUBDSP_dflt :
        process id=18290 ... Started.
6 processes started.

To quit out of the tuxedo command line, type quit or just q.

Integration Broker Logging

Integration Broker logging can be very useful for troubleshooting messaging issues. It can be enabled through:

PeopleTools > Integration Broker > Configuration > Gateways.

Open your local Gateway (usually called LOCAL) and click on the Gateway Setup Properties.

Login to the gateway properties using the default user ID and password (they are provided on the page). You may need to ask your system administrator for the appropriate credentials if the default values do not work.

Once you login, click on the Advanced Properties Page link. This will open the integrationGateway.properties file and allow you to edit it.

Scroll through the integrationGateway.properties file until you find the logging section. The following line is what sets the logging level (2 is the default):

ig.log.level=2

The logging levels are:

Level Value
-100 Suppress any logging
-1 Language Exception
1 Standard Gateway Exception
2 Errors and Warnings
3 Important information, errors and warnings
4 Standard and important information, errors and warnings
5 Low importance, standard, and important information, errors and warnings

The default log level of 2 should be sufficient for troubleshooting. Remember with logging less is more. It is easier to dig deeper once you've found the relevant issue rather than creating a haystock of log output.

You can view the message and error logs using your browsing by pointing to:

If you are having trouble determining the URL for these, look at your local gateway URL or ping it and use the same URL up to the PSIGW/ and then enter the appropriate message log page name.

NOTE: the names of the HTML files are case sensitive! So errorlog.html may not work while errorLog.html (capital L in log) will work.

If there have been no messages or errors logged then these files won't exist. Their locations/names may also have been changed in the integrationGateway.propertie file.

Check the values of:

Terminology

Integration broker is one of those areas of PeopleSoft that introduces a lot of terminology (jargon). You really need to get your head around this terminology before you can understand what's going on. Unfortunately, the documentation in PeopleBooks can be a little terse. Here's a quick summary.

Gateways: Are the pathways between systems. There is always a LOCAL gateway in PeopleSoft which refers to the source PeopleSoft system. Other gateways may exist - for example when there are differences between PeopleTools versions.
Domains: Refer to PeopleSoft application server (tuxedo) domains for publish/subscribe (pub/sub) services. These are turned on at the application server and required for Integration Broker to work.
Nodes: Define what application a message belongs to - any system that integration broker can talk to (including the local system) will have a node.
Messages: Define the structure of the XML sent between nodes. The key difference between asynchronous and synchronous messages is acknowledgement - synchronous messages need to be acknowledge by the receiver, asynchronous ones don't. Web services typically use synchronous messages.
Queues: Are used to manage and group messages so that all messages are sent through the multiple pipes. Queues are used for administration of integration broker to avoid stopping everything when something is wrong. For example you can pause a one queue for messages to the report architecture without say stopping messages to the finance system.
Handlers: Contain code and logic for sending/receiving/manipulating messages.
Service Operations: Bring together messages, queues, handlers and routings. They essentially define how the message moves between systems.
Services: Group service operations into the conceptual service provided. For example, you might have a service based on a component interface. The service operations would be the methods (create/find/get/update/update data) and the service would be the interface to that component interface.