[andyp222]

Hi,

Customer is using OTM v4.5 with very heavy HTTP integration, inbound and out, typically 20-30,000 messages per day. For whatever reason they are occasionally getting a "Broken Pipe" or ORA-00600 error. Once this happens generally most Agents will fail until there is a full restart of DB, App and Web server. The problem is that between the time of the error occuring and the restart, integrations are continuing to come it but failing to be processed correctly due to the Agent failures, requiring a lot of clean up afterwards.

We have in place an alert which will detect this issue within 5 minutes of it occuring and send emails, but it can still be up to an hour before integration feeds are turned off etc.

Can anyone think of something that can be done within GC3 (e.g. a change in the configuration) such that it immediatelty stops acknowledging the HTTP posts - in our situation this would means that the messages moved to the EAI suspended queue rather than continuing to pile into OTM.

Any thoughts, ideas greatly appreciated. And if anyone can stop the "Broken pipes", even better!

[chrisplough]

Hello,

The only way I can think of to force OTM to immediately stop accepting integration is to shutdown the OTM web server (Apache), which can take effect in a matter of seconds. If you have a separate OTM web server for integration vs. user UI, then this could work very effectively.

Otherwise, the Broken Pipe issue usually indicates a connectivity issue between the OTM app and DB servers. To investigate this, I'd begin looking at the network interfaces for errors, any Oracle trace files, and the OTM WebLogic console.log files (may be nohup.out in older versions of OTM), under $OTM_HOME/logs/weblogic/

--Chris

[sknmail@rediffmail.com]

Hi Chris,

In order to stop the integration to come to OTM and not to get processed temporarily, how about deleting all the threads for Integration queue from glog.webserver.event.EventDiagServlet.

Once you have zero threads present in Integration queue, they would get piled up in Backlog, however these won't be processed. Once if you add those deleted thread again the integration would startup as normal..

We found this to be useful when once we had an unusual bulk integration on one day, when the agentUtility was showing a huge backlog, so we streamlined the flood by stopping the integration this method, so that the agentutility queue backlog gets cleared for already integrated orders.

Let me know if this is a acceptable solution.

Regards,
Suresh

[chrisplough]

Suresh,

Personally, I wouldn't rely on this solution. The big issue is that the EventDiagServlet doesn't refresh in real-time and there's always a possibility that killing a thread will kill a transaction in process. I use the Kill and Add functions of the EventDiagServlet with caution.

--Chris

[sknmail@rediffmail.com]

Hi Chris,

I agree to your point. This issue can exists by this method.

Regards,
Suresh