Avoid full queues with MQ Series

Introduction :

The most reliable source of information for Websphere MQ is IBM documentation. But there is other sources that are difficult to find with Google.

The documentation is available online for the recent version of Websphere MQ Series :
https://www.ibm.com/support/knowledgecenter/SSFKSJ/com.ibm.mq.helphome.doc/product_welcome_wmq.htm. Choose correctly the version of Websphere MQ Series. It is also possible to download online documentation.

Other sources of information are forum. It seems there is only few communities out there but they are very active :

http://mqseries.net/
https://listserv.meduniwien.ac.at/archives/mqser-l.html

The problem :

We have an application in production which sends message to an Websphere output queue. A partner application reads message from this queue and replied to the input queue of our application.

The input queue keeps receiving messages which are not consumed in production. In the end the input queue is full and our application would responds with a timeout.

The code uses JMS API to send/receive messages from Websphere MQ.We send the initial message and then wait for a reply from the partner application.

If we don’t receive messages from the partner we send an abort message to announce we stop communicating with the partner application. When we send the abort request sometimes we don’t receive any message back from the partner. Later the partner sends a response in the queue but the message is not consumed because the process is finished.

 // SEND MESSAGE TO OUTPUT QUEUE
 sendJMSMessage(outputQueue, request);

            boolean cancelReadMessage = false;
boolean isTimeoutOccured= false;

// WAIT UNTIL THERE IS NO MESSAGES OR TIMEOUT
            while (!cancelReadMessage ) {
                long startTime = System.currentTimeMillis();

       // CONSUME MESSAGE
                final Message message = receiveMessage(inputQueue, id);

                long timeOutPartner = getJmsTemplate().getReceiveTimeout() - (System.currentTimeMillis() - startTime) <= 0 ? JmsTemplate.RECEIVE_TIMEOUT_NO_WAIT : getJmsTemplate().getReceiveTimeout() - (System.currentTimeMillis() - startTime);
               
// DECREASE TIMEOUT GLOBAL APPLICATION
getJmsTemplate().setReceiveTimeout(timeOutPartner);

                if (message != null) {
// MESSAGE RECEIVED : PROCESS IT
} else {
  
                    if (!sessionId.equals("-1") && !isTimeoutOccured) {
// TIMEOUT OCCURED SEND ABORT MESSAGE
                    final String abordtRequest = PartnerUtils.buildAbortRequest(contextId, contextCode, sessionId);
                        sendJMSMessage(outQueueName, abordtRequest );
                        isTimeoutOccured = true;
                    } else {
                    // NO MORE MESSAGES STOP THE LOOP
                        cancelReadMessage = true;
                    }

                }
            }
 

SOLUTIONS

After analysis there are several solutions possible but we choose only one because some were not possible at that time :

Temporary Fix to clear the queue only once.

This solution does not fix the problem for good. The next day the problem is the same and the queue need to be cleared again manually.

How to Clear a MQ Queue from a Script or Program

Use CAPEXPRY to set an expiry date on all messages entering the queue.

https://www.ibm.com/developerworks/community/blogs/messaging/entry/November_26_2015_at_9_21_34_PM?lang=en
This solution is only available for MQ version MQ 8.0.0.4.

Example of command :
ALTER QUEUE CAPEXPRY

Ask the partner application to send abort messages with a time to live(ttl) tag.

Therefore the abort message would die on his own after the expiry date. This solution is not possible because the partner cannot modify the code at the time.

Example in JMS how to add a tll message

public Message createMessage(Session session) throws JMSException {
 
                           message = session.createTextMessage(messageContent);
message.setJMSExpiration(100000)
 
                           return message;
                    }

Finally the fix was to add an additional timeout when waiting for abort messages.

We find out we were not waiting long enough for the partner in the case of abort messages. We have added an additional timeout to wait for the reponse.

// SEND MESSAGE TO OUTPUT QUEUE
sendJMSMessage(outQueueName, request);

// TIMEOUT ABORT MESSAGE
Boolean timeoutabort = getTimeoutAbort(); 
            boolean cancelReadMessage = false;
            boolean isTimeoutOccured= false;

// WAIT UNTIL THERE IS NO MESSAGES OR TIMEOUT
            while (!cancelReadMessage ) {
                long startTime = System.currentTimeMillis();

                final Message message = receiveMessage(inputQueue, idSelector + " = '" + messageId + "'");

                long timeOutPartner = getJmsTemplate().getReceiveTimeout() - (System.currentTimeMillis() - startTime) - timeoutabort <= 0 ? JmsTemplate.RECEIVE_TIMEOUT_NO_WAIT : getJmsTemplate().getReceiveTimeout()
                        - (System.currentTimeMillis() - startTime);
// DECREASE TIMEOUT GLOBAL APPLICATION
                getJmsTemplate().setReceiveTimeout(timeOutPartner);

                if (message != null) {
// MESSAGE RECEIVED : PROCESS IT
} else {
  
                    if (!sessionId.equals("-1") && !isTimeoutOccured) {
                       // TIMEOUT OCCURED SEND ABORT MESSAGE
                        final String abortRequest = PartnerUtils.buildAbortRequest(contextId, contextCode, sessionId);
                        sendJMSMessage(outQueueName, abortRequest );
// SET TIMEOUT ABORT MESSAGE
                        getJmsTemplate().setReceiveTimeout(timeoutabort);
                        isTimeoutOccured= true;
                    } else {
// NO MORE MESSAGES STOP THE LOOP
                        cancelReadMessage = true;
                    }

                                    }
            }

The downside of this solution is that we wait longer now for the partner that before but the queue do not get full anymore.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s