Atomikos Forum
Repeated "no XAResource to rollback" error never stops |
I'm using Atomikos 3.5.9 with Oracle 11g and ActiveMQ 5.2.0.
When Atomikos loses connectivity with ActiveMQ, I understandably get error messages such as this:
2010-02-16 03:32:50,388 WARN atomikos - XA resource 'APEX_ACTIVEMQ_BROKER': resume for XID '617065782D776F726B666C6F772D736572766963652D746D31363532353030303134:617065782D776F726B666C6F772D736572766963652D746D343030363132' raised -7: the XA resource has become unavailable
javax.transaction.xa.XAException: The JMS connection has failed: Channel was inactive for too long: localhost/127.0.0.1:61616
at org.apache.activemq.TransactionContext.toXAException(TransactionContext.java:628)
at org.apache.activemq.TransactionContext.setXid(TransactionContext.java:563)
at org.apache.activemq.TransactionContext.start(TransactionContext.java:301)
at com.atomikos.datasource.xa.XAResourceTransaction.resume(XAResourceTransaction.java:647)
at com.atomikos.datasource.xa.session.BranchEnlistedStateHandler.<init>(BranchEnlistedStateHandler.java:32)
at com.atomikos.datasource.xa.session.NotInBranchStateHandler.checkEnlistBeforeUse(NotInBranchStateHandler.java:41)
at com.atomikos.datasource.xa.session.TransactionContext.checkEnlistBeforeUse(TransactionContext.java:60)
at com.atomikos.datasource.xa.session.SessionHandleState.notifyBeforeUse(SessionHandleState.java:151)
at com.atomikos.jms.ConsumerProducerSupport.enlist(ConsumerProducerSupport.java:67)
at com.atomikos.jms.AtomikosJmsMessageConsumerProxy.receive(AtomikosJmsMessageConsumerProxy.java:45)
at com.atomikos.jms.AtomikosJmsMessageConsumerProxy.receive(AtomikosJmsMessageConsumerProxy.java:112)
...and this:
2010-02-16 03:33:21,899 WARN atomikos - atomikos xa session proxy for resource APEX_ACTIVEMQ_BROKER: could not close JMS session
javax.jms.JMSException: Channel was inactive for too long: localhost/127.0.0.1:61616
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
at org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1216)
at org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1208)
at org.apache.activemq.ActiveMQSession.close(ActiveMQSession.java:556)
at com.atomikos.jms.AtomikosJmsXaSessionProxy.destroy(AtomikosJmsXaSessionProxy.java:174)
at com.atomikos.jms.AtomikosJmsXaSessionProxy.onTerminated(AtomikosJmsXaSessionProxy.java:210)
at com.atomikos.datasource.xa.session.SessionHandleState.fireTerminatedEvent(SessionHandleState.java:244)
at com.atomikos.datasource.xa.session.SessionHandleState.notifySessionClosed(SessionHandleState.java:109)
at com.atomikos.jms.AtomikosJmsXaSessionProxy.invoke(AtomikosJmsXaSessionProxy.java:100)
...and this:
2010-02-16 03:33:28,108 WARN atomikos - atomikos connection pool 'APEX_ACTIVEMQ_BROKER': error creating proxy of connection atomikos pooled connection for resource APEX_ACTIVEMQ_BROKER
com.atomikos.datasource.pool.CreateConnectionException: atomikos pooled connection for resource APEX_ACTIVEMQ_BROKER: connection is erroneous
at com.atomikos.jms.AtomikosPooledJmsConnection.testUnderlyingConnection(AtomikosPooledJmsConnection.java:43)
at com.atomikos.datasource.pool.AbstractXPooledConnection.createConnectionProxy(AbstractXPooledConnection.java:43)
at com.atomikos.datasource.pool.ConnectionPool.borrowConnection(ConnectionPool.java:135)
at com.atomikos.jms.AtomikosConnectionFactoryBean.createConnection(AtomikosConnectionFactoryBean.java:560)
I expect some error messages when one of the XA resources is not available. But sometimes when the JVM starts up after connectivity to ActiveMQ has been restored, I get the following error repeatedly and it never stops:
2010-02-17 09:45:21,461 WARN atomikos - XAResourceTransaction apex-assign-tm0446000003apex-assign-tm37290: no XAResource to rollback - the required resource is probably not yet intialized?
2010-02-17 09:45:21,461 WARN atomikos - XAResourceTransaction apex-assign-tm0570600002apex-assign-tm5750: no XAResource to rollback - the required resource is probably not yet intialized?
2010-02-17 09:45:21,455 WARN atomikos - XAResourceTransaction apex-assign-tm0236300008apex-assign-tm195489: no XAResource to rollback - the required resource is probably not yet intialized?
...
Restarting the JVN when all XA resources are accessible does not clear it up. The only way I've been able to make it stop is to delete the transaction log.
Why does it complain that the required resource is not initialized when all resources are available? Any ideas?
Thanks,
Jim
When Atomikos loses connectivity with ActiveMQ, I understandably get error messages such as this:
2010-02-16 03:32:50,388 WARN atomikos - XA resource 'APEX_ACTIVEMQ_BROKER': resume for XID '617065782D776F726B666C6F772D736572766963652D746D31363532353030303134:617065782D776F726B666C6F772D736572766963652D746D343030363132' raised -7: the XA resource has become unavailable
javax.transaction.xa.XAException: The JMS connection has failed: Channel was inactive for too long: localhost/127.0.0.1:61616
at org.apache.activemq.TransactionContext.toXAException(TransactionContext.java:628)
at org.apache.activemq.TransactionContext.setXid(TransactionContext.java:563)
at org.apache.activemq.TransactionContext.start(TransactionContext.java:301)
at com.atomikos.datasource.xa.XAResourceTransaction.resume(XAResourceTransaction.java:647)
at com.atomikos.datasource.xa.session.BranchEnlistedStateHandler.<init>(BranchEnlistedStateHandler.java:32)
at com.atomikos.datasource.xa.session.NotInBranchStateHandler.checkEnlistBeforeUse(NotInBranchStateHandler.java:41)
at com.atomikos.datasource.xa.session.TransactionContext.checkEnlistBeforeUse(TransactionContext.java:60)
at com.atomikos.datasource.xa.session.SessionHandleState.notifyBeforeUse(SessionHandleState.java:151)
at com.atomikos.jms.ConsumerProducerSupport.enlist(ConsumerProducerSupport.java:67)
at com.atomikos.jms.AtomikosJmsMessageConsumerProxy.receive(AtomikosJmsMessageConsumerProxy.java:45)
at com.atomikos.jms.AtomikosJmsMessageConsumerProxy.receive(AtomikosJmsMessageConsumerProxy.java:112)
...and this:
2010-02-16 03:33:21,899 WARN atomikos - atomikos xa session proxy for resource APEX_ACTIVEMQ_BROKER: could not close JMS session
javax.jms.JMSException: Channel was inactive for too long: localhost/127.0.0.1:61616
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
at org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1216)
at org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1208)
at org.apache.activemq.ActiveMQSession.close(ActiveMQSession.java:556)
at com.atomikos.jms.AtomikosJmsXaSessionProxy.destroy(AtomikosJmsXaSessionProxy.java:174)
at com.atomikos.jms.AtomikosJmsXaSessionProxy.onTerminated(AtomikosJmsXaSessionProxy.java:210)
at com.atomikos.datasource.xa.session.SessionHandleState.fireTerminatedEvent(SessionHandleState.java:244)
at com.atomikos.datasource.xa.session.SessionHandleState.notifySessionClosed(SessionHandleState.java:109)
at com.atomikos.jms.AtomikosJmsXaSessionProxy.invoke(AtomikosJmsXaSessionProxy.java:100)
...and this:
2010-02-16 03:33:28,108 WARN atomikos - atomikos connection pool 'APEX_ACTIVEMQ_BROKER': error creating proxy of connection atomikos pooled connection for resource APEX_ACTIVEMQ_BROKER
com.atomikos.datasource.pool.CreateConnectionException: atomikos pooled connection for resource APEX_ACTIVEMQ_BROKER: connection is erroneous
at com.atomikos.jms.AtomikosPooledJmsConnection.testUnderlyingConnection(AtomikosPooledJmsConnection.java:43)
at com.atomikos.datasource.pool.AbstractXPooledConnection.createConnectionProxy(AbstractXPooledConnection.java:43)
at com.atomikos.datasource.pool.ConnectionPool.borrowConnection(ConnectionPool.java:135)
at com.atomikos.jms.AtomikosConnectionFactoryBean.createConnection(AtomikosConnectionFactoryBean.java:560)
I expect some error messages when one of the XA resources is not available. But sometimes when the JVM starts up after connectivity to ActiveMQ has been restored, I get the following error repeatedly and it never stops:
2010-02-17 09:45:21,461 WARN atomikos - XAResourceTransaction apex-assign-tm0446000003apex-assign-tm37290: no XAResource to rollback - the required resource is probably not yet intialized?
2010-02-17 09:45:21,461 WARN atomikos - XAResourceTransaction apex-assign-tm0570600002apex-assign-tm5750: no XAResource to rollback - the required resource is probably not yet intialized?
2010-02-17 09:45:21,455 WARN atomikos - XAResourceTransaction apex-assign-tm0236300008apex-assign-tm195489: no XAResource to rollback - the required resource is probably not yet intialized?
...
Restarting the JVN when all XA resources are accessible does not clear it up. The only way I've been able to make it stop is to delete the transaction log.
Why does it complain that the required resource is not initialized when all resources are available? Any ideas?
Thanks,
Jim
Please send a tm.out in DEBUG mode to our support email (see above). We will look at this as soon as we have time.
Thanks
Thanks
Guy Pardon 
Wednesday, February 17, 2010
Wednesday, February 17, 2010
any progress on this, same issue here ?
16:13:32,814 INFO atomikos:85 - XAResource.prepare ( 3132372E302E312E312E746D30303133333030303231:3132372E302E312E312E746D313333 ) returning XAResource.XA_RDONLY on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1133fd6
16:13:32,816 INFO atomikos:85 - createCompositeTransaction ( 30000 ): created new ROOT transaction with id 127.0.1.1.tm0013500021
16:13:32,817 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:1:2:1, started=true }: receive ( 5000 )...
16:13:32,817 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:1:2:1, started=true }: receive ( 5000 , null )...
16:13:32,818 INFO atomikos:85 - addParticipant ( XAResourceTransaction: 3132372E302E312E312E746D30303133353030303231:3132372E302E312E312E746D313335 ) for transaction 127.0.1.1.tm0013500021
16:13:32,819 INFO atomikos:85 - XAResource.start ( 3132372E302E312E312E746D30303133353030303231:3132372E302E312E312E746D313335 , XAResource.TMNOFLAGS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1133fd6
16:13:32,895 INFO atomikos:85 - atomikos xa session proxy for resource amq1: calling toString on JMS driver session...
16:13:32,895 DEBUG EndpointImplEx$1:347 - Consumer [atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:0:2:1, started=true }] of session [ActiveMQSession {id=ID:d820-33360-1268147270432-0:0:2,started=true}] did not receive a message
16:13:32,896 INFO atomikos:85 - commit() done (by application) of transaction 127.0.1.1.tm0013400021
16:13:32,898 INFO atomikos:85 - XAResource.end ( 3132372E302E312E312E746D30303133343030303231:3132372E302E312E312E746D313334 , XAResource.TMSUCCESS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1308275
16:13:32,916 INFO atomikos:85 - XAResource.prepare ( 3132372E302E312E312E746D30303133343030303231:3132372E302E312E312E746D313334 ) returning XAResource.XA_RDONLY on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1308275
16:13:32,923 INFO atomikos:85 - createCompositeTransaction ( 30000 ): created new ROOT transaction with id 127.0.1.1.tm0013600021
16:13:32,924 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:0:2:1, started=true }: receive ( 5000 )...
16:13:32,925 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:0:2:1, started=true }: receive ( 5000 , null )...
16:13:32,926 INFO atomikos:85 - addParticipant ( XAResourceTransaction: 3132372E302E312E312E746D30303133363030303231:3132372E302E312E312E746D313336 ) for transaction 127.0.1.1.tm0013600021
16:13:32,926 INFO atomikos:85 - XAResource.start ( 3132372E302E312E312E746D30303133363030303231:3132372E302E312E312E746D313336 , XAResource.TMNOFLAGS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1308275
16:13:34,574 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0046000009127.0.1.1.tm462: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:34,674 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0389100008127.0.1.1.tm3893: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:34,763 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0389000008127.0.1.1.tm3892: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:34,829 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0049100008127.0.1.1.tm491: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:34,919 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0107700008127.0.1.1.tm1077: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:34,953 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0107600008127.0.1.1.tm1076: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:35,033 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0049200008127.0.1.1.tm492: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:35,083 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0017200007127.0.1.1.tm172: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:37,820 INFO atomikos:85 - atomikos xa session proxy for resource amq1: calling toString on JMS driver session...
16:13:37,821 DEBUG EndpointImplEx$1:347 - Consumer [atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:1:2:1, started=true }] of session [ActiveMQSession {id=ID:d820-33360-1268147270432-0:1:2,started=true}] did not receive a message
16:13:37,822 INFO atomikos:85 - commit() done (by application) of transaction 127.0.1.1.tm0013500021
16:13:37,824 INFO atomikos:85 - XAResource.end ( 3132372E302E312E312E746D30303133353030303231:3132372E302E312E312E746D313335 , XAResource.TMSUCCESS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1133fd6
16:13:37,827 INFO atomikos:85 - XAResource.prepare ( 3132372E302E312E312E746D30303133353030303231:3132372E302E312E312E746D313335 ) returning XAResource.XA_RDONLY on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1133fd6
16:13:37,830 INFO atomikos:85 - createCompositeTransaction ( 30000 ): created new ROOT transaction with id 127.0.1.1.tm0013700021
16:13:37,830 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:1:2:1, started=true }: receive ( 5000 )...
16:13:37,831 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:1:2:1, started=true }: receive ( 5000 , null )...
16:13:37,832 INFO atomikos:85 - addParticipant ( XAResourceTransaction: 3132372E302E312E312E746D30303133373030303231:3132372E302E312E312E746D313337 ) for transaction 127.0.1.1.tm0013700021
16:13:37,833 INFO atomikos:85 - XAResource.start ( 3132372E302E312E312E746D30303133373030303231:3132372E302E312E312E746D313337 , XAResource.TMNOFLAGS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1133fd6
16:13:37,928 INFO atomikos:85 - atomikos xa session proxy for resource amq1: calling toString on JMS driver session...
16:13:37,929 DEBUG EndpointImplEx$1:347 - Consumer [atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:0:2:1, started=true }] of session [ActiveMQSession {id=ID:d820-33360-1268147270432-0:0:2,started=true}] did not receive a message
16:13:37,929 INFO atomikos:85 - commit() done (by application) of transaction 127.0.1.1.tm0013600021
16:13:37,931 INFO atomikos:85 - XAResource.end ( 3132372E302E312E312E746D30303133363030303231:3132372E302E312E312E746D313336 , XAResource.TMSUCCESS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1308275
16:13:37,934 INFO atomikos:85 - XAResource.prepare ( 3132372E302E312E312E746D30303133363030303231:3132372E302E312E312E746D313336 ) returning XAResource.XA_RDONLY on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1308275
16:13:37,936 INFO atomikos:85 - createCompositeTransaction ( 30000 ): created new ROOT transaction with id 127.0.1.1.tm0013800021
16:13:37,937 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:0:2:1, started=true }: receive ( 5000 )...
16:13:37,938 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:0:2:1, started=true }: receive ( 5000 , null )...
16:13:37,939 INFO atomikos:85 - addParticipant ( XAResourceTransaction: 3132372E302E312E312E746D30303133383030303231:3132372E302E312E312E746D313338 ) for transaction 127.0.1.1.tm0013800021
16:13:37,939 INFO atomikos:85 - XAResource.start ( 3132372E302E312E312E746D30303133383030303231:3132372E302E312E312E746D313338 , XAResource.TMNOFLAGS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1308275
16:13:42,834 INFO atomikos:85 - atomikos xa session proxy for resource amq1: calling toString on JMS driver session...
16:13:42,835 DEBUG EndpointImplEx$1:347 - Consumer [atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:1:2:1, started=true }] of session [ActiveMQSession {id=ID:d820-33360-1268147270432-0:1:2,started=true}] did not receive a message
16:13:42,836 INFO atomikos:85 - commit() done (by application) of transaction 127.0.1.1.tm0013700021
16:13:42,838 INFO atomikos:85 - XAResource.end ( 3132372E302E312E312E746D30303133373030303231:3132372E302E312E312E746D313337 , XAResource.TMSUCCESS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1133fd6
16:13:42,840 INFO atomikos:85 - XAResource.prepare ( 3132372E302E312E312E746D30303133373030303231:3132372E302E312E312E746D313337 ) returning XAResource.XA_RDONLY on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1133fd6
16:13:42,843 INFO atomikos:85 - createCompositeTransaction ( 30000 ): created new ROOT transaction with id 127.0.1.1.tm0013900021
16:13:42,843 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:1:2:1, started=true }: receive ( 5000 )...
16:13:42,844 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:1:2:1, started=true }: receive ( 5000 , null )...
16:13:42,845 INFO atomikos:85 - addParticipant ( XAResourceTransaction: 3132372E302E312E312E746D30303133393030303231:3132372E302E312E312E746D313339 ) for transaction 127.0.1.1.tm0013900021
16:13:42,846 INFO atomikos:85 - XAResource.start ( 3132372E302E312E312E746D30303133393030303231:3132372E302E312E312E746D313339 , XAResource.TMNOFLAGS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1133fd6
16:13:42,941 INFO atomikos:85 - atomikos xa session proxy for resource amq1: calling toString on JMS driver session...
16:13:42,942 DEBUG EndpointImplEx$1:347 - Consumer [atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:0:2:1, started=true }] of session [ActiveMQSession {id=ID:d820-33360-1268147270432-0:0:2,started=true}] did not receive a message
16:13:42,943 INFO atomikos:85 - commit() done (by application) of transaction 127.0.1.1.tm0013800021
16:13:42,944 INFO atomikos:85 - XAResource.end ( 3132372E302E312E312E746D30303133383030303231:3132372E302E312E312E746D313338 , XAResource.TMSUCCESS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1308275
16:13:42,947 INFO atomikos:85 - XAResource.prepare ( 3132372E302E312E312E746D30303133383030303231:3132372E302E312E312E746D313338 ) returning XAResource.XA_RDONLY on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1308275
16:13:42,963 INFO atomikos:85 - createCompositeTransaction ( 30000 ): created new ROOT transaction with id 127.0.1.1.tm0014000021
16:13:42,964 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:0:2:1, started=true }: receive ( 5000 )...
16:13:42,964 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:0:2:1, started=true }: receive ( 5000 , null )...
16:13:42,965 INFO atomikos:85 - addParticipant ( XAResourceTransaction: 3132372E302E312E312E746D30303134303030303231:3132372E302E312E312E746D313430 ) for transaction 127.0.1.1.tm0014000021
16:13:42,966 INFO atomikos:85 - XAResource.start ( 3132372E302E312E312E746D30303134303030303231:3132372E302E312E312E746D313430 , XAResource.TMNOFLAGS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1308275
16:13:44,575 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0046000009127.0.1.1.tm462: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:44,676 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0389100008127.0.1.1.tm3893: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:44,765 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0389000008127.0.1.1.tm3892: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:44,799 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0046000009127.0.1.1.tm462: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:44,830 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0049100008127.0.1.1.tm491: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:44,908 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0389100008127.0.1.1.tm3893: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:44,930 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0107700008127.0.1.1.tm1077: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:44,954 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0107600008127.0.1.1.tm1076: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:44,986 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0389000008127.0.1.1.tm3892: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:45,041 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0049200008127.0.1.1.tm492: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:45,053 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0049100008127.0.1.1.tm491: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:45,085 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0017200007127.0.1.1.tm172: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:45,154 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0107700008127.0.1.1.tm1077: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:45,221 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0107600008127.0.1.1.tm1076: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:45,328 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0049200008127.0.1.1.tm492: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:45,400 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0017200007127.0.1.1.tm172: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:47,847 INFO atomikos:85 - atomikos xa session proxy for resource amq1: calling toString on JMS driver session...
16:13:47,848 DEBUG EndpointImplEx$1:347 - Consumer [atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:1:2:1, started=true }] of session [ActiveMQSession {id=ID:d820-33360-1268147270432-0:1:2,started=true}] did not receive a message
16:13:47,849 INFO atomikos:85 - commit() done (by application) of transaction 127.0.1.1.tm0013900021
16:13:47,851 INFO atomikos:85 - XAResource.end ( 3132372E302E312E312E746D30303133393030303231:3132372E302E312E312E746D313339 , XAResource.TMSUCCESS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1133fd6
16:13:47,854 INFO atomikos:85 - XAResource.prepare ( 3132372E302E312E312E746D30303133393030303231:3132372E302E312E312E746D313339 ) returning XAResource.XA_RDONLY on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1133fd6
16:13:47,856 INFO atomikos:85 - createCompositeTransaction ( 30000 ): created new ROOT transaction with id 127.0.1.1.tm0014100021
16:13:47,857 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:1:2:1, started=true }: receive ( 5000 )...
16:13:47,857 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:1:2:1, started=true }: receive ( 5000 , null )...
16:13:47,858 INFO atomikos:85 - addParticipant ( XAResourceTransaction: 3132372E302E312E312E746D30303134313030303231:3132372E302E312E312E746D313431 ) for transaction 127.0.1.1.tm0014100021
16:13:47,859 INFO atomikos:85 - XAResource.start ( 3132372E302E312E312E746D30303134313030303231:3132372E302E312E312E746D313431 , XAResource.TMNOFLAGS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1133fd6
16:13:47,967 INFO atomikos:85 - atomikos xa session proxy for resource amq1: calling toString on JMS driver session...
16:13:47,968 DEBUG EndpointImplEx$1:347 - Consumer [atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:0:2:1, started=true }] of session [ActiveMQSession {id=ID:d820-33360-1268147270432-0:0:2,started=true}] did not receive a message
16:13:47,969 INFO atomikos:85 - commit() done (by application) of transaction 127.0.1.1.tm0014000021
16:13:47,971 INFO atomikos:85 - XAResource.end ( 3132372E302E312E312E746D30303134303030303231:3132372E302E312E312E746D313430 , XAResource.TMSUCCESS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1308275
16:13:32,814 INFO atomikos:85 - XAResource.prepare ( 3132372E302E312E312E746D30303133333030303231:3132372E302E312E312E746D313333 ) returning XAResource.XA_RDONLY on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1133fd6
16:13:32,816 INFO atomikos:85 - createCompositeTransaction ( 30000 ): created new ROOT transaction with id 127.0.1.1.tm0013500021
16:13:32,817 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:1:2:1, started=true }: receive ( 5000 )...
16:13:32,817 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:1:2:1, started=true }: receive ( 5000 , null )...
16:13:32,818 INFO atomikos:85 - addParticipant ( XAResourceTransaction: 3132372E302E312E312E746D30303133353030303231:3132372E302E312E312E746D313335 ) for transaction 127.0.1.1.tm0013500021
16:13:32,819 INFO atomikos:85 - XAResource.start ( 3132372E302E312E312E746D30303133353030303231:3132372E302E312E312E746D313335 , XAResource.TMNOFLAGS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1133fd6
16:13:32,895 INFO atomikos:85 - atomikos xa session proxy for resource amq1: calling toString on JMS driver session...
16:13:32,895 DEBUG EndpointImplEx$1:347 - Consumer [atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:0:2:1, started=true }] of session [ActiveMQSession {id=ID:d820-33360-1268147270432-0:0:2,started=true}] did not receive a message
16:13:32,896 INFO atomikos:85 - commit() done (by application) of transaction 127.0.1.1.tm0013400021
16:13:32,898 INFO atomikos:85 - XAResource.end ( 3132372E302E312E312E746D30303133343030303231:3132372E302E312E312E746D313334 , XAResource.TMSUCCESS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1308275
16:13:32,916 INFO atomikos:85 - XAResource.prepare ( 3132372E302E312E312E746D30303133343030303231:3132372E302E312E312E746D313334 ) returning XAResource.XA_RDONLY on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1308275
16:13:32,923 INFO atomikos:85 - createCompositeTransaction ( 30000 ): created new ROOT transaction with id 127.0.1.1.tm0013600021
16:13:32,924 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:0:2:1, started=true }: receive ( 5000 )...
16:13:32,925 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:0:2:1, started=true }: receive ( 5000 , null )...
16:13:32,926 INFO atomikos:85 - addParticipant ( XAResourceTransaction: 3132372E302E312E312E746D30303133363030303231:3132372E302E312E312E746D313336 ) for transaction 127.0.1.1.tm0013600021
16:13:32,926 INFO atomikos:85 - XAResource.start ( 3132372E302E312E312E746D30303133363030303231:3132372E302E312E312E746D313336 , XAResource.TMNOFLAGS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1308275
16:13:34,574 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0046000009127.0.1.1.tm462: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:34,674 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0389100008127.0.1.1.tm3893: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:34,763 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0389000008127.0.1.1.tm3892: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:34,829 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0049100008127.0.1.1.tm491: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:34,919 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0107700008127.0.1.1.tm1077: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:34,953 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0107600008127.0.1.1.tm1076: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:35,033 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0049200008127.0.1.1.tm492: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:35,083 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0017200007127.0.1.1.tm172: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:37,820 INFO atomikos:85 - atomikos xa session proxy for resource amq1: calling toString on JMS driver session...
16:13:37,821 DEBUG EndpointImplEx$1:347 - Consumer [atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:1:2:1, started=true }] of session [ActiveMQSession {id=ID:d820-33360-1268147270432-0:1:2,started=true}] did not receive a message
16:13:37,822 INFO atomikos:85 - commit() done (by application) of transaction 127.0.1.1.tm0013500021
16:13:37,824 INFO atomikos:85 - XAResource.end ( 3132372E302E312E312E746D30303133353030303231:3132372E302E312E312E746D313335 , XAResource.TMSUCCESS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1133fd6
16:13:37,827 INFO atomikos:85 - XAResource.prepare ( 3132372E302E312E312E746D30303133353030303231:3132372E302E312E312E746D313335 ) returning XAResource.XA_RDONLY on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1133fd6
16:13:37,830 INFO atomikos:85 - createCompositeTransaction ( 30000 ): created new ROOT transaction with id 127.0.1.1.tm0013700021
16:13:37,830 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:1:2:1, started=true }: receive ( 5000 )...
16:13:37,831 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:1:2:1, started=true }: receive ( 5000 , null )...
16:13:37,832 INFO atomikos:85 - addParticipant ( XAResourceTransaction: 3132372E302E312E312E746D30303133373030303231:3132372E302E312E312E746D313337 ) for transaction 127.0.1.1.tm0013700021
16:13:37,833 INFO atomikos:85 - XAResource.start ( 3132372E302E312E312E746D30303133373030303231:3132372E302E312E312E746D313337 , XAResource.TMNOFLAGS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1133fd6
16:13:37,928 INFO atomikos:85 - atomikos xa session proxy for resource amq1: calling toString on JMS driver session...
16:13:37,929 DEBUG EndpointImplEx$1:347 - Consumer [atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:0:2:1, started=true }] of session [ActiveMQSession {id=ID:d820-33360-1268147270432-0:0:2,started=true}] did not receive a message
16:13:37,929 INFO atomikos:85 - commit() done (by application) of transaction 127.0.1.1.tm0013600021
16:13:37,931 INFO atomikos:85 - XAResource.end ( 3132372E302E312E312E746D30303133363030303231:3132372E302E312E312E746D313336 , XAResource.TMSUCCESS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1308275
16:13:37,934 INFO atomikos:85 - XAResource.prepare ( 3132372E302E312E312E746D30303133363030303231:3132372E302E312E312E746D313336 ) returning XAResource.XA_RDONLY on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1308275
16:13:37,936 INFO atomikos:85 - createCompositeTransaction ( 30000 ): created new ROOT transaction with id 127.0.1.1.tm0013800021
16:13:37,937 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:0:2:1, started=true }: receive ( 5000 )...
16:13:37,938 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:0:2:1, started=true }: receive ( 5000 , null )...
16:13:37,939 INFO atomikos:85 - addParticipant ( XAResourceTransaction: 3132372E302E312E312E746D30303133383030303231:3132372E302E312E312E746D313338 ) for transaction 127.0.1.1.tm0013800021
16:13:37,939 INFO atomikos:85 - XAResource.start ( 3132372E302E312E312E746D30303133383030303231:3132372E302E312E312E746D313338 , XAResource.TMNOFLAGS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1308275
16:13:42,834 INFO atomikos:85 - atomikos xa session proxy for resource amq1: calling toString on JMS driver session...
16:13:42,835 DEBUG EndpointImplEx$1:347 - Consumer [atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:1:2:1, started=true }] of session [ActiveMQSession {id=ID:d820-33360-1268147270432-0:1:2,started=true}] did not receive a message
16:13:42,836 INFO atomikos:85 - commit() done (by application) of transaction 127.0.1.1.tm0013700021
16:13:42,838 INFO atomikos:85 - XAResource.end ( 3132372E302E312E312E746D30303133373030303231:3132372E302E312E312E746D313337 , XAResource.TMSUCCESS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1133fd6
16:13:42,840 INFO atomikos:85 - XAResource.prepare ( 3132372E302E312E312E746D30303133373030303231:3132372E302E312E312E746D313337 ) returning XAResource.XA_RDONLY on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1133fd6
16:13:42,843 INFO atomikos:85 - createCompositeTransaction ( 30000 ): created new ROOT transaction with id 127.0.1.1.tm0013900021
16:13:42,843 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:1:2:1, started=true }: receive ( 5000 )...
16:13:42,844 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:1:2:1, started=true }: receive ( 5000 , null )...
16:13:42,845 INFO atomikos:85 - addParticipant ( XAResourceTransaction: 3132372E302E312E312E746D30303133393030303231:3132372E302E312E312E746D313339 ) for transaction 127.0.1.1.tm0013900021
16:13:42,846 INFO atomikos:85 - XAResource.start ( 3132372E302E312E312E746D30303133393030303231:3132372E302E312E312E746D313339 , XAResource.TMNOFLAGS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1133fd6
16:13:42,941 INFO atomikos:85 - atomikos xa session proxy for resource amq1: calling toString on JMS driver session...
16:13:42,942 DEBUG EndpointImplEx$1:347 - Consumer [atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:0:2:1, started=true }] of session [ActiveMQSession {id=ID:d820-33360-1268147270432-0:0:2,started=true}] did not receive a message
16:13:42,943 INFO atomikos:85 - commit() done (by application) of transaction 127.0.1.1.tm0013800021
16:13:42,944 INFO atomikos:85 - XAResource.end ( 3132372E302E312E312E746D30303133383030303231:3132372E302E312E312E746D313338 , XAResource.TMSUCCESS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1308275
16:13:42,947 INFO atomikos:85 - XAResource.prepare ( 3132372E302E312E312E746D30303133383030303231:3132372E302E312E312E746D313338 ) returning XAResource.XA_RDONLY on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1308275
16:13:42,963 INFO atomikos:85 - createCompositeTransaction ( 30000 ): created new ROOT transaction with id 127.0.1.1.tm0014000021
16:13:42,964 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:0:2:1, started=true }: receive ( 5000 )...
16:13:42,964 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:0:2:1, started=true }: receive ( 5000 , null )...
16:13:42,965 INFO atomikos:85 - addParticipant ( XAResourceTransaction: 3132372E302E312E312E746D30303134303030303231:3132372E302E312E312E746D313430 ) for transaction 127.0.1.1.tm0014000021
16:13:42,966 INFO atomikos:85 - XAResource.start ( 3132372E302E312E312E746D30303134303030303231:3132372E302E312E312E746D313430 , XAResource.TMNOFLAGS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1308275
16:13:44,575 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0046000009127.0.1.1.tm462: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:44,676 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0389100008127.0.1.1.tm3893: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:44,765 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0389000008127.0.1.1.tm3892: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:44,799 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0046000009127.0.1.1.tm462: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:44,830 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0049100008127.0.1.1.tm491: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:44,908 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0389100008127.0.1.1.tm3893: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:44,930 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0107700008127.0.1.1.tm1077: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:44,954 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0107600008127.0.1.1.tm1076: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:44,986 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0389000008127.0.1.1.tm3892: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:45,041 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0049200008127.0.1.1.tm492: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:45,053 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0049100008127.0.1.1.tm491: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:45,085 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0017200007127.0.1.1.tm172: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:45,154 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0107700008127.0.1.1.tm1077: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:45,221 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0107600008127.0.1.1.tm1076: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:45,328 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0049200008127.0.1.1.tm492: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:45,400 WARN atomikos:82 - XAResourceTransaction 127.0.1.1.tm0017200007127.0.1.1.tm172: no XAResource to rollback - the required resource is probably not yet intialized?
16:13:47,847 INFO atomikos:85 - atomikos xa session proxy for resource amq1: calling toString on JMS driver session...
16:13:47,848 DEBUG EndpointImplEx$1:347 - Consumer [atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:1:2:1, started=true }] of session [ActiveMQSession {id=ID:d820-33360-1268147270432-0:1:2,started=true}] did not receive a message
16:13:47,849 INFO atomikos:85 - commit() done (by application) of transaction 127.0.1.1.tm0013900021
16:13:47,851 INFO atomikos:85 - XAResource.end ( 3132372E302E312E312E746D30303133393030303231:3132372E302E312E312E746D313339 , XAResource.TMSUCCESS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1133fd6
16:13:47,854 INFO atomikos:85 - XAResource.prepare ( 3132372E302E312E312E746D30303133393030303231:3132372E302E312E312E746D313339 ) returning XAResource.XA_RDONLY on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1133fd6
16:13:47,856 INFO atomikos:85 - createCompositeTransaction ( 30000 ): created new ROOT transaction with id 127.0.1.1.tm0014100021
16:13:47,857 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:1:2:1, started=true }: receive ( 5000 )...
16:13:47,857 INFO atomikos:85 - atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:1:2:1, started=true }: receive ( 5000 , null )...
16:13:47,858 INFO atomikos:85 - addParticipant ( XAResourceTransaction: 3132372E302E312E312E746D30303134313030303231:3132372E302E312E312E746D313431 ) for transaction 127.0.1.1.tm0014100021
16:13:47,859 INFO atomikos:85 - XAResource.start ( 3132372E302E312E312E746D30303134313030303231:3132372E302E312E312E746D313431 , XAResource.TMNOFLAGS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1133fd6
16:13:47,967 INFO atomikos:85 - atomikos xa session proxy for resource amq1: calling toString on JMS driver session...
16:13:47,968 DEBUG EndpointImplEx$1:347 - Consumer [atomikos TopicSubscriber proxy for ActiveMQMessageConsumer { value=ID:d820-33360-1268147270432-0:0:2:1, started=true }] of session [ActiveMQSession {id=ID:d820-33360-1268147270432-0:0:2,started=true}] did not receive a message
16:13:47,969 INFO atomikos:85 - commit() done (by application) of transaction 127.0.1.1.tm0014000021
16:13:47,971 INFO atomikos:85 - XAResource.end ( 3132372E302E312E312E746D30303134303030303231:3132372E302E312E312E746D313430 , XAResource.TMSUCCESS ) on resource amq1 represented by XAResource instance org.apache.activemq.TransactionContext@1308275
I've been distracted by other projects recently so I haven't spent enough time figuring out exactly how to reproduce this. However, it does seem to happen when the system has been idle for about a day.
Here is the tm.out log file in DEBUG mode. Note that there is no traffic through the system when this happens, just threads listening on JMS queues when the JMS broker connectivity is lost.
I will also send this to the support email.
10-03-26 14:59:26,500 [http-8081-1] Starting read of logfile /Users/jbreen/tools/apache-tomcat/apache-tomcat-6.0.20/data/tm/apex-workflow-service-tmlog0.log
10-03-26 14:59:26,500 [http-8081-1] Done read of logfile
10-03-26 14:59:26,500 [http-8081-1] Logfile closed: /Users/jbreen/tools/apache-tomcat/apache-tomcat-6.0.20/data/tm/apex-workflow-service-tmlog0.log
10-03-26 14:59:26,621 [http-8081-1] APEX_ACTIVEMQ_BROKER: refreshed XAResource
10-03-26 14:59:27,062 [http-8081-1] APEX_ORACLE_DB: refreshed XAResource
10-03-27 23:02:28,147 [jobImportCompleteEventListener-1] Local heuristic termination of coordinator apex-workflow-service-tm1390500020 with state ABORTING
10-03-27 23:02:41,068 [jobImportCompleteEventListener-1] Error in proxy
javax.jms.JMSException: Channel was inactive for too long: localhost/127.0.0.1:61616
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
at org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1216)
at org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1208)
at org.apache.activemq.ActiveMQSession.asyncSendPacket(ActiveMQSession.java:1762)
at org.apache.activemq.ActiveMQMessageConsumer.close(ActiveMQMessageConsumer.java:594)
at com.atomikos.jms.AtomikosJmsMessageConsumerProxy.close(AtomikosJmsMessageConsumerProxy.java:72)
at org.springframework.jms.support.JmsUtils.closeMessageConsumer(JmsUtils.java:151)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.clearResources(DefaultMessageListenerContainer.java:1019)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:887)
at java.lang.Thread.run(Thread.java:637)
Caused by: org.apache.activemq.transport.InactivityIOException: Channel was inactive for too long: localhost/127.0.0.1:61616
at org.apache.activemq.transport.InactivityMonitor.oneway(InactivityMonitor.java:225)
at org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:83)
at org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:100)
at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40)
at org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
at org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1214)
... 8 more
10-03-27 23:02:47,001 [jobImportCompleteEventListener-1] atomikos xa session proxy for resource APEX_ACTIVEMQ_BROKER: could not close JMS session
javax.jms.JMSException: Channel was inactive for too long: localhost/127.0.0.1:61616
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
at org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1216)
at org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1208)
at org.apache.activemq.ActiveMQSession.close(ActiveMQSession.java:556)
at com.atomikos.jms.AtomikosJmsXaSessionProxy.destroy(AtomikosJmsXaSessionProxy.java:174)
at com.atomikos.jms.AtomikosJmsXaSessionProxy.onTerminated(AtomikosJmsXaSessionProxy.java:210)
at com.atomikos.datasource.xa.session.SessionHandleState.fireTerminatedEvent(SessionHandleState.java:244)
at com.atomikos.datasource.xa.session.SessionHandleState.notifySessionClosed(SessionHandleState.java:109)
at com.atomikos.jms.AtomikosJmsXaSessionProxy.invoke(AtomikosJmsXaSessionProxy.java:100)
at $Proxy327.close(Unknown Source)
at org.springframework.jms.support.JmsUtils.closeSession(JmsUtils.java:108)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.clearResources(DefaultMessageListenerContainer.java:1020)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:887)
at java.lang.Thread.run(Thread.java:637)
Caused by: org.apache.activemq.transport.InactivityIOException: Channel was inactive for too long: localhost/127.0.0.1:61616
at org.apache.activemq.transport.InactivityMonitor.oneway(InactivityMonitor.java:225)
at org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:83)
at org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:100)
at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40)
at org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
at org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1214)
... 12 more
10-03-27 23:02:52,024 [jobImportCompleteEventListener-1] atomikos connection pool 'APEX_ACTIVEMQ_BROKER': error creating proxy of connection atomikos pooled connection for resource APEX_ACTIVEMQ_BROKER
com.atomikos.datasource.pool.CreateConnectionException: atomikos pooled connection for resource APEX_ACTIVEMQ_BROKER: connection is erroneous
at com.atomikos.jms.AtomikosPooledJmsConnection.testUnderlyingConnection(AtomikosPooledJmsConnection.java:43)
at com.atomikos.datasource.pool.AbstractXPooledConnection.createConnectionProxy(AbstractXPooledConnection.java:43)
at com.atomikos.datasource.pool.ConnectionPool.borrowConnection(ConnectionPool.java:135)
at com.atomikos.jms.AtomikosConnectionFactoryBean.createConnection(AtomikosConnectionFactoryBean.java:560)
at org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:184)
at org.springframework.jms.listener.AbstractJmsListeningContainer.createSharedConnection(AbstractJmsListeningContainer.java:378)
at org.springframework.jms.listener.AbstractJmsListeningContainer.refreshSharedConnection(AbstractJmsListeningContainer.java:363)
at org.springframework.jms.listener.DefaultMessageListenerContainer.refreshConnectionUntilSuccessful(DefaultMessageListenerContainer.java:785)
at org.springframework.jms.listener.DefaultMessageListenerContainer.recoverAfterListenerSetupFailure(DefaultMessageListenerContainer.java:767)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:898)
at java.lang.Thread.run(Thread.java:637)
10-03-27 23:02:52,090 [jobImportCompleteEventListener-1] atomikos pooled connection for resource APEX_ACTIVEMQ_BROKER: error closing XAConnection:
javax.jms.JMSException: Broken pipe
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
at org.apache.activemq.ActiveMQConnection.doSyncSendPacket(ActiveMQConnection.java:1289)
at org.apache.activemq.ActiveMQConnection.close(ActiveMQConnection.java:624)
at com.atomikos.jms.AtomikosPooledJmsConnection.destroy(AtomikosPooledJmsConnection.java:50)
at com.atomikos.datasource.pool.ConnectionPool.borrowConnection(ConnectionPool.java:141)
at com.atomikos.jms.AtomikosConnectionFactoryBean.createConnection(AtomikosConnectionFactoryBean.java:560)
at org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:184)
at org.springframework.jms.listener.AbstractJmsListeningContainer.createSharedConnection(AbstractJmsListeningContainer.java:378)
at org.springframework.jms.listener.AbstractJmsListeningContainer.refreshSharedConnection(AbstractJmsListeningContainer.java:363)
at org.springframework.jms.listener.DefaultMessageListenerContainer.refreshConnectionUntilSuccessful(DefaultMessageListenerContainer.java:785)
at org.springframework.jms.listener.DefaultMessageListenerContainer.recoverAfterListenerSetupFailure(DefaultMessageListenerContainer.java:767)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:898)
at java.lang.Thread.run(Thread.java:637)
Caused by: java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at org.apache.activemq.transport.tcp.TcpBufferedOutputStream.flush(TcpBufferedOutputStream.java:115)
at java.io.DataOutputStream.flush(DataOutputStream.java:106)
at org.apache.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:167)
at org.apache.activemq.transport.InactivityMonitor.oneway(InactivityMonitor.java:233)
at org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:185)
at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:203)
at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:185)
... 1 more
10-03-27 23:02:52,091 [jobImportCompleteEventListener-1] atomikos connection pool 'APEX_ACTIVEMQ_BROKER': no connection found - waiting a bit...
10-03-27 23:03:18,722 [Atomikos:307] Local heuristic termination of coordinator apex-workflow-service-tm1390500020 with state HEUR_HAZARD
10-03-27 23:04:08,877 [Atomikos:307] Local heuristic termination of coordinator apex-workflow-service-tm1390500020 with state HEUR_HAZARD
10-03-27 23:04:59,736 [Atomikos:307] Local heuristic termination of coordinator apex-workflow-service-tm1390500020 with state HEUR_HAZARD
10-03-27 23:05:18,388 [Thread-10] atomikos connection pool 'APEX_ORACLE_DB': destroying pool...
10-03-27 23:05:25,932 [Thread-10] Logfile closed: /Users/jbreen/tools/apache-tomcat/apache-tomcat-6.0.20/data/tm/apex-workflow-service-tmlog1.log
10-03-29 10:17:21,900 [main] Starting read of logfile /Users/jbreen/tools/apache-tomcat/apache-tomcat-6.0.20/data/tm/apex-workflow-service-tmlog1.log
.
.
.
.
.
10-03-29 10:17:21,993 [main] Done read of logfile
10-03-29 10:17:21,993 [main] Logfile closed: /Users/jbreen/tools/apache-tomcat/apache-tomcat-6.0.20/data/tm/apex-workflow-service-tmlog1.log
10-03-29 10:17:22,084 [main] APEX_ACTIVEMQ_BROKER: refreshed XAResource
10-03-29 10:17:22,088 [main] Coordinator not recoverable: apex-workflow-service-tm1390500020
10-03-29 10:17:22,088 [main] Coordinator not recoverable: apex-workflow-service-tm1390500020
10-03-29 10:17:22,089 [main] Coordinator not recoverable: apex-workflow-service-tm1390500020
10-03-29 10:17:22,258 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:17:22,767 [main] APEX_ORACLE_DB: refreshed XAResource
10-03-29 10:17:22,873 [main] Coordinator not recoverable: apex-workflow-service-tm1390500020
10-03-29 10:17:32,267 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:17:42,278 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:17:52,279 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:18:02,279 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:18:12,280 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:18:12,473 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:18:22,474 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:18:32,475 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:18:40,770 [Thread-10] atomikos connection pool 'APEX_ORACLE_DB': destroying pool...
10-03-29 10:18:40,802 [Thread-10] Logfile closed: /Users/jbreen/tools/apache-tomcat/apache-tomcat-6.0.20/data/tm/apex-workflow-service-tmlog2.log
10-03-29 10:31:37,785 [main] Starting read of logfile /Users/jbreen/tools/apache-tomcat/apache-tomcat-6.0.20/data/tm/apex-workflow-service-tmlog2.log
10-03-29 10:31:37,834 [main] Done read of logfile
10-03-29 10:31:37,834 [main] Logfile closed: /Users/jbreen/tools/apache-tomcat/apache-tomcat-6.0.20/data/tm/apex-workflow-service-tmlog2.log
10-03-29 10:31:37,893 [main] APEX_ACTIVEMQ_BROKER: refreshed XAResource
10-03-29 10:31:37,897 [main] Coordinator not recoverable: apex-workflow-service-tm1390500020
10-03-29 10:31:37,897 [main] Coordinator not recoverable: apex-workflow-service-tm1390500020
10-03-29 10:31:37,898 [main] Coordinator not recoverable: apex-workflow-service-tm1390500020
10-03-29 10:31:38,059 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:31:38,483 [main] APEX_ORACLE_DB: refreshed XAResource
10-03-29 10:31:38,594 [main] Coordinator not recoverable: apex-workflow-service-tm1390500020
10-03-29 10:31:48,082 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:31:58,082 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:32:08,083 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:32:18,083 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:32:28,084 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:32:28,240 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:32:38,241 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:32:48,241 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:32:58,242 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:33:08,242 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:33:18,243 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:33:18,396 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:33:28,396 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:33:38,397 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:33:48,397 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:33:58,398 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:34:08,398 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:34:08,552 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:34:18,553 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:34:28,553 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:34:38,554 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:34:48,554 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:34:58,555 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:34:58,708 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:35:08,708 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:35:09,034 [Thread-10] atomikos connection pool 'APEX_ORACLE_DB': destroying pool...
10-03-29 10:35:09,078 [Thread-10] Logfile closed: /Users/jbreen/tools/apache-tomcat/apache-tomcat-6.0.20/data/tm/apex-workflow-service-tmlog3.log
I will also send this to the support email.
10-03-26 14:59:26,500 [http-8081-1] Starting read of logfile /Users/jbreen/tools/apache-tomcat/apache-tomcat-6.0.20/data/tm/apex-workflow-service-tmlog0.log
10-03-26 14:59:26,500 [http-8081-1] Done read of logfile
10-03-26 14:59:26,500 [http-8081-1] Logfile closed: /Users/jbreen/tools/apache-tomcat/apache-tomcat-6.0.20/data/tm/apex-workflow-service-tmlog0.log
10-03-26 14:59:26,621 [http-8081-1] APEX_ACTIVEMQ_BROKER: refreshed XAResource
10-03-26 14:59:27,062 [http-8081-1] APEX_ORACLE_DB: refreshed XAResource
10-03-27 23:02:28,147 [jobImportCompleteEventListener-1] Local heuristic termination of coordinator apex-workflow-service-tm1390500020 with state ABORTING
10-03-27 23:02:41,068 [jobImportCompleteEventListener-1] Error in proxy
javax.jms.JMSException: Channel was inactive for too long: localhost/127.0.0.1:61616
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
at org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1216)
at org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1208)
at org.apache.activemq.ActiveMQSession.asyncSendPacket(ActiveMQSession.java:1762)
at org.apache.activemq.ActiveMQMessageConsumer.close(ActiveMQMessageConsumer.java:594)
at com.atomikos.jms.AtomikosJmsMessageConsumerProxy.close(AtomikosJmsMessageConsumerProxy.java:72)
at org.springframework.jms.support.JmsUtils.closeMessageConsumer(JmsUtils.java:151)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.clearResources(DefaultMessageListenerContainer.java:1019)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:887)
at java.lang.Thread.run(Thread.java:637)
Caused by: org.apache.activemq.transport.InactivityIOException: Channel was inactive for too long: localhost/127.0.0.1:61616
at org.apache.activemq.transport.InactivityMonitor.oneway(InactivityMonitor.java:225)
at org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:83)
at org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:100)
at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40)
at org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
at org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1214)
... 8 more
10-03-27 23:02:47,001 [jobImportCompleteEventListener-1] atomikos xa session proxy for resource APEX_ACTIVEMQ_BROKER: could not close JMS session
javax.jms.JMSException: Channel was inactive for too long: localhost/127.0.0.1:61616
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
at org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1216)
at org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1208)
at org.apache.activemq.ActiveMQSession.close(ActiveMQSession.java:556)
at com.atomikos.jms.AtomikosJmsXaSessionProxy.destroy(AtomikosJmsXaSessionProxy.java:174)
at com.atomikos.jms.AtomikosJmsXaSessionProxy.onTerminated(AtomikosJmsXaSessionProxy.java:210)
at com.atomikos.datasource.xa.session.SessionHandleState.fireTerminatedEvent(SessionHandleState.java:244)
at com.atomikos.datasource.xa.session.SessionHandleState.notifySessionClosed(SessionHandleState.java:109)
at com.atomikos.jms.AtomikosJmsXaSessionProxy.invoke(AtomikosJmsXaSessionProxy.java:100)
at $Proxy327.close(Unknown Source)
at org.springframework.jms.support.JmsUtils.closeSession(JmsUtils.java:108)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.clearResources(DefaultMessageListenerContainer.java:1020)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:887)
at java.lang.Thread.run(Thread.java:637)
Caused by: org.apache.activemq.transport.InactivityIOException: Channel was inactive for too long: localhost/127.0.0.1:61616
at org.apache.activemq.transport.InactivityMonitor.oneway(InactivityMonitor.java:225)
at org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:83)
at org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:100)
at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40)
at org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
at org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1214)
... 12 more
10-03-27 23:02:52,024 [jobImportCompleteEventListener-1] atomikos connection pool 'APEX_ACTIVEMQ_BROKER': error creating proxy of connection atomikos pooled connection for resource APEX_ACTIVEMQ_BROKER
com.atomikos.datasource.pool.CreateConnectionException: atomikos pooled connection for resource APEX_ACTIVEMQ_BROKER: connection is erroneous
at com.atomikos.jms.AtomikosPooledJmsConnection.testUnderlyingConnection(AtomikosPooledJmsConnection.java:43)
at com.atomikos.datasource.pool.AbstractXPooledConnection.createConnectionProxy(AbstractXPooledConnection.java:43)
at com.atomikos.datasource.pool.ConnectionPool.borrowConnection(ConnectionPool.java:135)
at com.atomikos.jms.AtomikosConnectionFactoryBean.createConnection(AtomikosConnectionFactoryBean.java:560)
at org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:184)
at org.springframework.jms.listener.AbstractJmsListeningContainer.createSharedConnection(AbstractJmsListeningContainer.java:378)
at org.springframework.jms.listener.AbstractJmsListeningContainer.refreshSharedConnection(AbstractJmsListeningContainer.java:363)
at org.springframework.jms.listener.DefaultMessageListenerContainer.refreshConnectionUntilSuccessful(DefaultMessageListenerContainer.java:785)
at org.springframework.jms.listener.DefaultMessageListenerContainer.recoverAfterListenerSetupFailure(DefaultMessageListenerContainer.java:767)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:898)
at java.lang.Thread.run(Thread.java:637)
10-03-27 23:02:52,090 [jobImportCompleteEventListener-1] atomikos pooled connection for resource APEX_ACTIVEMQ_BROKER: error closing XAConnection:
javax.jms.JMSException: Broken pipe
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
at org.apache.activemq.ActiveMQConnection.doSyncSendPacket(ActiveMQConnection.java:1289)
at org.apache.activemq.ActiveMQConnection.close(ActiveMQConnection.java:624)
at com.atomikos.jms.AtomikosPooledJmsConnection.destroy(AtomikosPooledJmsConnection.java:50)
at com.atomikos.datasource.pool.ConnectionPool.borrowConnection(ConnectionPool.java:141)
at com.atomikos.jms.AtomikosConnectionFactoryBean.createConnection(AtomikosConnectionFactoryBean.java:560)
at org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:184)
at org.springframework.jms.listener.AbstractJmsListeningContainer.createSharedConnection(AbstractJmsListeningContainer.java:378)
at org.springframework.jms.listener.AbstractJmsListeningContainer.refreshSharedConnection(AbstractJmsListeningContainer.java:363)
at org.springframework.jms.listener.DefaultMessageListenerContainer.refreshConnectionUntilSuccessful(DefaultMessageListenerContainer.java:785)
at org.springframework.jms.listener.DefaultMessageListenerContainer.recoverAfterListenerSetupFailure(DefaultMessageListenerContainer.java:767)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:898)
at java.lang.Thread.run(Thread.java:637)
Caused by: java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at org.apache.activemq.transport.tcp.TcpBufferedOutputStream.flush(TcpBufferedOutputStream.java:115)
at java.io.DataOutputStream.flush(DataOutputStream.java:106)
at org.apache.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:167)
at org.apache.activemq.transport.InactivityMonitor.oneway(InactivityMonitor.java:233)
at org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:185)
at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:203)
at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:185)
... 1 more
10-03-27 23:02:52,091 [jobImportCompleteEventListener-1] atomikos connection pool 'APEX_ACTIVEMQ_BROKER': no connection found - waiting a bit...
10-03-27 23:03:18,722 [Atomikos:307] Local heuristic termination of coordinator apex-workflow-service-tm1390500020 with state HEUR_HAZARD
10-03-27 23:04:08,877 [Atomikos:307] Local heuristic termination of coordinator apex-workflow-service-tm1390500020 with state HEUR_HAZARD
10-03-27 23:04:59,736 [Atomikos:307] Local heuristic termination of coordinator apex-workflow-service-tm1390500020 with state HEUR_HAZARD
10-03-27 23:05:18,388 [Thread-10] atomikos connection pool 'APEX_ORACLE_DB': destroying pool...
10-03-27 23:05:25,932 [Thread-10] Logfile closed: /Users/jbreen/tools/apache-tomcat/apache-tomcat-6.0.20/data/tm/apex-workflow-service-tmlog1.log
10-03-29 10:17:21,900 [main] Starting read of logfile /Users/jbreen/tools/apache-tomcat/apache-tomcat-6.0.20/data/tm/apex-workflow-service-tmlog1.log
.
.
.
.
.
10-03-29 10:17:21,993 [main] Done read of logfile
10-03-29 10:17:21,993 [main] Logfile closed: /Users/jbreen/tools/apache-tomcat/apache-tomcat-6.0.20/data/tm/apex-workflow-service-tmlog1.log
10-03-29 10:17:22,084 [main] APEX_ACTIVEMQ_BROKER: refreshed XAResource
10-03-29 10:17:22,088 [main] Coordinator not recoverable: apex-workflow-service-tm1390500020
10-03-29 10:17:22,088 [main] Coordinator not recoverable: apex-workflow-service-tm1390500020
10-03-29 10:17:22,089 [main] Coordinator not recoverable: apex-workflow-service-tm1390500020
10-03-29 10:17:22,258 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:17:22,767 [main] APEX_ORACLE_DB: refreshed XAResource
10-03-29 10:17:22,873 [main] Coordinator not recoverable: apex-workflow-service-tm1390500020
10-03-29 10:17:32,267 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:17:42,278 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:17:52,279 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:18:02,279 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:18:12,280 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:18:12,473 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:18:22,474 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:18:32,475 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:18:40,770 [Thread-10] atomikos connection pool 'APEX_ORACLE_DB': destroying pool...
10-03-29 10:18:40,802 [Thread-10] Logfile closed: /Users/jbreen/tools/apache-tomcat/apache-tomcat-6.0.20/data/tm/apex-workflow-service-tmlog2.log
10-03-29 10:31:37,785 [main] Starting read of logfile /Users/jbreen/tools/apache-tomcat/apache-tomcat-6.0.20/data/tm/apex-workflow-service-tmlog2.log
10-03-29 10:31:37,834 [main] Done read of logfile
10-03-29 10:31:37,834 [main] Logfile closed: /Users/jbreen/tools/apache-tomcat/apache-tomcat-6.0.20/data/tm/apex-workflow-service-tmlog2.log
10-03-29 10:31:37,893 [main] APEX_ACTIVEMQ_BROKER: refreshed XAResource
10-03-29 10:31:37,897 [main] Coordinator not recoverable: apex-workflow-service-tm1390500020
10-03-29 10:31:37,897 [main] Coordinator not recoverable: apex-workflow-service-tm1390500020
10-03-29 10:31:37,898 [main] Coordinator not recoverable: apex-workflow-service-tm1390500020
10-03-29 10:31:38,059 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:31:38,483 [main] APEX_ORACLE_DB: refreshed XAResource
10-03-29 10:31:38,594 [main] Coordinator not recoverable: apex-workflow-service-tm1390500020
10-03-29 10:31:48,082 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:31:58,082 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:32:08,083 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:32:18,083 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:32:28,084 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:32:28,240 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:32:38,241 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:32:48,241 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:32:58,242 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:33:08,242 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:33:18,243 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:33:18,396 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:33:28,396 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:33:38,397 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:33:48,397 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:33:58,398 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:34:08,398 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:34:08,552 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:34:18,553 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:34:28,553 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:34:38,554 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:34:48,554 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:34:58,555 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:34:58,708 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:35:08,708 [Atomikos:2] XAResourceTransaction apex-workflow-service-tm1390500020apex-workflow-service-tm141868: no XAResource to rollback - the required resource is probably not yet intialized?
10-03-29 10:35:09,034 [Thread-10] atomikos connection pool 'APEX_ORACLE_DB': destroying pool...
10-03-29 10:35:09,078 [Thread-10] Logfile closed: /Users/jbreen/tools/apache-tomcat/apache-tomcat-6.0.20/data/tm/apex-workflow-service-tmlog3.log
Other recent topics