Atomikos Forum
TX log corrupted if active TX in a thread pool being shutdown |
We have found that, in version 4.0.2, if a thread is in a thread pool inside an active transaction and shutdownNow() is invoked on the thread pool the active thread is interrupted and the TX log gets corrupted. Furthermore, no recovery is possible and the error logs indicate to restart the JVM and this also requires removing the TX logs. This error is always able to be reproduced and results in the following stack trace:
2016-04-07 13:19:04,706 [CallCacheManager-executor-thread-1] DEBUG: exception from @Transactional method protected java.util.List com.clj2.business.cache.CallCacheManager.loadObjects() on attempt #1: org.springframework.transaction.TransactionSystemException: JTA failure on commit; nested exception is com.atomikos.icatch.jta.ExtendedSystemException: Error in commit: could not flush state image Log corrupted - restart JVM com.atomikos.recovery.LogReadException (translates to null)
2016-04-07 13:19:04,706 [CallCacheManager-executor-thread-1] ERROR: exception in CallCacheManager
org.springframework.transaction.TransactionSystemException: JTA failure on commit; nested exception is com.atomikos.icatch.jta.ExtendedSystemException: Error in commit: could not flush state image Log corrupted - restart JVM com.atomikos.recovery.LogReadException
at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1037)
at com.clj2.business.lib.grid.tx.GridTransactionManager.doCommit(GridTransactionManager.java:88)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:757)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:726)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:478)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:272)
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:63)
at com.clj2.business.cache.CallCacheManager.loadObjects_aroundBody2(CallCacheManager.java:104)
at com.clj2.business.cache.CallCacheManager$AjcClosure3.run(CallCacheManager.java:1)
at org.dellroad.stuff.spring.RetryTransactionAspect.ajc$around$org_dellroad_stuff_spring_RetryTransactionAspect$1$878f3a34proceed(RetryTransactionAspect.aj:106)
at org.dellroad.stuff.spring.RetryTransactionAspect$RetryTransactionAspect$1.run(RetryTransactionAspect.aj:179)
at org.dellroad.stuff.java.ThreadLocalHolder.invoke(ThreadLocalHolder.java:111)
at org.dellroad.stuff.spring.RetryTransactionAspect.ajc$around$org_dellroad_stuff_spring_RetryTransactionAspect$1$878f3a34(RetryTransactionAspect.aj:176)
at com.clj2.business.cache.CallCacheManager.loadObjects(CallCacheManager.java:104)
at com.clj2.business.cache.AbstractCacheManager.reloadAndNotify(AbstractCacheManager.java:398)
at com.clj2.business.cache.AbstractCacheManager.access$100(AbstractCacheManager.java:56)
at com.clj2.business.cache.AbstractCacheManager$4.run(AbstractCacheManager.java:663)
at com.clj2.business.service.impl.AbstractThreadingService$1.run(AbstractThreadingService.java:96)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: com.atomikos.icatch.jta.ExtendedSystemException: Error in commit: could not flush state image Log corrupted - restart JVM com.atomikos.recovery.LogReadException
at com.atomikos.icatch.jta.TransactionImp.commit(TransactionImp.java:181)
at com.atomikos.icatch.jta.TransactionManagerImp.commit(TransactionManagerImp.java:414)
at com.atomikos.icatch.jta.UserTransactionImp.commit(UserTransactionImp.java:86)
at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1021)
... 24 more
We are attempting to downgrade to 4.0.0 to see if the error is resolved as 4.0.2 is unusable in its current form. Has anyone else encountered this issue and possibly have a fix or workaround?
2016-04-07 13:19:04,706 [CallCacheManager-executor-thread-1] DEBUG: exception from @Transactional method protected java.util.List com.clj2.business.cache.CallCacheManager.loadObjects() on attempt #1: org.springframework.transaction.TransactionSystemException: JTA failure on commit; nested exception is com.atomikos.icatch.jta.ExtendedSystemException: Error in commit: could not flush state image Log corrupted - restart JVM com.atomikos.recovery.LogReadException (translates to null)
2016-04-07 13:19:04,706 [CallCacheManager-executor-thread-1] ERROR: exception in CallCacheManager
org.springframework.transaction.TransactionSystemException: JTA failure on commit; nested exception is com.atomikos.icatch.jta.ExtendedSystemException: Error in commit: could not flush state image Log corrupted - restart JVM com.atomikos.recovery.LogReadException
at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1037)
at com.clj2.business.lib.grid.tx.GridTransactionManager.doCommit(GridTransactionManager.java:88)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:757)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:726)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:478)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:272)
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:63)
at com.clj2.business.cache.CallCacheManager.loadObjects_aroundBody2(CallCacheManager.java:104)
at com.clj2.business.cache.CallCacheManager$AjcClosure3.run(CallCacheManager.java:1)
at org.dellroad.stuff.spring.RetryTransactionAspect.ajc$around$org_dellroad_stuff_spring_RetryTransactionAspect$1$878f3a34proceed(RetryTransactionAspect.aj:106)
at org.dellroad.stuff.spring.RetryTransactionAspect$RetryTransactionAspect$1.run(RetryTransactionAspect.aj:179)
at org.dellroad.stuff.java.ThreadLocalHolder.invoke(ThreadLocalHolder.java:111)
at org.dellroad.stuff.spring.RetryTransactionAspect.ajc$around$org_dellroad_stuff_spring_RetryTransactionAspect$1$878f3a34(RetryTransactionAspect.aj:176)
at com.clj2.business.cache.CallCacheManager.loadObjects(CallCacheManager.java:104)
at com.clj2.business.cache.AbstractCacheManager.reloadAndNotify(AbstractCacheManager.java:398)
at com.clj2.business.cache.AbstractCacheManager.access$100(AbstractCacheManager.java:56)
at com.clj2.business.cache.AbstractCacheManager$4.run(AbstractCacheManager.java:663)
at com.clj2.business.service.impl.AbstractThreadingService$1.run(AbstractThreadingService.java:96)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: com.atomikos.icatch.jta.ExtendedSystemException: Error in commit: could not flush state image Log corrupted - restart JVM com.atomikos.recovery.LogReadException
at com.atomikos.icatch.jta.TransactionImp.commit(TransactionImp.java:181)
at com.atomikos.icatch.jta.TransactionManagerImp.commit(TransactionManagerImp.java:414)
at com.atomikos.icatch.jta.UserTransactionImp.commit(UserTransactionImp.java:86)
at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1021)
... 24 more
We are attempting to downgrade to 4.0.0 to see if the error is resolved as 4.0.2 is unusable in its current form. Has anyone else encountered this issue and possibly have a fix or workaround?
Mark Thomas 
Thursday, April 07, 2016
Thursday, April 07, 2016
Fixed in dev in the following way: we now allow the new recovery to tolerate corrupted log entries by treating them as the (unexpected) end of the log file and moving on as usual.
Can you confirm that this would be ok?
Can you confirm that this would be ok?
Guy Pardon 
Friday, April 22, 2016
Friday, April 22, 2016
I'm using essentials 4.0.4 and encountered this exact same problem. Is this supposed to be fixed in 4.0.4?
Thanks!
Thanks!
Natalia
Friday, July 15, 2016
Friday, July 15, 2016
Here's the log just in case
DEBUG 2016-07-15 12:12:38,208 [pool-18-thread-4] : Sync2Sync - beforeCompletion() called on Synchronization: SynchronizationAdapter{localTransaction=LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=null, backupKeyLocks=null, topologyId=-1, stateTransferFlag=null} org.infinispan.transaction.synchronization.SyncLocalTransaction@399d7} org.infinispan.transaction.synchronization.SynchronizationAdapter@399f6
...
ERROR 2016-07-15 12:12:38,336 [pool-18-thread-4] : CachedRepository - Corrupted log file - restart JVM
com.atomikos.recovery.LogWriteException: java.nio.channels.ClosedByInterruptException
at com.atomikos.recovery.imp.FileSystemRepository.writeCheckpoint(FileSystemRepository.java:233)
at com.atomikos.recovery.imp.CachedRepository.performCheckpoint(CachedRepository.java:84)
at com.atomikos.recovery.imp.CachedRepository.put(CachedRepository.java:77)
at com.atomikos.recovery.imp.OltpLogImp.write(OltpLogImp.java:31)
at com.atomikos.persistence.imp.StateRecoveryManagerImp.preEnter(StateRecoveryManagerImp.java:51)
at com.atomikos.finitestates.FSMImp.notifyListeners(FSMImp.java:164)
at com.atomikos.finitestates.FSMImp.setState(FSMImp.java:251)
at com.atomikos.icatch.imp.CoordinatorImp.setState(CoordinatorImp.java:312)
at com.atomikos.icatch.imp.CoordinatorStateHandler.commitFromWithinCallback(CoordinatorStateHandler.java:346)
at com.atomikos.icatch.imp.ActiveStateHandler$6.doCommit(ActiveStateHandler.java:267)
at com.atomikos.icatch.imp.CoordinatorStateHandler.commitWithAfterCompletionNotification(CoordinatorStateHandler.java:581)
at com.atomikos.icatch.imp.ActiveStateHandler.commit(ActiveStateHandler.java:262)
at com.atomikos.icatch.imp.CoordinatorImp.commit(CoordinatorImp.java:548)
at com.atomikos.icatch.imp.CoordinatorImp.terminate(CoordinatorImp.java:685)
at com.atomikos.icatch.imp.CompositeTransactionImp.commit(CompositeTransactionImp.java:282)
at com.atomikos.icatch.jta.TransactionImp.commit(TransactionImp.java:172)
at com.atomikos.icatch.jta.TransactionManagerImp.commit(TransactionManagerImp.java:414)
at com.atomikos.icatch.jta.UserTransactionImp.commit(UserTransactionImp.java:86)
at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1023)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:761)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:730)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:485)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:291)
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:70)
at ar.com.eox.idlogistics.test.HandheldPickerEmulator.getStoreLocationFeature(HandheldPickerEmulator.java:479)
at ar.com.eox.idlogistics.test.HandheldPickerEmulator.workPicker(HandheldPickerEmulator.java:192)
at ar.com.eox.idlogistics.test.HandheldPickerEmulator.start(HandheldPickerEmulator.java:131)
at ar.com.eox.idlogistics.test.EmulatorService.lambda$4(EmulatorService.java:126)
at ar.com.eox.idlogistics.test.EmulatorService$$Lambda$63/853683399.run(Unknown Source)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.nio.channels.ClosedByInterruptException
at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:200)
at com.atomikos.recovery.imp.FileSystemRepository.writeToFile(FileSystemRepository.java:91)
at com.atomikos.recovery.imp.FileSystemRepository.write(FileSystemRepository.java:86)
at com.atomikos.recovery.imp.FileSystemRepository.writeCheckpoint(FileSystemRepository.java:224)
... 33 more
WARN 2016-07-15 12:12:38,338 [pool-18-thread-4] : CoordinatorStateHandler - Error in committing: could not flush state image java.nio.channels.ClosedByInterruptException com.atomikos.recovery.LogWriteException - recovery will clean up in the background
java.lang.IllegalStateException: could not flush state image java.nio.channels.ClosedByInterruptException com.atomikos.recovery.LogWriteException
at com.atomikos.persistence.imp.StateRecoveryManagerImp.preEnter(StateRecoveryManagerImp.java:54)
at com.atomikos.finitestates.FSMImp.notifyListeners(FSMImp.java:164)
at com.atomikos.finitestates.FSMImp.setState(FSMImp.java:251)
at com.atomikos.icatch.imp.CoordinatorImp.setState(CoordinatorImp.java:312)
at com.atomikos.icatch.imp.CoordinatorStateHandler.commitFromWithinCallback(CoordinatorStateHandler.java:346)
at com.atomikos.icatch.imp.ActiveStateHandler$6.doCommit(ActiveStateHandler.java:267)
at com.atomikos.icatch.imp.CoordinatorStateHandler.commitWithAfterCompletionNotification(CoordinatorStateHandler.java:581)
at com.atomikos.icatch.imp.ActiveStateHandler.commit(ActiveStateHandler.java:262)
at com.atomikos.icatch.imp.CoordinatorImp.commit(CoordinatorImp.java:548)
at com.atomikos.icatch.imp.CoordinatorImp.terminate(CoordinatorImp.java:685)
at com.atomikos.icatch.imp.CompositeTransactionImp.commit(CompositeTransactionImp.java:282)
at com.atomikos.icatch.jta.TransactionImp.commit(TransactionImp.java:172)
at com.atomikos.icatch.jta.TransactionManagerImp.commit(TransactionManagerImp.java:414)
at com.atomikos.icatch.jta.UserTransactionImp.commit(UserTransactionImp.java:86)
at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1023)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:761)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:730)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:485)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:291)
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:70)
at ar.com.eox.idlogistics.test.HandheldPickerEmulator.getStoreLocationFeature(HandheldPickerEmulator.java:479)
at ar.com.eox.idlogistics.test.HandheldPickerEmulator.workPicker(HandheldPickerEmulator.java:192)
at ar.com.eox.idlogistics.test.HandheldPickerEmulator.start(HandheldPickerEmulator.java:131)
at ar.com.eox.idlogistics.test.EmulatorService.lambda$4(EmulatorService.java:126)
at ar.com.eox.idlogistics.test.EmulatorService$$Lambda$63/853683399.run(Unknown Source)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.atomikos.recovery.LogWriteException: java.nio.channels.ClosedByInterruptException
at com.atomikos.recovery.imp.FileSystemRepository.writeCheckpoint(FileSystemRepository.java:233)
at com.atomikos.recovery.imp.CachedRepository.performCheckpoint(CachedRepository.java:84)
at com.atomikos.recovery.imp.CachedRepository.put(CachedRepository.java:77)
at com.atomikos.recovery.imp.OltpLogImp.write(OltpLogImp.java:31)
at com.atomikos.persistence.imp.StateRecoveryManagerImp.preEnter(StateRecoveryManagerImp.java:51)
... 29 more
Caused by: java.nio.channels.ClosedByInterruptException
at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:200)
at com.atomikos.recovery.imp.FileSystemRepository.writeToFile(FileSystemRepository.java:91)
at com.atomikos.recovery.imp.FileSystemRepository.write(FileSystemRepository.java:86)
at com.atomikos.recovery.imp.FileSystemRepository.writeCheckpoint(FileSystemRepository.java:224)
... 33 more
ERROR 2016-07-15 12:12:38,348 [pool-18-thread-1] : TransactionImp - Error in rollback: could not flush state image Log corrupted - restart JVM com.atomikos.recovery.LogReadException
com.atomikos.icatch.SysException: Error in rollback: could not flush state image Log corrupted - restart JVM com.atomikos.recovery.LogReadException
at com.atomikos.icatch.imp.CoordinatorStateHandler.rollbackFromWithinCallback(CoordinatorStateHandler.java:512)
at com.atomikos.icatch.imp.ActiveStateHandler$7.doRollback(ActiveStateHandler.java:284)
at com.atomikos.icatch.imp.CoordinatorStateHandler.rollbackWithAfterCompletionNotification(CoordinatorStateHandler.java:563)
at com.atomikos.icatch.imp.ActiveStateHandler.rollback(ActiveStateHandler.java:279)
at com.atomikos.icatch.imp.CoordinatorImp.rollback(CoordinatorImp.java:577)
at com.atomikos.icatch.imp.TransactionStateHandler.rollback(TransactionStateHandler.java:165)
at com.atomikos.icatch.imp.TransactionStateHandler.rollbackWithStateCheck(TransactionStateHandler.java:180)
at com.atomikos.icatch.imp.CompositeTransactionImp.doRollback(CompositeTransactionImp.java:201)
at com.atomikos.icatch.imp.CompositeTransactionImp.rollback(CompositeTransactionImp.java:314)
at com.atomikos.icatch.jta.TransactionImp.rollback(TransactionImp.java:199)
at com.atomikos.icatch.jta.TransactionManagerImp.rollback(TransactionManagerImp.java:426)
at com.atomikos.icatch.jta.UserTransactionImp.rollback(UserTransactionImp.java:97)
at org.springframework.transaction.jta.JtaTransactionManager.doRollback(JtaTransactionManager.java:1050)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processRollback(AbstractPlatformTransactionManager.java:853)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.rollback(AbstractPlatformTransactionManager.java:830)
at org.springframework.transaction.interceptor.TransactionAspectSupport.completeTransactionAfterThrowing(TransactionAspectSupport.java:503)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:285)
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:70)
at ar.com.eox.idlogistics.test.HandheldPickerEmulator.pick(HandheldPickerEmulator.java:309)
at ar.com.eox.idlogistics.test.HandheldPickerEmulator.workPicker(HandheldPickerEmulator.java:220)
at ar.com.eox.idlogistics.test.HandheldPickerEmulator.start(HandheldPickerEmulator.java:131)
at ar.com.eox.idlogistics.test.EmulatorService.lambda$4(EmulatorService.java:126)
at ar.com.eox.idlogistics.test.EmulatorService$$Lambda$63/853683399.run(Unknown Source)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: could not flush state image Log corrupted - restart JVM com.atomikos.recovery.LogReadException
at com.atomikos.persistence.imp.StateRecoveryManagerImp.preEnter(StateRecoveryManagerImp.java:54)
at com.atomikos.finitestates.FSMImp.notifyListeners(FSMImp.java:164)
at com.atomikos.finitestates.FSMImp.setState(FSMImp.java:251)
at com.atomikos.icatch.imp.CoordinatorImp.setState(CoordinatorImp.java:312)
at com.atomikos.icatch.imp.CoordinatorImp.setStateHandler(CoordinatorImp.java:222)
at com.atomikos.icatch.imp.CoordinatorStateHandler.rollbackFromWithinCallback(CoordinatorStateHandler.java:506)
... 27 more
Caused by: com.atomikos.recovery.LogReadException: Log corrupted - restart JVM
at com.atomikos.recovery.imp.CachedRepository.assertNotCorrupted(CachedRepository.java:137)
at com.atomikos.recovery.imp.CachedRepository.get(CachedRepository.java:131)
at com.atomikos.recovery.imp.OltpLogImp.assertEntryIsAllowedInCurrentState(OltpLogImp.java:35)
at com.atomikos.recovery.imp.OltpLogImp.write(OltpLogImp.java:30)
at com.atomikos.persistence.imp.StateRecoveryManagerImp.preEnter(StateRecoveryManagerImp.java:51)
... 32 more
DEBUG 2016-07-15 12:12:38,208 [pool-18-thread-4] : Sync2Sync - beforeCompletion() called on Synchronization: SynchronizationAdapter{localTransaction=LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=null, backupKeyLocks=null, topologyId=-1, stateTransferFlag=null} org.infinispan.transaction.synchronization.SyncLocalTransaction@399d7} org.infinispan.transaction.synchronization.SynchronizationAdapter@399f6
...
ERROR 2016-07-15 12:12:38,336 [pool-18-thread-4] : CachedRepository - Corrupted log file - restart JVM
com.atomikos.recovery.LogWriteException: java.nio.channels.ClosedByInterruptException
at com.atomikos.recovery.imp.FileSystemRepository.writeCheckpoint(FileSystemRepository.java:233)
at com.atomikos.recovery.imp.CachedRepository.performCheckpoint(CachedRepository.java:84)
at com.atomikos.recovery.imp.CachedRepository.put(CachedRepository.java:77)
at com.atomikos.recovery.imp.OltpLogImp.write(OltpLogImp.java:31)
at com.atomikos.persistence.imp.StateRecoveryManagerImp.preEnter(StateRecoveryManagerImp.java:51)
at com.atomikos.finitestates.FSMImp.notifyListeners(FSMImp.java:164)
at com.atomikos.finitestates.FSMImp.setState(FSMImp.java:251)
at com.atomikos.icatch.imp.CoordinatorImp.setState(CoordinatorImp.java:312)
at com.atomikos.icatch.imp.CoordinatorStateHandler.commitFromWithinCallback(CoordinatorStateHandler.java:346)
at com.atomikos.icatch.imp.ActiveStateHandler$6.doCommit(ActiveStateHandler.java:267)
at com.atomikos.icatch.imp.CoordinatorStateHandler.commitWithAfterCompletionNotification(CoordinatorStateHandler.java:581)
at com.atomikos.icatch.imp.ActiveStateHandler.commit(ActiveStateHandler.java:262)
at com.atomikos.icatch.imp.CoordinatorImp.commit(CoordinatorImp.java:548)
at com.atomikos.icatch.imp.CoordinatorImp.terminate(CoordinatorImp.java:685)
at com.atomikos.icatch.imp.CompositeTransactionImp.commit(CompositeTransactionImp.java:282)
at com.atomikos.icatch.jta.TransactionImp.commit(TransactionImp.java:172)
at com.atomikos.icatch.jta.TransactionManagerImp.commit(TransactionManagerImp.java:414)
at com.atomikos.icatch.jta.UserTransactionImp.commit(UserTransactionImp.java:86)
at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1023)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:761)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:730)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:485)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:291)
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:70)
at ar.com.eox.idlogistics.test.HandheldPickerEmulator.getStoreLocationFeature(HandheldPickerEmulator.java:479)
at ar.com.eox.idlogistics.test.HandheldPickerEmulator.workPicker(HandheldPickerEmulator.java:192)
at ar.com.eox.idlogistics.test.HandheldPickerEmulator.start(HandheldPickerEmulator.java:131)
at ar.com.eox.idlogistics.test.EmulatorService.lambda$4(EmulatorService.java:126)
at ar.com.eox.idlogistics.test.EmulatorService$$Lambda$63/853683399.run(Unknown Source)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.nio.channels.ClosedByInterruptException
at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:200)
at com.atomikos.recovery.imp.FileSystemRepository.writeToFile(FileSystemRepository.java:91)
at com.atomikos.recovery.imp.FileSystemRepository.write(FileSystemRepository.java:86)
at com.atomikos.recovery.imp.FileSystemRepository.writeCheckpoint(FileSystemRepository.java:224)
... 33 more
WARN 2016-07-15 12:12:38,338 [pool-18-thread-4] : CoordinatorStateHandler - Error in committing: could not flush state image java.nio.channels.ClosedByInterruptException com.atomikos.recovery.LogWriteException - recovery will clean up in the background
java.lang.IllegalStateException: could not flush state image java.nio.channels.ClosedByInterruptException com.atomikos.recovery.LogWriteException
at com.atomikos.persistence.imp.StateRecoveryManagerImp.preEnter(StateRecoveryManagerImp.java:54)
at com.atomikos.finitestates.FSMImp.notifyListeners(FSMImp.java:164)
at com.atomikos.finitestates.FSMImp.setState(FSMImp.java:251)
at com.atomikos.icatch.imp.CoordinatorImp.setState(CoordinatorImp.java:312)
at com.atomikos.icatch.imp.CoordinatorStateHandler.commitFromWithinCallback(CoordinatorStateHandler.java:346)
at com.atomikos.icatch.imp.ActiveStateHandler$6.doCommit(ActiveStateHandler.java:267)
at com.atomikos.icatch.imp.CoordinatorStateHandler.commitWithAfterCompletionNotification(CoordinatorStateHandler.java:581)
at com.atomikos.icatch.imp.ActiveStateHandler.commit(ActiveStateHandler.java:262)
at com.atomikos.icatch.imp.CoordinatorImp.commit(CoordinatorImp.java:548)
at com.atomikos.icatch.imp.CoordinatorImp.terminate(CoordinatorImp.java:685)
at com.atomikos.icatch.imp.CompositeTransactionImp.commit(CompositeTransactionImp.java:282)
at com.atomikos.icatch.jta.TransactionImp.commit(TransactionImp.java:172)
at com.atomikos.icatch.jta.TransactionManagerImp.commit(TransactionManagerImp.java:414)
at com.atomikos.icatch.jta.UserTransactionImp.commit(UserTransactionImp.java:86)
at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1023)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:761)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:730)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:485)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:291)
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:70)
at ar.com.eox.idlogistics.test.HandheldPickerEmulator.getStoreLocationFeature(HandheldPickerEmulator.java:479)
at ar.com.eox.idlogistics.test.HandheldPickerEmulator.workPicker(HandheldPickerEmulator.java:192)
at ar.com.eox.idlogistics.test.HandheldPickerEmulator.start(HandheldPickerEmulator.java:131)
at ar.com.eox.idlogistics.test.EmulatorService.lambda$4(EmulatorService.java:126)
at ar.com.eox.idlogistics.test.EmulatorService$$Lambda$63/853683399.run(Unknown Source)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.atomikos.recovery.LogWriteException: java.nio.channels.ClosedByInterruptException
at com.atomikos.recovery.imp.FileSystemRepository.writeCheckpoint(FileSystemRepository.java:233)
at com.atomikos.recovery.imp.CachedRepository.performCheckpoint(CachedRepository.java:84)
at com.atomikos.recovery.imp.CachedRepository.put(CachedRepository.java:77)
at com.atomikos.recovery.imp.OltpLogImp.write(OltpLogImp.java:31)
at com.atomikos.persistence.imp.StateRecoveryManagerImp.preEnter(StateRecoveryManagerImp.java:51)
... 29 more
Caused by: java.nio.channels.ClosedByInterruptException
at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:200)
at com.atomikos.recovery.imp.FileSystemRepository.writeToFile(FileSystemRepository.java:91)
at com.atomikos.recovery.imp.FileSystemRepository.write(FileSystemRepository.java:86)
at com.atomikos.recovery.imp.FileSystemRepository.writeCheckpoint(FileSystemRepository.java:224)
... 33 more
ERROR 2016-07-15 12:12:38,348 [pool-18-thread-1] : TransactionImp - Error in rollback: could not flush state image Log corrupted - restart JVM com.atomikos.recovery.LogReadException
com.atomikos.icatch.SysException: Error in rollback: could not flush state image Log corrupted - restart JVM com.atomikos.recovery.LogReadException
at com.atomikos.icatch.imp.CoordinatorStateHandler.rollbackFromWithinCallback(CoordinatorStateHandler.java:512)
at com.atomikos.icatch.imp.ActiveStateHandler$7.doRollback(ActiveStateHandler.java:284)
at com.atomikos.icatch.imp.CoordinatorStateHandler.rollbackWithAfterCompletionNotification(CoordinatorStateHandler.java:563)
at com.atomikos.icatch.imp.ActiveStateHandler.rollback(ActiveStateHandler.java:279)
at com.atomikos.icatch.imp.CoordinatorImp.rollback(CoordinatorImp.java:577)
at com.atomikos.icatch.imp.TransactionStateHandler.rollback(TransactionStateHandler.java:165)
at com.atomikos.icatch.imp.TransactionStateHandler.rollbackWithStateCheck(TransactionStateHandler.java:180)
at com.atomikos.icatch.imp.CompositeTransactionImp.doRollback(CompositeTransactionImp.java:201)
at com.atomikos.icatch.imp.CompositeTransactionImp.rollback(CompositeTransactionImp.java:314)
at com.atomikos.icatch.jta.TransactionImp.rollback(TransactionImp.java:199)
at com.atomikos.icatch.jta.TransactionManagerImp.rollback(TransactionManagerImp.java:426)
at com.atomikos.icatch.jta.UserTransactionImp.rollback(UserTransactionImp.java:97)
at org.springframework.transaction.jta.JtaTransactionManager.doRollback(JtaTransactionManager.java:1050)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processRollback(AbstractPlatformTransactionManager.java:853)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.rollback(AbstractPlatformTransactionManager.java:830)
at org.springframework.transaction.interceptor.TransactionAspectSupport.completeTransactionAfterThrowing(TransactionAspectSupport.java:503)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:285)
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:70)
at ar.com.eox.idlogistics.test.HandheldPickerEmulator.pick(HandheldPickerEmulator.java:309)
at ar.com.eox.idlogistics.test.HandheldPickerEmulator.workPicker(HandheldPickerEmulator.java:220)
at ar.com.eox.idlogistics.test.HandheldPickerEmulator.start(HandheldPickerEmulator.java:131)
at ar.com.eox.idlogistics.test.EmulatorService.lambda$4(EmulatorService.java:126)
at ar.com.eox.idlogistics.test.EmulatorService$$Lambda$63/853683399.run(Unknown Source)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: could not flush state image Log corrupted - restart JVM com.atomikos.recovery.LogReadException
at com.atomikos.persistence.imp.StateRecoveryManagerImp.preEnter(StateRecoveryManagerImp.java:54)
at com.atomikos.finitestates.FSMImp.notifyListeners(FSMImp.java:164)
at com.atomikos.finitestates.FSMImp.setState(FSMImp.java:251)
at com.atomikos.icatch.imp.CoordinatorImp.setState(CoordinatorImp.java:312)
at com.atomikos.icatch.imp.CoordinatorImp.setStateHandler(CoordinatorImp.java:222)
at com.atomikos.icatch.imp.CoordinatorStateHandler.rollbackFromWithinCallback(CoordinatorStateHandler.java:506)
... 27 more
Caused by: com.atomikos.recovery.LogReadException: Log corrupted - restart JVM
at com.atomikos.recovery.imp.CachedRepository.assertNotCorrupted(CachedRepository.java:137)
at com.atomikos.recovery.imp.CachedRepository.get(CachedRepository.java:131)
at com.atomikos.recovery.imp.OltpLogImp.assertEntryIsAllowedInCurrentState(OltpLogImp.java:35)
at com.atomikos.recovery.imp.OltpLogImp.write(OltpLogImp.java:30)
at com.atomikos.persistence.imp.StateRecoveryManagerImp.preEnter(StateRecoveryManagerImp.java:51)
... 32 more
Natalia
Friday, July 15, 2016
Friday, July 15, 2016
Thanks,
Can you send an atomikos log in DEBUG mode to support@atomikos.com with "case 151740" in the subject line?
We'll have a look ASAP.
Can you send an atomikos log in DEBUG mode to support@atomikos.com with "case 151740" in the subject line?
We'll have a look ASAP.
Guy Pardon
Saturday, July 16, 2016
Saturday, July 16, 2016
Other recent topics