Re: Unexpected JVM crash, assistance requested

Hi there

After another JVM crash yesterday we've found some more information in std
err:

Assertion failed at /afs/
rchland.ibm.com/usr8/j9/src/rebuilt/jdk80/64bit/sr5fp20.v7r3/jre/lib/ppc64/compressedrefs/compiler/../omr/c
ompiler/codegen/OMRTreeEvaluator.cpp:348: false
VMState: 0x0005ff04
 badILOp BadILOp cannot be evaluated
compiling
com/internationalpresence/impscloudgateway/util/AS400LogHandler.flush()V at
level: hot
assumeDontCallMeDirectlyImpl(const char*,int,const char*,const
char*,char*,bool)+0x4d0
TR::fatal_assertion(const char*,int,const char*,const char*,...)+0x40
OMR::TreeEvaluator::badILOpEvaluator(TR::Node*,TR::CodeGenerator*)+0x4c
OMR::CodeGenerator::evaluate(TR::Node*)+0x510
J9::CodeGenerator::doInstructionSelection()+0x4ed8
OMR::CodeGenPhase::performInstructionSelectionPhase(TR::CodeGenerator*,TR::CodeGenPhase*)+0x5c0
OMR::CodeGenPhase::performAll()+0x2b4
OMR::CodeGenerator::generateCode()+0x484
OMR::Compilation::compile()+0x2188
TR::CompilationInfoPerThreadBase::compile(J9VMThread*,TR::Compilation*,TR_ResolvedMethod*,TR_J9VMBase&,TR_OptimizationPlan*,const
TR
::SegmentAllocator&)+0x161c
TR::CompilationInfoPerThreadBase::wrappedCompile(J9PortLibrary*,void*)+0x1cd4
omrsig_protect+0x5a8
TR::CompilationInfoPerThreadBase::compile(J9VMThread*,TR_MethodToBeCompiled*,J9::J9SegmentProvider&)+0x774
TR::CompilationInfoPerThread::processEntry(TR_MethodToBeCompiled&,J9::J9SegmentProvider&)+0x854
TR::CompilationInfoPerThread::processEntries()+0x7f8
protectedCompilationThreadProc(J9PortLibrary*,TR::CompilationInfoPerThread*)+0x4e0
omrsig_protect+0x5a8
compilationThreadProc(void*)+0x5b8
thread_wrapper+0x254
_pthread_body+0x138
+0x0
The assert subroutine failed: 0, file /afs/
rchland.ibm.com/usr8/j9/src/rebuilt/jdk80/64bit/sr5fp20.v7r3/jre/lib/ppc64/compressedrefs
/compiler/../omr/compiler/infra/Assert.cpp, line 72
JVMDUMP039I Processing dump event "abort", detail "" at 2018/11/15 07:41:09
- please wait.
JVMDUMP032I JVM requested System dump using
'//core.20181115.074109.435449.0001.dmp' in response to an event
JVMDUMP010I System dump written to //core.20181115.074109.435449.0001.dmp
JVMDUMP032I JVM requested Java dump using
'//javacore.20181115.074109.435449.0002.txt' in response to an event
JVMDUMP010I Java dump written to //javacore.20181115.074109.435449.0002.txt
JVMDUMP032I JVM requested Snap dump using
'//Snap.20181115.074109.435449.0003.trc' in response to an event
JVMDUMP010I Snap dump written to //Snap.20181115.074109.435449.0003.trc
JVMDUMP007I JVM Requesting JIT dump using
'//jitdump.20181115.074109.435449.0004.dmp'
#JITDUMP:  vmThread=000000003000EF00 Crashed while printing out current
IL.JVMDUMP010I JIT dump written to //jitdump.20181115.074109
.435449.0004.dmp
JVMDUMP013I Processed dump event "abort", detail "".

Does this help pinpoint the cause?

Thanks
Matt


On Thu, 15 Nov 2018 at 13:32, Matt Pryor <pryor-vppm1TVqICDMoOGh6ywv9JxocPoXdSqV0E9HWUfgJXw@public.gmane.org>
wrote:

> Hi again Steve
>
> From the customer:
>
> QJVACMDSRV authorities are *PUBLIC *USE.  I verified the TLXDRIVER
> authorities match what you noted below.
>
> Best wishes
> Matt
>
> On Thu, 15 Nov 2018 at 09:45, Matt Pryor <pryor-vppm1TVqICDMoOGh6ywv9JxocPoXdSqV0E9HWUfgJXw@public.gmane.org>
> wrote:
>
>> Hi Steve
>>
>> Thanks for your reply.
>>
>> Java is started via a CLP, this is the section that calls it:
>>
>> JAVA       +
>>             CLASS('com.internationalpresence.impscloudg+
>>             ateway.CloudQ') PARM(&DRIVER &INTERNET +
>>             &PRODLIB &CLDFLR &TYPE &ENV &HOSTIP +
>>             &CLDSSL &LOGTYP) CLASSPATH(*ENVVAR) +
>>             CHKPATH(*IGNORE) PROP((java.version &JVM) +
>>             (java.awt.headless true) +
>>             (os400.awt.native true) +
>>             (com.ibm.cacheLocalHost true) +
>>             (java.net.preferIPv4Stack &IP4) +
>>             (java.net.preferIPv6Addresses &IP6)) +
>>             OPTION(&OPT) OUTPUT(*PRINT)
>>
>> The program runs for several hours or days before the JVM crashes. We
>> have a 2 gb dump file and also a java core text file which shows all the
>> threat stack traces etc.
>>
>> All Java call stacks available show threads to either be in a wait state,
>> or a sleep state, or a socket read or accept state with the exception of
>> the following:
>>
>> 1)
>> 4XESTACKTRACE                at
>> com/ibm/misc/SignalDispatcher.waitForSignal(Native Method)
>> 4XESTACKTRACE                at
>> com/ibm/misc/SignalDispatcher.run(SignalDispatcher.java:73)
>>
>> 2)
>> 4XESTACKTRACE                at
>> com/ibm/tools/attach/target/IPC.waitSemaphore(Native Method)
>> 4XESTACKTRACE                at
>> com/ibm/tools/attach/target/CommonDirectory.waitSemaphore(CommonDirectory.java:251)
>>
>> 3)
>> 4XESTACKTRACE                at
>> com/ibm/lang/management/internal/MemoryNotificationThread.processNotificationLoop(Native
>> Method)
>> 4XESTACKTRACE                at
>> com/ibm/lang/management/internal/MemoryNotificationThread.run(MemoryNotificationThread.java:185)
>>
>> In case it's any use to you I've uploaded the java core text file to
>>
>>
>> http://www.international-presence.com/software/matt_transfer/javacore.20181104.102029.215708.0002.txt
>>
>> TLXDRIVER is the user profile that the CL program runs as, which should
>> belong to *SYSOPR user group and have special authority for *ALLOBJ,
>> *JOBCTL, *SECADM, *SPLCTL
>>
>> I will ask the customer to check on authority for QJVACMDSRV *pgm.
>>
>> Many thanks
>> Matt
>>
>>
>>
>>
>>>    1. Re: Unexpected JVM crash, assistance requested
>>>       (Steve Hinrichs via JAVA400-L)
>>>
>>>
>>> ----------------------------------------------------------------------
>>>
>>> message: 1
>>> date: Tue, 13 Nov 2018 19:09:50 +0000
>>> from: Steve Hinrichs via JAVA400-L <java400-l-Zwy7GipZuJhWk0Htik3J/[email protected]>
>>> subject: Re: Unexpected JVM crash, assistance requested
>>>
>>> How is the java process started, from RPG, RUNJVA, QSH?
>>> Is their any call stack or is it failing right away?
>>> What is the class/method being called?
>>> What authority does the user have that is running the java process,
>>> maybe try QSECOFR.
>>>
>>> I would check authority and existence of QJAVA/QJVACMDSRV *pgm.
>>> What is TLXDRIVER?  A user profile?  Searching for TLXDRIVER I find 2
>>> items related to user profiles.
>>>
>>> Steve
>>>
>>>
>
> --
> Matt Pryor
> Software Developer
>
> The International Presence Group of Companies
> EMAIL: pryor-pShvp1n0/[email protected]
> URL: www.International-presence.com
>
>
>

-- 
Matt Pryor
Software Developer

The International Presence Group of Companies
EMAIL: pryor-pShvp1n0/[email protected]
URL: www.International-presence.com
-- 
This is the Java Programming on and around the IBM i (JAVA400-L) mailing list
To post a message email: JAVA400-L-Zwy7GipZuJhWk0Htik3J/[email protected]
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/java400-l
or email: JAVA400-L-request-Zwy7GipZuJhWk0Htik3J/[email protected]
Before posting, please take a moment to review the archives
at https://archive.midrange.com/java400-l.

Help support midrange.com by shopping at amazon.com with our affiliate
link: https://amazon.midrange.com