Re: missing Java method information in operf profile

On 08/25/2014 01:01 PM, Brian Hall wrote:
> I'm running the operf from AT 7.0-5 on a POWER8 system:
>   # /opt/at7.0/bin/operf -v
>   /opt/at7.0/bin/operf: oprofile 1.0.0git compiled on Jul 24 2014 13:49:08
> using the command (as root):
>   /opt/at7.0/bin/operf --events PM_RUN_CYC:10290000 --separate-thread --separate-cpu --system-wide
> 
> There are 14 active JVM processes running on the system that I am trying to profile.  For some reason, the profiles only contain full JIT compiler method information for 2 or 3 of the processes (it varies across different runs).  I have checked that the agentlib option is being correctly passed to the JVMs, and that the JIT dump files for all 14 processes are available in /tmp/.oprofile/jitdump at the time that operf is run.
> 
> A casual look at verbose output from operf didn't indicate any problems in accessing the JIT dump files, it just didn't try to process all of them.  I tried adding --lazy-conversion to the operf options, and that run did show an issue at the end of the verbose output:
>   add sym: name=Ljava/utilchild received signal 9
Hi, Brian,
This sounds similar to a problem that was reported on this list on July 30 by Andrew Haley regarding an ill-advised hard-coded timeout.  Andrew's problem has been fixed upstream now, but if you were running into that exact problem, using --lazy-conversion would have resolved it.  Looking at the code now, I see there is a second hard-coded timeout.  The one I got rid of was where the main operf thread is waiting for the oper_read_pid to finish.  There's another one, where the operf_read_pid is waiting for yet another process to finish . . .  which was started via exec to do the JIT dump conversion.  The timeout for the actual JIT dump conversion is 2 seconds. It appears you've hit that limit.

I'm attaching a patch that should resolve this, if you could please test it.

Thanks.
-Maynard
> 
> Brian
> 
> 
> ------------------------------------------------------------------------------
> Slashdot TV.  
> Video for Nerds.  Stuff that matters.
> http://tv.slashdot.org/
> 
> 
> 
> _______________________________________________
> oprofile-list mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/oprofile-list
>

------------------------------------------------------------------------------
Slashdot TV.  
Video for Nerds.  Stuff that matters.
http://tv.slashdot.org/

_______________________________________________
oprofile-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/oprofile-list

Index: oprof-work/pe_profiling/operf.cpp
===================================================================
--- oprof-work.orig/pe_profiling/operf.cpp
+++ oprof-work/pe_profiling/operf.cpp
@@ -771,11 +771,15 @@ static void _jitconv_complete(int val __
 	if (WIFEXITED(child_status) && (!WEXITSTATUS(child_status))) {
 		cverb << vdebug << "JIT dump processing complete." << endl;
 	} else {
-		 if (WIFSIGNALED(child_status))
-			 cerr << "child received signal " << WTERMSIG(child_status) << endl;
-		 else
+		 if (WIFSIGNALED(child_status)) {
+			 if (ctl_c)
+				 cerr << "JIT conversion stopped by request of user via ctl-c" << endl;
+			 else
+				 cerr << "child received signal " << WTERMSIG(child_status) << endl;
+		 } else {
 			 cerr << "JIT dump processing exited abnormally: "
-			 << WEXITSTATUS(child_status) << endl;
+			      << WEXITSTATUS(child_status) << endl;
+		 }
 	}
 }
 
@@ -882,7 +886,6 @@ static void convert_sample_data(void)
 	int inputfd;
 	string inputfname;
 	int rc = EXIT_SUCCESS;
-	int keep_waiting = 0;
 	string current_sampledir = samples_dir + "/current/";
 	string previous_sampledir = samples_dir + "/previous";
 	string stats_dir = "";
@@ -953,12 +956,8 @@ static void convert_sample_data(void)
 	_set_signals_for_convert();
 	cverb << vdebug << "Calling _do_jitdump_convert" << endl;
 	_do_jitdump_convert();
-	while (jit_conversion_running && (keep_waiting < 2)) {
+	while (jit_conversion_running) {
 		sleep(1);
-		keep_waiting++;
-	}
-	if (jit_conversion_running) {
-		kill(jitconv_pid, SIGKILL);
 	}
 out:
 	if (!operf_options::post_conversion)