Re: operf evil memory consumption
Andrew Haley <[email protected]>
| Newsgroups | gmane.linux.oprofile |
|---|---|
| Message-ID | <[email protected]> |
Hi, On 05/12/2015 09:09 PM, William Cohen wrote: > On 05/12/2015 01:40 PM, Andrew Haley wrote: >> I was surprised to see this: >> >> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND >> 1968 aph 20 0 7657600 1.675g 13184 S 754.0 10.5 73:01.82 java >> 1970 aph 20 0 7533760 7.180g 4288 S 36.8 45.1 3:28.73 operf >> >> The RSS of operf really is extreme. Is it that it keeps all samples >> in memory, or somesuch? > > Hi Andrew, > > That does look excessive. This was just taking measurements on the > java process and its children or system-wide? What is the command > line being used to start up operf? aph 2999 27.7 27.0 4524416 4521728 pts/0 S+ 21:25 2:04 operf java -agentpath:/usr/lib64/oprofile/libjvmti_oprofile.so spec.jbb.JBBmain -propfile SPECjbb.props > I could imagine some options like --separate-thread and > --separate-cpu increasing the number of files open to record data > and increasing the resident memory use. Also the way that perf works > in system-wide mode each processor on the system needs to have a > perf open and buffer space allocated; for many processor machine > this overhead could be come more noticable. > > The main data collection loop for operf is in operf_counter.cpp the > method operf_record::recordPerfData(void) and this is calling > OP_perf_utils::op_get_kernel_event_data(struct mmap_data *md, > operf_record * pr). op_get_kernel_event_data() uses a fixed size > mmap'ed page to get data from the kernel. It looks like this should > be process immediately by OP_perf_utils::op_write_event(event_t * > event, u64 sample_type). > > What does the output of the following look like when this is happening? > > $ pmap `pgrep operf` 2995: operf java -agentpath:/usr/lib64/oprofile/libjvmti_oprofile.so spec.jbb.JBBmain -propfile SPECjbb.props 0000000000400000 256K r-x-- operf 0000000000440000 64K rw--- operf 0000000000450000 128K rw--- [ anon ] 00000000086b0000 192K rw--- [ anon ] 000003ffa82a0000 1408K r-x-- libc-2.17.so 000003ffa8400000 64K r---- libc-2.17.so 000003ffa8410000 64K rw--- libc-2.17.so 000003ffa8420000 128K r-x-- libgcc_s-4.8.3-20140911.so.1 000003ffa8440000 64K rw--- libgcc_s-4.8.3-20140911.so.1 000003ffa8450000 640K r-x-- libm-2.17.so 000003ffa84f0000 64K r---- libm-2.17.so 000003ffa8500000 64K rw--- libm-2.17.so 000003ffa8510000 960K r-x-- libstdc++.so.6.0.19 000003ffa8600000 64K r---- libstdc++.so.6.0.19 000003ffa8610000 64K rw--- libstdc++.so.6.0.19 000003ffa8620000 64K rw--- [ anon ] 000003ffa8630000 64K r-x-- libdl-2.17.so 000003ffa8640000 64K r---- libdl-2.17.so 000003ffa8650000 64K rw--- libdl-2.17.so 000003ffa8680000 64K r---- [ anon ] 000003ffa8690000 64K r-x-- [ anon ] 000003ffa86a0000 128K r-x-- ld-2.17.so 000003ffa86c0000 64K r---- ld-2.17.so 000003ffa86d0000 64K rw--- ld-2.17.so 000003ffc5a80000 192K rw--- [ stack ] total 5056K 2998: operf java -agentpath:/usr/lib64/oprofile/libjvmti_oprofile.so spec.jbb.JBBmain -propfile SPECjbb.props 0000000000400000 256K r-x-- operf 0000000000440000 64K rw--- operf 0000000000450000 128K rw--- [ anon ] 00000000086b0000 192K rw--- [ anon ] 000003ffa7e20000 576K rw-s- [ anon ] 000003ffa7eb0000 576K rw-s- [ anon ] 000003ffa7f40000 576K rw-s- [ anon ] 000003ffa7fd0000 576K rw-s- [ anon ] 000003ffa8060000 576K rw-s- [ anon ] 000003ffa80f0000 576K rw-s- [ anon ] 000003ffa8180000 576K rw-s- [ anon ] 000003ffa8210000 576K rw-s- [ anon ] 000003ffa82a0000 1408K r-x-- libc-2.17.so 000003ffa8400000 64K r---- libc-2.17.so 000003ffa8410000 64K rw--- libc-2.17.so 000003ffa8420000 128K r-x-- libgcc_s-4.8.3-20140911.so.1 000003ffa8440000 64K rw--- libgcc_s-4.8.3-20140911.so.1 000003ffa8450000 640K r-x-- libm-2.17.so 000003ffa84f0000 64K r---- libm-2.17.so 000003ffa8500000 64K rw--- libm-2.17.so 000003ffa8510000 960K r-x-- libstdc++.so.6.0.19 000003ffa8600000 64K r---- libstdc++.so.6.0.19 000003ffa8610000 64K rw--- libstdc++.so.6.0.19 000003ffa8620000 64K rw--- [ anon ] 000003ffa8630000 64K r-x-- libdl-2.17.so 000003ffa8640000 64K r---- libdl-2.17.so 000003ffa8650000 64K rw--- libdl-2.17.so 000003ffa8680000 64K r---- [ anon ] 000003ffa8690000 64K r-x-- [ anon ] 000003ffa86a0000 128K r-x-- ld-2.17.so 000003ffa86c0000 64K r---- ld-2.17.so 000003ffa86d0000 64K rw--- ld-2.17.so 000003ffc5a80000 192K rw--- [ stack ] total 9664K 2999: operf java -agentpath:/usr/lib64/oprofile/libjvmti_oprofile.so spec.jbb.JBBmain -propfile SPECjbb.props 0000000000400000 256K r-x-- operf 0000000000440000 64K rw--- operf 0000000000450000 128K rw--- [ anon ] 00000000086b0000 192K rw--- [ anon ] 00000000086e0000 4036800K rw--- [ anon ] 000003ffa7f10000 1344K rw-s- CPU_CYCLES.100000.0.all.all.all 000003ffa8150000 192K rw-s- CPU_CYCLES.100000.0.all.all.all 000003ffa81a0000 128K rw-s- CPU_CYCLES.100000.0.all.all.all 000003ffa81c0000 64K rw-s- CPU_CYCLES.100000.0.all.all.all 000003ffa81d0000 64K rw-s- CPU_CYCLES.100000.0.all.all.all 000003ffa81e0000 64K rw-s- CPU_CYCLES.100000.0.all.all.all 000003ffa81f0000 64K rw-s- CPU_CYCLES.100000.0.all.all.all 000003ffa8210000 64K rw-s- CPU_CYCLES.100000.0.all.all.all 000003ffa8220000 64K rw-s- CPU_CYCLES.100000.0.all.all.all 000003ffa8230000 64K rw-s- CPU_CYCLES.100000.0.all.all.all 000003ffa8240000 64K rw-s- CPU_CYCLES.100000.0.all.all.all 000003ffa8250000 64K rw-s- CPU_CYCLES.100000.0.all.all.all 000003ffa8260000 64K rw-s- CPU_CYCLES.100000.0.all.all.all 000003ffa8270000 64K rw-s- CPU_CYCLES.100000.0.all.all.all 000003ffa8290000 64K rw-s- CPU_CYCLES.100000.0.all.all.all 000003ffa82a0000 1408K r-x-- libc-2.17.so 000003ffa8400000 64K r---- libc-2.17.so 000003ffa8410000 64K rw--- libc-2.17.so 000003ffa8420000 128K r-x-- libgcc_s-4.8.3-20140911.so.1 000003ffa8440000 64K rw--- libgcc_s-4.8.3-20140911.so.1 000003ffa8450000 640K r-x-- libm-2.17.so 000003ffa84f0000 64K r---- libm-2.17.so 000003ffa8500000 64K rw--- libm-2.17.so 000003ffa8510000 960K r-x-- libstdc++.so.6.0.19 000003ffa8600000 64K r---- libstdc++.so.6.0.19 000003ffa8610000 64K rw--- libstdc++.so.6.0.19 000003ffa8620000 64K rw--- [ anon ] 000003ffa8630000 64K r-x-- libdl-2.17.so 000003ffa8640000 64K r---- libdl-2.17.so 000003ffa8650000 64K rw--- libdl-2.17.so 000003ffa8660000 64K rw-s- CPU_CYCLES.100000.0.all.all.all 000003ffa8680000 64K r---- [ anon ] 000003ffa8690000 64K r-x-- [ anon ] 000003ffa86a0000 128K r-x-- ld-2.17.so 000003ffa86c0000 64K r---- ld-2.17.so 000003ffa86d0000 64K rw--- ld-2.17.so 000003ffc5a80000 192K rw--- [ stack ] total 4044352K ------------------------------------------------------------------------------ One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y