Re: [PATCH] Add support for Intel Skylake events

On 07/03/2015 12:21 PM, Andi Kleen wrote:
> 
> Thanks for the detailed review. I reported the issues you pointed out and
> will send an incremential patch with the fixes.

Hi Andi,

I know how easy it is to miss an event setting or end up with the wrong number in one of the setting, so I figured that it would be good to do a sanity check of the events and get the changes sooner rather than having some event not work some months down the line when somewhen really needs it.
 
I will go ahead and check-in this patch and look forward to the update.

-Will


> 
>> Other Intel processors events and unit_mask files have the following 
>>
>> include i386/arch_perfmon 
>>
>> Any particular reasons that the skylake puts them directly in the events rather than using an include?
> 
> The new event list has some derived events from the fixed counters (e.g. an ANY version of the reference clock)
> So the unit masks of the fixed counters need to be extended. I didn't find a way to do that with 
> the include, so now it just duplicates the information.
> 
>>> +#
>>> +# Intel "Skylake" microarchitecture core events.
>>> +#
>>> +# See http://ark.intel.com/ for help in identifying Skylake based CPUs
>>> +#
>>> +# Note the minimum counts are not discovered experimentally and could be likely
>>> +# lowered in many cases without ill effect.
>>> +#
>>> +event:0x00 counters:1 um:inst_retired minimum:2000003 name:inst_retired :
>> Is this event number correct for inst_retired?  Is this attempting to get inst_retired.pred_dist?
> 
> It is not. I'll fix it.
> 
>>
>> The hle_retired and rtm_retired are not listed in the skylake table are those in the processor?
> 
> The SDM (awkwardly) lists the TSX events in a separate table shared with Haswell.
> It's in the processor.
> 
>>> +	0x1 extra: any Counts the total number when the front end is resteered, mainly when the BPU cannot provide a correct prediction and this is corrected by other branch handling mechanisms at the front end.
>>> +name:dsb2mite_switches type:mandatory default:0x2
>>> +	0x2 extra: penalty_cycles Decode Stream Buffer (DSB)-to-MITE switch true penalty cycles.
>>> +name:ept type:mandatory default:0x10
>>> +	0x10 extra: walk_pending Counts 1 per cycle for each PMH that is busy with a EPT (Extended Page Table) walk for any request type.
>>> +name:fp_assist type:mandatory default:0x1e
>>> +	0x1e extra:cmask=1 any Cycles with any input/output SSE or FP assist
>>> +name:hw_interrupts type:mandatory default:0x1
>>> +	0x1 extra: received Number of hardware interrupts received by the processor.
>>> +name:icache_16b type:mandatory default:0x4
>>> +	0x4 extra: ifdata_stall Cycles where a code fetch is stalled due to L1 instruction cache miss.
>>
>> It looks like there is a error on page 738 of http://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-system-programming-manual-325384.pdf there are two events listed for event 80H unit mask 04H. Should the icache_64b.iftag_stall be 83h event rather than 80H?
> 
> Yes. I'll report it. My listing should be correct.
> 
>>> +	0xe4 extra: all_code_rd L2 code requests
>>> +	0xf8 extra: all_pf Requests from the L1/L2/L3 hardware prefetchers or Load software prefetches
>>> +	0x38 extra: pf_miss Requests from the L1/L2/L3 hardware prefetchers or Load software prefetches that miss L2 cache
>>> +	0xd8 extra: pf_hit Requests from the L1/L2/L3 hardware prefetchers or Load software prefetches that hit L2 cache
>>> +	0x42 extra: rfo_hit RFO requests that hit L2 cache
>>> +	0x22 extra: rfo_miss RFO requests that miss L2 cache
>>> +	0x44 extra: code_rd_hit L2 cache hits when fetching instructions, code reads.
>>> +	0x24 extra: code_rd_miss L2 cache misses when fetching instructions
>>> +	0x27 extra: all_demand_miss Demand requests that miss L2 cache
>>> +	0xe7 extra: all_demand_references Demand requests to L2 cache
>>> +	0x3f extra: miss All requests that miss L2 cache
>>> +	0xff extra: references All L2 requests
>>
>> Should there be an 0xef in the l2_rqsts unit mask for L2_RQSTS.REFERENCES?
> 
> Why? 0xff is the correct mask for REFERENCES.
> 
>>
>>> +name:longest_lat_cache type:exclusive default:0x41
>>> +	0x41 extra: miss Core-originated cacheable demand requests missed L3
>>> +	0x4f extra: reference Core-originated cacheable demand requests that refer to L3
>>> +name:cpu_clk_thread_unhalted type:exclusive default:ref_xclk
>>> +	0x1 extra: ref_xclk Reference cycles when the thread is unhalted (counts at 100 MHz rate)
>>> +	0x2 extra: one_thread_active Count XClk pulses when this thread is unhalted and the other thread is halted.
>>> +	0x1 extra:any ref_xclk_any Reference cycles when the at least one thread on the physical core is unhalted (counts at 100 MHz rate)
>>
>> What about 0x0h unit mask for cpu_clk_unhalted for thread_p and thread_p_any?
> 
> I'll fix those.
> 
>>> +name:dtlb_store_misses type:exclusive default:miss_causes_a_walk
>>> +	0x1 extra: miss_causes_a_walk Store misses in all DTLB levels that cause page walks
>>> +	0x10 extra: walk_pending Counts 1 per cycle for each PMH that is busy with a page walk for a store.
>>> +	0x20 extra: stlb_hit Stores that miss the DTLB and hit the STLB.
>>> +	0xe extra: walk_completed Store misses in all TLB levels causes a page walk that completes. (All page sizes)
>>> +	0x10 extra:cmask=1 walk_active Cycles when at least one PMH is busy with a page walk for a store.
>>> +name:tx_mem type:exclusive default:0x1
>>> +	0x1 extra: abort_conflict Number of times a transactional abort was signaled due to a data conflict on a transactionally accessed address
>>> +	0x2 extra: abort_capacity Number of times a transactional abort was signaled due to a data capacity limitation for transactional reads or writes.
>>> +	0x4 extra: abort_hle_store_to_elided_lock Number of times a HLE transactional region aborted due to a non XRELEASE prefixed instruction writing to an elided lock in the elision buffer
>>> +	0x8 extra: abort_hle_elision_buffer_not_empty Number of times an HLE transactional execution aborted due to NoAllocatedElisionBuffer being non-zero.
>>> +	0x10 extra: abort_hle_elision_buffer_mismatch Number of times an HLE transactional execution aborted due to XRELEASE lock not satisfying the address and value requirements in the elision buffer
>>> +	0x20 extra: abort_hle_elision_buffer_unsupported_alignment Number of times an HLE transactional execution aborted due to an unsupported read alignment from the elision buffer.
>>> +	0x40 extra: hle_elision_buffer_full Number of times HLE lock could not be elided due to ElisionBufferAvailable being zero.
>>
>> Why does the system software manual only list TX_MEM.ABORT_CAPACITY for skylake (0x54 event, 0x02 unitmask)? The haswell tsx performance table lists seems to match up with the above.  
> 
> It only lists changed events. It's the only TSX event that changed.
> 
>>> +name:rs_events type:exclusive default:empty_cycles
>>> +	0x1 extra: empty_cycles Cycles when Reservation Station (RS) is empty for the thread
>>> +	0x1 extra:cmask=1,inv,edge empty_end Counts end of periods where the Reservation Station (RS) was empty. Could be useful to precisely locate Frontend Latency Bound issues.
>>> +name:offcore_requests_outstanding type:exclusive default:demand_data_rd
>>> +	0x1 extra: demand_data_rd Offcore outstanding Demand Data Read transactions in uncore queue.
>>> +	0x2 extra:cmask=1 demand_code_rd Cycles with offcore outstanding Code Reads transactions in the SuperQueue (SQ), queue to uncore, every cycle.
>>> +	0x4 extra:cmask=1 demand_rfo Offcore outstanding demand rfo reads transactions in SuperQueue (SQ), queue to uncore, every cycle
>>> +	0x8 extra: all_data_rd Offcore outstanding cacheable Core Data Read transactions in SuperQueue (SQ), queue to uncore
>>> +	0x10 extra: l3_miss_demand_data_rd Counts number of Offcore outstanding Demand Data Read requests who miss L3 cache in the superQ every cycle.
>>> +	0x1 extra:cmask=1 cycles_with_demand_data_rd Cycles when offcore outstanding Demand Data Read transactions are present in SuperQueue (SQ), queue to uncore
>>> +	0x8 extra:cmask=1 cycles_with_data_rd Cycles when offcore outstanding cacheable Core Data Read transactions are present in SuperQueue (SQ), queue to uncore
>>> +	0x2 extra:cmask=1 cycles_with_demand_code_rd Cycles with offcore outstanding Code Reads transactions in the SuperQueue (SQ), queue to uncore, every cycle.
>>> +	0x4 extra:cmask=1 cycles_with_demand_rfo Offcore outstanding demand rfo reads transactions in SuperQueue (SQ), queue to uncore, every cycle
>>
>> It looks like 0x2 and 0x4 have two events with cmask=1 in offcore_requests_outstanding unitmask. Shouldn't one of those be without cmask?
> 
> Yes you're right. One set should have no cmask. The descriptions are also wrong. I'll report and fix it.
>>
>>> +name:idq type:exclusive default:mite_uops
>>> +	0x4 extra: mite_uops Uops delivered to Instruction Decode Queue (IDQ) from MITE path
>>> +	0x8 extra: dsb_uops Uops delivered to Instruction Decode Queue (IDQ) from the Decode Stream Buffer (DSB) path
>>> +	0x20 extra: ms_mite_uops Uops initiated by MITE and delivered to Instruction Decode Queue (IDQ) while Microcode Sequenser (MS) is busy
>>> +	0x30 extra:cmask=1 ms_cycles Cycles when uops are being delivered to Instruction Decode Queue (IDQ) while Microcode Sequenser (MS) is busy
>>> +	0x4 extra:cmask=1 mite_cycles Cycles when uops are being delivered to Instruction Decode Queue (IDQ) from MITE path
>>> +	0x8 extra:cmask=1 dsb_cycles Cycles when uops are being delivered to Instruction Decode Queue (IDQ) from Decode Stream Buffer (DSB) path
>>> +	0x10 extra:cmask=1 ms_dsb_cycles Cycles when uops initiated by Decode Stream Buffer (DSB) are being delivered to Instruction Decode Queue (IDQ) while Microcode Sequenser (MS) is busy
>>> +	0x18 extra:cmask=4 all_dsb_cycles_4_uops Cycles Decode Stream Buffer (DSB) is delivering 4 Uops
>>> +	0x18 extra:cmask=1 all_dsb_cycles_any_uops Cycles Decode Stream Buffer (DSB) is delivering any Uop
>>> +	0x24 extra:cmask=4 all_mite_cycles_4_uops Cycles MITE is delivering 4 Uops
>>> +	0x24 extra:cmask=1 all_mite_cycles_any_uops Cycles MITE is delivering any Uop
>>> +	0x30 extra:cmask=1,edge ms_switches Number of switches from DSB (Decode Stream Buffer) or MITE (legacy decode pipeline) to the Microcode Sequencer
>>> +	0x30 extra: ms_uops Uops delivered to Instruction Decode Queue (IDQ) while Microcode Sequenser (MS) is busy
>>
>> is  IDQ.MS_DSB_UOPS setting suppose to be missing from idq unit mask?
> 
> That event is not in my list.
> 
>>> +	0x2 extra: retire_slots Retirement slots used.
>>> +	0x1 extra:cmask=1,inv stall_cycles Cycles without actually retired uops.
>>> +	0x1 extra:cmask=a,inv total_cycles Cycles with less than 10 actually retired uops.
>>> +name:machine_clears type:exclusive default:0x1
>>> +	0x1 extra:cmask=1,edge count Number of machine clears (nukes) of any type.
>>> +	0x2 extra: memory_ordering Counts the number of machine clears due to memory order conflicts.
>>> +	0x4 extra: smc Self-modifying code (SMC) detected.
>>
>> The table doesn't have edge for flag set for machine_clear.count.  Is that required for only that event?
> 
> I believe so.
> 
>>> +name:br_misp_retired type:exclusive default:all_branches
>>> +	0x0 extra: all_branches All mispredicted macro branch instructions retired.
>>> +	0x1 extra: conditional Mispredicted conditional branch instructions retired.
>>> +	0x1 extra:pebs conditional_pebs Mispredicted conditional branch instructions retired.
>>> +	0x20 extra: near_taken number of near branch instructions retired that were mispredicted and taken.
>>> +	0x20 extra:pebs near_taken_pebs number of near branch instructions retired that were mispredicted and taken.
>>> +	0x4 extra:pebs all_branches_pebs Mispredicted macro branch instructions retired.
>>
>> Any reasons for omitting all_branches non-pebs variant?
> 
> I'll fix it.
> 
> 
> -Andi
> 
<

------------------------------------------------------------------------------
Don't Limit Your Business. Reach for the Cloud.
GigeNET's Cloud Solutions provide you with the tools and support that
you need to offload your IT needs and focus on growing your business.
Configured For All Businesses. Start Your Cloud Today.
https://www.gigenetcloud.com/