Re: Accuracy of profiling

On 06/05/2015 04:09 AM, Mahmood N wrote:
>>I think these are the hardware counters: they don't sample anything,
>>and they are the absolute numbers.  So what causes differences?
>>Interrupts, interaction with I/O devices, and so on.  I would never
>>expect two runs to be identical.
> 
> I understand that but the problem is that in the first run (which took 37.9s), the hardware prefetcher was enabled while in the second run (which took 35.9s), the hardware prefetcher was disabled. So, I can not determine that the reduction in the execution time was the result of system wide caching or disabling the prefetcher. Note that there are some applications that are not positive to prefetching.
>  
> Regards,
> Mahmood

Hi Mahmood,

It can be hard to get repeatable results.  Keep in mind that the operating system is trying very hard to be efficient.  Depending on where the data is being stored for the bzip the first one may have had more IO to physical disk than the second one because the OS was caching files in memory.  You will want to do multiple runs of the experiment and take steps to make sure that the machine is in the same state for each run.  

You might want to get a book like "The Art of Computer Systems Performance Analysis" by Raj Jain to better understand issues with performance analysis of computer systems.

-Will


------------------------------------------------------------------------------