Re: How to benchmark memory managers?

On Jan 9, 2005, at 5:46 PM, Matthew Hertz wrote:

>>> We use exact object reachability traces [27, 28] as an oracle to 
>>> indicate when objects should be deallocated. By implementing this 
>>> oracle inside a detailed architectural simulator, we can execute and 
>>> precisely measure unaltered Java applications as if they were 
>>> written using explicit memory management.
>> by using an oracle, you eliminate the cost of the explicit management 
>> that a programmer using explicit memory management would have had to 
>> implement in order to determine when objects become garbage. Thus any 
>> cost of explicit destruction or reference counting which may have to 
>> be done is not accounted for. So I think this methodology is biased 
>> since it completely eliminates the cost on one side that is being 
>> counted on the other.
>
> As we describe in the paper, the system inserts calls to free whenever 
> the
> oracle (object reachability trace) determines that an object can be 
> deallocated.
>  Because the programs are run in a simulator, I capture the full cost 
> of these
> deallocation calls without adding any overhead for the oracle.  Since 
> the
> results in our paper include the full costs of both explicit and 
> implicit memory
> management, we stand by our conclusions.

the cost of free only equals the cost of explicit memory management if 
none of the objects ever had shared ownership. Otherwise there is some 
mechanism being hidden by the oracle that is not being accounted for.