Re: How to get large items from a table

On Wed, Apr 2, 2014 at 10:56 PM, shawn l.green <[email protected]> wrote:
> Hello Harry,

Hi Shawn,

> You appear to face the same resource dilemmas that all storage engines face:
> there will be more data stored on disk than you can possibly buffer in
> memory. There have been many strategies designed to keep only the 'most
> important' or 'most recently used' elements in memory. Please look to them
> for inspiration.
>
> To go along with trying to keep a 'warm' cache of data for faster retrieval,
> you need to decide how to manage scans of large quantities of data (such as
> your SELECT * ...  query) and if you want them to replace the warm buffer
> you have been cultivating since startup.  This may mean that you skip trying
> to buffer any scan that uses more than X percent of your table (you get to
> pick what the X is) to keep your cache nice and toasty.
>
> And you may have multiple cache types, hot, warm, cold, temporary, ...
> depending on how you want to manage them.  Again, this is a hugely broad
> topic with many different algorithms used in both commercial and open source
> products. There is no one 'correct' answer for all situations.
>
> Then you get to the subject of index management, file storage formats, row
> identification, and how to locate a row on disk.  That's an entirely
> different beast. Of course, if you store nothing on disk then those design
> questions become irrelevant.
>

Thanks very much, I would take your above suggestions into consideration.


Harry.

-- 
MySQL Internals Mailing List
For list archives: http://lists.mysql.com/internals
To unsubscribe:    http://lists.mysql.com/internals