Re: VFS GetAllData() and memory allocation

On 05.07.2012 17:10, Eunsoo Roh wrote:
> Currently, iFile::GetAllData() returns the buffer (or mmap()'d pointer)
> via iDataBuffer interface. The problem is, the same data is used for
> read and write operations as well. Since iDataBuffer interface do not
> support read-only
>  
> Apparently, documentation warns the user not to try to modify the
> content of buffer. Yet, I am uneasy because of its inherent danger (or
> imperfections) in design/linguistic perspective.
>  
> welltall suggested to leave it alone as it is user's fault if they try
> to do that. I'm wondering everyone agrees with this decision; or any
> other suggestions on this?

Well, I'm with weltall here...
At least for memory-mapped buffers the mapping should be set to
read-only. At least this will prevent from files on disk being
inadvertently change; IMO this is the most critical issue here.
For in-memory buffers, we can really only warn the user to behave nicely.

A perhaps better way would be to have an interface iDataBufferReadOnly
that grants only ‘const’ access to the data; however, the most difficult
aspect of this would be to have VFS return that *and* provide source
compatibility.

> In addition, old VFS uses its own heap allocator to allocate memory in
> iFile/VfsNode implementation. Specifically, they access a private heap
> object within csVFS class. It works as it is tightly coupled. New
> design, however, is more loosely coupled via interfaces. I'm not sure
> how I should approach this. I'm just using cs_malloc() for now, yet
> wondering what private heap is for and what advantages it can
> provide. Anyone have any ideas or suggestions?

The idea was to have a separate heap to reduce (main) heap fragmentation.
Because: files are usually only used during loading and their data
discarded after that. That means then you would have free heap blocks,
from the files, at the start of the heap (because loading is usually the
first thing done), with the processed level data (which is still in use)
afterwards. However, these free heap blocks at the start are “nasty”
because they take up virtual memory without much benefit. A separate
heap for files prevents that as, well, file data goes to a different
heap altogether.
(At least that was the reasoning that led me to believe a heap for VFS
files wasn't a bad idea.)
However, having one heap for all filesystems isn't really necessary.
Depending, on how many instances of objects providing file data you
would typically have (ie filesystem implementation), you could have one
heap per such object (if you have only few) or one heap per plugin (if
you have many).

-f.r.

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

_______________________________________________
Crystal-develop mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/crystal-develop

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk/2AioACgkQNCTzcE/07ez5XgCgywekh/7vy6kkdJK64lrkw8jZ
JjUAoJqA7zP4Vjb3SLMG2bORvW9dU4w0
=YST1
-----END PGP SIGNATURE-----