Re: How to generate a large file allocating space

--On 31 October 2010 10:23:51 -0500 Bruno Wolff III <[email protected]> wrote:

> On Sun, Oct 31, 2010 at 11:12:41 +0100,
>   Alex Bligh <[email protected]> wrote:
>> I want to generate or extend a large file in an ext4 filesystem
>> allocating space (i.e. not creating a sparse file) but not actually
>> writing any data. I realise that this will result in the file containing
>> the contents of whatever was there on the disk before, which is a
>> possible security problem in some circumstances, but it isn't a problem
>> here.
>
> There isn't going to be a way to do that through the file system, because
> as you note it is a security problem.
>
> What is the high level thing you are trying to accomplish here? Modifying
> the filesystem offline seems risky and maybe there is a safer way to
> accomplish your goals.

I am trying to allocate huge files on ext4. I will then read the extents
within the file and write to the disk at a block level rather than using
ext4 (the FS will not be mounted at this point). This will allow me to
have several iSCSI clients hitting the same LUN r/w safely. And at
some point when I know the relevant iSCSI stuff has stopped and been
flushed to disk, I may unlink the file.

As I have total control of what's on the disk, I don't really care
if previous content is exposed. If I write many Gigabyets of zeroes,
that's going to take a long time, and be totally unnecessary, since
I already have my own internal map of the data I will write into these
huge files.

Yes, I know this is deep scary voodoo, but that's ok. I can get the
extent list the same way as "filefrag -v" gets it. What I can't
currently work out (using either the library, or doing it with the
volume mounted) is how to extend a file AND allocate the extents
(as opposed to doing it sparse).

>> Supplementary question: can I assume that if a non-sparse file is on disk
>> and never opened, and never unlinked, then the sectors used to to store
>> that file's data will never change irrespective of other operations on
>> the ext4 filesystem? IE nothing is shuffling where ext4 files are stored.
>
> I think SSDs will move stuff around at a very low level. They would look
> like they are at the same place to stuff access the device like a disk,
> but physically would be stored in a different hardware location.
>
> With normal disks, you'd only see this if the device got a read error, but
> was able to successfully read a marginal sector and remap it to a spare
> sector. But again, stuff talking to the disk will see it at the same
> address.

Sure, that's no problem because the offset into the block device stays
the same, even if physically the file is in a different place. So the
extent list will stay the same for the file.

-- 
Alex Bligh