Re: Ext4 and large (>8TB) files
Arun Nair <[email protected]>
| Newsgroups | gmane.comp.file-systems.ext3.user |
|---|---|
| Message-ID | <[email protected]> |
Ok, so I guess ext4 with 64-bit, or another filesystem for us. Thanks Andreas & Eric for all the help. On Fri, Mar 26, 2010 at 3:38 PM, Andreas Dilger <[email protected]> wrote: > On 2010-03-26, at 16:05, Arun Nair wrote: > >> So I'm guessing switching the system to 64-bit would fix this for us. How >> about increasing the block size from the current 4k? Would that be an option >> too? >> > > Not in the near future, unless you are running on PPC/ARM/SPARC that can > also handle large pages. > > On Fri, Mar 26, 2010 at 2:10 PM, Eric Sandeen <[email protected]> wrote: >> On 03/26/2010 03:50 PM, Arun Nair wrote: >> > Eric, >> > >> > Thanks for the quick reply... see my responses inline... >> > >> > On Fri, Mar 26, 2010 at 12:16 PM, Eric Sandeen <[email protected] >> > <mailto:[email protected]>> wrote: >> > >> > On 03/26/2010 01:52 PM, Arun Nair wrote: >> > > Hi - >> > > >> > > (I apologize for the ext4 question in an ext3 mailer, but I >> couldn't >> > > find a user list for ext4.) >> > >> > [email protected] <mailto:[email protected]> :) >> > but that's ok. >> > >> > >> > Saw that but thought it was a dev-only list, sorry. Next time :) >> >> *shrug* I think user questions are welcome too. At least I don't mind. >> >> ... >> >> > dd fails as mentioned above. xfs_io errors too: >> > [root@camanoe]# xfs_io -F -f -c "pwrite 8T 1M" bigfile2 >> > pwrite64: File too large >> >> Oh. Well, then! Must be something else. >> >> oh, ok: >> >> sbi->s_bitmap_maxbytes = ext4_max_bitmap_size(); >> >> static loff_t ext4_max_size(int blkbits, int has_huge_files) >> { >> loff_t res; >> loff_t upper_limit = MAX_LFS_FILESIZE; >> >> <snip> >> >> /* Sanity check against vm- & vfs- imposed limits */ >> if (res > upper_limit) >> res = upper_limit; >> >> return res; >> } >> >> and: >> >> /* Page cache limit. The filesystems should put that into their s_maxbytes >> limits, otherwise bad things can happen in VM. */ >> #if BITS_PER_LONG==32 >> #define MAX_LFS_FILESIZE (((u64)PAGE_CACHE_SIZE << (BITS_PER_LONG-1))-1) >> >> so it's only giving us 31 bits of pages, not 32. This limits it to 8T >> on a 32-bit machine with 4k pages. >> >> I'm not honestly sure if there is anything in the vm that can't actually >> cope with a 32-bit offset... but until proven otherwise, probably not >> going to change this without a lot of testing & inspection. >> >> -Eric >> >> _______________________________________________ >> Ext3-users mailing list >> [email protected] >> https://www.redhat.com/mailman/listinfo/ext3-users >> > > > Cheers, Andreas > > > > > > _______________________________________________ Ext3-users mailing list [email protected] https://www.redhat.com/mailman/listinfo/ext3-users