View Single Post
  #33  
Old April 6th 07, 02:48 AM posted to comp.sys.ibm.pc.hardware.storage,comp.arch.storage
Torbjorn Lindgren
external usenet poster
 
Posts: 15
Default Raid0 or Raid5 for network to disk backup (Gigabit)?

Maxim S. Shatskih wrote:
I wouldn't know. Linux ext2/3 has a 2TB file size limit.


Sorry. See the cite from include/linux/ext2_fs.h below and
"__u32 i_size;" in it.

ext2's limit is 4GB. I remember ext3 being compatible with ext2 in
on-disk structures in everything except the transaction log, so,
looks like ext3 is also limited to 4GB per file.


Don't be silly, even a minimal amount of checking would have shown
this to be false for a long time now. The exact file size limit for
ext2/ext3 depends on blocksize, with the default 4 kB it's 2 TB (and
it's been rare to see any other blocksize for a long time now).

IIRC at least (some?) 2.2 kernels had this, though glibc support on
32-bit platforms lagged a bit. Since I was on 32-bit platforms back
then it might well have been MUCH earlier (2.0? 1.2?).

ext2/ext3 has a system of "features" which can be added, both fully
compatible and forward compatible flags are available so to avoid
corruption on incompatible features (ext3 is a set of IIRC two
options, one which says that it has a journal, one which is set when
mounted and removed when umounted, this is why EXT2 only mounts
*clean* EXT3 filesystems). IIRC NTFS has something not that
dissimilar...

The feature you are looking for is "large_size", this is set
automatically when the first 2GB file is created by a kernel which
supports this. I've not read the code, but the following line from the
same file you quoted makes me think they stash the upper bits of the
file size in dir_acl (which probably isn't used for files anyway).

From include/linux/ext2_fs.h:
struct ext2_inode {
....
__u32 i_size; /* Size in bytes */
....
__u32 i_dir_acl; /* Directory ACL */
};
#define i_size_high i_dir_acl

The 2 TB file size limit actually comes from i_blocks, Google found a
patch to extend this but I don't think anyone is really that
interested at the moment. There are a LARGE number of other
filesystems that supports this for Linux if someone actually need
this!


More so, if you will also find the superblock structure, you will see
that ext2 is also limited to 32bit block numbers in the volume. There
are good chances that this means the volume size limit of 2TB (if
"block" is really the disk sector and not a group of sectors).


I have no reason to doubt the statement in Wikipedia and other places
which for Linux 2.6 means 16 TB for ext3 assuming the standard 4kB
block size.

(It depends on block size but unpatched 2.4 and earlier has a hard
limit at 2TB, not sure if this applies to all 2.4 distributions, some
were heavily enhanced with some features from 2.6)

http://en.wikipedia.org/wiki/Ext2
http://en.wikipedia.org/wiki/Comparison_of_file_systems