#35 files above 2 GiB

[Pat LaVarre]

Paraphrasing Ben Fennema thoughts of 2004-05-24:

Somebody please make a disc, write a file of more than
2 GiB on to it without holes, confirm that the phgfsck
likes it, and then confirm or deny that Linux udf.ko
represents its length correctly.

We really should find we can do all of this in Linux,
done, but if need be we hear the dd of Mac OS X and the
pldd of Windows can do it. Key is acquiring a dd that
copes with lengths above the signed 32-bit limit of
x7FFF:FFFF B = 2 GiB - 1.

Higher priority because Ben's interested, gave us much
or most of udf.ko, and is now maintaining udf.ko.
Lower priority because as yet we have not reproduced
this trouble.

[Nobody/Anonymous]

Logged In: NO

Offline I see 2004-06-01 someone said: "reading files > 4
gig work, but there still seems to be a write issue in
creating them w/ lost blocks. Hmm."

[Pat LaVarre]

Logged In: YES
user_id=964855

Not simply reproducible in 2.6.8.1.

Here following is a log of creating a file of 5123 MiB
without holes in ten minutes, ending with metadata that an
ls -l and the phgfsck both like.

I suppose the next step is trying to write-read-compare a
pattern more interesting than all zeroes.

$ uname -msr
Linux 2.6.8.1 i686
$ sudo rrd scan /dev/scd1
/dev/scd1 is Iomega RRD 74.B rw
$ sudo plscsi /dev/sg1
export PLSCSI=/dev/sg1 // B Iomega RRD
74.B
$ mkudffs 2>&1 | head -1
mkudffs 1.0.0b3 for UDF FS 1.0.0-cvs, 2004/03/09
$
$ sudo mkudffs --udfrev=0x0102 /dev/scd1
start=0, blocks=16, type=RESERVED
start=16, blocks=3, type=VRS
start=19, blocks=237, type=USPACE
start=256, blocks=1, type=ANCHOR
start=257, blocks=16, type=PVDS
start=273, blocks=1, type=LVID
start=274, blocks=17090349, type=PSPACE
start=17090623, blocks=1, type=ANCHOR
start=17090624, blocks=239, type=USPACE
start=17090863, blocks=16, type=RVDS
start=17090879, blocks=1, type=ANCHOR
$
$ sudo mount /dev/scd1 /mnt/scd1
$ cd /mnt/scd1
$ time sudo dd of=gb.bin bs=1M count=5123 </dev/zero
5123+0 records in
5123+0 records out

real 9m15.537s
user 0m0.044s
sys 0m19.153s
$ time sync

real 0m0.626s
user 0m0.001s
sys 0m0.007s
$ cd
$ sudo umount /mnt/scd1
Password:
$
$ sudo tail -1 /var/log/messages
Aug 16 09:05:01 patlinux kernel: UDF-fs INFO UDF 0.9.8.1
(2004/29/09) Mounting volume 'LinuxUDF', timestamp
2004/08/16 09:04 (1e98)
$
$ sudo mount -r /dev/scd1 /mnt/scd1
$ ls -l /mnt/scd1
total 5245952
-rw-r--r-- 1 root root 5371854848 Aug 16 09:14 gb.bin
drwxr-xr-x 2 root root 40 Aug 16 09:04
lost+found
$ dc -e '5123 1024 * 1024 * p'
5371854848
$ sudo umount /dev/scd1
$
$ time sudo phgfsck -scsi /dev/sg1 >gb.txt

real 0m0.655s
user 0m0.291s
sys 0m0.024s
$ cat gb.txt | egrep -i '(note|warning|error|version).*:'
Application version: 1.1r0
UCT Core version : 1.1r0
INQUIRY/ANSI Version : 2
Version Descriptors : 0x03A0 0x1729 0x1730
FSD 28 Error: Interchange Level: 2, Maximum
Interchange Level: 3,
Note: LVID Min/Max Revision unequal to Medium Revision,
Error count: 1 total occurrences: 1
Warning count: 0 total occurrences: 0
$ grep -B1 -A8 'FE.*gb.bin' gb.txt
1321 read block
FE file type FILE name: "gb.bin"
cnt: extent type, size, location,part,
body, total alloc
1: long_ad 0 1073739776 1048 0
1073739776 1073739776
2: long_ad 0 1073739776 525335 0
2147479552 2147479552
3: long_ad 0 1073739776 1049622 0
3221219328 3221219328
4: long_ad 0 1073739776 1573909 0
4294959104 4294959104
5: long_ad 0 1073739776 2098196 0
#13FFFD800 #13FFFD800
6: long_ad 0 3155968 2622483 0
#140300000 #140300000

$ dc -e '1073739776 5 * 3155968 + p'
5371854848
$

[Pat LaVarre]

Logged In: YES
user_id=964855

PATAPI looks the same as USB, though my sample took more
like 16 minutes to create a file of 5123 MiB of zeroes.
Again I forgot to include the final sync and umount in the
measure of time to write.

[Pat LaVarre]

Logged In: YES
user_id=964855

> write-read-compare a pattern more interesting than all zeroes.

I don't have a good way to do this, in free or open source.

If I start with something like </dev/zero tr '.' $'\xAE'
then I find that dd sometimes writes less than whole bs blocks.

If I roll my own source code, then you have to trust it or
read it.

I guess I will roll my own source code.