Work at SourceForge, help us to make it a better place! We have an immediate need for a Support Technician in our San Francisco or Denver office.

Close

#35 files above 2 GiB

open
nobody
None
8
2004-09-14
2004-08-12
Pat LaVarre
No

Paraphrasing Ben Fennema thoughts of 2004-05-24:

Somebody please make a disc, write a file of more than
2 GiB on to it without holes, confirm that the phgfsck
likes it, and then confirm or deny that Linux udf.ko
represents its length correctly.

We really should find we can do all of this in Linux,
done, but if need be we hear the dd of Mac OS X and the
pldd of Windows can do it. Key is acquiring a dd that
copes with lengths above the signed 32-bit limit of
x7FFF:FFFF B = 2 GiB - 1.

Higher priority because Ben's interested, gave us much
or most of udf.ko, and is now maintaining udf.ko.
Lower priority because as yet we have not reproduced
this trouble.

Discussion

  • Pat LaVarre
    Pat LaVarre
    2004-08-12

    • priority: 8 --> 5
     
  • Logged In: NO

    Offline I see 2004-06-01 someone said: "reading files > 4
    gig work, but there still seems to be a write issue in
    creating them w/ lost blocks. Hmm."

     
  • Pat LaVarre
    Pat LaVarre
    2004-08-16

    Logged In: YES
    user_id=964855

    Not simply reproducible in 2.6.8.1.

    Here following is a log of creating a file of 5123 MiB
    without holes in ten minutes, ending with metadata that an
    ls -l and the phgfsck both like.

    I suppose the next step is trying to write-read-compare a
    pattern more interesting than all zeroes.

    $ uname -msr
    Linux 2.6.8.1 i686
    $ sudo rrd scan /dev/scd1
    /dev/scd1 is Iomega RRD 74.B rw
    $ sudo plscsi /dev/sg1
    export PLSCSI=/dev/sg1 // B Iomega RRD
    74.B
    $ mkudffs 2>&1 | head -1
    mkudffs 1.0.0b3 for UDF FS 1.0.0-cvs, 2004/03/09
    $
    $ sudo mkudffs --udfrev=0x0102 /dev/scd1
    start=0, blocks=16, type=RESERVED
    start=16, blocks=3, type=VRS
    start=19, blocks=237, type=USPACE
    start=256, blocks=1, type=ANCHOR
    start=257, blocks=16, type=PVDS
    start=273, blocks=1, type=LVID
    start=274, blocks=17090349, type=PSPACE
    start=17090623, blocks=1, type=ANCHOR
    start=17090624, blocks=239, type=USPACE
    start=17090863, blocks=16, type=RVDS
    start=17090879, blocks=1, type=ANCHOR
    $
    $ sudo mount /dev/scd1 /mnt/scd1
    $ cd /mnt/scd1
    $ time sudo dd of=gb.bin bs=1M count=5123 </dev/zero
    5123+0 records in
    5123+0 records out

    real 9m15.537s
    user 0m0.044s
    sys 0m19.153s
    $ time sync

    real 0m0.626s
    user 0m0.001s
    sys 0m0.007s
    $ cd
    $ sudo umount /mnt/scd1
    Password:
    $
    $ sudo tail -1 /var/log/messages
    Aug 16 09:05:01 patlinux kernel: UDF-fs INFO UDF 0.9.8.1
    (2004/29/09) Mounting volume 'LinuxUDF', timestamp
    2004/08/16 09:04 (1e98)
    $
    $ sudo mount -r /dev/scd1 /mnt/scd1
    $ ls -l /mnt/scd1
    total 5245952
    -rw-r--r-- 1 root root 5371854848 Aug 16 09:14 gb.bin
    drwxr-xr-x 2 root root 40 Aug 16 09:04
    lost+found
    $ dc -e '5123 1024 * 1024 * p'
    5371854848
    $ sudo umount /dev/scd1
    $
    $ time sudo phgfsck -scsi /dev/sg1 >gb.txt

    real 0m0.655s
    user 0m0.291s
    sys 0m0.024s
    $ cat gb.txt | egrep -i '(note|warning|error|version).*:'
    Application version: 1.1r0
    UCT Core version : 1.1r0
    INQUIRY/ANSI Version : 2
    Version Descriptors : 0x03A0 0x1729 0x1730
    FSD 28 Error: Interchange Level: 2, Maximum
    Interchange Level: 3,
    Note: LVID Min/Max Revision unequal to Medium Revision,
    Error count: 1 total occurrences: 1
    Warning count: 0 total occurrences: 0
    $ grep -B1 -A8 'FE.*gb.bin' gb.txt
    1321 read block
    FE file type FILE name: "gb.bin"
    cnt: extent type, size, location,part,
    body, total alloc
    1: long_ad 0 1073739776 1048 0
    1073739776 1073739776
    2: long_ad 0 1073739776 525335 0
    2147479552 2147479552
    3: long_ad 0 1073739776 1049622 0
    3221219328 3221219328
    4: long_ad 0 1073739776 1573909 0
    4294959104 4294959104
    5: long_ad 0 1073739776 2098196 0
    #13FFFD800 #13FFFD800
    6: long_ad 0 3155968 2622483 0
    #140300000 #140300000

    $ dc -e '1073739776 5 * 3155968 + p'
    5371854848
    $

     
  • Pat LaVarre
    Pat LaVarre
    2004-08-16

    Logged In: YES
    user_id=964855

    PATAPI looks the same as USB, though my sample took more
    like 16 minutes to create a file of 5123 MiB of zeroes.
    Again I forgot to include the final sync and umount in the
    measure of time to write.

     
  • Pat LaVarre
    Pat LaVarre
    2004-08-16

    Logged In: YES
    user_id=964855

    > write-read-compare a pattern more interesting than all zeroes.

    I don't have a good way to do this, in free or open source.

    If I start with something like </dev/zero tr '.' $'\xAE'
    then I find that dd sometimes writes less than whole bs blocks.

    If I roll my own source code, then you have to trust it or
    read it.

    I guess I will roll my own source code.

     
  • Pat LaVarre
    Pat LaVarre
    2004-09-14

    • priority: 5 --> 6
     
  • Pat LaVarre
    Pat LaVarre
    2004-09-14

    • priority: 6 --> 7
     
  • Pat LaVarre
    Pat LaVarre
    2004-09-14

    • priority: 7 --> 8