#200 foo_dsd_converter: incorrect CHUNK sizes

[Jay Moore]

Hello,

This is going to be a pretty obscure bug report that I'm not entirely sure is a bug but it sure feels and acts like it. To put it simply: foo_dsd_converter does not seem to be handling DSDIFF chunk sizes properly in a couple of places. I'd noticed, since it's inception, that my authoring tools such as dsdmerge and even superauthor was not happy with them. I blamed Philips and just ran things through TASCAM to get nice valid files. But I started digging in to the format for reasons of wanting to strip COMT chunks out; which requires adjusting chunk sizes. I will state my findings; but as I do not know how this is written or operates...I don't know where this error occurs or how far it carries over to the process.

The first glaring problem is in the Compression Type Chunk within the Property Chunk.

Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

00000050                    43 4D 50 52 00 00 00 00 00 00        CMPR......
00000060  00 13 44 53 44 20 0E 6E 6F 74 20 63 6F 6D 70 72  ..DSD .not compr
00000070  65 73 73 65 64 00                                essed.

The byte at offset 0x61 should 0x14 and not 0x13. While the required null to keep the chunk an even byte-count exists; it's not being accounted for in the size. For the most part Foobar and most software doesn't seem to care about this; maybe it sees that 0x13 and assumes it's supposed to be 0x14.

Everything is fine until we get down to the ID3 tag. While not part of the specification; DSDIFF spec does allow for invalid/custom/unknown chunks and spec says software should ignore it; and that's actually never been the problem all along.

Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

052F0580        49 44 33 20 00 00 00 00 00 00 09 07          ID3 ........

So, a 4 byte Chunk ID and 8-byte integer for chunk size; except the number of bytes from 52F058E to the end of the file is actually 0x908 bytes in length.

Sometimes the ID3 count is correct, sometimes it's off by a byte. It feels like there's the null pad getting written to the file but sometimes not account for in the chunk? DSDIFF spec says every chunk should be an even byte size.

I first noticed the master form CHUNK size was a byte off; and in fact it seems that it too can be off; but it's only off by a byte when the ID3 chunk size is incorrect.

If I correct offset 0x61, strip out anything after the DSTI, then write a new value for the master form chunk size that's the total byte count minus 12; the files are fine.

Ok, that's it. Again...I'm probably the only one to notice this. Thanks.

[Maxim V.Anisiutkin]

Thank you for pointing that out. Checked that both Philips tools AFC and DST Encoder give 0x14 value due to padding. I'll fix that.

[Maxim V.Anisiutkin]

Philips "Direct Stream Digital Interchange File Format" doc says:

ckData[] is the data stored in the chunk. The format of this data is determined by ckID. If
the data is an odd number of bytes in length, a pad byte must be added at the end. The pad
byte is not included in ckDataSize.

Looks like the contradiction to what Philips really does in AFC and DST Encoder.

[Jay Moore]

The confusing part is the spec also says this:

The ckDataSize is the summation of the sizes of the local chunks plus the size of formType.
This is equal to the total file size in bytes minus the length of ckID and ckDataSize . It is
always an even number because all chunks cover an even number of bytes.

So it's kind of weird it says not to include the pad byte but then says all chunks cover an even number of bytes. I can't say I did a LOT of testing of DST encoded files to check this; but I did compare files output with the 20 year old ProTech DST Encoder 4.0.3 and the compression chunk included the null.

[Maxim V.Anisiutkin]

This is said about chunks (like FRM8 or PROP) that contain the other chunks. And because of that their length should be even. The low-level chunks (like DSTF) have odd or even ckDataSize value.
In the new version I've fixed CMPR chunk to 20 bytes. Let it be like Philips and others.

[Jay Moore]

I thought this format was difficult enough to work with authoring; the deeper I get it's almost like they didn't want us doing this. ;)

Thanks for the fix. Everything looks like it aligns now.