TiVo File Decoder

This patch is being superceded by #1851705 (tdmake2).

Logged In: NO

Thanks for taking the time to look at this patch and thanks for the
explanations, especially the packed struct info. I found your code well
structured and easy to follow and I didn't want to muck around outside of
tdmake. My patch on the otherhand, well like I said; quick and dirty.
Feel free to use anything from the patch that you like. Personally I find
design much harder than implementation, so once you've settled on the
calling convention, this patch will be of little use.

I don't see a problem setting the minor version back to 5 just to play it
safe. I haven't compared the file structures to see if there is additional
benefits with the new version, but I would think it might allow for
additional attributes in chunk1 and chunk2. The only reason I can think of
not to go to version 0xf would be if someone was running an old version of
TiVoDesktop or other app, that expected an older version of TiVo files and
wouldn't serve them otherwise. I would think that the TiVo units
themselves would handle either format since they *should* be at the latest
SW release, unless they're hacked of course:)

My original chunk struct did not include the data. I was writing out the
chunk, data, and filler separately. Then I found your perl script and it
was much more strait forward. I just never got around to changing my code
to match your script for padding the chunk, although I did use it for the
mpeg offset.

I used the packed struct as a convienence. Thanks for the information, I
hadn't really considered anything except x86 and MIPS. I will definately
have a look at how you handled this issue.

As for the pipes, I didn't really see a need for stdin since there were 3
chunks to handle and stdin is not working on Windows. Stdout might be
useful, if a caller wanted to write out the header and then append an mpeg
file to the end. I guess at this point, tdmake isn't really that useful as
a standalone application. A few more options might make it more
appealing:

tdmake -m MAK [-c] [-f mpegfile] <chunk0> <chunk1> <chunk2> <outputfile or
stdout>
where -f would append an mpeg file to the header to make a complete .TiVo
file and -c would optionally cleanup the input files when completed.

On the API side, my first thought is one call to build the complete header
and one call to encrypt the chunks. It doesn't look like the chunks are
very large, so I could pass all three at once. I could then read in all
three chunks from the files and not worry about the file length before
hand. Getting the complete header at once would also allow for stdout.

Ok, I think I'll have another go at this. I'll avoid packed structs and
cleanup up the padding routine. I'll add some additioanl options to
tdmake, and see if I can come up with any ideas on the api side.

I have a couple questions/comments for you.

First off, after reading your patch, I remembered why I never got around
to doing this myself: I never really figured out a good interface for this,
either API or command line. So it is fairly likely that I will hack this
around quite a bit before I settle on something I like. Hope you don't
mind :)

I noticed you changed the minor format version from 5 to 0xF, which
corresponds to the Fall 2007 TiVo update. I'm not sure what impact this
would have on a TiVo that was still running older software, but it may be a
good idea to leave the version at 5 since TiVo is pretty likely to preserve
backwards compatibility for the foreseeable future (making old files
incompatible would anger customers who have collected recordings).

Why did you change the algorithm for calculating the chunk padding? Were
you observing different values in the new file format?

I see you used a packed struct for the header. You may have wondered why
the version in tivo-parse.h is not packed. This is because packed structs
are not portable. I tried this originally, until I discovered that it
caused problems on architectures with
strict alignment rules. On x86, you merely incur a performance penalty
for unaligned access, but on SPARC it generates a hardware trap, which
could be handled to emulate the access in software (at a HUGE performance
penalty), or could be passed on to kill the program (which is the behavior
I observed on Solaris 9 on SPARC). Take a look at tivo-parse.c to see how
I read the header in, and just do it the other way round.

You should avoid seeking as much as possible. It precludes the use of
pipes in the future, and messes with caching. You should probably stat()
(or fstat()) the files upfront to get their sizes so you can avoid seeking
to the end and doing an ftell(). The functions are called _stat and _fstat
on windows. These are not necessarily large file safe, but the chunk files
should never be anywhere near 2GB anyway ;)

You don't necessarily need to do anything about any of this, since I need
to come up with a nice API before I do anything with this anyway I can make
these changes then. This is just so you know why I did some of the odd
things I did elsewhere in the code...