#14 Would dfu-util support reading a metadata table?

[Richard Hughes]

Hi,

I'm in need of storing some metadata in the DFU footer. I was going to store it in a small binary metadata table above the existing DFU header and increasing the header size appropriately. This ensures my metadata never gets sent to the device using all the existing tools, and my tools can extract the metadata as required. The metadata I want to add is the file copyright, licence and the cipher type used.

I'm happy to draft up an informal specification if this sounds interesting, although I also understand if dfu-util just wants to ignore the data as well. I would have liked to take the proposal to the USB consortium, but given the footer length is only one byte (!!! so frustrating !!!) I don't think it's worth all the extra emails to the committee just to store up to 160 bytes in the section. If we had two or more bytes for future expandability I'd be talking to all the other interested parties too.

Of course, the other option is that we extend the DfuSe prefix specification, but I don't know how well dfu-util copes with flashing a single-element, single-image dfuse file to a DFU 1.1 device. I also don't know how interested ST would be in the additional metadata fields.

You could argue perhaps it's better just to put the DFU file into a container like .cab with an additional XML metadata file (like we do for UEFI firmware updates) although then it does mean we should just ship a raw .bin file instead). Ideas welcome.

[Richard Hughes]

This is what I came up with; comments very welcome: https://github.com/hughsie/fwupd/blob/master/docs/dfu-metadata-store.md

[Tormod Volden]

Hi Richard,
This is a reasonable request, and it would be nice to have some kind of standard established. I have looked at your RFC and it seems to make sense. However you don't explain explicitly how/where the values are located. Are the "key(n)" and "value(n)" numbers offsets from the end of the file? So the file will be: Payload, metadata store (any size), metadata table (up to 160 bytes), DFU footer (16 bytes)? Why all keys, then all values, instead of key-value, key-value,...? Maybe one could put the lengths into the metadata store (as first word), so that there is space for more keys in the table?

On the other hand, yes, I would argue that this could stay in a container along with the "real" DFU file :) The existing fields in the DFU footer are potentially interesting for the dfu-util or equivalent tool, like VID and PID values as simple guards for pushing the payload into the right device. The metadata you speak of can be ignored by dfu-util (as long as it plays the role of a simple hardware-communicating vehicle) and the device, so something outside dfu-util could pick out the metadata and use it to its heart's content and then just hand the dfu file to dfu-util. Granted, this would still benefit from some standardisation of the container and its metadata, but we wouldn't have to involve dfu-util in it. Of course, if there is a need for standardised treatment of this metadata, like displaying copyrights or checking crypto and signing information that would be something that /could/ be integrated into dfu-util, but we could also make another utility that works together with dfu-util.

BTW, regarding dfuse files, dfu-util will refuse to download dfuse (DFU "1.1a") to DFU 1.1 devices. It might not be easy to get ST to extend their dfuse format, and the world might do not want yet another vendor-specific kludge. But it would be relatively easy to stuff metadata into a new image segment of a dfuse file. Each image segment has a "target" header that includes the alternate setting and a "target name" which would be a good candidate for flagging a metadata segment.

Best regards,
Tormod

[Anonymous]

Hi Tormod,

I've added an example file here: https://github.com/hughsie/fwupd/blob/2aa112465d5f5515bac506996d0690fc20144fa1/docs/dfu-metadata-store.md#example -- I hope that makes things clearer.

I think that for such a limited amount of data it makes sense to put the metadata table in the existing 255 space for a footer, and then maybe for more advanced usage steer people towards the metainfo specification instead. e.g. See http://www.fwupd.org/vendors.html for what I've been asking vendors to provide for the LVFS.

If you think it's worthwhile, I can talk to a ST representative and discuss (an additional) metadata table in the DfuSe prefix; I think that may well be fruitless like you suggest.

Thanks,

Richard.

[Tormod Volden]

If you would be willing to go the DfuSe way, you wouldn't need to add anything in the DfuSe prefix, just put your metadata in any format you like into a separate image segment of the .dfu file, with an alternative interface number that does not match any on the device. dfu-util currently only loads the image corresponding to the -a argument. If in the future we will automatically load all images in the .dfu file, it will of course not load one without matching alternate interface number in the device. What we could standardize on, is maybe a number for this use, and which dfu-util would silently ignore, instead of complaing about missing match in the device.

If you do it as described in your document, there is no special support needed from dfu-util if I understand correctly, just that it should respect the variable footer length when it strips it off.

[Anonymous]

Hi Tormod,

I did look into using the DfuSe extension, but I thought it was a bit of a hack to arbitarily choose an alt setting that hardware could potentially use in the future. If specifed, this would need to come from ST rather than us. The way I've implemented this in fwupd at the moment means that if indeed dfu-util resepcts the length of the header then the metadata table gets ignored, the problem is only apparent when dfu-util wants to either display things present in this table or modify the contents. This might or might-not be something you wanted to do for future releases.

Richard.

[Tormod Volden]

OK. If this becomes a standard, or at least used by a few devices, I will happily take patches for manipulating these in the dfu-suffix program, or even printing out things in dfu-util.

[Anonymous]

Please see https://sourceforge.net/p/dfu-util/dfu-util/merge-requests/16/ which provides a solution to this open ticket.