> There's nothing wrong with whole-file/stream compression :-) But from what I
>understand, Paul is looking for something else, which is mucking with the internal
>structure, such as allowing for non-printable binary blobs.
Yes – I’m looking at a little more than just zip-like compression.
No - I’m hoping to not muck with the internal structures too much, but instead adding an alternate encoding. The main idea is that you can have both human readable (YAML text representation) and efficient binary encoding that are isomorphic. This adds a new representation type but it would be directly convertible to a readable YAML text format (assuming you have the encoding rules).
All data types would have both a canonical text and binary encoding defined. The human readable form as currently envisioned would be YAML – or at least a subset that conforms to stricter conventions, but that could be read and parsed by existing YAML libraries.
Validation would be possible using the encoding schema for both the human readable and binary versions.
The application space is small embedded devices, sensors, wireless protocols, etc. Some of the motivation comes from being tired of working in standards that are hard to read that use TLV formats and long prose to describe the protocol. I’m trying to get the most readable and efficient representation of a binary protocol.
The specific protocol I’m working on is for “privilege management” - basically defining who can do what. The expressions for these statements need to be human readable and understandable – YAMl would be ideal. Parameter validity checking is required (e.g. there are a bunch of security issues with dns names if you only treat the dns name as a string). TLV encoding and “efficient” representation of enumerated options is necessary for the embedded applications.
Paul A. Lambert | Marvell | +1 650 787 9141