[Pytables-users] advice on using PyTables

[<ben...@lf...>]

Hi,

I'm working with Air Traffic Management and would like to perform checks / compute statistics on ASTERIX data.
ASTERIX is an ATM Surveillance Data Binary Messaging Format (http://www.eurocontrol.int/asterix/public/standard_page/overview.html)

The data consist of a concatenation of consecutive data blocks.
Each data block consists of data category + length + records.
Each record is of variable length and consists of several data items (that are well defined for each category).
Some data items might be present or not depending on a field specification (bitfield).

I started to write a parser using hachoir (https://bitbucket.org/haypo/hachoir/overview) a pure python library.
But the parsing was really too slow and taking a lot of memory.
That's not really useable.

>From what I read, PyTables could really help to manipulate and analyze the data.
So I've been thinking about writing a tool (probably in C) to convert my ASTERIX format to HDF5.

Before I start, I'd like confirmation that this seems like a suitable application for PyTables.
Is there another approach than writing a conversion tool to HDF5?

Thanks in advance

Benjamin