From: Robert Lipe <robertlipe@us...> - 2005-04-01 23:00:40
I was recently approached by a consulting customer that needed to move
bazillions of waypoints through a handful of custom CSV formats. They
were unhappy with performance and wanted faster input processing for
Profiling showed one of the pieces of low-hanging fruit in this case
(and hardly measurable in the general one) to be the myriad of string
compares for each token in the style sheet. I bought about a 20% speed
gain in thier case this by implementing a table of tokens that I could
feed to gperf so we can get a perfect hash that we can just switch on.
Thus the pertinent fragements of csv_util started go look more like:
csv_parse_val(const char *s, waypoint *wpt, const field_map_t *fmp)
wpt->shortname = csv_stringtrim(s, "", 0);
wpt->description = csv_stringtrim(s, "", 0);
Further performance improvements are possible (and may be pursued) but
this was a pretty mechanical thing.
This approach does have the unfortunate requirement of requiring Gperf
to add or change tokens. I've tried to not require external tools for
our builds, but this seemed less painful than either bringing in a code
base to do minimal perfect hashing like Bob Burtle's stuff or (horrors!)
trying to open code our own.
Once we had the infrastructure, there are a handful of other places we
could employ it like the various icon table lookups. That said, it does
raise the entry price for a developer to work on that code (gotta have
gperf) but does buy back some readibility, IMO.
I could go either way on checking this in. How do the rest of you feel
about requiring gperf to modify this stuff? (I'd check in the generated
files so _most_ people wouldn't need it anyway...)
Support GPSBabel by helping to improve it or fund those that that have
done so. Visit: