Re: [Gpsbabel-code] Re: RFC - simple hashing?

[Robert L. <rob...@us...>]

Alex Mottram wrote:

> Would this be a global filtering pass or format specific?  That
> sounds a bit like my old back-end approach.  Kicking off a filter
> before the first ovec is processed..  I'd be interested in what other

That's sort of what I'm thinking, yes.

> functionality would be a candidate for the filtering pass.  Right now,
> I can think of track optimization and duplicate expunging (essentially

Track optimization and global renames are the ones that come up.  So
we'd inherit a table of filters that could be invoked before output.

> I wonder if dragging mkshort() in is feasible, even though it's
> heavily format specific? I can see something in a filtering pass
> possibly setting up for avoiding shortname collisions in mkshort,

I really do think the mkshort stuff has to be done at output, but maybe
we could stack the deck for it.

For example, if you live in an area where you have three caches named
"Geocache", life will be bad; but maybe we special case this in the
"preconditioning" pass to change append a number that mkshort would then
know how to keep.  It could also look for names that end in ", too" or
roman numerals or other mutant approaches.

> The more I think about it, perhaps the solution to true duplicate
> whacking is to take the position, as well as some form of the
> character data and run a 32 bit CRC across it.  

I'll confess that I've solve this in the problem by spinning everything
down to a format that allows single-line output without headers or
footers (I use gpsutil; while it's magellan-oriented, it's probably
"good enough" for most common cases) and then just pass them through
"sort -u", creating a uniq list as we go.

Some of the CSV formats could be used for this, too.

> I'm probably making this much more complicated than it actually is.

Maybe, but I like talking this sort of think through in advance instead
of just blundering through a half-dozen related things that we realized
later needs to be unified.

RJL