Re: [PyIndexer] Notes on Docs and Testing

[Chris W. <ch...@ni...>]

Casey Duncan wrote:
> 
> We probably don't need a specific index type for dates and times (because
> they can be distilled down to floats), unless there is some date-specific
> search functionality that is needed (none comes to mind).

Hmmm... maybe not at first, but indexing Dates and Times in this way feels very unnatural to me.
In addition, DateTime combined objects also feel pretty unnatural :-(

What do other people think?

> It would be beneficial for the index to have a lower level identifer (a 64
> bit int probably) that is exposed in the indexer interface. That way a
> reverse mapping to the string identifier would not be necessary for
> applications that don't need it (such as the ZODB, which has (at least
> currently) an 8-byte object id).

Yeah, this is a toughy. Dunno how to nicely expose it in the interface.

Ideas?

Personally, I think a string covers it all, just turn your number into a string if you need to use
it as an identifier.

Of course, I'd like to chaneg my mind later ;-)

> Whether the string ids are supported should probably be decided at the time
> the index is instantiated. We could acheive that by having two index classes.
> A basic class that only supports integer ids and a subclass that supports
> strings.

Well, what the Indexing Engine does internally is entirely up to it ;-)

> On the testing front, I found in my travels some big old piles of text data
> for use in testing IR software. perhaps a sample database such as one of
> these can be used for scalability testing.
> 
> see: http://192.115.216.71/webir/resources.html
> under "Free for all text/web files collection"

Coooool... I wonder if they mind having them hit every time you run the scalability test?

Chris