Re: [Archive-access-discuss] indexing best practices Wayback 1.x.x Indexers

[Colin R. <cs...@st...>]

On 06/04/2013 08:27 PM, Jones, Gina wrote:
>
> -Wayback 1.6.0 can handle both indexes, so it doesn't matter if you have your content indexed with either of the two. However, if you plan to combine the indexes into one big index, they need to match.
>
> -The specific problem we had was with sections of an ongoing crawl. 2009 content was indexed with 1.4.X, but 2009+2010 content was indexed with 1.6.X, so if we merge and sort, we would get the 2009 entries twice, because they do not match exactly (different number of fields).
>
> -The field configurations for the two versions (as we have them are)
>
> 1.4.2: CDX N b h m s k r V g
> 1.6.1: CDX N b a m s k r M V g
>
> For definitions of the fields here is an old reference: http://archive.org/web/researcher/cdx_legend.php
>
Thank you, Gina, that is extremely interesting!

Colin Rosenthal
Netarkivet