#275 Calling genxref Multiple Times Results in Broken Directory Structure

[Paramjot Oberoi]

Calling genxref multiple times ("./genxref --url=http://127.0.0.1/lxr --allversions --tree=neovim") results in a broken directory structure. It appears that the Glimpse directory is being added within the old directory as a subdirectory. This clearly is not correct behavior.

glimpsedb/lxr/neovim/
└── 1
├── 1
│   ├── 1
│   │   ├── 1
│   │   │   ├── .glimpse_filenames
│   │   │   ├── .glimpse_filenames_index
│   │   │   ├── .glimpse_filetimes
│   │   │   ├── .glimpse_index
│   │   │   ├── .glimpse_messages
│   │   │   ├── .glimpse_partitions
│   │   │   ├── .glimpse_statistics
│   │   │   └── .glimpse_turbo
│   │   ├── .glimpse_filenames
│   │   ├── .glimpse_filenames_index
│   │   ├── .glimpse_filetimes
│   │   ├── .glimpse_index
│   │   ├── .glimpse_messages
│   │   ├── .glimpse_partitions
│   │   ├── .glimpse_statistics
│   │   └── .glimpse_turbo
│   ├── .glimpse_filenames
│   ├── .glimpse_filenames_index
│   ├── .glimpse_filetimes
│   ├── .glimpse_index
│   ├── .glimpse_messages
│   ├── .glimpse_partitions
│   ├── .glimpse_statistics
│   └── .glimpse_turbo
├── .glimpse_filenames
├── .glimpse_filenames_index
├── .glimpse_filetimes
├── .glimpse_index
├── .glimpse_messages
├── .glimpse_partitions
├── .glimpse_statistics
└── .glimpse_turbo

I called genxref four times and you can see that genxref created a nested directory structure.

[Andre-Littoz]

I frequently call genxref several times in a row and don't get this erroneous behaviour. The gimpse directories are created only if they don't exist. Consequently, something in your configuration causes genxref to think they are not present.

Please attach your lxr.conf so that I can study it.

[Paramjot Oberoi]

Attached my lxr.conf.

[Andre-Littoz]

There's nothing weird in your lxr.conf, it is fully "standard".

Since I don't understand this strange behaviour, I looked at my own glimpse directory and found the same bug (which I never noticed because the glimpse directory are usually hidden). I investigate and report back here.

Thanks for notifying this bug.

[Andre-Littoz]

I found the culprit.

When I modified genxref to allow uninterrupted service on huge trees (i.e. users can still access the LXR service while reindexing the tree, which lasted in this cases up to 5 hours), I enabled glimpseindex incremental mode. Since it is incremental, I make a copy of the glimpse databases and indexing occurs on this copy, not disturbing live LXR service. When this is done, the new databases are substituted for the older ones.

This is where I made a mistake: I used a "standard" cp command while it requires an mv for a directory one level deeper.

I'm preparing a new release and I'll notify here when it is ready.

[Andre-Littoz]

Just released new 2.3.3 version to fix this bug (and another one with event Apache MPM module).

Download it and replace your existing LXR root directory. Backup first lxr.conf and reinstall it. This will save you the configuration step.

Run genxref with --reindexall option to get rid of all spurious subdirectories.

For the record: this bug plagues releases 2.2.0 to 2.3.2.