Galago: UniversalParser should be robust to exceptions

Search engine and data mining applications and ClueWeb datasets.

Brought to you by: cammiemw, david_fisher, gregorybrooks, jamiecallan, sm-harding

#113 Galago: UniversalParser should be robust to exceptions

Milestone: Next_Release

Status: open

Owner: John Foley

Labels: None

Priority: 1

Updated: 2014-05-16

Created: 2014-05-16

Creator: John Foley

Private: No

Sometimes, documents are broken. Instead of crashing the build, we should log the broken documents so they can be looked at in the future.

We need to catch all exceptions around any point we interact with a DocumentStreamParser; and log this somewhere. Might be nice to have a "build.errlog" or something that gets output to an index by default. Some Tupleflow thought would be needed there.

Galago: UniversalParser should be robust to exceptions

Search engine and data mining applications and ClueWeb datasets.

Group

Searches

Help

#113 Galago: UniversalParser should be robust to exceptions

Discussion