#184 QueryParser chokes on large queries

[McCann]

Using the following code with data equal to a large string (more than 2048 bytes in a quoted string, or lots of quoted strings separated by a single space):

KeywordAnalyzer analyzer;
queryParser::QueryParser queryParser( L"_All_", &analyzer );
pTermQuery = queryParser.parse( data );

The result is a parsing error:

Cannot parse '<data string here>': Lexical error at line 1, column 11526. Encountered: <EOF> after : "\"ypes d"

I looked at the code and couldn't figure out what was causing the error. The error seems to be directly related to a query size issue. The original exception being thrown is "_CLTHROWA(CL_ERR_IO, "read past eof");" in FastCharStream::refill() [src/core/CLucene/queryParser/FastCharStream.cpp]. This ends up rippling to the error code at the end of QueryParserTokenManager::getNextToken() [src/core/CLucene/queryParser/QueryParserTokenManager.cpp] which throws a lexical error. This is then caught and re-thrown by QueryParser::parse() [src/core/CLucene/queryParser/QueryParser.cpp].