Unexpected end of ZLIB input stream

  • rwking

    rwking - 2012-01-13

    First, thanks for the great work on EclipseFP.

    I just upgraded to version 2.2.1, and now when Eclipse starts up, and the Hoogle database is being rebuilt, after awhile I get the error "'Rebuilding local database' has encountered a problem.  Error loading local databse".  The detailed error is "Unexpected end of ZLIB input stream".  Yet when I go to a command line and run "Hoogle data" it appears that a good database is built in the "..\AppData\Roaming\cabal\hoogle-4.2.8\databases" subdirectory, and hoogle works from the command line.  I'm on Windows 7, Eclipse Indigo Classic Win32, Haskell Platform 2011.4.0.0, and ghc version 7.0.4.  Hoogle was working from within eclipse prior to this upgrade. 

    Any ideas?

  • JP Moresmau

    JP Moresmau - 2012-01-13

    If you go to the console view in Eclipse, and choose the console that displays the scion-browser output, you should see the command that is used to build and load the local database.
    In the preferences, you can turn on the logging of error messages for scion-browser (the "debug mode"), maybe if you check that box and restart Eclipse you'll see a meaningful error message.
    Otherwise, you can copy the JSON command line you see in the console, and run scion-browser yourself outside of EclipseFP. Launch scion-browser, and paste the full JSON command in the console. Maybe then you'll see the real error.
    Scion-Browser has its own database that is unrelated to the Hoogle database (though some operations use Hoogle)
    Hope this helps, let me know …

  • rwking

    rwking - 2012-01-13

    I'm not seeing any JSON in the  console, but the last few steps and error that I do see are:

    Download http://hackage.haskell.org/packages/archive/array/
    GET /packages/archive/array/ HTTP/1.1

    Host: hackage.haskell.org

    Content-Length: 0

    User-Agent: hs-HTTP-4000.0.9

    Creating new connection to hackage.haskell.org
    HTTP/1.1 200 OK

    Date: Fri, 13 Jan 2012 22:07:03 GMT

    Server: Apache/2.2.9 (Debian) mod_python/3.3.1 Python/2.5.2

    Last-Modified: Thu, 18 Nov 2010 12:23:57 GMT

    ETag: "1eba71c-5ac7-49552dd26b140"

    Accept-Ranges: bytes

    Content-Length: 23239

    Vary: Accept-Encoding

    Content-Type: text/plain

    scion-browser.exe: user error (SQLite3 returned ErrorBusy while attempting to perform step.)

  • rwking

    rwking - 2012-01-13

    I did turn on "Show scion-browser responses (debug mode)" and noticed that "Gather information from Hackage" is also checked.  I assume this is correct?

  • rwking

    rwking - 2012-01-13

    So I found the JSON command before it rolled off the console.  Here it is along with the first few lines after it:

    >> {"filepath":"C:\\Users\\king\\Documents\\Eclipse\\workspace\\.metadata\\.plugins\\net.sf.eclipsefp.haskell.browser\\scion-browser-0.2-dbs\\local.db","command":"load-local-db","rebuild":true}
    Download http://hackage.haskell.org/packages/archive/warp/
    GET /packages/archive/warp/ HTTP/1.1

    The last item in the command is "array" which is the last item processed in the console before the failure. 
    The very last item in the console is "", and the first item in the console after the JSON command is "Removing ".

    Note that the next command in the console after "Removing " is "Adding ….".  Is that what is generating the SQLite error?

  • rwking

    rwking - 2012-01-13

    Sorry for the play by play but here's the latest.

    "busy" gave me an idea.  I went into the eclipse metadata directory and found where scion-browser was keeping its db.  Tried to erase it to start fresh and got "in use".  Stopped eclipse.   Same thing.  I got it.  Rebooted, and then was able to delete it.  Re-started Eclipse.  Now I get further.. "Local database loaded".   Then a different error "Error loading hackage database  incorrect header check".  Here's the last few lines of the console, I think I'm done for the night.

    Local database loaded
    >> {"new-db":"_all","command":"set-current-db"}
    >> {"module":"","command":"get-modules"}
    >> {"filepath":"C:\\Users\\king\\Documents\\Eclipse\\workspace\\.metadata\\.plugins\\net.sf.eclipsefp.haskell.browser\\scion-browser-0.2-dbs\\hackage.db","command":"load-hackage-db","rebuild":true}

  • rwking

    rwking - 2012-01-14

    Partial success.

    Ok, repeated restarts of Eclipse intermixed with manual deletes of hackage.db and hackage.db-journal, and checking and unchecking of "Gather information from Hackage" yielded a clean start.  Searches with only Local checked work but I've not yet gotten a successful search for types with both Local and Hackage checked, or just Hackage checked, for types that are not on my machine yet which the command line Hoogle can find on Hackage.  I notice in the Eclipse workspace I now have local.db, hackage.db, and hackage.db-journal, after several restarts of Eclipse.

  • JP Moresmau

    JP Moresmau - 2012-01-14

    There is a bug in scion-browser on the Hackage db rebuilding. Version 0.2.2 should fix it. Close eclipseFP to make sure scion-browser is not running, run cabal update && cabal install scion-browser. Delete hackage.db and hackage.db-journal and restart EclipseFP. It takes a little while (on my machine it generates a 54MB db file).

  • rwking

    rwking - 2012-01-15

    I performed your steps regarding version 0.2.2.  Now I too have a 54M hackage.db in the eclipse scion-browser-0.2-dbs directory with no journal file.  Local lookups seem to work, Hackage no.  Example console output:

    >> {"query":"oneOf","command":"hoogle-query"}
    scion-browser.exe: user error (SQLite3 returned ErrorError while attempting to perform prepare "SELECT DbDecl.id, DbDecl.declType, DbDecl.name, DbDecl.doc, DbDecl.kind, DbDecl.signature, DbDecl.equals, DbDecl.moduleId, DbPackage.name, DbPackage.version FROM DbDecl, DbModule, DbPackage WHERE DbDecl.moduleId = DbModule.id AND DbModule.packageId = DbPackage.id AND DbDecl.name = 'timeZoneOffsetString'' AND DbModule.name = 'Data.Time.LocalTime'": near "Data": syntax error)

  • JP Moresmau

    JP Moresmau - 2012-01-15

    I'm really sorry, I didn't test that feature myself and it seems there are issues with it. I'll have a look and will ask the developer that implemented it to look at it too and hopefully we can release a new version soon if there's a bug…

  • rwking

    rwking - 2012-01-15

    No need to apologize, I appreciate the function you have provided.  I only spent as much time on this problem as I did to provide you information to help you improve it.  To be honest, I don't use the embedded Hoogle feature that often, and just turned off the Hackage checkbox for now.

  • Alejandro

    Alejandro - 2012-01-15

    I finally found the source of the error: I forgot to escape some parts of SQL and the double quote in 'timeZoneOffsetString'' was creating errors. I've just uploaded a new version of scion-browser to Hackage, so just running "cabal update && cabal install scion-browser" should fix your problems.
    Thanks for reporting this.

  • Anonymous - 2012-01-15

    I forgot to escape some parts of SQL and the double quote in 'timeZoneOffsetString'' was creating errors.

    Haskell's strong typing would help manage SQL escaping :-/

    Something like (quick and dirty attempt):

    newtype SQL = SQL String 
    getString (SQL s) = s
    escapeSql :: String -> SQL
    escapeSql [] = SQL ""
    escapeSql ('\'':cs) = SQL ('\'':'\'':(getString $ escapeSql cs))
    escapeSql (c:cs) = SQL (c:(getString $ escapeSql cs))
    queryDb :: SQL -> ([PersistValue] -> a) -> SqlPersist IO [a]
    queryDb sql action = withStmt (T.pack $ getString sql) [] (withPopper action)

    so that code like this would not type check without calls to escapeSql:

    queryDb "Select blah WHERE name LIKE '" ++ modName - …

    would require an explicit conversion of modName from String to EscapedSQL to satisfy the compiler.

  • Alejandro

    Alejandro - 2012-01-16

    The problem with that approach it that it will escape everything, not only the things betweeen quotes.
    Usually, Persistent helps with that, creating really good wrappers. But the support for joins is not that good, so I just made a quick-and-dirty hack to make it as fast as possible.


Log in to post a comment.