Vicaya 4b13b release and future

Developers
Alex
2004-11-14
2013-04-24
  • Alex

    Alex - 2004-11-14

    MO,

    That was certainly a thourough test response. Thank you!

    Lynx

    I don't suspect Vicaya.jar would run on lynx
    because even if lynx could run a java applet
    (I doubt it), it would not be able to display
    the graphics. However, if lynx can run
    javascript, I will consider supporting
    lynx in some way. It's certainly not a
    priority at the moment.

    Navigation Bug

    I am aware that the navigation buttons do not
    work. I simply have not attached any code to them.

    Query Persistance

    The browser 'back' failure that you describe
    (query hits lost when the user returns from
    a search) points out a flaw in my design.
    Thank you for raising this issue as it sheds
    light on a better implementation.

    However, you mentioned a history of queries, which I think might
    be difficult considering we are dealing with read-only media,
    and trying to avoid temp files. To handle search
    query history would be to write to disk (perhaps a cookie),
    or accumulate huge URLs. I think it is fair to remember
    one previous query, but not much more.

    I have drawn up some architecture in which
    an applet can gain initial parameters from
    the underlying html (appletAll.html). Without
    getting too technical: A simple html/get form
    would open a new page (just like google)
    containing the hits. The trick is that
    the hits are displayed in Vicaya (without
    the user controls on the top).

    The downside is that the 'response time' will
    be a bit slow (launch, query, parse, display)
    for every new query, hit page request (next,
    prev, first, last), or browser back history.
    On the plus side, more of the code will
    be html (javascript document.write() ).
    However, knowing we can only display one hit page
    at a time, might afford some query optimizations.

    Vicaya Initial Parameters

    initialQuery =  null         (default)
    hitsPerPage  =  10           (default)
    initialPage  =  1            (default)
    numberPages  =  1            (default)
    summarySize  =  100          (default)
    userControls =  hidden       (default for applet)
                    internal     (default for stand-alone)
                    external
    hitDisplay =    internal    (default)
                    external
                    feelLucky
                    pageAppend
                    pageReplace
                    pageNew
    resultDisplay = pageReplace (default for applet)
                    internal    (default for stand-alone)
                    external
                    pageAppend
                    pageNew

    "internal" refers to a component inside the
    applet (as in version 4b13b). "external" displays
    a java component as a java popup window. "feelLucky"
    jumps over the hitDisplay, immediately
    displaying the first hit result. "pageReplace",
    is the method used (version 4b13b) to display
    results. Note that "pageAppend", "pageReplace",
    and "pageNew" are not supported in hitDisplay nor
    is "pageAppend" supported in the
    resultDisplay because they would require temp  files
    or DOM manipulation (not supported by many browsers
    or platforms). "pageNew" opens
    the result in a new browser window (might be
    perceived as a pop-up), leaving the applet
    untouched. "pageReplace" uses the same browser
    window (and is not support in stand-alone as it
    is not embedded in a browser window)..

    technical implementation

    I imagine a search page would also contain
    Vicaya (hits). When the user initially
    opens the search page, there will be no
    http/get variables in the URL. Therefore,
    the query field will be blank and the
    Vicaya applet will perform a test. The
    form action will return the same page
    (this time with http/get variables).
    This url will be stored in the browser
    history and will represent the hit state.
    A query hit page could therefore be saved
    as a bookmark.

    The javascript may be quite ugly. It already
    determines the platform and browser, and
    will now be responsible for extracting the
    http/get variables and populating the
    appropriate parameters. While not a
    technical issue, it does present a concern
    for a web/cdrom developer who might like
    to mark-up the interface with graphics
    and style. I'll try to separate logic,
    from content, from style, as much as
    possible.

    Vicaya on read-only media

    I have done simple tests of Vicaya using read-only files,
    but have not yet tested Vicaya from a CDROM. Early versions
    of Lucene use lock files in the index to ensure
    that one process doesn't write to the index while another
    is reading. This would only be a problem if the index
    were dynamically updated (such as with a global internet
    web site (such as google) which continually crawls the
    web). Obviously, this will not be the case with a CDROM.

    I suspect this has been handled more gracefully in 1.4
    if not already.

    Project cooperation

    Vicaya struggles to perform searches on multiple platforms,
    in multiple browsers, from local (read-only) media. That
    specification has been the primary focus of the project.
    While better, faster queries, mutlilanguage, Unicode,
    and other features have been of lower priority. There
    are dozens (if not hundreds) of other projects based on
    Lucene that perform web searches and handle more common
    search concerns. I would like to find another (license
    compatible) project to merge or cooperate alongside.

    Concurrent Version Control (CVS)

    I would like to separate the Apache Lucene library
    from the build. Since I am considering using a more
    recent version of Lucene, the time is ripe to
    decouple the code. There might be a license violation
    in keeping them together anyway.

    Because massive changes to the underlying structure
    is possible I am considering creating a module representing
    a major version as sourceforge/CVS has directory structure
    modification limitations. Comments?

    Directories and files in (parenthesis) will not be stored
    in CVS, but will be part of the development or deployment
    structure.

    Module: Vicaya_01:
    |
    |-- (appletAll.html)
    |
    |-- bin
    |  |-- runIndex.sh
    |  |-- runSearch.sh
    |  |-- runMakeCleanIndex.sh
    |  |-- runMakeCleanSrc.sh
    |
    |-- docs
    |  |-- ApacheLicense2.0.txt
    |  |-- lgpl.txt
    |  |-- README.txt
    |  |-- notes
    |  |  |-- ...
    |
    |-- (build)
    |  |-- (classes)
    |  |  |-- ...
    |
    |-- build.xml
    |
    |-- (index)
    |  |-- ...
    |
    |-- (lib)
    |  |-- (lucene-1.4-final.jar)
    |
    |-- samples
    |  |-- ...
    |
    |-- src
    |  |-- html
    |  |  |-- appletAll.html
    |  |
    |  |-- java
    |  |  |-- org
    |  |  |  |-- accesstoinsight
    |  |  |  |  |-- ...
    |
    |-- (Vicaya.jar)

    Considering known bugs:

    Windows 2000 Pro (sp4)
       stand-alone   : OK (mo)
       IExplorer 6.0 : OK (mo)
       Firefox 1.0   : OK (mo)
       Opera 7.23    : applet opens, would not search (mo)

     
    • Hugo Gayosso

      Hugo Gayosso - 2004-11-16

      I'd like to have a persistent bookmark capability, better if it is in-page bookmarking.

      Do you see this very difficult to accomplish?

       
    • Alex

      Alex - 2004-11-18

      I have compiled email responses...

      Vicaya 4b13b (2004-Nov-13 b)

      Platform/Browser Test Results, bugs, and suggestions.

      Definitions:

      Query: A string of text one wishes to search for.
         Example: "water fire"
      Search page: the user interface (HTML page) that
         lets a user type in a query.
      Hits: The results of a query. A list of possible
         pages. Likely to contain a summary of ten results.
      Result: When a user clicks on a hit, they will
         be presented with an HTML page. An original
         document.
        

      Known Bugs:

      1) The navigation buttons do not work
      2) Hits to Result page and Browser Back
         Button results in blank hits and query
      3) Stand-alone opens hits in new browser windows.

      Notes:

      CDROM some OS's *may* have trouble with
         8 nested directories.
      CVS Modules need not have version number.
         "Vicaya" for the source code, and
         "website" for a website.

      Test Results:

      --- Windows 2000 Pro Sp4:
      IExplorer 6.0 :  OK
      Firefox 1.0 :    OK
      Opera 7.23 :     Opens, but unusable
      Stand-alone:     OK

      --- Macintosh OS X:
      IExplorer 5.2.3: Opens, but unusable (tiny hits page)
      Safari 1.2.4:    OK
      Firefox 1.0:     tiny hits page, works otherwise.
      Camino 0.8b:     Error

      --- Linux:
      No test results.

      Alex' Response:

      Thanks for all the feedback and patience. I seem only to be able to
      work on this one day each fortnight.

      I ripped previous code using one graphics toolkit to another
      mostly-incompatible toolkit (AWT - Swing). I will rewrite the
      UI framework from scratch in the next version. This should
      clear up the sizing issues with Firefox.

      I appreciate testing on Mac/IE, but unless it magically works,
      I consider it an "unsupported" browser. Same for Camino until
      1.0. Camino uses the Gecko (Mozilla) engine. I consider the
      bugs in the way Camino handles Java too great to support at this
      time. Safari is based on the very stable Konquer engine. Thus
      Vicaya will aim to support both of these products. I know very
      little about Opera and other browsers.

      I have described a detailed technical solution to the browser
      history 'back' button complaints (herein called "browser history
      problem"). With 4b13b, the applet is run with no knowledge of
      previous states. I can solve this by placing the search query
      in HTML which calls another page containing the applet, sending the
      query (and other variables) via GET URL and Java Applet Params.

      The query history in a pulldown is a bit more complex. I may be
      able to support this to a very limited degree (one or two previous
      generations). However, hitting the browser 'back' button brings
      the user back to many many previous states (although this is
      unlikely to be intuitive to most users).

      The stand-alone application opens all results in new browser
      windows. "Fixing" this might be out of scope, as I ask the operating
      system to open the file for the application. We don't have alot of
      control there. However, I can add a feature to open the result in
      the same Java window that displays the hits.

      I simply have not programmed the navigation buttons
      ( |< << >> >| ).

      I agree with the CVS file structure suggestion. Any Java
      restructuring can occur in the module vicaya under:
      /src/javaX (where X is a major version number). I have removed
      Apache from the src tree and link to (a more recent) library jar
      in /lib/. Expect to see this in the next release.

      Simply zipping the source tree in distributions will eliminate
      the problem of burning large nested directories.

      I think text map conversions is out of scope. I would like to
      support Unicode, but even that is on the back-burner.

      Thanks again,
      Alex

      =============================================================
      Reference Scrap Quotes Below:
      =============================================================

      MO:

      Have images: Brahmi and stylized VICAYA images

      Bugs: Navigation buttons (<<,>>) do not work on any platform.

      Windows 2000 Pro Sp4

      IE 6.0:
      --> Clicking the back button from a link returns one to the search page
      with the search term edit box blank. I suggest two things:
      1. It return with the previous search term (even Google doesn't do this and
      it should) and
      2. the previous x searches should be available in a drop-down box.
      (Eventually it would be good to have a 'save search' option.

      Firefox 1.0:
      I would say the results were identical to those for I.E. 6.0

      Opera 7.23 (I believe there are later versions of Opera)
      The 'appletALL.html' file opened in Opera, but would not search; buttons
      did not function.

      Stand-alone:
      Clicking on [Vicaya.jar] brought up the search box and searching was possible
      but otherwise the results were the same as above. Yes we have no buttons.

      +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
      --> Clicking the back button from a link returns one to the search page
      with the search term edit box blank. I suggest two things:
      1. It return with the previous search term (even Google doesn't do this and
      it should) and
      2. the previous x searches should be available in a drop-down box.
      (Eventually it would be good to have a 'save search' option.

      Burned to CDROM:
      I just did a test run burning Vicaya4b13bobcanobe 2 CD. It worked just as as
      it did on the hard disk. Running Java.jar had this quirk...when clicking on
      a result it opens the page in a separate window with no 'back' button active
      (with the 'back' button inactive). [...] I got this message:
      Same problems otherwise...no retention of
      search term, no expanded list of search results. "The directory
      "Vicaya4b13b\src\java\org\accesstoinsight\vicaya\index\bak\FileDocument.java
      " exceeds the maximum depth of directories (8) and may cause problems
      reading the CD on some OSs."

      FLUID Dynamics, 8-bit Text Conversion:
      I also read with interest the doc on building one's own index. I will try
      this myself later, but I would like to insert yet another factor to be
      considered in this project: the Fluid Dynamics Search Engine
      (http://www.xav.com) has the cabability of searching a customized font (not
      talking about UNICODE here). This is a table that instructs the search tool
      to make certain substitutions when they are encountered which allows one to
      use a custom font by way of some convention, such as the PTS Pali character
      convention. If not UNICODE this much is essential for searching BuddhaDust
      and every other site that uses one of the 'bastard' fonts such as Norman,
      etc.

      JB:

      Macintosh OS X

      Macintosh test results (with pictures!) for four browsers at 
      http://www.notdotorgnot.com/vicaya/index.html#2004.11.15 . Bottom line: 
      It works pretty well with Safari. With IE and Firefox, the search 
      results window is the size of a postage stamp. On IE the search button 
      is inoperative. In all browsers, the other buttons seem to serve the 
      same function as the "Search" button (are they still under 
      development?). I hope we can eventually make it all work with Firefox, 
      since that seems to be an increasingly popular browser (and currently 
      my favorite, despite its bugginess).

      Thanks for addressing some of my pet peeves. Now here's another one for 
      you: The links in the search results panel don't behave quite as I'd 
      expect. Specifically:

      1. When I click on a link, the page opens in the same browser window, 
      just as I'd expect. But when I hit my browser's "back" button and I 
      return to the search results page, all the previous search results are 
      gone.
      2. I am not able to open the link in a new browser window by 
      right-clicking on the link (the browser's usual pop-up menu does not 
      appear).
      3. I am unable to drag a link over to another browser window.

      Anyone else observe this behavior, or is it just me?

      Hugo:

      * File structure
      > I foresee rearranging the structure extensively every once in a blue moon
      (such as if/when we merge with a more professional search/index tool). I will
      simply create a new module. For now I foresee two modules:
      >Vicaya_00 website

      Why don't call it just 'website', it would show up as

      >If a massive restructuring must take place, we'll just create a new module:
      >Vicaya_01

      What about using just 'vicaya', if there needs to be a new one, then
      we start with the numbering.

      The structure that you stated in your post at the forum is fine with me.

      * Forums
      I made the 'Developers' forum public, do you want to keep it private?

      * Wiki
      Create yourself an account so your contributions get under your name.
      There is a link on the upper right hand of the main page to create it.

       

Log in to post a comment.