Menu

Large databases

2013-02-25
2013-05-25
  • Simon Levermann

    Simon Levermann - 2013-02-25

    I'm currently migrating our ancient biborb installation to wikindx.

    The xml file biborb uses contains over 5000 publications and about 8000 authors. Since this is a rather large database, I am wondering wether wikindx is suitable for this kind of data. Main concern is that database queries will take long to execute, especially when performing searches.

    Anyone else run a similarly big (or even bigger) database and can report any issues?

     
  • Stephan Matthiesen

    I have 3400 publications with 10290 authors now, and don't have any problems with searches or such.

    It will of course depend on your server, and the number of users whoa re accessing it at the same time. If you have a lot of users, it might be an idea to limit the number of resources that are shown in a list view (somewhere in the admin settings). I have it set to infinite and don't have issues (but I'm basically a single user).

    Hope this helps
    Stephan

     
  • Simon Levermann

    Simon Levermann - 2013-02-25

    That sounds good.
    Probably won't be more than 10 or 20 users using this, so I guess it should be alright. And if it is too slow, I guess I can always pump up the resources on the VM it is running on :)

     
  • Mark Grimshaw

    Mark Grimshaw - 2013-02-25

    Hi Lee,

    I have a test database that has:
    22,494 records
    12,232 creators
    4,769 publishers

    Mac OSX 2.3GHz i7, 4GB RAM, Firefox

    A quick search on title produces a result in 1.6 seconds
    Search on creator name: 1.2 seconds
    The front page displaying the latest 10 records: 0.6 seconds
    The latest 1000 records in 4.5 seconds

    New resource, add an author (therefore display of +12,000 authors in select box: c.0.5 seconds.

    Probably most of the display 'delays' are due to browser rendering.

    v3.8.2 of wikindx was slow with large databases but v4 requires a more uptodate MySQL server which allows me to use fancy things such as unions to speed things up.

    Regards and enjoy yourself with WIKINDX,

    Mark

     

    Last edit: Mark Grimshaw 2013-02-25
  • Notis Toufexis

    Notis Toufexis - 2013-05-25

    I have just put online a WIKINDX with 111334 records here: http://www.biblio-uellner.uni-hamburg.de/biblio

    The database has 11734 creators, and 51185 keywords.

    A quick search on creator name produces a result in 7.2 secs
    Displaying the category tree: 8.68 secs

    Thank you Mark for this absolutely amazing tool!

    Notis

     
  • Mark Grimshaw

    Mark Grimshaw - 2013-05-25

    Very nice Notis and an interesting customization.

    It must be possible to get it faster. Something for me to look at in the future.

    But it's good to see wikindx robustly handling such large data sets.

    Regards,

    Mark

     

Log in to post a comment.