Opensearchserver database crawl

Help
2014-06-10
2014-06-12
  • Shingionline

    Shingionline - 2014-06-10

    Hello everyone,

    I have a MySQL table (tbl_test) with 4 columns - id, url, title & description. There are 39409 rows in the table. I would like to index this table into Opensearchserver v1.5.3, I have created a new index with the 'web crawler' template and my database crawler settings are as follows:-

    General settings
    

    Name: test_crawl

    Driver class: com.mysql.jdbc.Driver

    JDBC url: jdbc:mysql://localhost/test_db

    Isolation level: TRANSACTION_READ_COMMITTED

    User: root

    Password: ••••••

    Language: English

    Buffer size: 100

    Sleep (ms): 0

    Primary key: id

    Unique key (for deletion): id

    SQL Select: SELECT * FROM tbl_test

    SQL Update: (blank)

    FieldMap settings
    
    SQL Column    Index Field
    title         title
    description   metaDescription
    url           url
    

    When I save settings and execute the crawl process i get the following:-

    Name        Execution time    Duration    Status     Info
    test_crawl  8:47:44 PM        0:00:00     Complete   0 (0) / 39409 (0)
    

    No documents are added to the index. Please will you let me know where I'm going wrong.

    Thank you.

     
  • Alexandre Toyer

    Alexandre Toyer - 2014-06-10

    Hello,

    Could you please select TRANSACTION_SERIALIZABLE for field "Isolation level", and leave empty field "Unique key (for deletion)" and then try again indexation?

    Regards,
    Alexandre

     
  • Alexandre Toyer

    Alexandre Toyer - 2014-06-10

    Hi,

    You also have to add a field mapping between the primary key of your table with a field of your schema.

    Can you tell me if those changes work?

    Regards,
    Alexandre

     
  • Shingionline

    Shingionline - 2014-06-12

    Thank you very much Alexandre, the new settings are working perfectly!

     

Log in to post a comment.