Pre-Processing Error

Anonymous
2013-02-08
2014-10-13
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2013-02-08

    Hello,

    In attempting to pre-process a text file, I receive the error, "Something wrong with the database: bun_r table KH Coder will exit now. How can I fix this error?

    Thanks for your time!

    Kent

     
    Last edit: Anonymous 2013-02-08
  • HIGUCHI Koichi

    HIGUCHI Koichi - 2013-02-09

    Thank you for the post!

    Did you try "Botchan" tutorial? If you can follow the tutorial without any errors, it seems that your KH Coder installation is working properly.

    In that case, the data you tried could contain some incompatibilities. Please prepare a plain text file (.txt) as a "target file" for KH Coder. Not Word (.doc .docx) or PDF (.pdf). The encoding of the text file should be something like "Latin-1," "ISO 8859-1," "US-ASCII," or "Plain ASCII." And be sure that the text file doesn't include any tab characters or any other control characters other than "line feed." And you should delete "<" and ">" from your data unless you are using tags for KH Coder.

    Hope it helps.
    Best regards.

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2013-02-09

    Thanks for your help.

    I have had success with the Botchan file and smaller portions of my file that is creating the errors. I have attempted to create a clean ASCII text file but I must have missed some errant characters.

     
  • HIGUCHI Koichi

    HIGUCHI Koichi - 2013-02-10

    Thank you for the post!

    You can try "CLEAN" function of EXCEL / CALC to remove all control characters. Copy and paste the data to Excel / Calc, apply "CLEAN" function, then copy the data back to a text file.
    http://www.techrepublic.com/blog/msoffice/clean-up-your-data-with-this-easy-to-use-excel-function/896

    About "<" and ">," use search or replace function of your favorite text editor to delete them all.

    Hope it helps.
    Best regards.

     
    Last edit: HIGUCHI Koichi 2013-02-10
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2013-05-07

    I am having the same issue as Kent, while I have had success with the Botchan file and smaller portions of my file, other parts are creating errors. I have removed every < and > besides the ones in \<h1> or \</h1>, using notepad++ I asked it to remove everything defined by the regex [\x00-\x09\x0B-\x1F\x7F] (which wikipedia tells me are the control characters expect for (in hex) 0A which is new line, and reading about the clean function is what it removes). Notepad++ tells me it is encoded as UTF-8 without BOM, which is the same as a smaller portion that works.

    Is there some other character that that regex is not picking up that might be causing problems?

    ETA: I also tried using the clean function, it is now ANSI, and then removing all < and > and still had this error.

     
    Last edit: Anonymous 2013-05-07
  • HIGUCHI Koichi

    HIGUCHI Koichi - 2013-05-07

    Thank you for the post!

    1) Would you give us the EXACT error message you got? Dose it say "bun_r?" Isn't it bun-bun_r1? or bun-bun_r2?

    2) Can you send me the data by e-mail?

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2013-05-07

    1) The exact message is "Something wrong with the database:bun_r table KH Coder will exit now" .

    2) I'll send an email. Thank you!

     
    Last edit: HIGUCHI Koichi 2013-05-08
  • HIGUCHI Koichi

    HIGUCHI Koichi - 2013-05-08

    Thank you for the post and e-mail.

    I see a lot of "\" (backslash) in your data.
    Please delete them all and it will be OK.

    Good luck with your research.

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2013-06-14

    Koichi San konnichiwa !

    KH coder is one of the best text mining tools I have come across. Thanks.

    I had issues when I tried to run pre processing for Japanese text files. KH coder seems to work only for ANSI Encoded text files. Whereas if a text file is saved in ANSI the kanjis turn into question mark. How can I keep my Japanese text file in Unicode and run the preprocess without getting the following
    Fatal error: Could not execute ChaSen.

     
  • HIGUCHI Koichi

    HIGUCHI Koichi - 2013-06-14

    Hello from Japan and thank you for the post!

    Please try saving Japanese text files in Shift-JIS (SJIS) or EUC-JP (EUC) encoding. Currently, KH Coder can not hundle Unicode.

    Best regards.

     
    Last edit: HIGUCHI Koichi 2013-06-14
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2014-09-26

    Greetings

    I have just started using the coder so, I am not very familiar with it but, I keep getting the same error.

    "Fatal error: could not execute ChaSen"

    A window appears with this message right after I try to run the Pre-Processing. I click on OK and the whole program shuts down.

    Any suggestion on how to solve this?

    Thank you very much for your time, and sorry for the bother.

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2014-09-30

    Greetings, Mr Higuchi

    Thank you very much for your reply. I am attempting to analyze a bilingual text that has both Japanese and English characters. Is it possible to do this kind of procedure with two languages at the same time?

    Thank you very much for your attention, and sorry for the nuisance.

     
  • HIGUCHI Koichi

    HIGUCHI Koichi - 2014-10-01

    No, KH Coder is not ready for a bilingual text. Sorry. If you use ChaSen, all English words will be treated as "unknown" POS words.

    But the error is not normal. Not expected. You should be able to process/analyze the data.

     
    Last edit: HIGUCHI Koichi 2014-10-01
  • David-O. Mercier

    Although it is impossible to use the program to analyse two different languages at once, as briefly discussed in another thread, I have been able to process heavily multilingual data with KH Coder by:
    1) removing all non-ASCII characters;
    2) using the English "left3words" Tagger and simply treating foreign words as "foreign"/"unknown" with regard to part-of-speech, as prof. Higuchi just explained.

    I have found that in that way, most tools in KH Coder will work as desired (for the English parts of the data only!).

    You could use a similar procedure with a Japanese Tagger - minus the deletion of non-ASCII characters, of course - then interpret the complementary results of both analyses at once? I am not familiar with parsing Japanese data, but if you want to ignore English words altogether for that part of the analysis, you could set up an exhaustive list of words to temporarily ignore for that purpose ('Pre-Processing' --> 'Select Words to Analyze').

     
    Last edit: David-O. Mercier 2014-10-14


Anonymous

Cancel  Add attachments





Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

JavaScript is required for this form.





No, thanks