Menu

#1 Skipped records and incorrect schema

Database_MySQL
open-accepted
1
2005-09-23
2005-09-17
Anonymous
No

Following your instructions, I noticed issues with
cases of some things, so I changed all table names to
all lowercase.

I also noticed that on MySQL 4.0 and 4.1, you can't
have a varchar(512), as the UL is 255. I changed that
to text.

Once I had the tables created, I proceeded to load up
the txt files:

mysql> load data local infile 'FOOTNOTE.txt' into
table footnotetbl FIELDS TERMINATED BY '^' ENCLOSED BY
'~' LINES TERMINATED BY '\r\n' ;
Query OK, 207 rows affected (0.02 sec)
Records: 212 Deleted: 0 Skipped: 5 Warnings: 0

mysql> load data local infile 'DATA_SRC.txt' into table
datasrctbl FIELDS TERMINATED BY '^' ENCLOSED BY '~'
LINES TERMINATED BY '\r\n' ;
Query OK, 1 row affected (0.02 sec)
Records: 366 Deleted: 0 Skipped: 365 Warnings: 366

I had warnings like crazy on the other txt files, but
these ones here have skipped records, which does
concern me.

Can you assist?

Discussion

  • Conrad Fredericks

    Logged In: YES
    user_id=929472

    Hi,

    Sorry for the late response.
    Thanks for the feedback.

    Yeah, there are heaps of warnings when uploading the data.
    This is because the datafiles provided by the USDA often skip
    fields that have no data in them.

    It took me a while to figure it out, but you can safely ignore all
    the warnings. I will post specific info on why these warning
    are generated soon. If you want I can email you directly.

    Further feedback is most welcome. You can contact me
    directly at conrad4peace@yahoo.com

    Also, I'm currently re-testing all my database schema scripts
    for MySQL (I develop with PostgreSQL).

    Thanks.
    Conrad

     
  • Conrad Fredericks

    • labels: --> Install Problem (example)
    • milestone: --> 527364
    • assigned_to: nobody --> conrad4peace
     
  • Conrad Fredericks

    • milestone: 527364 --> Database_MySQL
    • priority: 5 --> 1
    • status: open --> open-accepted
     
  • Conrad Fredericks

    Logged In: YES
    user_id=929472

    I've added some help regarding the warnings when importing
    the USDA data files. See File Releases -> Database.

     

Log in to post a comment.

Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.