Menu
▾
▴
#51 Upload CSV - Migration helper
This patch adds a biblio CSV import which can be used for library migration to openbiblio.
The CSV import file must be a tab separated file (use any parser-converter to convert your CSV file to the usable format). It can include barcode numbers to
ease migration but it does not yet handle duplicates cleanly (currently, it will add a complete entry, not just a copy).
The import page describes the column names understood by the import tool. Most are represented by the MARC identifier, other (openbiblio-special enries) carry
own column names.
Before import, a dry-run feature is available (like in the MARC import) to check for a lot of error conditions (missing mandatory data).
Translation-ready (en part of this patch).
Logged In: NO
While imports are receiving error: "Fatal error: Maximum execution time of 30 seconds exceeded in D:\wamp\www\openbiblio\catalog\upload_csv.php on line 205".
Why get this error?
Please give us and a CSV file format for tests.
Logged In: YES
user_id=2125563
Originator: YES
The error message "Maximum execution time of 30 seconds exceeded" can be circumvented by:
a) increasing the maximum execution time in php.ini and restarting the web server or
b) using a smaller CSV file (try the first 10, 100, 1000 entries for a test).
For the reference library, an import of 16000 entries used 10-15 minutes with a 1GHz CPU, 1GB RAM.
Logged In: YES
user_id=2125563
Originator: YES
I uploaded a patch update,including:
- A one-line sample CSV file
- A update of the table on the welcome page (Call number was missing)
Thanks for testing.
H.-J. Tappe
File Added: upload_csv.patch-0.6.X-1.zip
Logged In: NO
I used this file(http://openbiblioromania.xhost.ro/exampleUpload.zip) but without positive results.Get the same error:"Fatal error: Maximum execution time of 200 seconds exceeded in D:\wamp\www\openbiblio\catalog\upload_csv.php on line 254".
I do not know what I'll try to solve the problem.
It would be nice to add in "upload_csv.patch-0.6.X-1" and a test file.
Logged In: NO
Fatal error: Maximum execution time of 60 seconds exceeded in C:\xampp\htdocs\openbiblio\classes\BiblioCopy.php on line 120
This is the error transmitted by XAMPP.
Logged In: YES
user_id=2125563
Originator: YES
Update 2 of the patch includes an updated dependency handling and the example CSV file (one biblio item). Especially, the situation of bad entries in the collection or material column now are a warning instead of an error and a fallback as given on the main page is applied.
With this file, the import cannot use more than one or two seconds, even on slow systems. On the top left of the import page, there is a counter, indicating how many items have been imported. Most other error cases will show a distinct error message. Please try another browser if this does not help. Another try could be the insertion of (echo "HOOK";) in the php code to find out where it loops or stalls on your system (not reproducible here; Linux environment, Firefox).
About the file mentioned in the comment: You should remove the leading "+", which is a patch "inserted line" indication.
File Added: upload_csv.patch-0.6.X-2.zip
Upload CSV 0.6.X Update #3
Logged In: YES
user_id=2125563
Originator: YES
Update #3 adds a hint text that all MARC fields as known to openbiblio can also be imported by use of a CSV column with the according heading.
File Added: upload_csv.patch-0.6.X-3.zip
Logged In: NO
My default browser is Firefox
Fatal error: Maximum execution time of 60 seconds exceeded in C:\xampp\htdocs\openbiblio\classes\BiblioCopy.php on line 120
Opera
Fatal error: Maximum execution time of 60 seconds exceeded in C:\xampp\htdocs\openbiblio\catalog\upload_csv.php on line 211
Internet Explorer
Fatal error: Maximum execution time of 60 seconds exceeded in C:\xampp\htdocs\openbiblio\catalog\upload_csv.php on line 204
Please send me the two files upload_csv_form.php and upload_csv.php to daviodan at yahoo dot com .
Maybe importation can not be done in WIndows?
Logged In: NO
I found where the problem was.I used the function Replace "+" with "" (space) and delete + from lines:
$recCount++;
for($colCount = 0; $colCount < count($targets); $colCount ++) {
Thank you very much for this patch.
Logged In: NO
It is possible that the CSV file to have the form:
barCo;100$a;245$a;Coll.;mType;showO;Call1;Call2
1;Mihai,Eminescu;Luceafarul;;;;LUC;1
for import direct in Excel
or
"barCo","100$a","245$a","Coll.","mType","showO","Call1","Call2"
1,"Mihai,Eminescu","Luceafarul",,,,"LUC",1
for import direct in Calc ?
Currently after the file is saved in CSV Calc must be removed quotes in an editor.
Logged In: NO
1)It is possible that the CSV file to have the form:
barCo;100$a;245$a;Coll.;mType;showO;Call1;Call2
1;Mihai,Eminescu;Luceafarul;;;;LUC;1
for import direct in Excel
or
"barCo","100$a","245$a","Coll.","mType","showO","Call1","Call2"
1,"Mihai,Eminescu","Luceafarul",,,,"LUC",1
for import direct in Calc ?
Currently after the file is saved in CSV Calc must be removed quotes in an editor.
Logged In: YES
user_id=2125563
Originator: YES
There are several reasons why the "tab with no text indication" format is preferred:
- There is no field which contains a line break. Even a description of an item would need the HTML line break tag instead of a line break character to separate paragraphs.
- the ";" and "," character may well be part of e.g. a title, so they don't always separate fields in the file, while the tab character will not occur in any data.
- Only when using the ";" "," or line break character, the text indication ("") is necessary. To extract data, there is no standard PHP function and a more sophisticated parser is necessary (not impossible but significantly more work) to strip and respect the field boundaries as indicated.
- For most migration purposes, a self-made parser will be necessary to convert another library database into the CSV values (i.e. split the call number).
Last but not least, it is easy to convert from one CSV format into another.
Excel has become much more sophisticated (irony) during the years and does not ask for the format any more (at least, I can't find it), try:
- Save as "Text (tab separated, *.txt)"
OpenOffice:
- Save as "Text CSV (.csv)"; check "Edit Filter"
- select {tab} and remove " from text indication
Logged In: NO
I have made a test.
I have imported data from CSV file then I have a backup database with phpMyAdmin.
Get the same error with two programs: WAMP and XAMPP.
Fatal error: Maximum execution time of 300 seconds exceeded in D:\wamp\apps\phpmyadmin2.11.5\libraries\import\sql.php on line 118
Fatal error: Maximum execution time of 300 seconds exceeded in C:\xampp\phpMyAdmin\libraries\import\sql.php on line 121
I made changes in php.ini but I did not initiate managed to solve the problem.
http://openbiblioromania.xhost.ro/openbiblio.zip
Zip archive contains CSV file and backup.
Why do I get the same error?Smaller backup is imported without problems.
Logged In: NO
Accuracy: error occurs when I want to restore the database
Logged In: YES
user_id=2125563
Originator: YES
For problems involving MySQL and phpmyadmin, please try Google for available solutions. The solutions again are various:
- increase the time limit in php.ini and phpmyadmin config.php (http://wiki.cihar.com/pma/Config#ExecTimeLimit)
- Split the data and use partial import as available in phpmyadmin
- Use any other tool which does not restrict the execution time to import back the SQL.
For openbiblio backup purposes, search the web of a simple, frequent, automated mysql database backup.
Logged In: NO
Thanks for the advice.
Logged In: YES
user_id=1605054
Originator: NO
Hi!
What's this patch for? Is it to import mysql data, or is it to import media saved in *.csv files? I'm a newbie and not sure about this.
Is there a function in OpenBiblio to import media saved in *.csv? At the moment our library is managed by Excel and I'd like to import these data to OpenBiblio.
TIA!
Clemens
it must be the grammar focus so i thought it was an unforgettable story wich means awful and heartwarming americas frontier...it would be great for everyone of us
Could someone kindly provide instructions for non-technically minded person. I don't know how to install this patch. Also if the cvs files does not have the same entries will the import work anyway? Apologies to coders but I need to get this working to keep my job.
This patch adds a biblio CSV *import*, not any kind of export. Exports *to* CSV can be done via reports.
This patch modifies several files. There is no less-technical way of including the patch than understanding the patch file format or "simply" using a patch program (ask wikipedia or hire an IT consultant).
Hi,
Can you please show me how to install this patch, i couldn't find any HowTo file inside the zip file? This is how i did but it didn't work.
krator@laptop:/var/www/openbiblio$ patch -p1 < upload_csv.patch-0.6.X-3/upload_csv.patch
patching file openbiblio/catalog/upload_csv_form.php
patching file openbiblio/catalog/upload_csv.php
can't find file to patch at input line 638
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|Index: quilt/openbiblio/navbars/cataloging.php
|===================================================================
|--- quilt.orig/openbiblio/navbars/cataloging.php 2008-06-16 21:06:52.000000000 +0200
|+++ quilt/openbiblio/navbars/cataloging.php 2008-06-16 21:07:02.000000000 +0200
--------------------------
File to patch:
Thank you
Patch Update #3 applied to 0.7.X
For easier installing on version 0.7.X: applied the patch to 0.7.1. The changed and new files are available in upload_csv.files-0.7.X-3.zip
Hi,
I've been getting this warning and I was wondering if anyone had some advice.
Warning: Missing argument 1 for BiblioCopy::validateData(), called in C:\xampp\htdocs\openbiblio\catalog\upload_csv.php on line 275 and defined in C:\xampp\htdocs\openbiblio\classes\BiblioCopy.php on line 43
Thanks!