For KeePass 2.19, I've completely rewritten the generic CSV importer and added a few other features to support importing more CSV files. In detail:
* New generic CSV importer (now supports multi-line fields, '\' as escape character, field & record separators and the text qualifier can be specified, white space characters can be removed from the beginning/end of fields, the fields and their order can be defined, supported fields now are group name & standard fields like e.g. title & custom strings & times & ignore column).
* Added support for reading texts encoded using UTF-32 Big Endian.
* Enhanced text encoding detection (now detects UTF-32 LE/BE and UTF-16 LE/BE by zeros, improved UTF-8 detection, …).
* Unified text encoding handling for internal data viewer and editor, generic CSV importer and text encoding selection dialog.
Funny that you just posted this yesterday, as it solved a problem that I just found today! :) It's fantastic to be able to import custom string fields now.
I wonder how much work would be involved in reading the first row for headings, and if it might be possible to include an option to not import the first row.
Thanks very much!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Great tool. As fpr the UI, I think you should try to put the whole preview stuff to where now "specify the layout" is - this way we could see the results of syntax changes in realtime.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I've now added the following:
* Added an option to ignore the first row.
* KeePass tries to guess the fields and their order based on the first row (this e.g. works well with the KeePass 1.x CSV format).
Unfortunately there is too few space for a preview on the structure tab page.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
-
2012-03-09
This is absolutely awesome… I was racking my brain playing the 2.18 version and reading the documentation on how to expand the fields in the database to match the fields in my csv. Thanks for doing this, with the custom fields
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Very good feature, that I was looking for:,Can this feature be adressed by scripting? The KPScript Tool does not like this version.
My task is is to make such a job as a script:
1) Make Backup of keepass-dB
2) Import Data (overwrite Entry) from a CSV-File, with constant parameters for matching columns (will be a long line, but allways the same)
Thanks in advance!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
What might I be doing wrong? Or do I not understand how this option works? (I wouldn't be shocked if so! :-))
When I open the Import option on a KeePass Windows 10 desktop, and select the file I want to import into KeePass, I see this list: "Field, Title, Username, Password, URL, Notes" and I can delete from the list of fields (except NOT "Field"). Field doesn't seem like it ought to show in that box, but should be a label ON the input box? "Group" shows in the "Add" field name box by default, and I see that I can choose among a variety of field names, and if I delete all the pre-existing field names I am able to match up the field names with the items in my import .csv. So, perhaps this is how it's intended to work.
I had hoped that the "Add" could offer the opportunity to import into a temporary Keepass group (for example, "Import from Chrome browser") but that's not apparently how the importer was designed, am I right? My data does not include a group name, so I guess I cannot generate this as a group on Import.
Thanks for a great password manager tool. I've used KeePass now for several years, and I'm just working to explain how to use it, to a friend who has been relying on paper lists and browser password memory for some time.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Field doesn't seem like it ought to show in that box, but should be a label ON the input box?
You're right, "Field" is only a caption and should have a different appearance than the names of the fields below.
If you want to import all entries into a specific group, all (data) lines of the CSV file must have an extra column with the group name. The safest way to do this, is to load the CSV file into a spreadsheet program (Excel, Libre Office Calc), add a new column, write the group name into its first data row and propagate it down to the last row (auto fill). If the first line contains field names, put the string "Group" into the extra column. Then save. Additionally, you can easily delete columns which have no meaning for KeePass (may not be necessary, see below).
CORRECTION - Sorry, the following paragraph is wrong ... *About the first line of the CSV file: * Very often the first line doesn't contain table data, but a sequence of descriptive field names. If (you make) these names match KeePass field names, KeePass will recognize them and automatically adjust the field name list to the sequence present in the CSV file (including "Group"). Columns with names unknown to KeePass will be skipped (field name "(ignore)"). Recommended: If the CSV file comes without such a descriptive line, add one.
I tested variations on a CSV file re-using the same open instance of KeePass. Although I started separate import actions, KeePass remembered my manual changes to the field list and presented it over and over again. This gave me the WRONG impression, that KeePass automatically adapts to the field names specified on the first line. It does NOT - in fact, it ignores them! A case of wishful thinking! I apologize.
While the file is loaded in a spreadsheet program to add the Group column, superfluous columns should be deleted, which is easier than inserting "(ignore)" fields in KeePass. The sequence of columns can be easily changed (drag-and-drop) to fit the standard sequence of KeePass. If the first line contains field names it can safely be removed; it is not needed.
By default, KeePass creates a new group. Repeated imports create separate groups with the same name. Optionally, an existing group with a matching name can be re-used. Repetitions will create duplicate or look-alike entries. Some care required there.
Last edit: Heribert Slama 2021-09-18
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hello,
when KeePass can't process some rows from a CSV-file, what happens?
On two computers, I exported CSV from MSecure, then edited it with LibreOffice and finally imported it with the Generic CVS importer. For both times I had the strange effect, that after a while we found out, that a handful of entries was missing.
Unfortunately I can't say if MSecure didn't export everything or the Keepass import dropped them. (I don't think it's a sync problem.)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The Generic CSV import will interpret each line in a CSV file as an entry or a group with or without an entry. Blank lines aren't imported. A header line is treated like another entry unless the Ignore first row option is checked. If your CSV file does not follow the guidelines in https://keepass.info/help/kb/imp_csv.html you can get some unexpected results, but the importer will still import each line. For example, a line that does not contain any separators will be imported into the first field listed in the import layout, typically the Title field. The import is undefined if the importer crashes. However, since only a handful of entries are missing, it is unlikely that the importer crashed. The easiest way to determine what might have happened during the import is to inspect the imported CSV file and compare its contents to the database contents.
Correction: empty groups are imported. Only blank lines are not imported.
Last edit: wellread1 2022-02-24
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thank you, wellread1.
As I have erased the CSV-file immediately after importing for security reasons, I can't compare it any more. But if there is an opportunity again, I'll have a look on it earlier.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
For KeePass 2.19, I've completely rewritten the generic CSV importer and added a few other features to support importing more CSV files. In detail:
* New generic CSV importer (now supports multi-line fields, '\' as escape character, field & record separators and the text qualifier can be specified, white space characters can be removed from the beginning/end of fields, the fields and their order can be defined, supported fields now are group name & standard fields like e.g. title & custom strings & times & ignore column).
* Added support for reading texts encoded using UTF-32 Big Endian.
* Enhanced text encoding detection (now detects UTF-32 LE/BE and UTF-16 LE/BE by zeros, improved UTF-8 detection, …).
* Unified text encoding handling for internal data viewer and editor, generic CSV importer and text encoding selection dialog.
The latest development snapshot is available here:
http://keepass.info/filepool/KeePass_120219b.zip
Testing, feedback and suggestions are welcome :-) If you find any CSV file that can't be imported using the new importer, please let me know.
Best regards
Dominik
Funny that you just posted this yesterday, as it solved a problem that I just found today! :) It's fantastic to be able to import custom string fields now.
I wonder how much work would be involved in reading the first row for headings, and if it might be possible to include an option to not import the first row.
Thanks very much!
Great tool. As fpr the UI, I think you should try to put the whole preview stuff to where now "specify the layout" is - this way we could see the results of syntax changes in realtime.
Thanks for the feedback! :-)
I've now added the following:
* Added an option to ignore the first row.
* KeePass tries to guess the fields and their order based on the first row (this e.g. works well with the KeePass 1.x CSV format).
Unfortunately there is too few space for a preview on the structure tab page.
Here's the latest development snapshot for testing:
http://keepass.info/filepool/KeePass_120221.zip
Thanks and best regards
Dominik
This is absolutely awesome… I was racking my brain playing the 2.18 version and reading the documentation on how to expand the fields in the database to match the fields in my csv. Thanks for doing this, with the custom fields
Very good feature, that I was looking for:,Can this feature be adressed by scripting? The KPScript Tool does not like this version.
My task is is to make such a job as a script:
1) Make Backup of keepass-dB
2) Import Data (overwrite Entry) from a CSV-File, with constant parameters for matching columns (will be a long line, but allways the same)
Thanks in advance!
KPScript does not support importing yet (and thus importing using the CSV importer isn't supported either).
We already have feature requests for an import command (e.g. FR 2790286). I think this would be great, and will probably add it in a future version.
Best regards
Dominik
Great utility. Two things would make it even better from my point of view:
Import into a group
Save custom structures
Best Regards
Stuart
What might I be doing wrong? Or do I not understand how this option works? (I wouldn't be shocked if so! :-))
When I open the Import option on a KeePass Windows 10 desktop, and select the file I want to import into KeePass, I see this list: "Field, Title, Username, Password, URL, Notes" and I can delete from the list of fields (except NOT "Field"). Field doesn't seem like it ought to show in that box, but should be a label ON the input box? "Group" shows in the "Add" field name box by default, and I see that I can choose among a variety of field names, and if I delete all the pre-existing field names I am able to match up the field names with the items in my import .csv. So, perhaps this is how it's intended to work.
I had hoped that the "Add" could offer the opportunity to import into a temporary Keepass group (for example, "Import from Chrome browser") but that's not apparently how the importer was designed, am I right? My data does not include a group name, so I guess I cannot generate this as a group on Import.
Thanks for a great password manager tool. I've used KeePass now for several years, and I'm just working to explain how to use it, to a friend who has been relying on paper lists and browser password memory for some time.
You're right, "Field" is only a caption and should have a different appearance than the names of the fields below.
If you want to import all entries into a specific group, all (data) lines of the CSV file must have an extra column with the group name. The safest way to do this, is to load the CSV file into a spreadsheet program (Excel, Libre Office Calc), add a new column, write the group name into its first data row and propagate it down to the last row (auto fill).
If the first line contains field names, put the string "Group" into the extra column.Then save. Additionally, you can easily delete columns which have no meaning for KeePass(may not be necessary, see below).CORRECTION - Sorry, the following paragraph is wrong ...
*About the first line of the CSV file: * Very often the first line doesn't contain table data, but a sequence of descriptive field names. If (you make) these names match KeePass field names, KeePass will recognize them and automatically adjust the field name list to the sequence present in the CSV file (including "Group"). Columns with names unknown to KeePass will be skipped (field name "(ignore)"). Recommended: If the CSV file comes without such a descriptive line, add one.I tested variations on a CSV file re-using the same open instance of KeePass. Although I started separate import actions, KeePass remembered my manual changes to the field list and presented it over and over again. This gave me the WRONG impression, that KeePass automatically adapts to the field names specified on the first line. It does NOT - in fact, it ignores them! A case of wishful thinking! I apologize.
While the file is loaded in a spreadsheet program to add the Group column, superfluous columns should be deleted, which is easier than inserting "(ignore)" fields in KeePass. The sequence of columns can be easily changed (drag-and-drop) to fit the standard sequence of KeePass. If the first line contains field names it can safely be removed; it is not needed.
By default, KeePass creates a new group. Repeated imports create separate groups with the same name. Optionally, an existing group with a matching name can be re-used. Repetitions will create duplicate or look-alike entries. Some care required there.
Last edit: Heribert Slama 2021-09-18
You should always have a header line in a CSV to make it easier for both you and KeePass.
There is a more advanced CSV converter that may help.
https://sourceforge.net/p/keepass/discussion/329220/thread/f7f9fd03/
cheers, Paul
Hello,
when KeePass can't process some rows from a CSV-file, what happens?
On two computers, I exported CSV from MSecure, then edited it with LibreOffice and finally imported it with the Generic CVS importer. For both times I had the strange effect, that after a while we found out, that a handful of entries was missing.
Unfortunately I can't say if MSecure didn't export everything or the Keepass import dropped them. (I don't think it's a sync problem.)
The Generic CSV import will interpret each line in a CSV file as an entry or a group with or without an entry. Blank lines aren't imported. A header line is treated like another entry unless the Ignore first row option is checked. If your CSV file does not follow the guidelines in https://keepass.info/help/kb/imp_csv.html you can get some unexpected results, but the importer will still import each line. For example, a line that does not contain any separators will be imported into the first field listed in the import layout, typically the Title field. The import is undefined if the importer crashes. However, since only a handful of entries are missing, it is unlikely that the importer crashed. The easiest way to determine what might have happened during the import is to inspect the imported CSV file and compare its contents to the database contents.
Correction: empty groups are imported. Only blank lines are not imported.
Last edit: wellread1 2022-02-24
The short answer is: if it is a line in a text file. The importer can import it.
Thank you, wellread1.
As I have erased the CSV-file immediately after importing for security reasons, I can't compare it any more. But if there is an opportunity again, I'll have a look on it earlier.