#494 Load WKT file with set options

[János Tamás Kis]

I have a simple WKT file with an attribute data (like 'proba.wkt' file) where the geomerty is in the second data field/column.
I have open it with the "wkt (set options)" function into a new project.
I have set the "Column containing WKT geometry" listbox to "2".
That was great: I could see the map and data.
I have saved the project as proba.jmp and I saw the

<key class="java.lang.String">WKT-Column</key>
<value class="java.lang.String">2</value>

lines in JMP file so I thought everyhing is OK, but when I tried reopen the JMP file I got a java.lang.Exception:

 Field 1 is needed for geometry but [] has only 0 fields.

What was wrong...?

It's OK, I tried other way...
Let be the geometry in the first and the attribute data in the second data field/column (like 'reverse.wkt' file) and open it (with "Column containing WKT geometry" listbox to "1" option) and save the project (reverse.jmp). There are the

<key class="java.lang.String">WKT-Column</key>
<value class="java.lang.String">1</value>

rows.
When I tried reopen then I got the

 Field 0 is needed for geometry but [] has only 0 fields

exception, similar before but the index is 0...

I am afraid, the JMP files are correct but the loader have wrong about indexing besause it try load geometry from other column what I have set and stored in JMP file or what am i wrong with?

[michael michaud]

Hi János,
Thank you for the very clear description of the problem, I'll have a look ASAP.

[michael michaud]

János,
Problem was in the serialization of the field separator. Please, check that it works in r6365+
Notice that you need both this new version of OJ and the new version of the csv driver.

[János Tamás Kis]

Thank you very much! The opening 'WKT file (set options)' function it works but i have some remark (I am using the "OpenJUMP-Portable-20200805-r6370-PLUS"):

1. The CSV driver needs for translated properties file else the user get error message (in the background at console or Output window).

[WARN] 10:15:20.179 No translation for key 'fr.michaelm.jump.drivers.csv.CSVDriverConfiguration$2.Encoding' in bundle 'language/jump'.
[WARN] 10:15:20.179 Can't find resource for bundle java.util.PropertyResourceBundle, key fr.michaelm.jump.drivers.csv.CSVDriverConfiguration$2.Encoding no default value, the resource key is used: Encoding
[WARN] 10:15:20.194 No translation for key 'fr.michaelm.jump.drivers.csv.CSVDriverConfiguration$2.Comment line (regex)' in bundle 'language/jump'.
[WARN] 10:15:20.194 Can't find resource for bundle java.util.PropertyResourceBundle, key fr.michaelm.jump.drivers.csv.CSVDriverConfiguration$2.Comment line (regex) no default value, the resource key is used: Comment line (regex)
[WARN] 10:15:20.194 No translation for key 'fr.michaelm.jump.drivers.csv.CSVDriverConfiguration$2.Field separator' in bundle 'language/jump'.
[WARN] 10:15:20.194 Can't find resource for bundle java.util.PropertyResourceBundle, key fr.michaelm.jump.drivers.csv.CSVDriverConfiguration$2.Field separator no default value, the resource key is used: Field separator
[WARN] 10:15:20.194 No translation for key 'fr.michaelm.jump.drivers.csv.CSVDriverConfiguration$2.Header (column names)' in bundle 'language/jump'.
[WARN] 10:15:20.194 Can't find resource for bundle java.util.PropertyResourceBundle, key fr.michaelm.jump.drivers.csv.CSVDriverConfiguration$2.Header (column names) no default value, the resource key is used: Header (column names)
[WARN] 10:15:20.194 No translation for key 'fr.michaelm.jump.drivers.csv.CSVDriverConfiguration$2.Data types' in bundle 'language/jump'.
[WARN] 10:15:20.194 Can't find resource for bundle java.util.PropertyResourceBundle, key fr.michaelm.jump.drivers.csv.CSVDriverConfiguration$2.Data types no default value, the resource key is used: Data types
[WARN] 10:15:20.194 No translation for key 'fr.michaelm.jump.drivers.csv.CSVDriverConfiguration$2.Column containing WKT geometry' in bundle 'language/jump'.
[WARN] 10:15:20.194 Can't find resource for bundle java.util.PropertyResourceBundle, key fr.michaelm.jump.drivers.csv.CSVDriverConfiguration$2.Column containing WKT geometry no default value, the resource key is used: Column containing WKT geometry

So I suggest handling these error messages.

1. I saw the "csv.properties" and "csv_en.properties" are different. It is OK...?

2. I have translated the "csv.properties" and the main "jump_hu.properties" files and I am going to attach. (I have attached the last file at other ticket [#268] but maybe there was not a good idea...)

3. When I would like to load an WKT file (with set option) it is all right but if I want to save the layer to WKT (where I cannot set any options...) then file stored only geometry without attribute data. If I save it as CSV then I get back the original WKT file.
   It is strange in my eyes...

Sincerely,
János

[Jukka Rahkonen]

It may feel strange but after all WKT handles just geometries as defined in the standard https://portal.ogc.org/files/?artifact_id=25355. Perhaps the issue is calling file with WKT and attributes as "WKT file" instead of calling it as "CSV file with WKT".

Saving geometries into WKT is possible without any options because everything is defined in the standard. There is no standard about CSV and therefore options for selecting the separator etc. are needed.

But of course we want that it is easy to understand the meaning of different file format options and appreciate all suggestions for improvements.

As a comparison in GDAL the single CSV driver offers the capabilities with and without WKT https://gdal.org/drivers/vector/csv.html.

[michael michaud]

Thank you for feedback.
I removed csv-driver.en and committed jump.hu and for csv-driver.hu.
Should be available in next release (r6374+)
I modified a bit the driver so that it can now import a file with .csv extension and including wkt geometries.
To make it clear, there are two drivers.

• WKT (uppercase) is the original one. It can only load and save pure wkt file without attribute.
• The new driver (csv-driver) can save geometry as x,y,z or as wkt along with attributes in a .csv file, but in the other way, it could only read x,y columns from file with csv extension. Files with wkt and attributes was accepted by the wkt loader if they had .wkt extension. Now, wkt loader accepts also .csv extension.
  It is more symmetric (can read what it wrote). Hope it is more clear.
  I also cleaned up the code a bit. Please test and report.

[michael michaud]

Hi János, can we close this ticket ?

[János Tamás Kis]

H,
I am sorry... I forget it a bit...
Please try open the attached equal files with "csv (set options)" function.
If i opened the data.csv everything was OK (i get some WARN message) but with uj_wkt.csv i got some other WARN message, for examlpe
Can't parse string '57' as double with this CSVFile using fieldSeparator ';'
I dont understand what happend...?

[michael michaud]

Janos,
About the warnings, I think I've found the cause, but it will be a bit tricky to solve. Will open another ticket for that..
About the parsing, I could not reproduce. Any how, your files are the same, so that it is highly improbable that they are not imported the same way. I could reproduce an error similar to yours if I try to import as wkt, but you can't do that as geometry is not in th ewkt format in this case.
Anyway, the message is a bit misleading, beacuse it says that 57 cannot be parse as double but it should be 57 cannot be parsed as wkt. I will fix that.

[János Tamás Kis]

Maybe... I has been wrong in the previous example... I am sorry.

i get and i have lot of similar file as data.csv (data with yx(z) OR with wkt) often so I must be able to process efficiently so i would like to solve/avoid all fo problem...
in my oppinion there is two (or more) parts of the problem: the WKT file format and the CSV file format.
I can accept the restriction of the WKT format, that it can be stored only the geometry but then should not be allowed the "WKT (set options)" chance. The WKT file with other attribute datas is not a valid WKT file.
Everything, what is not valid WKT format (storing not geometry only) that we need to be able to read with CSV driver, independent what it stores: xyz or WKT string and optional attribute data(s) instead of "wkt (set options)".
Maybe we could need a "Column containing WKT value" option at "csv (set options)" panel...

(I beg you pardon for my poor English knowledge. The precise wording of things isn't too easy for me but fortunately easier as Chinese...)

[ The xy has been replaced to yx in Hungarian Geodetic System but the formulas are the same... it is our individual social problem... :P ]

[michael michaud]

Janos
"I can accept the restriction of the WKT format, that it can be stored only the geometry but then should not be allowed the "WKT (set options)" chance."

Indeed, you can save the wkt with attribute. It is just a little bit hidden : save as csv, choose attributes to export and in the attribute list, check GEOMETRY ;-)

Event if most code is the same for xyz or wkt import/export, I prefered to simulate two different drivers because the list of options id slightly different (you have to define a single column for geometry in the case of wkt and at least two columns in the case of xy(z) or any other combinaison. But now, it is just a matter of selecting the right driver csv (set options) or wkt (set options when you import your files (anyway, you must now what is inside). IN the current version, wkt (set option) is not proposed for files with extension txt. I'll add it in the next version.

[János Tamás Kis]

Hi,
Ok, I understand.
We can close this ticket, in my opinion.
There are more things yet but these aren't for this topic.
Thank you for your efforts and support
:)

[michael michaud]