Menu
▾
▴
#74 vstar does not support space delimited files
Download all data for S Ori from the aavso website choosing "Space Delimited (not column-delimited)" format. Opening the file in vstar gives "...error in observation source."
Personally, I find the thought of space delimited data to be abhorrent. But if this is a supported format to download from the aavso website, then vstar should open it. I had no problem opening csv or tsv formated data files for the same star.
Hi Mike. It's interesting to read the current spec, discussion, and requirements for input file support in VStar, which along with field validation and creating a generic framework to be able to handle database access and file input sources uniformly within VStar, consumed a lot of my time early on. See these pages:
https://sourceforge.net/apps/mediawiki/vstar/index.php?title=Input
https://sourceforge.net/apps/mediawiki/vstar/index.php?title=Talk:Input
https://sourceforge.net/apps/mediawiki/vstar/index.php?title=Phase_I_VStar_Requirements
The summary is that only CSV and TSV are supported for AAVSO download files (I've gone further than the spec and requirements by supporting both because once you've handled one, the other is trivial), but not space delimited files. I started out trying to support that too but gave up after awhile. For simple format files, space delimiters are doable (and in the spec), but for AAVSO download format, space delimited files are human readable but not really machine readable. Due to the optionality of some fields and the free-form text allowed in comments fields, not only would a context-free grammar be impossible, a context-sensitive parse of each line of such a file would be very challenging, to say the least.
So, although I have not taken a close look at examples of such space-delimited (e.g. The same as TSV and CSV but just a single space, or is it multiple spaces per delimiter, in which case how to handle optional fields? Are comment fields quoted?) downloaded files, I do not believe we need to concern ourselves with this format. If you think we should pursue this with AAVSO, then let's keep this issue open until resolved.
However, this tracker has raised two issues:
1. Is the Help documentation clear about this?
2. Does the simple file format (as currently implemented) permit space delimiters?
The answer to 1. is almost certainly "no". Do you want to have a go at this?
The answer to 2. is no. Simple format also requires TSV and CSV. That's interesting. The requirements I derived from the spec say that simple format file rows will be TSV or CSV, but the spec and discussion pages suggest that space delimiters should permitted. Even here however, any of the last 3 (of 5) fields may be optional. Sorting out space delimited rows when parsing each simple format file line is no big drama, but arguably unnecessary. Again, we can raise this with Sara et al again. Given that this simple file format is intended for use when creating one's own observation files, or perhaps later, importing data from other sources (e.g. Photometrica), we should be able to define what this looks like. So, I would be arguing for TSV and CSV even here, and that is indeed what the format requirement states. There were individual, ad hoc informal reviews of the phase I requirements I derived but nothing formal. I recall some discussions about this, but I think this may have fallen through the cracks.
I agree that space delimited is not something we want to support and that this is a documentation issue. I'll assign this to myself. I've also modified the summary of this tracker to reflect this.
There is a note in help about space delimited file being unsupported in svn 298
We might want to gather some information about what formats are produced by default, or optionally, using various software that reduces images. For example MaximDL or other packages that are commonly used. However, csv and tsv are by far the most common formats for data such as this and are handled by all spreadsheet software and other common tools.