#289 Editable form of first 10 errors

[Steve Keen]

We need to change the fail mode on import from "I'm sorry, you can't get in without a ticket" to "now that you're in, how would you like to pay?". This file fails for numerous reasons, not least of which is a weird date entry of "TQ" on row 41.

What we need to do in every case is import and then identify the problems for the user to repair within the program, whether that's by deleting a row (my decision here), editing the dimension or data, etc.

[High Performance Coder]

What happens if there are a hundred such errors? That's why we have the "generate error report" - the spreadsheet generated can be edited and then used to import the data.

Of course the report form needs work - it is actually out of date with respect to the main import code, and its a problem that it is implemented as two different code paths, which means the report and the main import code can easily diverge.

[Steve Keen]

My issue with this is that, with several such errors, the user is likely to give up, and try to analyze in Excel instead, where at least he/she can see the data.

Correct me if I'm wrong, but the program throws up the error message on the first error discovered. So the other 99 would still be there to be discovered on the next import attempt?

We should instead go through the file and find every error, then give the user the option of an exported CSV file with the errors identified. Then the exported file should have 3 additional columns: Error (as now) on the relevant row; Column (which column contains the error); Fix (suggested fix to error).

At a later stage, we should enable these to be identified and applied within the import system itself. So at each error, a tabular view of the data would stop on the error, with it highlighted and the suggested fix shown. Then the user would have 3 options: apply; edit; report. If the user applied or manually edited all errors, the file would be imported; if they chose report on any, they would get an error CSV export at the end of the process.

[High Performance Coder]

I agree the error report process needs improvement, and there are ways of finessing it. The design was effectively ported over from the TCL version, which had limitations from that toolkit, which don't apply in the Javascript world. I just haven't had the time to revisit that functionality.

Maybe we can show the first 10 errors in an editable table, and if there are more than 10, provide an export to CSV button.

[Steve Keen]

That sounds like a decent compromise.

[High Performance Coder]

This is a suggestion to show first 10 errors in an editable form. Backlogging this for now.
Currently report is working for this sort of cleanup operation.

[High Performance Coder]

Ticket moved from /p/minsky/ravel/386/