I get the point. I'm just saying that for some cases I think its OK to toss a error or drop data. We don't have to be 100% on every combination.  But the baseline has to get to zero to hold it there.

I have the same stance on warnings.  Some if the things GCC warns about are just dumb, but its hard too hold ground at 1,234 warnings. You wave to child it at zero and that sometimes means busy work.

On Dec 8, 2012 6:12 AM, "tsteven4" <tsteven4@gmail.com> wrote:
My motivation in solving what I consider the more serious issues
(crashes) is to allow more complete automated testing.  I like the idea
in test-all to try all the allowed translations, but currently it is of
marginal benefit because it is little used and there are too many false
positives.  If we can establish a criteria for test-all success based on
a subset of the test-all findings then we can add the test to the
automatic regression I recently implemented.  So even though the formats
involved in these issues might be of little importance these issues
prevent the establishment of a criteria such as "test-all will be
considered successful if there are no SIGABRT or SIGSEGV events."   An
alternate criteria might be "test-all is considered successful if there
are no return codes > 128."

I am still learning to count, but Robert's last couple of changesets
considerably improve the situation.  The 8 SIGSEGVs with pocketfms_wp
are fixed.  There are 290 less total exceptions.  I consider an
exception anything with a either a non-zero return code from gpsbabel or
any gpsbabel run that produces output to standard error or standard
out.  There are many many more invocations of gpsbabel than this during
the test and the runs that aren't considered an exception are not
counted.  The percentages I stated yesterday are all erroneous and
pessimistic.

We could cover Bortel Lazslos discovery by including a point without a
name in the test-all reference file.  I bet this would show many
SIGSEGVs, even though the name is optional in the gpx schema.

r4214:
> cmd(134): ./gpsbabel -t -i lowranceusr4 -f
> /tmp/gb-test-all/t-igc.lowranceusr4 -o igc -F
> /tmp/gb-test-all/t-igc.lowranceusr4.igc
> cmd(134): ./gpsbabel -t -i lowranceusr4 -f
> /tmp/gb-test-all/t-pocketfms_bc.lowranceusr4 -o pocketfms_bc -F
> /tmp/gb-test-all/t-pocketfms_bc.lowranceusr4.pocketfms_bc
> cmd(139): ./gpsbabel -w -i coastexp -f
> /tmp/gb-test-all/w-enigma.coastexp -o enigma -F
> /tmp/gb-test-all/w-enigma.coastexp.enigma
> cmd(139): ./gpsbabel -w -i geoniche -f
> /tmp/gb-test-all/w-enigma.geoniche -o enigma -F
> /tmp/gb-test-all/w-enigma.geoniche.enigma
> cmd(139): ./gpsbabel -w -i quovadis -f
> /tmp/gb-test-all/w-enigma.quovadis -o enigma -F
> /tmp/gb-test-all/w-enigma.quovadis.enigma
> cmd(139): ./gpsbabel -r -i enigma -f /tmp/gb-test-all/r-enigma -o
> enigma -F /tmp/gb-test-all/r-enigma.enigma
> cmd(139): ./gpsbabel -w -i geoniche -f
> /tmp/gb-test-all/w-naviguide.geoniche -o naviguide -F
> /tmp/gb-test-all/w-naviguide.geoniche.naviguide
> cmd(139): ./gpsbabel -w -i quovadis -f
> /tmp/gb-test-all/w-naviguide.quovadis -o naviguide -F
> /tmp/gb-test-all/w-naviguide.quovadis.naviguide
> cmd(139): ./gpsbabel -t -i lowranceusr4 -f
> /tmp/gb-test-all/t-stmsdf.lowranceusr4 -o stmsdf -F
> /tmp/gb-test-all/t-stmsdf.lowranceusr4.stmsdf
> SIGABRT 2
> SIGSEGV 7
> total exceptions 377

For comparison with the same scoring r4211:
> cmd(134): ./gpsbabel -t -i lowranceusr4 -f
> /tmp/gb-test-all/t-igc.lowranceusr4 -o igc -F
> /tmp/gb-test-all/t-igc.lowranceusr4.igc
> cmd(134): ./gpsbabel -t -i lowranceusr4 -f
> /tmp/gb-test-all/t-pocketfms_bc.lowranceusr4 -o pocketfms_bc -F
> /tmp/gb-test-all/t-pocketfms_bc.lowranceusr4.pocketfms_bc
> cmd(139): ./gpsbabel -w -i pocketfms_wp -f
> /tmp/gb-test-all/w-csv.pocketfms_wp -o csv -F
> /tmp/gb-test-all/w-csv.pocketfms_wp.csv
> cmd(139): ./gpsbabel -w -i pocketfms_wp -f
> /tmp/gb-test-all/w-an1.pocketfms_wp -o an1 -F
> /tmp/gb-test-all/w-an1.pocketfms_wp.an1
> cmd(139): ./gpsbabel -w -i pocketfms_wp -f
> /tmp/gb-test-all/w-xmap.pocketfms_wp -o xmap -F
> /tmp/gb-test-all/w-xmap.pocketfms_wp.xmap
> cmd(139): ./gpsbabel -w -i coastexp -f
> /tmp/gb-test-all/w-enigma.coastexp -o enigma -F
> /tmp/gb-test-all/w-enigma.coastexp.enigma
> cmd(139): ./gpsbabel -w -i geoniche -f
> /tmp/gb-test-all/w-enigma.geoniche -o enigma -F
> /tmp/gb-test-all/w-enigma.geoniche.enigma
> cmd(139): ./gpsbabel -w -i quovadis -f
> /tmp/gb-test-all/w-enigma.quovadis -o enigma -F
> /tmp/gb-test-all/w-enigma.quovadis.enigma
> cmd(139): ./gpsbabel -r -i enigma -f /tmp/gb-test-all/r-enigma -o
> enigma -F /tmp/gb-test-all/r-enigma.enigma
> cmd(139): ./gpsbabel -w -i pocketfms_wp -f
> /tmp/gb-test-all/w-ggv_ovl.pocketfms_wp -o ggv_ovl -F
> /tmp/gb-test-all/w-ggv_ovl.pocketfms_wp.ggv_ovl
> cmd(139): ./gpsbabel -w -i pocketfms_wp -f
> /tmp/gb-test-all/w-mapconverter.pocketfms_wp -o mapconverter -F
> /tmp/gb-test-all/w-mapconverter.pocketfms_wp.mapconverter
> cmd(139): ./gpsbabel -w -i geoniche -f
> /tmp/gb-test-all/w-naviguide.geoniche -o naviguide -F
> /tmp/gb-test-all/w-naviguide.geoniche.naviguide
> cmd(139): ./gpsbabel -w -i quovadis -f
> /tmp/gb-test-all/w-naviguide.quovadis -o naviguide -F
> /tmp/gb-test-all/w-naviguide.quovadis.naviguide
> cmd(139): ./gpsbabel -w -i pocketfms_wp -f
> /tmp/gb-test-all/w-dna.pocketfms_wp -o dna -F
> /tmp/gb-test-all/w-dna.pocketfms_wp.dna
> cmd(139): ./gpsbabel -t -i lowranceusr4 -f
> /tmp/gb-test-all/t-stmsdf.lowranceusr4 -o stmsdf -F
> /tmp/gb-test-all/t-stmsdf.lowranceusr4.stmsdf
> cmd(139): ./gpsbabel -w -i pocketfms_wp -f
> /tmp/gb-test-all/w-tomtom.pocketfms_wp -o tomtom -F
> /tmp/gb-test-all/w-tomtom.pocketfms_wp.tomtom
> cmd(139): ./gpsbabel -w -i pocketfms_wp -f
> /tmp/gb-test-all/w-tiger.pocketfms_wp -o tiger -F
> /tmp/gb-test-all/w-tiger.pocketfms_wp.tiger
> SIGABRT 2
> SIGSEGV 15
> total exceptions 567


On 12/7/2012 8:06 PM, tsteven4 wrote:
> actually even a bit less prevalent.  SIGABRT and SGISEGV errors were
> counted correctly but the total was not.  The corrected rates are
> SIGABRT 2/567 (.352%)
> SIGSEGV 15/567 (2.645%)
>
> On 12/7/2012 8:04 PM, tsteven4 wrote:
>> I overcounted by not clearing the log. The problems are in the same
>> places, they just are as prevalent as I originally stated.
>>
>> ~/work/trackfilter% ./test-all --tally
>> cmd(134): ./gpsbabel -t -i lowranceusr4 -f
>> /tmp/gb-test-all/t-igc.lowranceusr4 -o igc -F
>> /tmp/gb-test-all/t-igc.lowranceusr4.igc
>> cmd(134): ./gpsbabel -t -i lowranceusr4 -f
>> /tmp/gb-test-all/t-pocketfms_bc.lowranceusr4 -o pocketfms_bc -F
>> /tmp/gb-test-all/t-pocketfms_bc.lowranceusr4.pocketfms_bc
>> cmd(139): ./gpsbabel -w -i pocketfms_wp -f
>> /tmp/gb-test-all/w-csv.pocketfms_wp -o csv -F
>> /tmp/gb-test-all/w-csv.pocketfms_wp.csv
>> cmd(139): ./gpsbabel -w -i pocketfms_wp -f
>> /tmp/gb-test-all/w-an1.pocketfms_wp -o an1 -F
>> /tmp/gb-test-all/w-an1.pocketfms_wp.an1
>> cmd(139): ./gpsbabel -w -i pocketfms_wp -f
>> /tmp/gb-test-all/w-xmap.pocketfms_wp -o xmap -F
>> /tmp/gb-test-all/w-xmap.pocketfms_wp.xmap
>> cmd(139): ./gpsbabel -w -i coastexp -f
>> /tmp/gb-test-all/w-enigma.coastexp -o enigma -F
>> /tmp/gb-test-all/w-enigma.coastexp.enigma
>> cmd(139): ./gpsbabel -w -i geoniche -f
>> /tmp/gb-test-all/w-enigma.geoniche -o enigma -F
>> /tmp/gb-test-all/w-enigma.geoniche.enigma
>> cmd(139): ./gpsbabel -w -i quovadis -f
>> /tmp/gb-test-all/w-enigma.quovadis -o enigma -F
>> /tmp/gb-test-all/w-enigma.quovadis.enigma
>> cmd(139): ./gpsbabel -r -i enigma -f /tmp/gb-test-all/r-enigma -o
>> enigma -F /tmp/gb-test-all/r-enigma.enigma
>> cmd(139): ./gpsbabel -w -i pocketfms_wp -f
>> /tmp/gb-test-all/w-ggv_ovl.pocketfms_wp -o ggv_ovl -F
>> /tmp/gb-test-all/w-ggv_ovl.pocketfms_wp.ggv_ovl
>> cmd(139): ./gpsbabel -w -i pocketfms_wp -f
>> /tmp/gb-test-all/w-mapconverter.pocketfms_wp -o mapconverter -F
>> /tmp/gb-test-all/w-mapconverter.pocketfms_wp.mapconverter
>> cmd(139): ./gpsbabel -w -i geoniche -f
>> /tmp/gb-test-all/w-naviguide.geoniche -o naviguide -F
>> /tmp/gb-test-all/w-naviguide.geoniche.naviguide
>> cmd(139): ./gpsbabel -w -i quovadis -f
>> /tmp/gb-test-all/w-naviguide.quovadis -o naviguide -F
>> /tmp/gb-test-all/w-naviguide.quovadis.naviguide
>> cmd(139): ./gpsbabel -w -i pocketfms_wp -f
>> /tmp/gb-test-all/w-dna.pocketfms_wp -o dna -F
>> /tmp/gb-test-all/w-dna.pocketfms_wp.dna
>> cmd(139): ./gpsbabel -t -i lowranceusr4 -f
>> /tmp/gb-test-all/t-stmsdf.lowranceusr4 -o stmsdf -F
>> /tmp/gb-test-all/t-stmsdf.lowranceusr4.stmsdf
>> cmd(139): ./gpsbabel -w -i pocketfms_wp -f
>> /tmp/gb-test-all/w-tomtom.pocketfms_wp -o tomtom -F
>> /tmp/gb-test-all/w-tomtom.pocketfms_wp.tomtom
>> cmd(139): ./gpsbabel -w -i pocketfms_wp -f
>> /tmp/gb-test-all/w-tiger.pocketfms_wp -o tiger -F
>> /tmp/gb-test-all/w-tiger.pocketfms_wp.tiger
>> SIGABRT 2/232 (.862%)
>> SIGSEGV 15/232 (6.465%)
>>
>>
>> On 12/7/2012 5:18 PM, tsteven4 wrote:
>>> test-all is finding serious issues:
>>> 31 return codes of 134, sigabrt, all with -i lowranceusr4
>>> 1594 return codes of 139, sigsegv:
>>>     1522 with -i pocketfms_wp
>>>     54 with -o enigma
>>>     18 with -o naviguide
>>> -----------------------------------
>>> out of 7626 tests
>>>
>>> so 1625 tests (21%) show serious issues.  it may only boil down to 4
>>> bugs involving 4 formats.
>>>
>>> only 2575 tests (34%) return error codes of 0.  the rest have other
>>> issues, possibly false failures.
>>>
>>> Bortel Lazslo has recently demonstrated another sigsegv not caught
>>> by test-all that should be easy to fix.
>>
>


------------------------------------------------------------------------------
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
Gpsbabel-code@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code