You are running into precision limits.  The numbers you are dealing with are very small and your doing a lot of multiplications and trig calculations to create intermediate results.  Each intermediate result has to be rounded to fit into the 8 byte double.  The fact that the robotstix doesn't give you results below 1.6 nautical miles shows that you're hitting these limits and your intermediate results are being rounded.

Additionally, while the Amtel double is 8 bytes, its only an 8-bit microcontroller vs. the 32/64-bit desktop you have.  If you're pushing precision limits this is probably what gives the difference between the robostix and your desktop.  As for the differences between your final results, ( 2.9 NM vs 2.5 NM) the differences are only big because the units are nautical miles.  Considering how small your numbers are before the final calculation its not surprising that they could differ by a few tenths from machine to machine.

To get around this there are a few things you can do.  The first it to try to rearrange the math to either cancel out operations that aren't necessary or to avoid creating results that can't be represented.  For example, I could do this:

  a = 1 / 3;
  b = 21 * a;

however the calculation in 'a' will be 0.3333..., an number which simply cannot be represented on any computer.  However, if I combined those steps to divide by 3 directly and avoid storing an intermediate value:

  b = 21 / 3;

there is a defined, finite result that can be easily stored with no loss of precision.

Another thing you can do is work in the smallest units possible.  For example, if I wanted to write a program that calculated the position of a spacecraft, I might choose AU (1 astronomical unit == distance between the earth and the sun) as my unit of measure.  This gives me good results when calculating my distance to nearby planets, ( e.g. Mars is 1.29 AU away) however if I'm trying to dock with a satelite this unit way to large, (e.g. "we are 0.000000000006684 AU and closing").

In your case if you care about tenths of miles, then you probably want to work in a smaller unit like feet or inches.  I don't know how this changes your calculations but working with a smaller unit can help you avoid amplifying the 'noise' of precision loss.

Finally, what you're doing has probably already been done many times before.  Try to find a real code example (non-textbook) of this calculation someplace where somebody has already considered the limitations of computer math.

On 5/12/07, Jon Keller <> wrote:
I've asked this same question around some of the avr lists but am having
troubles getting an answer so I thought I may as well try this list as
well, sorry if its a little off topic.

I'm trying to calculate the distance between two gps coordinates on the
robostix and am running into some weird issues. I'm using the code below
for the calc. When I run this same code within perl and C on my standard
workstation I get the correct values, which for the coordinates below are

"NMiles: 2.5005995199488 SMiles: 2.87763854023694 SKm: 4.63111031889908"

However when running the same code over the robostix I get the following

"NMiles: 2.9072017670 SMiles: 3.3455481529 SKm: 5.3841376305"

Obviously theres quite a big difference in the results, could anyone
shed some light on why? Another very strange problem I've noticed is
that if I make the two coords closer together, the results on the
robostix stop decreasing when they hit a certain value, for instances
the nautical miles result stops at 1.678 no matter how close the coords
get to each other. Any help is greatly appreciated, Cheers.


        double lon1 = 174.681821;
        double lon2 = 174.629821;
        double lat2 = deg2rad(-36.728889);
        double lat1 = deg2rad(-36.728889);

        double dlon = lon2 - lon1;
        if (dlon > 180) {
                dlon -= 360;
        if (dlon < -180) {
                dlon += 360;

        double rdlon = deg2rad(dlon);
        double cosdist = sin(lat1) * sin(lat2) + cos(lat1) * cos(lat2) *
        double rdist = acos(cosdist);

        double nautical_miles = rad2deg(rdist) * 60;
        double statue_miles = fabs((rad2deg(rdist) * 60.) * 1.15077945);
        double statue_km = fabs((rad2deg(rdist) * 60.) *

        print_double("Distance -  NMiles: %.2lf SMiles: %.2lf SKm:
%.2lf\n", nautical_miles, statue_miles, statue_km);

This email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
gumstix-users mailing list