Scaling error in ds2450.c

2014-01-27
2014-01-28
  • Hi folks,

    I already posted a ticket in the support bin, but also want to put this into open discussion here:

    There seems to be a scaling error in the source code addressing the DS2450 A/D converter.

    According to the data sheet of the DS2450, p.22 , its minimum full scale input voltage at 16 bit resolution and 5.12 V range is 5.1199 V.

    This means, that at

    (65535-0.5)/65536 * 5.120 = 5.11988 (rounded 5.1199) V the binary value becomes 65535.

    Each least significant bit therefore weighs 1/65536 * 5.120 = 0.000078125 (rounded 0.000078) V in agreement with the table on page 22 of the data sheet

    However, in line 412 ff of owfs-xxx/module/owlib/src/c/ow_2450.c the code reads

    V[0] = 7.8126192E-5 * ((((UINT) data[1]) << 8) | data[0]);
    

    (3 lines following are the same).

    The floating point number here is

    0.000078126192 = 1/65535*5.120 instead of the 0.000078125 we obtained above

    Consequently, OWFS weighs each least significant bit slightly more than it should do, and at a binary count of 65535 displays the value 5.120 V, whereas it should display only 65535/65536*5.120 = 5.11992 (rounded 5.1199 ) V.

    The data sheet is not written very clearly here, but one may crosscheck the understanding of the data sheet with another resolution: Using 8 bit resolution, the binary count of the DS2450 will be set to FF00=255*256 at an input voltage of

    (255-0.5)256/653565120 = 5.090 V (as listed in the data sheet)

    and would be set to 65536 (e.g. ZERO plus carry bit) at

    (255+0.5)256/653565120 = 5.110 V

    Regards

    pah

     
    • Paul Alfille
      Paul Alfille
      2014-01-28

      Instead of:
      V[0] = 7.8126192E-5 * ((((UINT) data[1]) << 8) | data[0]);

      Can I put:
      V[0] = ((((UINT) data[1]) << 8) | data[0]) / ( 65535*5.1199) ;

      Or should it be
      V[0] = ((((UINT) data[1]) << 8) | data[0]) / ( 65535*5.11988) ;

      To get the proper bit rounding?

      On Mon, Jan 27, 2014 at 2:39 AM, Peter A. Henning pahenning@users.sf.netwrote:

      Hi folks,

      I already posted a ticket in the support bin, but also want to put this
      into open discussion here:

      There seems to be a scaling error in the source code addressing the DS2450
      A/D converter.

      According to the data sheet of the DS2450, p.22 , its minimum full scale
      input voltage at 16 bit resolution and 5.12 V range is 5.1199 V.

      This means, that at

      (65535-0.5)/65536 * 5.120 = 5.11988 (rounded 5.1199) V the binary value
      becomes 65535.

      Each least significant bit therefore weighs 1/65536 * 5.120 = 0.000078125
      (rounded 0.000078) V in agreement with the table on page 22 of the data
      sheet

      However, in line 412 ff of owfs-xxx/module/owlib/src/c/ow_2450.c the code
      reads

      V[0] = 7.8126192E-5 * ((((UINT) data[1]) << 8) | data[0]);

      (3 lines following are the same).

      The floating point number here is

      0.000078126192 = 1/65535*5.120 instead of the 0.000078125 we obtained above

      Consequently, OWFS weighs each least significant bit slightly more than it
      should do, and at a binary count of 65535 displays the value 5.120 V,
      whereas it should display only 65535/65536*5.120 = 5.11992 (rounded 5.1199
      ) V.

      The data sheet is not written very clearly here, but one may crosscheck
      the understanding of the data sheet with another resolution: Using 8 bit
      resolution, the binary count of the DS2450 will be set to FF00=255*256 at
      an input voltage of

      (255-0.5)256/653565120 = 5.090 V (as listed in the data sheet)

      and would be set to 65536 (e.g. ZERO plus carry bit) at

      (255+0.5)256/653565120 = 5.110 V

      Regards

      pah

      Scaling error in ds2450.chttps://sourceforge.net/p/owfs/discussion/292717/thread/b147e05a/?limit=25#c09d

      Sent from sourceforge.net because you indicated interest in
      https://sourceforge.net/p/owfs/discussion/292717/

      To unsubscribe from further messages, please visit
      https://sourceforge.net/auth/subscriptions/

       
      Attachments