Scaling error in ds2450.c

2014-01-27
2014-01-28
• Hi folks,

I already posted a ticket in the support bin, but also want to put this into open discussion here:

There seems to be a scaling error in the source code addressing the DS2450 A/D converter.

According to the data sheet of the DS2450, p.22 , its minimum full scale input voltage at 16 bit resolution and 5.12 V range is 5.1199 V.

This means, that at

(65535-0.5)/65536 * 5.120 = 5.11988 (rounded 5.1199) V the binary value becomes 65535.

Each least significant bit therefore weighs 1/65536 * 5.120 = 0.000078125 (rounded 0.000078) V in agreement with the table on page 22 of the data sheet

However, in line 412 ff of owfs-xxx/module/owlib/src/c/ow_2450.c the code reads

```V[0] = 7.8126192E-5 * ((((UINT) data[1]) << 8) | data[0]);
```

(3 lines following are the same).

The floating point number here is

0.000078126192 = 1/65535*5.120 instead of the 0.000078125 we obtained above

Consequently, OWFS weighs each least significant bit slightly more than it should do, and at a binary count of 65535 displays the value 5.120 V, whereas it should display only 65535/65536*5.120 = 5.11992 (rounded 5.1199 ) V.

The data sheet is not written very clearly here, but one may crosscheck the understanding of the data sheet with another resolution: Using 8 bit resolution, the binary count of the DS2450 will be set to FF00=255*256 at an input voltage of

(255-0.5)256/653565120 = 5.090 V (as listed in the data sheet)

and would be set to 65536 (e.g. ZERO plus carry bit) at

(255+0.5)256/653565120 = 5.110 V

Regards

pah

• Paul Alfille
2014-01-28

Instead of:
V[0] = 7.8126192E-5 * ((((UINT) data[1]) << 8) | data[0]);

Can I put:
V[0] = ((((UINT) data[1]) << 8) | data[0]) / ( 65535*5.1199) ;

Or should it be
V[0] = ((((UINT) data[1]) << 8) | data[0]) / ( 65535*5.11988) ;

To get the proper bit rounding?

On Mon, Jan 27, 2014 at 2:39 AM, Peter A. Henning pahenning@users.sf.netwrote:

Hi folks,

I already posted a ticket in the support bin, but also want to put this
into open discussion here:

There seems to be a scaling error in the source code addressing the DS2450
A/D converter.

According to the data sheet of the DS2450, p.22 , its minimum full scale
input voltage at 16 bit resolution and 5.12 V range is 5.1199 V.

This means, that at

(65535-0.5)/65536 * 5.120 = 5.11988 (rounded 5.1199) V the binary value
becomes 65535.

Each least significant bit therefore weighs 1/65536 * 5.120 = 0.000078125
(rounded 0.000078) V in agreement with the table on page 22 of the data
sheet

However, in line 412 ff of owfs-xxx/module/owlib/src/c/ow_2450.c the code
reads

V[0] = 7.8126192E-5 * ((((UINT) data[1]) << 8) | data[0]);

(3 lines following are the same).

The floating point number here is

0.000078126192 = 1/65535*5.120 instead of the 0.000078125 we obtained above

Consequently, OWFS weighs each least significant bit slightly more than it
should do, and at a binary count of 65535 displays the value 5.120 V,
whereas it should display only 65535/65536*5.120 = 5.11992 (rounded 5.1199
) V.

The data sheet is not written very clearly here, but one may crosscheck
the understanding of the data sheet with another resolution: Using 8 bit
resolution, the binary count of the DS2450 will be set to FF00=255*256 at
an input voltage of

(255-0.5)256/653565120 = 5.090 V (as listed in the data sheet)

and would be set to 65536 (e.g. ZERO plus carry bit) at

(255+0.5)256/653565120 = 5.110 V

Regards

Scaling error in ds2450.chttps://sourceforge.net/p/owfs/discussion/292717/thread/b147e05a/?limit=25#c09d

Sent from sourceforge.net because you indicated interest in
https://sourceforge.net/p/owfs/discussion/292717/

To unsubscribe from further messages, please visit
https://sourceforge.net/auth/subscriptions/

Attachments