Add a sensor to I2CTools?

[Jean-Michel]

Hi,

I love your work!

I am going to try adding the SCD41 sensor.

Could you please resume the necessary steps?

1/ add a dictionary in the glob
2/ create a class
3/ modify the unix script

Do you have any example on how to run the script from a Python console directly? I mean, instead of a unix script.

thanks

(btw you should migrate to Github ;-)

[ullix]

Hi Jean-Michel,
thanks for the message. I looked up the SCD41 sensor, and it sure looks like an interesting thing! This photo-acoustic principle, although claimed to be over 100 years old, is new to me. The chip would fill an important niche. If they can deliver!

I am a bit biased against Sensirion. I have worked with their SGP30 sensor, which can measure VOC (Volatile organic compounds) in the air, and from there conclude on CO2 concentration. This turned out - to say it politely - a load of BS. But I am open for good news! If you can get good data out of this SCD4X, that would be great!

To measure CO2 I am using the MiniMon device, which is based on the NDIR principle. You find software from me also on sourceforge: https://sourceforge.net/projects/minimon/. I have also implemented this as a device under GeigerLog, https://sourceforge.net/projects/geigerlog/ where this is in this moment running for 319 days straight! Works really well. I was amazed to see how quickly CO2 builds up, even when doing only office work.

Of course, the NDIR principle requires a much larger device, see the pics on the linked-to web sites. The small size of the SCD4X is indeed attractive; the accuracy is good enough.

However, I would no longer pursue the approach taken with my I2Cpytools. In the meantime I have used the ESP32 microchips quite successfully on related projects, which are so much more powerful than any of these dongles AND bring WiFi to the game (and are cheaper on top!). Even the simple ESP8266 might work well enough to log data from a sensor and send it per WiFi. It is almost embarrassingly simple ;-))

Sensirion offers "Arduino drivers" for the SCD4X as downloads. This might be a good start. The simple combination of an ESPXXX with this chip would suffice for a data collector. The processing, saving, displaying of the data could be done in my GeigerLog, which is ready (in the next version I am preparing for release right now) to handle such data and such transmission pathways.

If you need help on Arduino-type programming, I am sure I can help!

[Jean-Michel]

Actually the light-matter interaction that takes place in the SCD4x sensor is the same as in a NDIR device. So it's not BS. i mean, it is quite specific, with low hysteresis.

The thing is that I have this old setup with a SHT75 sensor and a dongle, run under Labview, and recently I bought this SCD one, and I wanted to migrate the code to use Python, withiut changing everything.

thanks anyway!

[ullix]

I could not resist the temptation to try out this new sensor, thanks for pointing it out, and here is some result (see pic).

I ran it in a modified GeigerLog side-by-side with my MiniMon https://sourceforge.net/projects/minimon/. MiniMon is an NDIR device.

Both were in a loosely covered card board box( to limit speed of air exchange) sitting on my desk in the office. The ups and downs came from my presence or absence in the room, and the degree of ventilation by opening doors and/or windows.

The CO2 curves are in decent agreement. The temperature curves are a consistent 1 °C apart, which is still ok given the specs.

However, there is a catch with the CO2 sensing by the SCD41: While the MiniMon - and I believe every NDIR based system - has a built in reference, the SCD41 has not. Its initial offset had been 330 ppm! I then calibrated with the MiniMon as reference.

To use a SCD41 you either need to have a reference device, or you need to equilibrate with outside air and call it 400 ppm.

[Jean-Michel]

that is very interesting, thank you.
They could have implemented a reference cell, but I think it would have required two microphone sensors, so a drift could still have been possible.
i think that the sensor is intended to stay very low cost, hence that is why.

[ullix]

I am afraid a reference cell in the SCD41 is principally impossible.

In an NDIR system a beam-splitter is used to guide the beam either to the ambient-gas cell, or through the reference-cell with a defined gas-mixture. All other parts exist only once. So there is a true intrinsic reference.

A reference in an SCD41 cell would be a complete second, and separate, additional system. Which needs its own calibration, defeating the purpose for a 2nd system.

Sensirion has their own NDIR system https://www.sensirion.com/de/umweltsensoren/kohlendioxidsensor/kohlendioxidsensoren-scd30/ (nicely showing the double-chamber) For both they see applications in air conditioning and greenhouses. But those are big systems, which don't benefit from an even smaller sensor. But would you want to need a weekly calibration, as the imply in their manual? (" ... sensor is exposed to the atmospheric CO 2 concentration of 400 ppm at least once per week.")

Cost-wise the their two systems (SCD41 and SCD30) are roughly on-par with USD 50 to 60 for each. It does not even matter for the above applications. And for Smartphone applications the sensor is still too big, unlike e.g. a BME280.

I don't see the marketing value.

Nevertheless, it is fun to play with this sensor. Here an example where I exhaled into an empty plastic bag with the sensor inside. Exhaled air should be up to 40000 ppm; I got 34000 ppm. My MiniMon NDIR system saturates near 10000 ppm. Pure CO2 is 1 Mio ppm.

The SCD41 yields a new value every 5 sec, which is still fast compared to MiniMon.

[ullix]

After these initial experiments it took on a life of its own, resulting in fully integrating all dongles (including a new one) and I2C devices into GeigerLog in its not-yet-released version 1.3.0.

For now I can offer a review on the performance of dongles and devices in GeigerLog, available as "Review of USB-To-I2C Dongles as used by GeigerLog" in my Articles folder https://sourceforge.net/projects/geigerlog/files/Articles/

I find the outcome very surprising - "Grand-daddy runs circles around the hip youngster". Comments welcome.

[Jean-Michel]

I had almost got I2Ctools working with SCD41 in the meantime.
it was very close to be working under windows, just a few linux specific command flags to be changed and some troubles with locating the right win64 library.

i suppose I'll switch to GeigerTool then, when you release the 1.3 version.
and when I get back to the office.

Sometimes a GUI is nice, but I also want to be able to call the log from an external Python program (I can still read the log files as they are being written though).

thank you.

[ullix]

I do my best to make GeigerLog ready for prime-time, but I 'm not there yet. I am sure you can extract the code from GeigerLog, just mind the license conditions.

What dongle are you actually using?

[Jean-Michel]

Hi,

just to let you know that I have modified your I2Cpytools code to work with Sensirion SHT75 + IOWarrior24 + Windows.

I know that SHT75 is obsolete and that you don't like Sensirion, but I write it nonetheless for reference.
SHT75 is not I2C compliant, so I had to create a special function to initialize it, and another one to interrogate it with a command byte inside a ReportID=3, which is the only way to get data from a subsequent read call with this sensor.

Here is a non-exhaustive list of changes:

• create I2Cpytools_Sensor_SHT7x.py
• add keyword parameters in IOWsetIOWtoI2Cmode(voltage_3o3V=False, sensibus=False), to optionally initialize the dongle with Sensirion-specific settings
• modify IOWaskDongle() to detect if the sensor is sensirion, and if it is, skip the first call to glob.iow.IOWwriteData(), so that no ReportID=2 is written; and then call the new method glob.iow.IOWreadCommand() instead of glob.iow.IOWinitializeRead()
• create IOWreadCommand(), which is actually almost identical to IOWinitializeRead(), except that instead of always sending the Sensor I2C address as the 'command' byte, it sends either the I2C address as before, or a user-supplied command byte to address 0, which is the way the SHT75 sensor works (because it is not I2C compliant, it requires a command byte but does not care about I2C address).
• changed the init call:

if glob.SHT75["dngl"] or glob.SHT71["dngl"]:
    glob.iow = IOW.IOWdongle(voltage_3o3V=True,sensibus=True)
  else:
    glob.iow = IOW.IOWdongle()

I also fixed some bugs:

• corrected a buggy call in IOWaskDongle(): fecprint("wrong report ID") => util.fecprint("wrong report ID")
• changed some methods so that if glob.debug is True, print traceback instead of Exception
• changed the signal handlers in the main file for windows (but not yet Util.signal_handler()):

if platform.system() == 'Windows':
    signal.signal(signal.SIGTERM,  util.signal_handler)
  else:

• added os.path.normpath() around file paths to make them work under windows (which uses backslash instead of slash in path names)
• changed the hour format in Util.py to use underscore _ instead of colon :, because the latter is a forbidden character in Windows paths.
• added some lines in to check whether the Python interpreter is 32bit or 64bits, because it defines where the DLL must be put (some people run a 32bit interpreter under a 64bit windows):

if sys.maxsize > 2**32:
    print('You are running a 64 bit Python interpreter')
else:
    print('You are running a 32 bit Python interpreter')

• I expect other windows-specific bugs when plotting; not tested yet

Next step will be to include the SCD41. As I understand, not only this sensor uses 16-bit commands, but it requires the CRC to be sent upon writing + a 500 ms sleep after some commands.

[ullix]

Jean,
quite a job, nice! I don't know what you plan to do with your code, but if you want to publish it, and have no place to do it, I'd be happy to post it here. Under your name, of course.

Did I read this right, you are doing all of this under Windows? Which iowarrior lib do you call, and how do you do it?

Using the IOWarrior24 may be something to think about, because it is deprecated. I checked with Code Mercenaries on their suggested replacement with IoWarrior28. It is not quite easy, but given the things you have managed, I am sure you can do it. Their support on my questions is certainly pleasing, see this forum post: IO-Warrior 28 replacing IO Warrior 24.

I have now settled for the USB-ISS. It is fast, easy to program, and compatible with all OS without installation. One strange thing: while it is recognized as a serial ACMx connection, it does not matter what baud (and other) settings you use, it transmits with the full USB 12MBit/s! Not sure how they do it, but they confirmed, and I verified.

Both the SCD30 and the SCD41 work quite well, actually on all dongles ELV, ISS, IOW, FTD, except the SCD30 not at all on the FTD. Their performance is quite respectable, I am in the process of summarizing.

[ullix]

I just uploaded my assessment of the SCD30, SCD41, and MiniMon devices for measuring CO2. In short, they all work for the intended purposes.

Download from the GeigerLog site: Measuring CO2 with SCD30, SCD41, and MiniMon

For something cross-platform capable and to experiment with, choose the SCD41 on the USB-ISS dongle.

Enjoy.

[ullix]

I have released GeigerLog 1.3.0, which has all the new I2C stuff (and lots more).

The dongle USB-ISS is my choice - the fastest, the easiest, and cross platform (Lin, Win Mac,...) compatible, even without the need for driver installation - but the ELV and IOW dongles are still supported. The FTD dongle is just not worth it, I removed it from GeigerLog.

Fully supported I2C devices in GeigerLog are: LM75 (temperature), BME280 (temperature, barometric pressure, humidity), TSL2591 (light sensor visible and infrared), SCD30 (CO2, temperature, humidity), SCD41 (CO2, temperature, humidity).

Extending this list is not too difficult. Any suggestion?

[Jean-Michel]

looks great!
i will test that next week.
tx

[Jean-Michel]

Ullix,

I have not tested GeigerLog yet, probably today I will.

In the meantime I have finished modifying I2CTools to work with the SCD4x and the SHT75. I dunno if you're still interested in that software but if that's the case, it would be good to test it back with the other sensors to be sure I haven't broken things.

I had to change the way dongles and sensors are stored in the glob file, so that a close() function can be automatically called upon shutdown(). Indeed, the SCD4x sensor has to be told to stop its periodical data retrieval when the dongle is closed, otherwise when the sensor is called again it will issue an error.

Also, the SCD4x only refresh its data every 5 sec, and if you try to read from it sooner, it will send an error. What's strange is that the "is_data_ready" function of the sensor returns "Yes" even though the sensor is not ready yet. So, I think that if you set a cycle time below 5 sec in the glob file, it will generate errors. On the whole this sensor will slow down global acquisition of multiple sensors. To handle that would correctly, it would require either: * a new loop in the main script to force wait 5 sec before interrogating the SDC4x (but the way loops are handled in that script do no make this modification easy) * a "is_sensor_ready" property in the Sensor object, that would be set to True only once 5 sec have elapsed since the last query. If False, then the main script would skip the call to the Get_All function

I have also added a forced_calibration method for the SCD4x. The problem is that it takes 3min (in compliance with the sensor datasheet) and thus blocks acquisition of other sensors data. To prevent that, we could use Python's asyncio module. But I wonder what would be the point of starting all sensors but one?

[ullix]

Jean-Michel,

I have my hands full with GeigerLog and other stuff. As I have spend much effort to the I2C code in the new GeigerLog 1.3, and I think shave straightened it out quite a bit, I wont be spending more efforts on the I2Ctools.

So far I am pleased with how my I2C code runs, and I don't see the problems you mention. In particular the "is_data_ready" function works just fine; it is key for how I use the sensor. I use a cycle time of mostly 1 sec, and simply return NAN values when the data are not ready.

Which Dongle are you using? My preference is the USB-ISS. By some miracle coding of the manufacturer one can connect it with 9600 baud, but it ignores the setting, and it is always using the maximum bandwidth of 12 MBit/s (USB "Fullspeed")! And it shows:

# measurement duration: SCD41: 
#   with dongle ISS:  CO2:829.0, Temp:24.1, Humid:32.0  duration:  4.3 ...  5.5 ms (avg: 4.8 ms)  1.0x
#   with dongle ELV:  CO2:922.0, Temp:24.5, Humid:31.8  duration: 11.7 ... 24.7 ms (avg:13.1 ms)  2,7x
#   with dongle IOW:  CO2:837.0, Temp:26.8, Humid:30.2  duration: 39.2 ... 40.4 ms (avg:39.9 ms)  8.3x
#   with dongle FTD:  CO2:837.0, Temp:26.8, Humid:30.2  duration: 65.8 ... 70.1 ms (avg:67.1 ms) 14.0x

I also used the SCD30 sensor, which is also for CO2 but based on the NDIR principle. Its handling is basically the same as with the SCD41, and it also works well.

I just think that both sensors are a bit too expensive for broad integration, while the typical applications (air conditioning, green houses) which easily can afford the price, won't care too much about miniaturization. I doubt commercial success; we'll see.

The SHT75/SHT85 are for Temp and Humidity. For these variables I use my darling BME280, which additionally offers barometric pressure. Programming is a bit complex, though, but it is done. And it is really fast: typically 3 ms for all 3 vars on the ISS!

I find the CO2 sensors a success for me, thanks for pointing it out. I am open for testing new sensors.

[Jean-Michel]

On Windows Geigerlog ask for admin rights, after the GUI window opens.

I'll run a step by step run of the program to identify where it is needed.

I think it is bad practice to require admin rights for a python program that does not even require installation

About I2CTools: what can I do with my modified version?
- keep it for myself?
- distribute it under the hood?
- publish it on Github?

I am not good with software copy rights.

[ullix]

GeigerLog will NEVER ASK FOR A PASSWORD !
There is a problem somewhere else, perhaps due to the Python installation?

For investigation please start GeigerLog with "geigerlog -dvwR devel" and send me the resulting geigerlog.stdlog file (find it in GL's data directory).

All 3 options are possible. Plus a 4th one: I'll put it on my site under Files as a "User contributed modification" with your name (or any other of your choice).

When you publish elsewhere, then please rename it, since you are free to use the code under the license (with reference to the original source), but not to "I2Cpytools", as this is considered kind of a trade mark.

[Jean-Michel]

sorry, I haven't been clear.
that's windows firewall that is blocking.
I'll do the test

[ullix]

Huh, why would the firewall act up when GeigerLog starts? The only thing related to net at start is the Monitor Server, but as a server it is just waiting for calls, and not reaching out to the internet.

I don't think it comes from GeigerLog, but if it does I would very much like to understand what is happening.

Some interesting data to share: I haven't calibrated my SCD30/41 sensors within the last few weeks, and I now run them for some other tests. The sensors sit in my office; I noticed this result, see pic.

The NDIR based SCD30 gets down to 400 ppm overnight; I would expect this, since it basically has its reference built in. But the SCD41 does not go below 650 ppm!

My MiniMon device (NDIR) currently shows 610 ppm (it had the lowest in the morning at 424 ppm), the SCD30 shows 588 ppm, the SCD41 is at 906 ppm. Clearly, the SCD41 needs calibration in regular intervals (weekly?), while the NDIRs may go for much periods. The MiniMon has even no provision for user calibration.

[ullix]

Just discovered: a CO2 sensor from Infineon based on the - ta,da! - Photoacoustic principle.
https://www.mouser.de/pdfDocs/Infineon-EVAL_PASCO2_SENSOR-DataSheet-v01_02-EN.pdf

at half-price from the SCDXYs
https://www.mouser.de/ProductDetail/Infineon-Technologies/EVALPASCO2MINIBOARDTOBO1?qs=sGAEpiMZZMv0NwlthflBi7sKZsPhjcFAG0Zyu4K4hvQ%3D

But it needs 3.3V and 12V supply :-(

[Jean-Michel]

yes, I was aware of it.
i didn't know about the 12V though. Maybe it uses a more powerful IR source and thus provides a better limit of detection (I haven't checked if that's the case).
I had been wondering about the automatic calibration procedure and I think that it takes the lowest recorded 3-hour average recorded within the 7 days as the CO2 baseline. It assumes this value to be 400 ppm and calibrates the sensor over that.
Meaning that if you live in Mexico or Bombay (where the CO2 value never goes down 1000 ppm; I'm joking but that's the idea), the sensot will underestimate the CO2 level.
On the contrary, if you store the sensor in a CO2 deprived area (that mat happen in an industrial process, in a N2 pressurized enclosure for instance), the sensor will overestimate the CO2 level.
That's what forced calibration is for I guess.

[ullix]

The datasheet is rather quiet on the automatic self calibration of the SCD41, I only see this:

The automatic self-calibration algorithm assumes that the sensor is exposed to the atmospheric CO2 concentration of 400 ppm at least once per week.

I have the auto-calibration NOT enabled, and use force-calib when needed. I did this 2 days ago, and all my three sensors (MiniMon, SCD30, SCD41) were in agreement. Two days later the SCD30 and MiniMon are still in agreement, while the SCD41 has already wandered off by well over 100 ppm. It does not reach the 400 ppm anymore, which the others do.

I'll set up a long term run, but I am becoming more and more suspicious on the reliability of the SCD41.

The NDIR systems need an initial calibration but then remain set. There is no user calibration for the MiniMon at all. These folks even claim a 5 year (!) Calibration interval https://www.epluse.com/fileadmin/data/product/ee895/Datenblatt_EE895.pdf for their I2C NDIR system

[ullix]

I was testing GeigerLog on Raspi 4, and was wondering: why not dump all these "USB-Dongles" and use Raspi as "I2C Dongle" and transmit the data by WiFi?

Tested with the LM75 sensor - it takes 5 lines of I2C specific code!

import smbus
I2C_TMP_LM75_ADRESS = 0x48              # I2C-Adresse des LM75 Temperatursensors (0x48)
LM75_TMP_BYTE       = 0x00              # Adresse im LM75, ab der Daten ausgelesen werden
i2c                 = smbus.SMBus(1)    # initialize I2C-Bus
raw = i2c.read_i2c_block_data(I2C_TMP_LM75_ADRESS, LM75_TMP_BYTE)
# raw has the data

Run time (of the raw line):
on Raspi: 3.3 ms.
on Desktop: with USB-ISS dongle: 1.4 ms, ELV dongle: 8.7 ms, IOW dongle: 13.6 ms, FTD dongle: 33.1 ms.