#5 Renard on Arduino with Lumos - Serial Purge Issue?

[C0D3C101]

Testing Lumos currently on Windows 7 machine, also have it installed on a Linux server. I am using an Arduino Mega 2650 controller that I programmed to accept the Renard Protocol. The Arduino functions as expected with receiving Renard data in Xlights and manually (HTERM). It does not respond to data from Lumos. I believe it may be due to "IOCTL_SERIAL_PURGE Purge: TXABORT RXABORT TXCLEAR RXCLEAR", specifically where it is in the process.

Serial Data as seen by portmon from Lumos:

11:18:01 PM python.exe IRP_MJ_CREATE USBSER000 SUCCESS Options: Open
11:18:01 PM python.exe IOCTL_SERIAL_SET_QUEUE_SIZE USBSER000 SUCCESS InSize: 4096 OutSize: 4096
11:18:01 PM python.exe IOCTL_SERIAL_GET_TIMEOUTS USBSER000 SUCCESS
11:18:01 PM python.exe IOCTL_SERIAL_SET_TIMEOUTS USBSER000 SUCCESS RI:0 RM:0 RC:0 WM:0 WC:0
11:18:01 PM python.exe IOCTL_SERIAL_SET_WAIT_MASK USBSER000 SUCCESS Mask: ERR
11:18:01 PM python.exe IOCTL_SERIAL_GET_BAUD_RATE USBSER000 SUCCESS
11:18:01 PM python.exe IOCTL_SERIAL_GET_LINE_CONTROL USBSER000 SUCCESS
11:18:01 PM python.exe IOCTL_SERIAL_GET_CHARS USBSER000 SUCCESS
11:18:01 PM python.exe IOCTL_SERIAL_GET_HANDFLOW USBSER000 SUCCESS
11:18:01 PM python.exe IOCTL_SERIAL_GET_BAUD_RATE USBSER000 SUCCESS
11:18:01 PM python.exe IOCTL_SERIAL_GET_LINE_CONTROL USBSER000 SUCCESS
11:18:01 PM python.exe IOCTL_SERIAL_GET_CHARS USBSER000 SUCCESS
11:18:01 PM python.exe IOCTL_SERIAL_GET_HANDFLOW USBSER000 SUCCESS
11:18:01 PM python.exe IOCTL_SERIAL_SET_BAUD_RATE USBSER000 SUCCESS Rate: 115200
11:18:01 PM python.exe IOCTL_SERIAL_SET_RTS USBSER000 SUCCESS
11:18:01 PM python.exe IOCTL_SERIAL_SET_DTR USBSER000 SUCCESS
11:18:01 PM python.exe IOCTL_SERIAL_SET_LINE_CONTROL USBSER000 SUCCESS StopBits: 1 Parity: NONE WordLength: 8
11:18:01 PM python.exe IOCTL_SERIAL_SET_CHAR USBSER000 SUCCESS EOF:0 ERR:0 BRK:0 EVT:0 XON:11 XOFF:13
11:18:01 PM python.exe IOCTL_SERIAL_SET_HANDFLOW USBSER000 SUCCESS Shake:1 Replace:40 XonLimit:0 XoffLimit:0
11:18:01 PM python.exe IOCTL_SERIAL_PURGE USBSER000 SUCCESS Purge: TXABORT RXABORT TXCLEAR RXCLEAR
11:18:01 PM python.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 2: 7E 80
11:18:01 PM python.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:18:01 PM python.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:18:01 PM python.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:18:01 PM python.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:18:01 PM python.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:18:01 PM python.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:18:01 PM python.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:18:01 PM python.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:18:02 PM python.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:18:02 PM python.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:18:02 PM python.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:18:02 PM python.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:18:02 PM python.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:18:02 PM python.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:18:02 PM python.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:18:02 PM python.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:18:02 PM python.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 18: 7E 80 FE 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
11:18:02 PM python.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 18: 7E 80 FE FE 00 00 00 00 00 00 00 00 00 00 00 00 00 00
11:18:03 PM python.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 18: 7E 80 FE FE FE 00 00 00 00 00 00 00 00 00 00 00 00 00
11:18:03 PM python.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 18: 7E 80 FE FE FE FE 00 00 00 00 00 00 00 00 00 00 00 00

Here is the same data set but sent in HTERM and the controller works properly:

11:35:07 PM HTerm.exe IOCTL_SERIAL_SET_BAUD_RATE USBSER000 SUCCESS Rate: 115200
11:35:07 PM HTerm.exe IOCTL_SERIAL_CLR_RTS USBSER000 SUCCESS
11:35:07 PM HTerm.exe IOCTL_SERIAL_CLR_DTR USBSER000 SUCCESS
11:35:07 PM HTerm.exe IOCTL_SERIAL_SET_LINE_CONTROL USBSER000 SUCCESS StopBits: 1 Parity: NONE WordLength: 8
11:35:07 PM HTerm.exe IOCTL_SERIAL_SET_CHAR USBSER000 SUCCESS EOF:0 ERR:0 BRK:0 EVT:0 XON:0 XOFF:0
11:35:07 PM HTerm.exe IOCTL_SERIAL_SET_HANDFLOW USBSER000 SUCCESS Shake:0 Replace:0 XonLimit:0 XoffLimit:0
11:35:07 PM HTerm.exe IOCTL_SERIAL_GET_BAUD_RATE USBSER000 SUCCESS
11:35:07 PM HTerm.exe IOCTL_SERIAL_GET_LINE_CONTROL USBSER000 SUCCESS
11:35:07 PM HTerm.exe IOCTL_SERIAL_GET_CHARS USBSER000 SUCCESS
11:35:07 PM HTerm.exe IOCTL_SERIAL_GET_HANDFLOW USBSER000 SUCCESS
11:35:07 PM HTerm.exe IOCTL_SERIAL_CLR_DTR USBSER000 SUCCESS
11:35:07 PM HTerm.exe IOCTL_SERIAL_CLR_RTS USBSER000 SUCCESS
11:35:12 PM HTerm.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 2: 7E 80
11:35:16 PM HTerm.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:35:17 PM HTerm.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:35:17 PM HTerm.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:35:17 PM HTerm.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:35:18 PM HTerm.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:35:19 PM HTerm.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:35:19 PM HTerm.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:35:20 PM HTerm.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:35:20 PM HTerm.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:35:20 PM HTerm.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:35:21 PM HTerm.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:35:22 PM HTerm.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:35:22 PM HTerm.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:35:23 PM HTerm.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 1: 00
11:35:34 PM HTerm.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 18: 7E 80 FE 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
11:35:36 PM HTerm.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 18: 7E 80 FE FE 00 00 00 00 00 00 00 00 00 00 00 00 00 00
11:35:38 PM HTerm.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 18: 7E 80 FE FE FE 00 00 00 00 00 00 00 00 00 00 00 00 00

Again, if I play a Vixen sequence in Xlights with it setup for a Renard controller, it works fine.

I would also like to add by default you have in the RenardContollerUnit.py script this line:

"def init(self, id, power_source, network, address=0, resolution=256, num_channels=64):"

This made the packet length at least 66 bytes long, longer if Renard escape bytes are sent.
In order to make sure I didn't overflow the Arduino's serial buffer which is only 64 (63) Bytes, I edited that line to reflect 16 channels and dropping the length down to a more manageable size of 18 Bytes (plus whatever escape bytes are sent but theoretically even if escape bytes are needed for all 16 channels in a packet, we are only at 34 Bytes, still very manageable)

If there were somewhat to set that above default number of channels for the controller based on the show config file where you set the other controller params up, that might be better?

Thanks for your hard work and it is my goal to use Lumos from a Linux server CLI to run a show in 2014!

[Steve Willoughby]

[Steve Willoughby]

Can you include the parts from your show config file which set up the serial port and controller parameters?

Lumos uses PySerial to manage the serial port. I believe the IOCTL_SERIAL_PURGE is part of how it opens and initializes the port before Lumos starts using it, but I'll check into that more.

I do see a couple of other differences in the two outputs. I do note that the port configurations look different:

Lumos:
IOCTL_SERIAL_SET_CHAR USBSER000 SUCCESS EOF:0 ERR:0 BRK:0 EVT:0 XON:11 XOFF:13
IOCTL_SERIAL_SET_HANDFLOW USBSER000 SUCCESS Shake:1 Replace:40 XonLimit:0 XoffLimit:0

HTerm:
IOCTL_SERIAL_SET_CHAR USBSER000 SUCCESS EOF:0 ERR:0 BRK:0 EVT:0 XON:0 XOFF:0
IOCTL_SERIAL_SET_HANDFLOW USBSER000 SUCCESS Shake:0 Replace:0 XonLimit:0 XoffLimit:0

You may want to try setting
xonxoff=no
rtscts=no

in the serial port configuration part of your show config file and see if that helps.

Also, you can set the number of Renard channels in the show configuration file, like this:

[unit renard1]
type=renard
num_channels=16

the code you refer to simply sets the default value in case you don't give it explicitly in the config file. However, in trying that just now, I discovered there is a bug in that part of the code for Renard controllers specifically, so I'll fix that and release an update.

[C0D3C101]

Here is the config section pertaining to the serial port:

[net leds]
description=leds
type=serial
port=2
bits=8
stop=1
baudrate=115200
xonxoff=no
rtscts=no
units=arduino

As for the XON:11 XOFF:13, Doesn't matter if I specify xonxoff or not, it still shows the same in portmon. Default says it's supposed to be off anyway. It's also set to no flow control on the port setting in windows. I've set Lumos up on my Ubuntu Server also, same exact way and behaves the same way as on windows though I do not have a way to monitor the serial data like in windows since the server is not loaded with a GUI.

Note, with XLights, the setup of the serial connection as seen by portmon is:

9:18:23 PM xScheduler.exe IOCTL_SERIAL_SET_BAUD_RATE USBSER000 SUCCESS Rate: 115200
9:18:23 PM xScheduler.exe IOCTL_SERIAL_CLR_RTS USBSER000 SUCCESS
9:18:23 PM xScheduler.exe IOCTL_SERIAL_CLR_DTR USBSER000 SUCCESS
9:18:23 PM xScheduler.exe IOCTL_SERIAL_SET_LINE_CONTROL USBSER000 SUCCESS StopBits: ERROR Parity: NONE WordLength: 8
9:18:23 PM xScheduler.exe IOCTL_SERIAL_SET_CHAR USBSER000 SUCCESS EOF:0 ERR:0 BRK:0 EVT:0 XON:11 XOFF:13
9:18:23 PM xScheduler.exe IOCTL_SERIAL_SET_HANDFLOW USBSER000 SUCCESS Shake:0 Replace:0 XonLimit:1536 XoffLimit:1536
9:18:23 PM xScheduler.exe IOCTL_SERIAL_SET_TIMEOUTS USBSER000 SUCCESS RI:-1 RM:0 RC:0 WM:0 WC:0
9:18:23 PM xScheduler.exe IOCTL_SERIAL_SET_QUEUE_SIZE USBSER000 SUCCESS InSize: 3072 OutSize: 6144
9:18:28 PM xScheduler.exe IOCTL_SERIAL_GET_COMMSTATUS USBSER000 SUCCESS
9:18:28 PM xScheduler.exe IRP_MJ_WRITE USBSER000 SUCCESS Length 18: 7E 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

and it does work with xlights just fine.

If it helps at all, I did find out that whenever a serial connection is made, the Arduino resets. It does this on both Linux and Windows. I am still looking into a way of getting around the reset but it seems that Xlights allows enough time between making the connection and sending the data from the sequence. Since I know some PHP, I have been playing around with PHP and the Arduino, sending it data formatted for the Renard Protocol, I simply have the PHP script delay like 3 - 5 seconds before sending it the data after it setups up the connection and it works just fine. I read that you can set a delay in Lumos to help with syncing the music but that doesn't seem to delay between setting up the connection and sending the data.

[Steve Willoughby]

[Steve Willoughby]

Looking through the code, I don't see anything that Lumos is doing in there that is directly influencing that timing. When you run lplay it opens the serial port as soon as the show configuration file is loaded, then reads the sequence file, building it into a list of scheduled events to send to the serial port. Assuming that the PySerial library which Lumos uses doesn't delay the actual hardware port opening until data starts arriving (which might be happening, I suppose--we could experiment with comparing lplay with the -s option vs. playing a sequence consisting of a single event (say, turning on an unused channel) with a long gap before the next event happens).

As far as a delay to allow the Arduino to reset, lplay does have a -s (--skew) option which adds a constant value (in milliseconds) to the scheduled event time for each event in the sequence. Assuming that PySerial actually opens the serial port when Lumos tells it to, that should translate directly to a delay between port opening and the first sequence arriving).

That would look something like:

$ lplay -s 1000 -c show.conf sequence.lseq

for a 1-second delay before the sequenced events started showing up.