Hardware address and timestamp endianess etc

[Robert]

Hello Professor Delbruck,

I was just chasing some more information on the hardware data output of the DVS128.

Currently my understanding is that the information for each event comes out in a 4 byte package, with byte 1 & 2 being the address bytes (byte 2 being the most significant byte) and bytes 3 & 4 being the timestamp (with byte 4 being the most significant byte of the timestamp)

From what I have read I think I may understand the bit order, but it appears to be giving me some very questionable data.

Firstly, I am working off the assumption that bit-15 is the most significant bit of each 2 byte package and bit 0 is the least significant bit

So for the 2byte/16 bit address

The most significant bit (bit 15) is the sync bit
The next 7 (14 -> 8) are the y co-ordinate
The next 7 (7 -> 1) are the x co-ordinate
The last bit (bit 0) is the polarity bit

For the 2 byte/ 16 bit timestamp

The most significant bit of the timestamp (bit 15) is the "sync" bit
The next bit (bit 14) is the "wrap" bit
The last 14 bits (bits 13 -> 0) are the timestamp).

I have searched on this forum find out which bit is what, but I am still having difficulty just determining for certain which bit belongs where and thought it would be better to ask you directly.

Thank you for your time,

Sincerely,

Robert

[Tobi Delbruck]

You got it Robert. The place to look is the class CypressFX2DVS128HardwareInterface.
There in the translateEvents method is the authoritative place to see how the raw USB data is converted to jAER AEPacketRaw packets of events with 32 bit timestamps and addresses.

Unfortunately this wiki is pretty lousy at editing or displaying code, but here is the method. The code display is chopped off rather than giving you a scrolling pane... Lame... You can see how after all the other checks are done, the address is extracted and the timestamp is built by combining the 16 bit timestamp with the wrapAdd value.

The wrapAdd is incremented by 14 bits every time a wrap event is received. These wrap events are sent even if there are no DVS events so that the time is maintained correctly on the host side.

Remember that java can only use signed integers and that is why byte values are ANDed with 0xff, because java does this in 32 bit integer and thus correctly handles the sign bit. E.g. negative byte values are cast to unsigned positive int values properly.

Best, Tobi

~~~~~~~~~~~~~~~~~~~~

:::java
* Does the translation, timestamp unwrapping and reset. Prints a message when a SYNC event is detected.
* @param b the raw buffer
/
@Override
protected void translateEvents(UsbIoBuf b) {

// System.out.println("buf has "+b.BytesTransferred+" bytes");
synchronized (aePacketRawPool) {
AEPacketRaw buffer = aePacketRawPool.writeBuffer();
// if(buffer.overrunOccuredFlag) return; // don't bother if there's already an overrun, consumer must get the events to clear this flag before there is more room for new events
int shortts;
// int NumberOfWrapEvents;
// NumberOfWrapEvents=0;

            byte[] aeBuffer = b.BufferMem;
            //            byte lsb,msb;
            int bytesSent = b.BytesTransferred;
            if (bytesSent % 4 != 0) {
                log.warning("CypressFX2.AEReader.translateEvents(): warning: " + bytesSent + " bytes sent, which is not multiple of 4");
                bytesSent = (bytesSent / 4) * 4; // truncate off any extra part-event
            }

            int[] addresses = buffer.getAddresses();
            int[] timestamps = buffer.getTimestamps();

            // write the start of the packet
            buffer.lastCaptureIndex = eventCounter;

            for (int i = 0; i < bytesSent; i += 4) {

// if(eventCounter>aeBufferSize-1){
// buffer.overrunOccuredFlag=true;
// // log.warning("overrun");
// return; // return, output event buffer is full and we cannot add any more events to it.
// //no more events will be translated until the existing events have been consumed by acquireAvailableEventsFromDriver
// }

                if ((aeBuffer[i + 3] & 0x80) == 0x80) { // timestamp bit 15 is one -> wrap
                    // now we need to increment the wrapAdd

                    wrapAdd += 0x4000L; //uses only 14 bit timestamps

                    //System.out.println("received wrap event, index:" + eventCounter + " wrapAdd: "+ wrapAdd);

// NumberOfWrapEvents++;
} else if ((aeBuffer[i + 3] & 0x40) == 0x40) { // timestamp bit 14 is one -> wrapAdd reset
// this firmware version uses reset events to reset timestamps
this.resetTimestamps();
if (resetTimestampWarningCount < RESET_TIMESTAMPS_INITIAL_PRINTING_LIMIT || resetTimestampWarningCount % RESET_TIMESTAMPS_WARNING_INTERVAL == 0) {
log.info(this + ".translateEvents got reset event from hardware, timestamp " + (0xffff & ((short) aeBuffer[i + 2] & 0xff | ((short) aeBuffer[i + 3] & 0x3f) << 8)));
}
if (resetTimestampWarningCount == RESET_TIMESTAMPS_INITIAL_PRINTING_LIMIT) {
log.warning("will only print reset timestamps message every " + RESET_TIMESTAMPS_WARNING_INTERVAL + " times now\nCould it be that you are trying to inject sync events using the DVS128 IN pin?\nIf so, select the \"Enable sync events output\" option in the DVS128 menu");
}
resetTimestampWarningCount++;
} else if ((eventCounter > aeBufferSize - 1) || (buffer.overrunOccuredFlag)) { // just do nothing, throw away events
buffer.overrunOccuredFlag = true;
} else {
// address is LSB MSB
addresses[eventCounter] = (int) ((aeBuffer[i] & 0xFF) | ((aeBuffer[i + 1] & 0xFF) << 8));

                    // same for timestamp, LSB MSB
                    shortts = (aeBuffer[i + 2] & 0xff | ((aeBuffer[i + 3] & 0xff) << 8)); // this is 15 bit value of timestamp in TICK_US tick

                    timestamps[eventCounter] = (int) (TICK_US * (shortts + wrapAdd)); //*TICK_US; //add in the wrap offset and convert to 1us tick

                    if (timestamps[eventCounter] < lastTimestampTmp) {
                        log.info("nonmonotonic timestamp: lastTimestamp=" + lastTimestampTmp + " timestamp=" + timestamps[eventCounter]);
                    }
                    lastTimestampTmp = timestamps[eventCounter];
                    // this is USB2AERmini2 or StereoRetina board which have 1us timestamp tick
                    if ((addresses[eventCounter] & SYNC_EVENT_BITMASK) != 0) {
                        if (printedSyncEventWarningCount < 10) {
                            if (printedSyncEventWarningCount < 10) {
                                log.info("sync event at timestamp=" + timestamps[eventCounter]);
                            } else {
                                log.warning("disabling further printing of sync events");
                            }
                            printedSyncEventWarningCount++;
                        }
                    }
                    eventCounter++;
                    buffer.setNumEvents(eventCounter);
                }
            } // end for

            // write capture size
            buffer.lastCaptureLength = eventCounter - buffer.lastCaptureIndex;
            buffer.systemModificationTimeNs=System.nanoTime();

            // if (NumberOfWrapEvents!=0) {
            //System.out.println("Number of wrap events received: "+ NumberOfWrapEvents);
            //}
            //System.out.println("wrapAdd : "+ wrapAdd);
        } // sync on aePacketRawPool

    }
}

~~~~~~~~~~~~~~~~~~~~~~

[Tobi Delbruck]

By the way, are you accessing the raw data from USB by the linux driver?
The above post was for the windows UsbIo driver.
For the linux driver, the hardware is sending the same data but the raw USB buffers are translated to AEPacketRaw by the method translateEvents_code in the class CypressFX2RetinaLinux.

You can locate these classes most quickly in netbeans with ctl-o and type the class name.

[Robert]

Yes, I am using the linux driver, and am managing to display in binary a data feed off the camera with a very simple program I have written in C (as this interfaces best with the device we are using). So just checking, every time the wrap bit is set (i.e. everytime the 14 bit timestamp overflows), the camera sends out an "event" to ensure that the "Wrap" is recorded? (i.e. we are getting "events" that aren't actually events every time the timestamp overflows?)

[Tobi Delbruck]

Yes, exactly. Otherwise the host would not be able to advance time. So
you will get one of these wrap events every 2^14us which is 16.38ms, or
about 61 per second.
Does this make sense? I hope I got the arithmetic correct.
Tobi

[Robert]

Oh wait... ok so when the "wrap" event is sent, the camera sends through zeros in all other bits of the timestamp except the wrap bit... which is set to one

[Tobi Delbruck]

Yes, but those zero bits are not guaranteed. At the moment only the wrap bit needs to be checked and the other bits are reserved and may be used in the future, with newer cameras.

[Robert]

Great! That's what we appear to be getting now (except that the wrap bit we realized was bit 15), and our timestamp actually looks more realistic as well! The problem we are now having is that every-time we read in the values from the camera through the linux driver into the buffer, the timestamp jumps ahead roughly 2000us, as if we are missing a chunk of events or something similar... I was wondering if this was a known bug with the linux driver or if you knew of others encountering this issue?

[Tobi Delbruck]

This may indeed be a problem of the current linux driver that it does not do overlapped IO, so that there may not be a buffer available to the host controller to write data to while the driver is processing the data in the buffer.
However we have not investigated this. We need to discuss.

Can you tell if the kernel-mode driver (the one you are using) has multiple buffers for the host controller to write to?

This problem (if it is really a problem) should be fixed by the new unified libUSB driver which is being developed now. This is an experimental driver at this point however.

Can you send a short data file to me separately? You can use the email tobi@ini.phys.ethz.ch. I guess that this file will not be in the .aedat format so it may be some work to interpret it.

[Robert]

As far as I can tell, the kernel-mode driver doesn't have multiple buffers for the host controller, I will send you the data files now. Also, there appears to be a point where the data just goes crazy... the wrap bit gets set to 1 continuously... and then the data returns to counting like normal for the rest of the program.

[Tobi Delbruck]

Could that business about going crazy (at what timestamp value does that occur?) possibly have something to do with handling signed/unsigned ints? On windows the timestamps wrap properly for days at a time and it is exactly the same hardware. It may be a problem with handling signed / unsigned in the linux driver.

[Robert]

I'm really not sure, but the "going crazy" errors seem to occur on event 149 read in, rather than a particular time stamp value, and then continues for a random number of events, before returning back to normal. Also, do you have an approximate release time for the new unified linux driver?