[Pvmanager-devel] PVManager and changes in the numeric types

[Carcassi, G. <car...@bn...>]

Hi all,

(Sorry for the cross posting on both CSS and PVManager lists.)

As part of the new release of pvManager, I am reviewing the numeric types.

The first requirement was to improve the ability to work on numeric types regardless of the specific type (byte, short, int, float, double). For scalar, we originally said we'd constraint to int and double, but we had no solution for arrays. I have been testing some new ideas which I think work much better.

For scalars: java already have a super class for numbers called Number. Regardless of the original type, you can use toDouble() or toInt() to get the value. I now have a VNumber which defines
    Number getValue();
VDouble, thanks to covariant type returns, overrides that to:
    Double getValue();
So, if you have a VDouble, you can work with that. Or, you could simply request a VNumber, use getValue().toDouble() and that would work regardless of the type. This is already implemented and tested, and it works better than what we had before.

In practical terms, this means you can write:
PVReader<VNumber> pv = PVManager.read(vNumber("mypv")).maxRate(ofMillis(100));
And forget about the concrete implementations.

For arrays: in epics.util we defined a series of classes that borrow some ideas from Number and from the Collection classes
http://pvmanager.sourceforge.net/EpicsUtil/apidocs/index.html
They can't be 100% compatible with the collection classes because of java generics issues. But the general idea is that you can have an IteratorNumber, which has:
boolean hasNext();
byte nextByte();
short nextShort();
...
We also have IteratorByte, IteratorShort, ... which implement IteratorNumber, so you can know which concrete implementation you have, in the same way that Byte, Short, ... implement Number.
Then we have collections that can return the iterator, CollectionNumber with CollectionXxx, lists for random access, ListNumber with Listxxx, and wrappers of arrays, ArrayNumber with ArrayXxx. The ArrayXxx are final classes, and using them directly gives no performance degradation (methods get inlined and is as if you are using the array directly). The other upshot, is that these can work as read-only views on arrays, making the code a lot more resistant. And we can provide standard computation on top of them.

The VTypes have been retrofitted to use this, so now we have a VNumberArray that defines:
    ListNumber getData();
VDouble overrides that to:
    ListDouble getData();
As before, if you have a VDoubleArray you can work with that directly. Or, you could simply request a VNumberArray, and work with any numeric type.
This is already implemented and tested, especially with Graphene, and it really works very well (I only write one binding to all numeric graphs). There is no more issues of converting types from one to the other. You can have read only views. You can make computation without having to copy data over and over.

In practical terms, this means you can write:
PVReader<VNumberArray> pv = PVManager.read(vNumberArray("mypv")).maxRate(ofMillis(100));
You request it to be an array of numbers, without caring too much of what number type.

Another requirement I got is the ability to see a scalar as an array of one. This is what some "physics/math" environment do, since arrays are usually n-dimensional, and a scalar is a 0-dimensional array. This would make the type system closer to that. There are some issues, though, and I am trying to understand whether I am shooting myself in the foot by trying to support that. So, I encourage you to think through this thing, and tell me I should not be doing it if you so feel.

The simple way to do this is simply to have VNumber extend VNumberArray. That is, a scalar number is simply an array of size 1. One problem is that, since we limited only to have Integer/Double on scalars, we do have Byte/Short/Int/Float/Double on array (for performance reasons). So far, arrays have always matched the type on the server. We have to choices:
1) Keep only Integer/Double for scalars. Some arrays now will not match the type (the ones that have length 1)
2) Implement all types for scalars. All arrays match. With the changes above, you can still use VNumber anyway for the general case.
My preference is the second at this point. The other (bigger) problem is for dynamic arrays. While an array of size 0 is fine, a scalar with no number is not. We have the following choices:
1) If the elementCount on the server is 1, I setup a monitor of fixed size 1, and always return a scalar (array of 1). This means: you cannot have a dynamic array of 0 or 1.
2) I always setup a monitor to dynamic size. If elementCount is 1, it means I can produce VNumbers, but if I get an array of 0, the VNumber.getValue() will return null. Which means, clients will need to test for null.
Not really happy with either choice, I have to say. Probably the second one is better because the behavior is more expected, while the first behavior seems more arbitrary. But I would always look for the channel elementCount to see whether it's a scalar (i.e. 1) or a proper array (i.e. >1).
This I haven't implemented yet, because of these concerns.

The Scalar and Array interfaces are, after these changes, of little use. My feeling is that they should be retired. So, I will most likely deprecate them.

The array types, like VDoubleArray, still return a direct access to the array. I am inclined to remove that, but have facilities in place to easily provide the array in case you need it (i.e. to use in some math library). There will be two options: get a copy or get the array without a copy. The second one may not always be available (e.g. type conversion, etc...), is unsafe (arrays are mutable), but provides the best performance. The first will always be available.

I think that should be it.

Gabriele