pvmanager-devel — Development discussion

Re: [Pvmanager-devel] alarm

[Dalesio, L. <da...@bn...>]

Redundancy of a transducer can be handled a number of ways - and I think that we understand none of them well enough to build in mechanisms to support it.

My favorite is this one
At the device layer --- read two addresses - and determine there if either or both are good. Forward the reading determined to be good. 

It is part of the hardware configuration, not part of a configurable database.

In any case, this is not something we are going to solve in the next 4 months.

Bob

Re: [Pvmanager-devel] alarm

[Andrew J. <an...@ap...>]

On Tuesday 31 August 2010 11:29:57 Ralph Lange wrote:
>
> One more point for UNDEFINED: the Gateway.
> If an IOC goes away, all direct clients would lose connection and thus
> change the severity to UNDEFINED.
> A client behind a Gateway does not lose the connection, but the Gateway
> would simply forward the UNDEFINED severity, and thus act transparently.

It is a mistake to confuse the meta-data about the state of the connection 
with the alarm severity from the device.  If a record is allowed to put itself 
in UNDEFINED severity (because its serial connection to the PLC is dead say) a 
client connected through a Gateway would have no way to distinguish between 
the IOC being down and the PLC's serial cable having been cut.

I think the Gateway needs a way to pass on its 'disconnected' status to the 
client (but without triggering another search), which should cause the 
connection callback to be run just as if the client were not using a Gateway 
at all.  I don't think you can use the alarm severity of the channel for 
reporting the connection status since the client may not even be monitoring 
channel alarms.

> "Maximize severity" is a well-established and widely used concept in
> EPICS 3.
> Moving that to the client will not work. In that case every client would
> have to know that a certain calc record is combining redundant signals
> and another one is a controller - that would replicate that information
> and allow for greatly inconsistent views of the system.
> How will a generic application (panel/alarm color mode) show a value
> that has (NO_ALARM & MINOR & MAJOR & INVALID)?

A generic application that can only display one severity (or a V3 one through 
a Gateway) would use the maximum severity bit reported, in your case INVALID 
(NO_ALARM is not a bit, it's defined as the value zero, meaning the complete 
absence of any set bits).  However a more capable application might show that 
the record is currently reporting MINOR, MAJOR and INVALID alarms.

- Andrew
-- 
The best FOSS code is written to be read by other humans -- Harald Welte

Re: [Pvmanager-devel] alarm

[Andrew J. <an...@ap...>]

On Tuesday 31 August 2010 06:48:00 Dalesio, Leo wrote:
> Pushing through any string to an operator has one problem -
> they are not managed and have no readily understood meaning.

In what way does presenting the operator with one of a limited set of alarm 
messages (as was originally described) make understanding an alarm any easier 
than a more specific error message written by the programmer who knows the 
most about the problem, i.e. from the original code that found it?

I don't believe any of the V3 EPICS display managers have used the alarm 
*status* of a channel in any way other than to display it in the PV Info 
dialog.  ALH does display and log the alarm status, but talking to Randy Flood 
our chief operator just now he says the only time the operators use the alarm 
status from ALH is when looking back at the log, and that a string description 
of the error might be just as helpful or even more so since it could contain 
more clues to the problem.

> In a use case ----
> A PV from a PLC - what do I want to know about it........
...
> There is not some endless set of conditions that have to be handled. A
> limited set per device support per record type is very achievable. My
> expectation is that there will be a large common set that will be used
> everywhere. Some handful will be needed to support devices.

The set may be limited, but in my experience as a driver writer it's usually a 
different set for every device, and there is often additional information that 
I want to include and log which could help with fixing the problem — for a 
communications failure I would want to include the text of the message 
received that I can't parse, for a limit violation the actual value that was 
out of range, for a disconnected link which of INPA-INPK it was, etc.

What could you do with the alarm classification information (which is how I 
think about what you're talking about) that you can't do with a string 
message?  I think that's the important question here; if it's just for logging 
then recording the full error string is better.

> A connection does not send across any unlimited set of unforeseen
> conditions.

But in some cases you can't know what those conditions might be in advance, 
especially if they're being reported to you by a manufacturer's poorly 
documented software (and by now you're probably using a later version than the 
original programmer had so it reports errors he didn't know about).  In those 
cases the driver writer might have to lump the errors from that library into a 
single classification, which is now almost useless.

> The error logging system, which was never formalized nor implemented, could
> be a new topic. There is need for it. Again though - some endless number of
> messages at least needs some standard information to make them useful.

I guess I'm saying that I see no real-time need for something like the V3 
alarm status, and a V4 alarm system would be much better if it incorporates 
the error logger rather than keeping the two separate.  A client like a 
display manager is usually only interested in the alarm severity, and it 
should not have to fetch the error strings if it doesn't want them.  The alarm 
manager however should be able to correlate alarms with error messages.

- Andrew
-- 
The best FOSS code is written to be read by other humans -- Harald Welte

Re: [Pvmanager-devel] alarm

[Ralph L. <Ral...@be...>]

  On 30.08.2010 18:02, Andrew Johnson wrote:
> On Monday 30 August 2010 15:11:34 Ralph Lange wrote:
>> The UNDEFINED enum option was just added to allow clients to distinguish
>> between:
>> * a working connection to a PV/record that has a value and set its
>> severity to INVALID (for one of several reasons, which can be found out
>> reading the status)
>> * no working connection to a PV/record
>> The alarm handler (maybe other clients) already implement this
>> additional severity, so we added it as it is needed.
 >
> Which is more severe, the Undefined status of a broken PV-link to a secondary
> temperature sensor on an IOC that got turned off for maintenance, or the Major
> alarm from the primary sensor indicating that the equipment is overheating?  I
> agree that this is a made-up example and that we would usually expect the
> application developer to ensure that the broken link would not override the
> overheating status, but I'm showing that Undefined is not always more severe,
> so it shouldn't necessarily overshadow a Major alarm.  The application
> designer is the only person who knows which matters more.

Excellent example, but it proves a different point. ;-)

The thing that is really special in your example is the *relation* 
between the two PVs, i.e. them being two separate sensors for the same 
signal. This is a redundancy example, and the point you are proving is 
that "maximize severity" is an inadequate algorithm for combining 
redundant signals. Which is definitely true. Thinking longer about this, 
you will probably have to create a new record type for combining 
redundant signals, and have fields to configure the combination 
algorithms for values as well as alarms:
Given three redundant analog inputs for the same signal, should the 
value be a) the lowest, b) the highest, c) the average? Should the 
severity be a) the lowest, b) the highest, c) the severity of the 
majority of inputs? There are valid use cases for each of these options.

[Actually, redundancy as a concept is only poorly supported in EPICS, 
and we should probably change that in EPICS 4 - but that's a different 
discussion.]

The traditionally more common use case is a controller, with its inputs 
being unrelated. A controller needs to have special actions for 
severities of INVALID or higher (IVOA etc.), and any input being INVALID 
must immediately put the controller into INVALID and execute the special 
actions. "Maximize severity" or the new "maximize severity INVALID" are 
perfectly adequate algorithms for that. UNDEFINED (i.e. an input with no 
connection) has to be treated the same as INVALID for a controller, as 
both mean the input value is unusable, and the result of the control 
algorithm is the same.

One more point for UNDEFINED: the Gateway.
If an IOC goes away, all direct clients would lose connection and thus 
change the severity to UNDEFINED.
A client behind a Gateway does not lose the connection, but the Gateway 
would simply forward the UNDEFINED severity, and thus act transparently.

> My point of suggesting using a bitset was that maybe the client should be the
> one that looks at the different outstanding alarm severities and messages; it
> could still use the Maximize Severity algorithm, but it doesn't have to.  In
> V3 we use a 16 bit enum to send 2 bits of severity information, having thrown
> away all data about lower priority alarms which might also exist; if you use a
> bitset, you can still send a 16 bit quantity, but now you're sending 3 or 4
> bits of severity information and letting the client decide what to discard.

"Maximize severity" is a well-established and widely used concept in 
EPICS 3.
Moving that to the client will not work. In that case every client would 
have to know that a certain calc record is combining redundant signals 
and another one is a controller - that would replicate that information 
and allow for greatly inconsistent views of the system.
How will a generic application (panel/alarm color mode) show a value 
that has (NO_ALARM & MINOR & MAJOR & INVALID)?

I do see a record having more than one alarm status (hence the bitset), 
but I think it should still have one severity.

Adding more algorithm choices to the severity inheritance might be useful.

Ralph

Re: [Pvmanager-devel] alarm

[Ralph L. <Ral...@be...>]

  On 31.08.2010 07:29, Marty Kraimer wrote:
>    What Andrew is describing sounds like an alarm and error reporting
> system rather than just a simple alarm system.
>
> [...]
> Let me ask another question.
>
>   From discussions about alarm handlers at various meetings I got the
> impression that an alarm handler system based on pushing alarms is a
> better solution than the EPICS alarm handler, which monitors a set of
> records specified in a configuration file.
>
> If this is true then do we want?
>
> 1) something really simple for normal clients.
> 2) Something more like Andrew describes for the alarm handler system.

I wouldn't say a pushing alarm system is a better solution, as that 
strictly depends on your installation.

Multiple institutes have created different versions of an alarm pushing 
framework, most notably Fermilab/D0 and DESY. Attempts to merge these 
different approches failed.

Creating a pushing alarm framework is an interesting task, but well 
beyond the horizon of this project. (I.e. if we start this, EPICS 4 will 
never be finished.)
If we start implementing the lower end "as we think it might be useful", 
it will end up not being used, since any existing framework will be 
different.

For EPICS 4 I think we should:
* Provide something simple that can be used with the generic clients 
(like the alarm handler)
* Provide the hooks necessary for an easy integration into a pushing 
alarm framework, without making assumptions of how such a framework will 
work.

Ralph

Re: [Pvmanager-devel] alarm

[Andrew J. <an...@ap...>]

On Tuesday 31 August 2010 07:27:06 Benjamin Franksen wrote:
>
> Let's talk about specification first. I think the client should never need
> to explicitly request from the server the string belonging to an id (and
> neither v.v.). Also, user code on the client side needs to know nothing
> about the whole issue.

Agreed this can/should be hidden from the client code completely.

> As I see it, the requirements are
>
> server:
>
> (1) should send each string only once (first time),
>     together with int to identify it
> (2) on later occasions sends only id
>
> client:
>
> (3) remembers associations (server,string,id)
> (4) user code calls client lib to get the actual string value
>
> Do we need servers to be able to invalidate id/string associations? I think
> it is not necessary:

True it's not /necessary/, but if a server knows that a particular string was 
a 1-off (i.e. it was generated using sprintf() and contains variable data that 
might never repeat exactly) it is nice for it to let a long-lived client such 
as an archiver know that it can release the cache for this string, rather than 
them both having to keep a copy of it in RAM and locking up that ID for the 
next 6 months until one of them reboots.

Without something like this the server has to look through all of its non-
fixed error strings every time a new sprintf()-generated string is to be sent 
(it would probably use a hash, but it still has to check for collisions) to 
see if this message is already in the cache.

An alternative would be to let the server send generated strings every time; 
the driver/record could indicate when it sets the alarm message whether this 
particular message should be cached or not, thus avoiding the need to support 
a cache-invalidate operation.

- Andrew
-- 
The best FOSS code is written to be read by other humans -- Harald Welte

Re: [Pvmanager-devel] alarm

[Benjamin F. <ben...@be...>]

On Tuesday, August 31, 2010, Marty Kraimer wrote:
> If a client asks to get/monitor a scalar double field together with
> alarm then the client gets LOTS of stuff in addition to the simple
> double value.
> 
> To prevent an excessive number of strings being passed Andrew provides
> 
> the suggestion:
> > In Data Access Jeff allows a client and/or server to dynamically add
> > new property names to their circuit at runtime; they get sent over the
> > wire once when they are needed, along with an ID number that will
> > refer to them in the future.
> 
> One way I see to implement this [...]

Let's talk about specification first. I think the client should never need 
to explicitly request from the server the string belonging to an id (and 
neither v.v.). Also, user code on the client side needs to know nothing 
about the whole issue.

As I see it, the requirements are

server:

(1) should send each string only once (first time),
    together with int to identify it
(2) on later occasions sends only id

client:

(3) remembers associations (server,string,id)
(4) user code calls client lib to get the actual string value

Do we need servers to be able to invalidate id/string associations? I think 
it is not necessary:

(5) if connection to a server breaks down, clients invalidate associations
    belong to this server

Although perhaps redundant, I would like to add for emphasis:

(6) for a given client, server must never send an id that has not been
    associated with a string to this client before

If client detects violation of (6) it means there is a protocol error and 
should be reported as such.

Cheers
Ben

Re: [Pvmanager-devel] alarm

[Dalesio, L. <da...@bn...>]

Pushing through any string to an operator has one problem -
they are not managed and have no readily understood meaning.

In a use case ----
A PV from a PLC - what do I want to know about it........

>From the PLC
Open circuit detect
Over range
Perhaps out of software range
violates interlock limit

>From the driver
No communication
No connection for this tag (point)
Loop time limit violated

>From the database
software limits

There is not some endless set of conditions that have to be handled. A limited set per device support per record type is very achievable. My expectation is that there will be a large common set that will be used everywhere. Some handful will be needed to support devices.

A connection does not send across any unlimited set of unforeseen conditions.

The error logging system, which was never formalized nor implemented, could be a new topic. There is need for it. Again though - some endless number of messages at least needs some standard information to make them useful.

Push or Pull
-- both should be supported. Push makes it possible to send them to many people over multicast. It's more efficient. Monitors in either case have problems. They are not reliable. How do you specify queues in the case of an alarm storm? How do we do this now? Do all queue grow as needed? How do you keep track of multicast listeners to make sure that they did not overrun their receive buffers? Polling is a good way to make sure you always see what's going on - but you miss very short alarm conditions. There are benefits and downfalls to both push and pull as well as monitor or poll. For now we have a monitor/pull system. Dropping that would be a huge mistake - as we have tools that depend on it. If we are going to add or change that model, a separate effort is needed. 

Shall we have a small meeting to discuss alarm systems?

Bob 

Re: [Pvmanager-devel] alarm

[Marty K. <mrk...@co...>]

  What Andrew is describing sounds like an alarm and error reporting 
system rather than just a simple alarm system.

Do we want this?
How will it be used?
     a) By ordinary display widgets. Consider just text and plot widgets.
     b) By an alarm handler.
     c) By an archiver.

If a client asks to get/monitor a scalar double field together with 
alarm then the client gets LOTS of stuff in addition to the simple 
double value.

To prevent an excessive number of strings being passed Andrew provides 
the suggestion:

> In Data Access Jeff allows a client and/or server to dynamically add new
> property names to their circuit at runtime; they get sent over the wire once
> when they are needed, along with an ID number that will refer to them in the
> future.

One way I see to implement this is to have the server keep a 
map<Integer,String). On the server code could register new strings via a 
call something like:

     int registerString(String);



Interface Channel could have some method like:

     String getString(int id);

The Channel implementation also keeps a map<Integer,String>. When 
getString is called it looks at it's local map and if found returns it. 
If not found it asks the server for it. But then a non-blocking version 
of getString is also required.

But this also means that clients must  know when to call getString.

But is this a good solution to?

> However my main objection was with the plan to force alarm status to be
> selected from a fixed set of strings.  That really limits what a driver writer
> can do if she has no way of knowing what error strings might come back from
> her network-connected device, or from the interface library that a device
> manufacturer gives her.

If this results in a call to registerString then the string value will 
either live in the map until the ioc is rebooted or else some way to 
remove elements from the map must exist. But consider the above case. 
Who will know when the string should be removed?

Let me ask another question.

 From discussions about alarm handlers at various meetings I got the 
impression that an alarm handler system based on pushing alarms is a 
better solution than the EPICS alarm handler, which monitors a set of 
records specified in a configuration file.

If this is true then do we want?

1) something really simple for normal clients.
2) Something more like Andrew describes for the alarm handler system.

Marty

On 08/30/2010 06:02 PM, Andrew Johnson wrote:
> On Monday 30 August 2010 15:11:34 Ralph Lange wrote:
>> The UNDEFINED enum option was just added to allow clients to distinguish
>> between:
>> * a working connection to a PV/record that has a value and set its
>> severity to INVALID (for one of several reasons, which can be found out
>> reading the status)
>> * no working connection to a PV/record
>> The alarm handler (maybe other clients) already implement this
>> additional severity, so we added it as it is needed.
> Which is more severe, the Undefined status of a broken PV-link to a secondary
> temperature sensor on an IOC that got turned off for maintenance, or the Major
> alarm from the primary sensor indicating that the equipment is overheating?  I
> agree that this is a made-up example and that we would usually expect the
> application developer to ensure that the broken link would not override the
> overheating status, but I'm showing that Undefined is not always more severe,
> so it shouldn't necessarily overshadow a Major alarm.  The application
> designer is the only person who knows which matters more.
>
> My point of suggesting using a bitset was that maybe the client should be the
> one that looks at the different outstanding alarm severities and messages; it
> could still use the Maximize Severity algorithm, but it doesn't have to.  In
> V3 we use a 16 bit enum to send 2 bits of severity information, having thrown
> away all data about lower priority alarms which might also exist; if you use a
> bitset, you can still send a 16 bit quantity, but now you're sending 3 or 4
> bits of severity information and letting the client decide what to discard.
>
>> By definition severity ("degree of seriousness") has an importance
>> ordering, which *must* be uniform throughout the system to allow a
>> consistent view of the alarm situation across devices.
>>
>> Should devices have their own set of severity terms and definitions,
>> device support must map the device's internal severity set to the EPICS
>> severity set.
> I agree, but an unconnected PV shouldn't always be mapped higher than Major.
>
>>> Note that severity is used by many widgets to display mote than just a
>>> string. For example  none is green, minor is yellow, major is red. What
>>> happens if severity is a bit set and what if it's choices are also
>>> extendable?
>> Exactly.
>> Also: What would "maximize severity" mean if any record can have the
>> severities in different order?
> I'm not saying that we should allow the severities to be re-ordered, I am
> objecting to adding Undefined as a severity that means not connected and
> putting it above Invalid.  Connection status needs to be filtered by the
> application to turn it into severity data.
>
> Combining two severity bitsets just means ORing them together; it's /simpler/
> than the maximize-severity algorithm.  Providing you define the bits properly
> you can also compare two severities using<  or>  to find which is more severe.
>
> Combining the status messages would be a little harder, but I can think of
> several possibilities.
>
>
> However my main objection was with the plan to force alarm status to be
> selected from a fixed set of strings.  That really limits what a driver writer
> can do if she has no way of knowing what error strings might come back from
> her network-connected device, or from the interface library that a device
> manufacturer gives her.
>
> In Data Access Jeff allows a client and/or server to dynamically add new
> property names to their circuit at runtime; they get sent over the wire once
> when they are needed, along with an ID number that will refer to them in the
> future.  Adopting this model for alarm status messages would be much better
> than requiring that all alarm strings be known and passed up to the client at
> connection time (slow and unnecessary).  A driver writer looking for high
> performance could register all their static alarm strings and just give an ID
> when posting an alarm, but you can also be flexible for drivers that can't do
> that (you would allow the server to withdraw or overwrite an ID so it can be
> re-used for a different alarm string in that case).
>
>
> Looking back, I see Marty discussing a "pvManager getPossibleAlarms method" as
> the reason for the static strings implementation; I'd want the author of that
> specification to explain why it's needed and so restrictive.
>
> - Andrew

Re: [Pvmanager-devel] alarm

[Andrew J. <an...@ap...>]

On Monday 30 August 2010 15:11:34 Ralph Lange wrote:
>
> The UNDEFINED enum option was just added to allow clients to distinguish
> between:
> * a working connection to a PV/record that has a value and set its
> severity to INVALID (for one of several reasons, which can be found out
> reading the status)
> * no working connection to a PV/record
> The alarm handler (maybe other clients) already implement this
> additional severity, so we added it as it is needed.

Which is more severe, the Undefined status of a broken PV-link to a secondary 
temperature sensor on an IOC that got turned off for maintenance, or the Major 
alarm from the primary sensor indicating that the equipment is overheating?  I 
agree that this is a made-up example and that we would usually expect the 
application developer to ensure that the broken link would not override the 
overheating status, but I'm showing that Undefined is not always more severe, 
so it shouldn't necessarily overshadow a Major alarm.  The application 
designer is the only person who knows which matters more.

My point of suggesting using a bitset was that maybe the client should be the 
one that looks at the different outstanding alarm severities and messages; it 
could still use the Maximize Severity algorithm, but it doesn't have to.  In 
V3 we use a 16 bit enum to send 2 bits of severity information, having thrown 
away all data about lower priority alarms which might also exist; if you use a 
bitset, you can still send a 16 bit quantity, but now you're sending 3 or 4 
bits of severity information and letting the client decide what to discard.

> By definition severity ("degree of seriousness") has an importance
> ordering, which *must* be uniform throughout the system to allow a
> consistent view of the alarm situation across devices.
>
> Should devices have their own set of severity terms and definitions,
> device support must map the device's internal severity set to the EPICS
> severity set.

I agree, but an unconnected PV shouldn't always be mapped higher than Major.

> > Note that severity is used by many widgets to display mote than just a
> > string. For example  none is green, minor is yellow, major is red. What
> > happens if severity is a bit set and what if it's choices are also
> > extendable?
>
> Exactly.
> Also: What would "maximize severity" mean if any record can have the
> severities in different order?

I'm not saying that we should allow the severities to be re-ordered, I am 
objecting to adding Undefined as a severity that means not connected and 
putting it above Invalid.  Connection status needs to be filtered by the 
application to turn it into severity data.

Combining two severity bitsets just means ORing them together; it's /simpler/ 
than the maximize-severity algorithm.  Providing you define the bits properly 
you can also compare two severities using < or > to find which is more severe.

Combining the status messages would be a little harder, but I can think of 
several possibilities.


However my main objection was with the plan to force alarm status to be 
selected from a fixed set of strings.  That really limits what a driver writer 
can do if she has no way of knowing what error strings might come back from 
her network-connected device, or from the interface library that a device 
manufacturer gives her.

In Data Access Jeff allows a client and/or server to dynamically add new 
property names to their circuit at runtime; they get sent over the wire once 
when they are needed, along with an ID number that will refer to them in the 
future.  Adopting this model for alarm status messages would be much better 
than requiring that all alarm strings be known and passed up to the client at 
connection time (slow and unnecessary).  A driver writer looking for high 
performance could register all their static alarm strings and just give an ID 
when posting an alarm, but you can also be flexible for drivers that can't do 
that (you would allow the server to withdraw or overwrite an ID so it can be 
re-used for a different alarm string in that case).


Looking back, I see Marty discussing a "pvManager getPossibleAlarms method" as 
the reason for the static strings implementation; I'd want the author of that 
specification to explain why it's needed and so restrictive.

- Andrew
-- 
The best FOSS code is written to be read by other humans -- Harald Welte

Re: [Pvmanager-devel] alarm

[Ralph L. <Ral...@be...>]

  On 30.08.2010 15:49, Marty Kraimer wrote:
> [...]
> In a separate message Andrew suggests that the severity also be a bitset
> instead of an enum.
>
> Do we really want this?
> Do we want the number of severity values to be extendable?

No! No!

Ehm ... I don't think this is practical nor necessary.

The UNDEFINED enum option was just added to allow clients to distinguish 
between:
* a working connection to a PV/record that has a value and set its 
severity to INVALID (for one of several reasons, which can be found out 
reading the status)
* no working connection to a PV/record
The alarm handler (maybe other clients) already implement this 
additional severity, so we added it as it is needed.

By definition severity ("degree of seriousness") has an importance 
ordering, which *must* be uniform throughout the system to allow a 
consistent view of the alarm situation across devices.

Should devices have their own set of severity terms and definitions, 
device support must map the device's internal severity set to the EPICS 
severity set.

> Note that severity is used by many widgets to display mote than just a
> string. For example  none is green, minor is yellow, major is red. What
> happens if severity is a bit set and what if it's choices are also
> extendable?

Exactly.
Also: What would "maximize severity" mean if any record can have the 
severities in different order?
In the sense of: if every input of a CALC record points to something 
with a different order of severities, which order does the calc record 
have? Would the calc record's severity actually mean anything when 
displayed in a panel?

>   Sounds like some rules are required.

I go for: fixed enum, no rules.

~Ralph

Re: [Pvmanager-devel] alarm

[Marty K. <mrk...@co...>]

  FOR pvmanager but not pvdata listeners

The discussion is about alarm severity and status. pvManager currently 
defines these as:
public enum AlarmSeverity {NONE,MINOR,MAJOR,INVALID,UNDEFINED{

public interface Alarm {
     AlarmSeverity getAlarmSeverity();
     Set<String> getAlarmStatus();  // get current enabled status values
     List<String> getPossibleAlarms(); // get all possible status values
}

The discussion is how can this be implemented efficently.

ALL,


On 08/30/2010 11:11 AM, Ralph Lange wrote:
>    Hi Marty,
>
> On 30.08.2010 09:13, Marty Kraimer wrote:
>>     I did not know if this message should go to the pvmanager mail list or
>> the pvdata mail list so I just picked pvdata.
> Please send to both lists in those cases, else people will be missing
> things.
> Deleting a double mail is a lot easier than reading one you didn't get.

OK starting with this reply.
>> My concern is the two methods:
>>
>> Set<String>   getAlarmStatus():
>> List<String>   getPossibleAlarms();
>>
>> I want to make the case that these should just be the single method:
>>
>> String getMessage();
>> [...]
> Please remember our discussions with Matej at the last meeting:
>
> Each status with its strings will be represented as an enum within
> pvAccess. So, in the worst case, the server sends the set of strings
> once per channel (at connection to a channel).
> Matej was also saying that if modules (device supports) define these
> string enums as immutable (and most of them will), they will be shared
> between channels, so that the server only sends the enum strings once
> per client (when the client connects to the first channel using that enum).
> In both cases, the updates will always be simple ints (bitsets).
>

Thanks for the reminder. I did forget. Sorry.
> I do not see any performance issue with this approach.
>
> Also, the bitset format for statuses was widely agreed-upon, and it
> certainly does *not* only apply to digital I/O. (Why should it? I really
> don't understand that point of your argumentation.)
> Complex devices may have a set of error conditions, combinations of
> which may be true at one moment. While the old (enum) approach was
> forcing EPICS to show just one error (e.g. "READ error", which tells you
> next to nothing), the bitset approach allows you to show a more
> realistic impression of your device's status.
>

OK I think I can start seeing an implementation. The trick is to get a 
set of immutable string choices but allow each support have a chance to 
add new choices.

Since Support has methods initialize and start what could be done is the 
following:

During initialize and during start any support module can add new 
choices. After a record instance has been started all alarm status 
fields will be set immutable. Since the pvAccess server will not allow 
client to connect to a record until after it is started then the 
pvAccess optimization can be used to limit the passing of strings.

Let me try again.

There is a need for a structure that defines a bitset structure in 
addition to the enumerated structure we have now in order to allow 
enabling of multiple statuses.
Let me come up with a proposed definition of a standard bitset structure 
and a proposed interface that support can call to set alarm severity 
choices and to raise alarms. Better to think about it overnight instead 
of making up something quickly.

In a separate message Andrew suggests that the severity also be a bitset 
instead of an enum.

Do we really want this?
Do we want the number of severity values to be extendable?

Note that severity is used by many widgets to display mote than just a 
string. For example  none is green, minor is yellow, major is red. What 
happens if severity is a bit set and what if it's choices are also 
extendable? Sounds like some rules are required.

Marty

> Cheers,
> Ralph
>
>
> ------------------------------------------------------------------------------
> Sell apps to millions through the Intel(R) Atom(Tm) Developer Program
> Be part of this innovative community and reach millions of netbook users
> worldwide. Take advantage of special opportunities to increase revenue and
> speed time-to-market. Join now, and jumpstart your future.
> http://p.sf.net/sfu/intel-atom-d2d
>

Re: [Pvmanager-devel] org.epics.pvmanager.data.Display

[Marty K. <mrk...@co...>]

  On 08/30/2010 10:28 AM, Carcassi, Gabriele wrote:
> Hi Marty,
>
>>   The method
>> NumberFormat getFormat();
>> Is very Java specific.
> Yes, it's intended. This way we can "hide" the different implementations
> for CA 3 and CA V. The data on the wire (and on the server) is indeed a
> String for pvData (the current precision for CA 3).
>


OK now I get it. Thus implement something like the following:

public class MumberFormatTest extends TestCase {

     public void testNumberFormat() {
         NumberFormat nf = new NumberFormatDouble("%12.2f");
         StringBuffer sb = new StringBuffer();
         FieldPosition fp = new FieldPosition(0);
         sb = nf.format(1.12, sb, fp);
         System.out.println(sb.toString());
         ParsePosition pp = new ParsePosition(0);
         double xxx = (nf.parse(sb.toString(),pp)).doubleValue();
         System.out.println("parse is "+ xxx);
     }

     static class NumberFormatDouble extends NumberFormat {
         private final String format;

         private NumberFormatDouble (String format) {
             this.format = format;
         }
         @Override
         public StringBuffer format(double arg0, StringBuffer 
arg1,FieldPosition arg2) {
             return arg1.append(String.format(format, arg0));
         }
         @Override
         public StringBuffer format(long arg0, StringBuffer 
arg1,FieldPosition arg2) {
             throw new IllegalArgumentException("long not supported");
         }
         @Override
         public Number parse(String arg0, ParsePosition arg1) {
             return Double.parseDouble(arg0.substring(arg1.getIndex()));
         }
     }
}

Marty

> Gabriele
>
> -----Original Message-----
> From: Marty Kraimer [mailto:mrk...@co...]
> Sent: Monday, August 30, 2010 10:14 AM
> To: pvm...@li...
> Subject: [Pvmanager-devel] org.epics.pvmanager.data.Display
>
>
>    The method
>
> NumberFormat getFormat();
>
> Is very Java specific.
>
> Instead how about
>
> String getFormat();
>
> Use C style formats. Examples are  "14.2e", "14.2f","10d"
>
> All of these will work on both c, c++, and Java (since java supports
> printf).
>
> Marty
>
> ------------------------------------------------------------------------
> ------
> Sell apps to millions through the Intel(R) Atom(Tm) Developer Program
> Be part of this innovative community and reach millions of netbook users
>
> worldwide. Take advantage of special opportunities to increase revenue
> and
> speed time-to-market. Join now, and jumpstart your future.
> http://p.sf.net/sfu/intel-atom-d2d
> _______________________________________________
> Pvmanager-devel mailing list
> Pvm...@li...
> https://lists.sourceforge.net/lists/listinfo/pvmanager-devel
>

Re: [Pvmanager-devel] org.epics.pvmanager.data.Display

[Carcassi, G. <car...@bn...>]

Hi Marty,

>  The method
>NumberFormat getFormat();
>Is very Java specific.
Yes, it's intended. This way we can "hide" the different implementations
for CA 3 and CA V. The data on the wire (and on the server) is indeed a
String for pvData (the current precision for CA 3).

Gabriele

-----Original Message-----
From: Marty Kraimer [mailto:mrk...@co...] 
Sent: Monday, August 30, 2010 10:14 AM
To: pvm...@li...
Subject: [Pvmanager-devel] org.epics.pvmanager.data.Display


  The method

NumberFormat getFormat();

Is very Java specific.

Instead how about

String getFormat();

Use C style formats. Examples are  "14.2e", "14.2f","10d"

All of these will work on both c, c++, and Java (since java supports 
printf).

Marty

------------------------------------------------------------------------
------
Sell apps to millions through the Intel(R) Atom(Tm) Developer Program
Be part of this innovative community and reach millions of netbook users

worldwide. Take advantage of special opportunities to increase revenue
and 
speed time-to-market. Join now, and jumpstart your future.
http://p.sf.net/sfu/intel-atom-d2d
_______________________________________________
Pvmanager-devel mailing list
Pvm...@li...
https://lists.sourceforge.net/lists/listinfo/pvmanager-devel

[Pvmanager-devel] org.epics.pvmanager.data.Display

[Marty K. <mrk...@co...>]

  The method

NumberFormat getFormat();

Is very Java specific.

Instead how about

String getFormat();

Use C style formats. Examples are  "14.2e", "14.2f","10d"

All of these will work on both c, c++, and Java (since java supports 
printf).

Marty

Re: [Pvmanager-devel] Partital Working Draft, Epics Scientific Service Interfaces

[Shen, G. <sh...@bn...>]

Hi Greg,
It will be nice if we can capture all detailed information.
How do you use those information? And how heavy?
How many applications do you already have to consume those info?
Could you please give an application example?

Guobao

White, Greg wrote:
> Thanks Nikolay, comments line,
>
> On Jul 29, 2010, at 7:13 PM, Malitsky, Nikolay D wrote:
>
>   
>> Hello,
>>
>> At this time, I have two comment about modeling part, particularly,
>> Twiss and Magnet readback.
>>
>> I am using the same set of Twiss parameters which are designed after and
>> consistent with MAD.
>> But I would replace new columns i)-v) with two entries:
>>
>> - element name
>> - s position from the beginning of this element
>>     
>
> Giving only S means that beamline elements which have >1 lattice slice (like Quads commonly, and Klystrons with > cavity) means that a client couldn't formally associate the data (twiss or R) with precisely which slice it associates with. 
>
> I agree my first proposal as detailed below was maybe too complicated for elements with > 1 slice. It's based on what we've done at SLAC for years. But I'd be happy to simplify it to this:
>
> 1) For thin elements (BPMs etc) return 1 record, marked "END".
>
> 2) For thick elements (QUADs etc), return 2 records for the first slice (for its beginning and end) and then 1 record for each slice thereafter. So, for instance a typical 2-slice quad with an embedded BPM would look like this:
>
>   1 QUAD:AREA1:234 BEG [data for entry to the first lattice slice of quad]
>   2 QUAD:AREA1:234 END [data for exit to the first lattice slice of quad]
>   3 BPM:AREA1:234  END [data for the BPM often embedded in the quad]
>   4 QUAD:AREA1:234 END [data for the exit of the 2nd lattice slice of the quad] 
>
> How's this? I think consensus at SLAC is this what we should have done. 
>
> Greg
>
>   
>> In this case, this header as well as other associated information would
>> be maintained and available separately. The approach will add
>> flexibility and portability, resolve mess with Begin/Mid/End, etc.
>>
>> About Magnet. From my point of view, physicists prefer the MAD strengths
>> rather than currents.
>> In this case, I like the RHIC approach using the dedicated middle layer
>> server which
>> provides a machine state of the element strengths. Designing a generic
>> solution is not easy.
>> I am working on that.
>>
>> Regards,
>> Nikolay
>>
>> -----Original Message-----
>> From: White, Greg [mailto:gr...@sl...]
>> Sent: Wednesday, July 28, 2010 7:50 AM
>> To: pvm...@li...
>> Subject: [Pvmanager-devel] Partital Working Draft,Epics Scientific
>> Service Interfaces
>>
>> [Partial working draft, July 28 2010]
>>
>> Colleagues,
>>
>> Please find below a partial working draft of a minimum set of 4
>> scientific service interfaces for accelerator applications;
>>  A) Modelling
>>  B) BPM Orbit
>>  C) Magnet readback/control
>>  D) Archive history.
>>
>> This documents the discussion in our meeting Tuesday 20th July 2010. As
>> such it only includes the basic data *returned* by a minimum set of
>> scientific services.
>>
>> A following draft will include the parameters required as *inputs*, a
>> more complete API for each service, and document the association between
>> data services and the applications which use them.
>>
>> These interfaces are based on those in existence in AIDA [1], where
>> further clarification can be sought.
>>
>>
>> A) Modelling
>> ============
>>
>> The following 3 output data types for modelling, are each parameterized
>> by an identifier which specifies the modelling run that produced the
>> model. The identifier most used the "GOLD", meaning the modelling run
>> identified by the control room operations, as most descriptive of the
>> present operating conditions, or which describes the objective
>> operational conditions. Different labs may have different names for this
>> "best" short term model.
>>
>> Required Data:
>>
>> 1. The 1st order transfer matrix.
>> ---------------------------------
>> This is the full plane coupled (ie the 6x6) matrix, from the beamline
>> beginning to each lattice element, given as a table. Ie 36 values for
>> each row.
>>
>> 1 row = 1 lattice element. That is, one beamline device may well
>> correspond to more than one lattice element eg, quads may well be split
>> into 2 lattice elements, other magnets may be >2 slices; one klystron
>> may be represented as a sequence of cavities, etc. Elements given in
>> phase advance beamline order from the beginning of the beamline.
>>
>> If the beamline is a transport line, then the 6x6 should be interpreted
>> as a linear transport matrix (i.e. an R aka T matrix). If the beamline
>> is a ring, then the 6x6 should be interpreted as a closed orbit transfer
>> matrix (i.e. a C matrix).
>>
>> Use cases: Basis for the system matrix of orbit correction software.
>> Also can be used for client side caching the whole model, making
>> repeated model operations faster than using the element by element
>> interface (3 below).
>>
>> Multiplicity: 1 such table for each modelling run of each beamline.
>>
>> Columns:
>>
>> i)    Ordinal position of the lattice slice described by this row (as
>> defined by phase advance)
>> ii)   EPICS device name, if there is one, of the lattice slice described
>> by this row
>> iii)  The MAD or otherwise offline modelling code name, if different to
>> ii.
>> iv)   S position of the lattice slice described by this row
>> v)    Where in the lattice element the row pertains to, "BEGIN|MID|END."
>>      + Default (ie for thin elements) should be END.
>>      + Thick elements, e.g. quadrupoles, will require >1 row, since the
>> transfer matrix will be required at least at their BEGINning, MIDdle,
>> and END. MIDdle means the end of the median lattice slice of the
>> element.
>>      + Klystrons with many cavities will need a convention. I propose
>> this convention, whose advantage is that it limits entries for klystrons
>> to at most 3 rows:
>>         - 1st row describes the entry of the 1st cavity, labeled BEG.
>>         - If == 1 cavity then
>>             2nd row describes the exit plan of the cavity, labelled
>> END.
>>           else if == 2 cavities then
>>             2nd row describes the end of the 1st cavity, labelled MID.
>>             3rd row described the end of the 2nd cavity, labelled END.
>>           else if > 2 cavities then
>>             2nd row is for the end of the medial cavity, labelled MID
>>             3rd row is the end of the last cavity, labelled END.
>>
>>      + Another perfectly reasonable convention would be that
>>        all cavities are included;
>>           1st row = entry to first cavity
>>           2nd and all subsequent rows describe the exit of each cavity.
>>
>> vi) The 36 elements of the 6x6 transfer matrix (R or C depending on
>> whether linac or circular).
>>
>> [Example in next draft.]
>>
>>
>> 2. The basic Twiss / aka Courant Snyder parameters.
>> --------------------------------------------------
>> Given as a table, for each lattice element, in phase advance beamline
>> order from the beginning of the beamline.
>>
>> Use cases: control room lattice diagnostics.
>>
>> Multiplicity: 1 such table for each modelling run of each beamline.
>>
>> Columns:
>>
>> i) - v) as above for the transfer matrix.
>>
>> v) The Twiss parms themselves: Kinetic Energy (GeV), psi x, beta x (m),
>> alpha x (rad), eta x (m), eta x' (rad), psi y, beta y (m), alpha y
>> (rad), eta y (m), eta y' (rad).
>>
>> vi) Slice Effective Length (m). Include this since only the model really
>> knows this, and
>> not worth creating a different interface for it.
>>
>> [Example in next draft]
>>
>>
>> 3. The transfer matrix (6x6) of a single given lattice element.
>> ---------------------------------------------------------------
>> Given as the 36 floats of the 6x6 transfer matrix of the given lattice
>> element, in row major order.
>>
>> Use cases: This is the basic interface for building bump software,
>> feedbacks etc. Eg this is the way you get the "R12" of a device.
>>
>> Multiplicity: 1 such 6x6 table for each lattice element in each
>> beamline, computed by a model run. So, for instance, there would be 1
>> for a BPM; but there would be 3 for a single quad split into 2 slices
>> (the entry of the first slice, the exit of the first slice, and the exit
>> of the second slice).
>>
>> If no "B" parameter is given, then the transfer matrix returned should
>> be the R matrix from the the beamline start to the given lattice element
>> (or in case of circular machine, the C matrix from injection to given
>> element).
>>
>> If a B parameter is given, the transfer matrix returned should be the R
>> matrix from the given device to B.
>>
>>
>> B. BPM Orbit
>> ============
>>
>> Given as a table of the X and Y offsets from the calibrated center of
>> each beam monitor (BPMs and Toroids). Each value should be accompanied
>> by an RMS, giving the measured error. Valued 0 if only 1 reading was
>> made.
>>
>> Note, this interface is independent of the question of whether all BPM
>> readings were taken on the same pulse, or whether the RMS is over
>> consecutive pulses/turns of ring etc. This same output interface can be
>> used.
>>
>> Use cases: BPM orbit plots, orbit fitting, orbit correction.
>>
>> Columns:
>>
>> [columns i - iv as A.1 above]
>> i)    Ordinal position of the lattice slice described by this row (as
>> defined by phase advance)
>> ii)   EPICS device name, if there is one, of the lattice slice described
>> by this row
>> iii)  The MAD or otherwise offline modelling code name, if different to
>> ii.
>> iv)   S position (m) of the lattice slice described by this row
>> v)    X offset (mm) (null if Y only bpm)
>> vi)   Y offset (mm) (null if x only bpm)
>> vii)  TMIT (coulombs)
>> viii) X rms (null if Y only bpm)
>> ix)   Y rms (null if x only bpm)
>> x)    BPM status (codified cause of most important severity or alarm).
>>
>> Note that a client might well merge the above table with a table of all
>> beamline devices by S (not included here), so that it can display an
>> orbit plot that includes beamline obstructions, stoppers etc.
>>
>> For AIDA example see [2].
>>
>>
>> C. Magnet readback/control
>> ==========================
>>
>> Given as a table of magnet B fields [I'm not sure we can push B on
>> places that operate in KG/m etc. Also not sure all places integrate
>> field strength of different magnet types equally.]
>>
>> Use case: operational diagnostics. Orbit correction.
>>
>> [Columns i - iv as A.1 above]
>> i)    Ordinal position of the magnet by this row (as defined by phase
>> advance)
>> ii)   EPICS device name, if there is one, of the magnet.
>> iii)  The MAD or otherwise offline modelling code name, if different to
>> ii.
>> iv)   S position of the magnet
>> v)    B field
>>
>> B field may be the B "desired" (the control setpoint), B "actual", or B
>> "configured" (the last loaded configuration value, eg SCORE CON),
>> depending on an input parameter, eg (field=DES or field=CON).
>>
>> The above data type may be used for magnet readback or control.
>>
>>
>> D. Archive/History
>> ==================
>>
>> The archived data points for a given device.
>>
>> Given as a table of timestamped values, containing a rectangular matrix
>> of 5 arrays:
>>
>> Columns:
>> i)   the values (double)
>> ii)  the times as strings "dd-mmm-yyyy hh:mm:ss" (matlab dateform '0',
>> suitable for datenum);
>> iii) repeat count;
>> iv)  the times as numeric timestamps in UNIX system time format
>> v)   the pulse id;
>> vi)  the count (1 for a scalar and greater than 1 for a waveform-- count
>> may be used to index into the values array for waveforms to extract the
>> waveform values for each sample
>> vii) a flag indicating whether the times represent Daylight Savings
>> Time.
>>
>> For example see [3].
>>
>>
>> E. Klystron
>> ===========
>> [To follow in next draft - for now see example in
>> http://www.slac.stanford.edu/grp/cd/soft/aida/slcKlysDpGuide.html]
>>
>>
>> Assumptions and Definitions.
>> ============================
>>
>> At each facility a naming convention will map EPICS PVs to beamline
>> devices (say the first 3 fields of the EPICS PV name identify the
>> device), and some other field identifies a property of the device. Eg In
>> PV name "XCOR:LI21:245:BDES," XCOR:LI21:245 identifies a device, and
>> BDES identifies a property.
>>
>> Element. A modelled element in the MAD sense. A element may correspond
>> to an EPICS device, but not necessarily. Eg Mad Element "X245" may be
>> EPICS device "XCOR:LI21:245". However, there may well be elements that
>> have no corresponding EPICS device, eg "MARK4" might be a modelled
>> marker point, for which no EPICS PV is needed. *So, don't expect all
>> rows in a table of all the R-matrices of an accelerator to map to a PV,
>> and for those that do, don't expect only 1 row to match a single PV.
>>
>> References
>> ==========
>> [1] Accelerator Independent Data Access (AIDA) [1],
>> http://www.slac.stanford.edu/grp/cd/soft/aida/. See in particular links
>> under the section "Individual Service Data Users Guides."
>>
>> [2] AIDA SLC Bpm Data Provider Guide
>> http://www.slac.stanford.edu/grp/cd/soft/aida/slcBpmDpGuide.html
>>
>> [3] AIDA EPICS Channel Archiver Data Provider Guide
>> http://www.slac.stanford.edu/grp/cd/soft/aida/epicsChannelArchiverDpGuid
>> e.html
>>
>>
>>
>>
>> ------------------------------------------------------------------------
>> ------
>> The Palm PDK Hot Apps Program offers developers who use the
>> Plug-In Development Kit to bring their C/C++ apps to Palm for a share
>> of $1 Million in cash or HP Products. Visit us here for more details:
>> http://ad.doubleclick.net/clk;226879339;13503038;l?
>> http://clk.atdmt.com/CRS/go/247765532/direct/01/
>> _______________________________________________
>> Pvmanager-devel mailing list
>> Pvm...@li...
>> https://lists.sourceforge.net/lists/listinfo/pvmanager-devel
>>     
>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by 
>
> Make an app they can't live without
> Enter the BlackBerry Developer Challenge
> http://p.sf.net/sfu/RIM-dev2dev 
> _______________________________________________
> Pvmanager-devel mailing list
> Pvm...@li...
> https://lists.sourceforge.net/lists/listinfo/pvmanager-devel
>
>   

Re: [Pvmanager-devel] Partital Working Draft, Epics Scientific Service Interfaces

[White, G. <gr...@sl...>]

Thanks Nikolay, comments line,

On Jul 29, 2010, at 7:13 PM, Malitsky, Nikolay D wrote:

> Hello,
> 
> At this time, I have two comment about modeling part, particularly,
> Twiss and Magnet readback.
> 
> I am using the same set of Twiss parameters which are designed after and
> consistent with MAD.
> But I would replace new columns i)-v) with two entries:
> 
> - element name
> - s position from the beginning of this element

Giving only S means that beamline elements which have >1 lattice slice (like Quads commonly, and Klystrons with > cavity) means that a client couldn't formally associate the data (twiss or R) with precisely which slice it associates with. 

I agree my first proposal as detailed below was maybe too complicated for elements with > 1 slice. It's based on what we've done at SLAC for years. But I'd be happy to simplify it to this:

1) For thin elements (BPMs etc) return 1 record, marked "END".

2) For thick elements (QUADs etc), return 2 records for the first slice (for its beginning and end) and then 1 record for each slice thereafter. So, for instance a typical 2-slice quad with an embedded BPM would look like this:

  1 QUAD:AREA1:234 BEG [data for entry to the first lattice slice of quad]
  2 QUAD:AREA1:234 END [data for exit to the first lattice slice of quad]
  3 BPM:AREA1:234  END [data for the BPM often embedded in the quad]
  4 QUAD:AREA1:234 END [data for the exit of the 2nd lattice slice of the quad] 

How's this? I think consensus at SLAC is this what we should have done. 

Greg

> 
> In this case, this header as well as other associated information would
> be maintained and available separately. The approach will add
> flexibility and portability, resolve mess with Begin/Mid/End, etc.
> 
> About Magnet. From my point of view, physicists prefer the MAD strengths
> rather than currents.
> In this case, I like the RHIC approach using the dedicated middle layer
> server which
> provides a machine state of the element strengths. Designing a generic
> solution is not easy.
> I am working on that.
> 
> Regards,
> Nikolay
> 
> -----Original Message-----
> From: White, Greg [mailto:gr...@sl...]
> Sent: Wednesday, July 28, 2010 7:50 AM
> To: pvm...@li...
> Subject: [Pvmanager-devel] Partital Working Draft,Epics Scientific
> Service Interfaces
> 
> [Partial working draft, July 28 2010]
> 
> Colleagues,
> 
> Please find below a partial working draft of a minimum set of 4
> scientific service interfaces for accelerator applications;
>  A) Modelling
>  B) BPM Orbit
>  C) Magnet readback/control
>  D) Archive history.
> 
> This documents the discussion in our meeting Tuesday 20th July 2010. As
> such it only includes the basic data *returned* by a minimum set of
> scientific services.
> 
> A following draft will include the parameters required as *inputs*, a
> more complete API for each service, and document the association between
> data services and the applications which use them.
> 
> These interfaces are based on those in existence in AIDA [1], where
> further clarification can be sought.
> 
> 
> A) Modelling
> ============
> 
> The following 3 output data types for modelling, are each parameterized
> by an identifier which specifies the modelling run that produced the
> model. The identifier most used the "GOLD", meaning the modelling run
> identified by the control room operations, as most descriptive of the
> present operating conditions, or which describes the objective
> operational conditions. Different labs may have different names for this
> "best" short term model.
> 
> Required Data:
> 
> 1. The 1st order transfer matrix.
> ---------------------------------
> This is the full plane coupled (ie the 6x6) matrix, from the beamline
> beginning to each lattice element, given as a table. Ie 36 values for
> each row.
> 
> 1 row = 1 lattice element. That is, one beamline device may well
> correspond to more than one lattice element eg, quads may well be split
> into 2 lattice elements, other magnets may be >2 slices; one klystron
> may be represented as a sequence of cavities, etc. Elements given in
> phase advance beamline order from the beginning of the beamline.
> 
> If the beamline is a transport line, then the 6x6 should be interpreted
> as a linear transport matrix (i.e. an R aka T matrix). If the beamline
> is a ring, then the 6x6 should be interpreted as a closed orbit transfer
> matrix (i.e. a C matrix).
> 
> Use cases: Basis for the system matrix of orbit correction software.
> Also can be used for client side caching the whole model, making
> repeated model operations faster than using the element by element
> interface (3 below).
> 
> Multiplicity: 1 such table for each modelling run of each beamline.
> 
> Columns:
> 
> i)    Ordinal position of the lattice slice described by this row (as
> defined by phase advance)
> ii)   EPICS device name, if there is one, of the lattice slice described
> by this row
> iii)  The MAD or otherwise offline modelling code name, if different to
> ii.
> iv)   S position of the lattice slice described by this row
> v)    Where in the lattice element the row pertains to, "BEGIN|MID|END."
>      + Default (ie for thin elements) should be END.
>      + Thick elements, e.g. quadrupoles, will require >1 row, since the
> transfer matrix will be required at least at their BEGINning, MIDdle,
> and END. MIDdle means the end of the median lattice slice of the
> element.
>      + Klystrons with many cavities will need a convention. I propose
> this convention, whose advantage is that it limits entries for klystrons
> to at most 3 rows:
>         - 1st row describes the entry of the 1st cavity, labeled BEG.
>         - If == 1 cavity then
>             2nd row describes the exit plan of the cavity, labelled
> END.
>           else if == 2 cavities then
>             2nd row describes the end of the 1st cavity, labelled MID.
>             3rd row described the end of the 2nd cavity, labelled END.
>           else if > 2 cavities then
>             2nd row is for the end of the medial cavity, labelled MID
>             3rd row is the end of the last cavity, labelled END.
> 
>      + Another perfectly reasonable convention would be that
>        all cavities are included;
>           1st row = entry to first cavity
>           2nd and all subsequent rows describe the exit of each cavity.
> 
> vi) The 36 elements of the 6x6 transfer matrix (R or C depending on
> whether linac or circular).
> 
> [Example in next draft.]
> 
> 
> 2. The basic Twiss / aka Courant Snyder parameters.
> --------------------------------------------------
> Given as a table, for each lattice element, in phase advance beamline
> order from the beginning of the beamline.
> 
> Use cases: control room lattice diagnostics.
> 
> Multiplicity: 1 such table for each modelling run of each beamline.
> 
> Columns:
> 
> i) - v) as above for the transfer matrix.
> 
> v) The Twiss parms themselves: Kinetic Energy (GeV), psi x, beta x (m),
> alpha x (rad), eta x (m), eta x' (rad), psi y, beta y (m), alpha y
> (rad), eta y (m), eta y' (rad).
> 
> vi) Slice Effective Length (m). Include this since only the model really
> knows this, and
> not worth creating a different interface for it.
> 
> [Example in next draft]
> 
> 
> 3. The transfer matrix (6x6) of a single given lattice element.
> ---------------------------------------------------------------
> Given as the 36 floats of the 6x6 transfer matrix of the given lattice
> element, in row major order.
> 
> Use cases: This is the basic interface for building bump software,
> feedbacks etc. Eg this is the way you get the "R12" of a device.
> 
> Multiplicity: 1 such 6x6 table for each lattice element in each
> beamline, computed by a model run. So, for instance, there would be 1
> for a BPM; but there would be 3 for a single quad split into 2 slices
> (the entry of the first slice, the exit of the first slice, and the exit
> of the second slice).
> 
> If no "B" parameter is given, then the transfer matrix returned should
> be the R matrix from the the beamline start to the given lattice element
> (or in case of circular machine, the C matrix from injection to given
> element).
> 
> If a B parameter is given, the transfer matrix returned should be the R
> matrix from the given device to B.
> 
> 
> B. BPM Orbit
> ============
> 
> Given as a table of the X and Y offsets from the calibrated center of
> each beam monitor (BPMs and Toroids). Each value should be accompanied
> by an RMS, giving the measured error. Valued 0 if only 1 reading was
> made.
> 
> Note, this interface is independent of the question of whether all BPM
> readings were taken on the same pulse, or whether the RMS is over
> consecutive pulses/turns of ring etc. This same output interface can be
> used.
> 
> Use cases: BPM orbit plots, orbit fitting, orbit correction.
> 
> Columns:
> 
> [columns i - iv as A.1 above]
> i)    Ordinal position of the lattice slice described by this row (as
> defined by phase advance)
> ii)   EPICS device name, if there is one, of the lattice slice described
> by this row
> iii)  The MAD or otherwise offline modelling code name, if different to
> ii.
> iv)   S position (m) of the lattice slice described by this row
> v)    X offset (mm) (null if Y only bpm)
> vi)   Y offset (mm) (null if x only bpm)
> vii)  TMIT (coulombs)
> viii) X rms (null if Y only bpm)
> ix)   Y rms (null if x only bpm)
> x)    BPM status (codified cause of most important severity or alarm).
> 
> Note that a client might well merge the above table with a table of all
> beamline devices by S (not included here), so that it can display an
> orbit plot that includes beamline obstructions, stoppers etc.
> 
> For AIDA example see [2].
> 
> 
> C. Magnet readback/control
> ==========================
> 
> Given as a table of magnet B fields [I'm not sure we can push B on
> places that operate in KG/m etc. Also not sure all places integrate
> field strength of different magnet types equally.]
> 
> Use case: operational diagnostics. Orbit correction.
> 
> [Columns i - iv as A.1 above]
> i)    Ordinal position of the magnet by this row (as defined by phase
> advance)
> ii)   EPICS device name, if there is one, of the magnet.
> iii)  The MAD or otherwise offline modelling code name, if different to
> ii.
> iv)   S position of the magnet
> v)    B field
> 
> B field may be the B "desired" (the control setpoint), B "actual", or B
> "configured" (the last loaded configuration value, eg SCORE CON),
> depending on an input parameter, eg (field=DES or field=CON).
> 
> The above data type may be used for magnet readback or control.
> 
> 
> D. Archive/History
> ==================
> 
> The archived data points for a given device.
> 
> Given as a table of timestamped values, containing a rectangular matrix
> of 5 arrays:
> 
> Columns:
> i)   the values (double)
> ii)  the times as strings "dd-mmm-yyyy hh:mm:ss" (matlab dateform '0',
> suitable for datenum);
> iii) repeat count;
> iv)  the times as numeric timestamps in UNIX system time format
> v)   the pulse id;
> vi)  the count (1 for a scalar and greater than 1 for a waveform-- count
> may be used to index into the values array for waveforms to extract the
> waveform values for each sample
> vii) a flag indicating whether the times represent Daylight Savings
> Time.
> 
> For example see [3].
> 
> 
> E. Klystron
> ===========
> [To follow in next draft - for now see example in
> http://www.slac.stanford.edu/grp/cd/soft/aida/slcKlysDpGuide.html]
> 
> 
> Assumptions and Definitions.
> ============================
> 
> At each facility a naming convention will map EPICS PVs to beamline
> devices (say the first 3 fields of the EPICS PV name identify the
> device), and some other field identifies a property of the device. Eg In
> PV name "XCOR:LI21:245:BDES," XCOR:LI21:245 identifies a device, and
> BDES identifies a property.
> 
> Element. A modelled element in the MAD sense. A element may correspond
> to an EPICS device, but not necessarily. Eg Mad Element "X245" may be
> EPICS device "XCOR:LI21:245". However, there may well be elements that
> have no corresponding EPICS device, eg "MARK4" might be a modelled
> marker point, for which no EPICS PV is needed. *So, don't expect all
> rows in a table of all the R-matrices of an accelerator to map to a PV,
> and for those that do, don't expect only 1 row to match a single PV.
> 
> References
> ==========
> [1] Accelerator Independent Data Access (AIDA) [1],
> http://www.slac.stanford.edu/grp/cd/soft/aida/. See in particular links
> under the section "Individual Service Data Users Guides."
> 
> [2] AIDA SLC Bpm Data Provider Guide
> http://www.slac.stanford.edu/grp/cd/soft/aida/slcBpmDpGuide.html
> 
> [3] AIDA EPICS Channel Archiver Data Provider Guide
> http://www.slac.stanford.edu/grp/cd/soft/aida/epicsChannelArchiverDpGuid
> e.html
> 
> 
> 
> 
> ------------------------------------------------------------------------
> ------
> The Palm PDK Hot Apps Program offers developers who use the
> Plug-In Development Kit to bring their C/C++ apps to Palm for a share
> of $1 Million in cash or HP Products. Visit us here for more details:
> http://ad.doubleclick.net/clk;226879339;13503038;l?
> http://clk.atdmt.com/CRS/go/247765532/direct/01/
> _______________________________________________
> Pvmanager-devel mailing list
> Pvm...@li...
> https://lists.sourceforge.net/lists/listinfo/pvmanager-devel

Re: [Pvmanager-devel] Partital Working Draft, Epics Scientific Service Interfaces

[White, G. <gr...@sl...>]

Right, as Ralph was pointing out, there is a difference between statistically reduced data that is then archived, raw data which is reduced by the archiver as part of the archiving process, and archived data which is reduced by the archive server as part of making a reply.

I think common practice is to reduce data at the stage of archiving it - at least to record repeat counts of same valued data. Eg, if a PV's value doesn't change between archiver polls/monitor callbacks, we just increase the repeat count (and don't bother recording the times at which those same values were found). It's this "repeat count" that I included in my interface definition.

So I'll add to the interface also a way to include what Bob wants - a way for the archiver to tell a client that what it recorded was itself reduced data, and the meta data of that reduction (eg Standard deviation).

Cheers
Greg
 
On Jul 29, 2010, at 10:55 AM, Ralph Lange wrote:

> On 29.07.2010 12:43, Dalesio, Leo wrote:
>> 
>> Before going further with the proposal to filter, let's check the performance is actually network bound.
>> --- imagine I want to look at one year of temperature variation. Computation over network bandwidth seems obvious. More importantly though, I would like to introduce the archiving of statistical samples - this would allow me to have my bpm fpga compute the statistical information on turn by turn data that I could then archive or study for errant behavoir.
>> 
> 
> Note that in this case - the BPM FPGA calculates the statistical values - those values would simply be the raw archived numbers.
> 
> Which is different from the proposal that had the statistics calculated by the archive retrieval daemon when sending binned data back to the client.
> (Except that for your case - where the raw data are statistical values - the algorithm that calculates statistics on these when binning might have to be different.)
> 
> Ralph
> 
> ps. Please: send answers only to the mailing list, and configure your mail client to use the standard "> " quoting.
> ------------------------------------------------------------------------------
> The Palm PDK Hot Apps Program offers developers who use the
> Plug-In Development Kit to bring their C/C++ apps to Palm for a share
> of $1 Million in cash or HP Products. Visit us here for more details:
> http://p.sf.net/sfu/dev2dev-palm_______________________________________________
> Pvmanager-devel mailing list
> Pvm...@li...
> https://lists.sourceforge.net/lists/listinfo/pvmanager-devel

Re: [Pvmanager-devel] Data structure update

[White, G. <gr...@sl...>]

+1. 

Cool. Thanks Gabriele. 

Also, I'm sorry to be away from these discussions for so long. The usual excuses apply. 

Greg

On Aug 6, 2010, at 12:29 PM, Carcassi, Gabriele wrote:

> Ok, to close this thread: I’ll call the data structures VXxx. The main reason is that it makes all data interfaces be below all the other interfaces, so the javadocs looks nicer. :-) I am starting to rename the classes and move the pvmanager in org.epics.pvmanager.
>  
> Gabriele
>  
> From: Ralph Lange [mailto:Ral...@be...] 
> Sent: Friday, July 23, 2010 1:44 PM
> To: pvm...@li...
> Subject: Re: [Pvmanager-devel] Data structure update
>  
> +1
> 
> BTW - on a different topic:
> We can safely assume that everyone that is posting on the list is also subscribed to it (as the list rejects postings from non-members).
> Please reply to the list only to avoid the inbox noise of double messages.
> 
> Thanks,
> Ralph
> 
> 
> On 23.07.2010 13:31, Carcassi, Gabriele wrote:
> Hi Bob:
>  
> The classes are going to be used for data gathered through Epics 3 too… So I would not put 4 in the name.
>  
> Gabriele
>  
> From: Dalesio, Leo 
> Sent: Friday, July 23, 2010 10:00 AM
> To: Marty Kraimer; pvm...@li...
> Subject: Re: [Pvmanager-devel] Data structure update
>  
>  
> Two new proposals:
> >
> > 1. E4xx (as in Epics)
> >
> > 2. CD4xx (as in Client Data)
> >
> > 3. EV4xx (as in Epics Value, or Epics Variable)
> >
> > 4. VX4x (as in Variable)
> >
> 
>  
>  
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Sprint
> What will you do first with EVO, the first 4G phone?
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
>  
>  
> _______________________________________________
> Pvmanager-devel mailing list
> Pvm...@li...
> https://lists.sourceforge.net/lists/listinfo/pvmanager-devel
>  
> --
> Ralph Lange                            Bldg.902-B, Rm.18 tel: x8060
> NSLS-II Controls Group                                   fax: x8085
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by 
> 
> Make an app they can't live without
> Enter the BlackBerry Developer Challenge
> http://p.sf.net/sfu/RIM-dev2dev _______________________________________________
> Pvmanager-devel mailing list
> Pvm...@li...
> https://lists.sourceforge.net/lists/listinfo/pvmanager-devel

Re: [Pvmanager-devel] Data structure update

[Carcassi, G. <car...@bn...>]

Ok, to close this thread: I’ll call the data structures VXxx. The main reason is that it makes all data interfaces be below all the other interfaces, so the javadocs looks nicer. :-) I am starting to rename the classes and move the pvmanager in org.epics.pvmanager.

 

Gabriele

 

From: Ralph Lange [mailto:Ral...@be...] 
Sent: Friday, July 23, 2010 1:44 PM
To: pvm...@li...
Subject: Re: [Pvmanager-devel] Data structure update

 

+1

BTW - on a different topic:
We can safely assume that everyone that is posting on the list is also subscribed to it (as the list rejects postings from non-members).
Please reply to the list only to avoid the inbox noise of double messages.

Thanks,
Ralph


On 23.07.2010 13:31, Carcassi, Gabriele wrote: 

Hi Bob:

 

The classes are going to be used for data gathered through Epics 3 too… So I would not put 4 in the name.

 

Gabriele

 

From: Dalesio, Leo 
Sent: Friday, July 23, 2010 10:00 AM
To: Marty Kraimer; pvm...@li...
Subject: Re: [Pvmanager-devel] Data structure update

 

 

Two new proposals:
>
> 1. E4xx (as in Epics)
>
> 2. CD4xx (as in Client Data)
>
> 3. EV4xx (as in Epics Value, or Epics Variable)
>
> 4. VX4x (as in Variable)
>

 
 
------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
 
 
_______________________________________________
Pvmanager-devel mailing list
Pvm...@li...
https://lists.sourceforge.net/lists/listinfo/pvmanager-devel

 

-- 

Ralph Lange                            Bldg.902-B, Rm.18 tel: x8060
NSLS-II Controls Group                                   fax: x8085

Re: [Pvmanager-devel] Partital Working Draft, Epics Scientific Service Interfaces

[Malitsky, N. D <mal...@bn...>]

Hello,

At this time, I have two comment about modeling part, particularly,
Twiss and Magnet readback.

I am using the same set of Twiss parameters which are designed after and
consistent with MAD.
But I would replace new columns i)-v) with two entries:

- element name
- s position from the beginning of this element

In this case, this header as well as other associated information would 
be maintained and available separately. The approach will add 
flexibility and portability, resolve mess with Begin/Mid/End, etc. 

About Magnet. From my point of view, physicists prefer the MAD strengths
rather than currents. 
In this case, I like the RHIC approach using the dedicated middle layer
server which 
provides a machine state of the element strengths. Designing a generic
solution is not easy.
I am working on that.

Regards,
Nikolay

-----Original Message-----
From: White, Greg [mailto:gr...@sl...] 
Sent: Wednesday, July 28, 2010 7:50 AM
To: pvm...@li...
Subject: [Pvmanager-devel] Partital Working Draft,Epics Scientific
Service Interfaces

[Partial working draft, July 28 2010]

Colleagues, 

Please find below a partial working draft of a minimum set of 4
scientific service interfaces for accelerator applications;
  A) Modelling
  B) BPM Orbit
  C) Magnet readback/control
  D) Archive history. 

This documents the discussion in our meeting Tuesday 20th July 2010. As
such it only includes the basic data *returned* by a minimum set of
scientific services. 

A following draft will include the parameters required as *inputs*, a
more complete API for each service, and document the association between
data services and the applications which use them.

These interfaces are based on those in existence in AIDA [1], where
further clarification can be sought.


A) Modelling
============

The following 3 output data types for modelling, are each parameterized
by an identifier which specifies the modelling run that produced the
model. The identifier most used the "GOLD", meaning the modelling run
identified by the control room operations, as most descriptive of the
present operating conditions, or which describes the objective
operational conditions. Different labs may have different names for this
"best" short term model.

Required Data:

1. The 1st order transfer matrix. 
---------------------------------
This is the full plane coupled (ie the 6x6) matrix, from the beamline
beginning to each lattice element, given as a table. Ie 36 values for
each row.

1 row = 1 lattice element. That is, one beamline device may well
correspond to more than one lattice element eg, quads may well be split
into 2 lattice elements, other magnets may be >2 slices; one klystron
may be represented as a sequence of cavities, etc. Elements given in
phase advance beamline order from the beginning of the beamline. 

If the beamline is a transport line, then the 6x6 should be interpreted
as a linear transport matrix (i.e. an R aka T matrix). If the beamline
is a ring, then the 6x6 should be interpreted as a closed orbit transfer
matrix (i.e. a C matrix).

Use cases: Basis for the system matrix of orbit correction software.
Also can be used for client side caching the whole model, making
repeated model operations faster than using the element by element
interface (3 below).

Multiplicity: 1 such table for each modelling run of each beamline.

Columns:

i)    Ordinal position of the lattice slice described by this row (as
defined by phase advance)
ii)   EPICS device name, if there is one, of the lattice slice described
by this row
iii)  The MAD or otherwise offline modelling code name, if different to
ii.
iv)   S position of the lattice slice described by this row
v)    Where in the lattice element the row pertains to, "BEGIN|MID|END."
      + Default (ie for thin elements) should be END. 
      + Thick elements, e.g. quadrupoles, will require >1 row, since the
transfer matrix will be required at least at their BEGINning, MIDdle,
and END. MIDdle means the end of the median lattice slice of the
element.
      + Klystrons with many cavities will need a convention. I propose
this convention, whose advantage is that it limits entries for klystrons
to at most 3 rows:
         - 1st row describes the entry of the 1st cavity, labeled BEG.
         - If == 1 cavity then 
             2nd row describes the exit plan of the cavity, labelled
END.
           else if == 2 cavities then 
             2nd row describes the end of the 1st cavity, labelled MID.
             3rd row described the end of the 2nd cavity, labelled END.
           else if > 2 cavities then
             2nd row is for the end of the medial cavity, labelled MID
             3rd row is the end of the last cavity, labelled END.

      + Another perfectly reasonable convention would be that 
        all cavities are included;
           1st row = entry to first cavity
           2nd and all subsequent rows describe the exit of each cavity.

vi) The 36 elements of the 6x6 transfer matrix (R or C depending on
whether linac or circular).

[Example in next draft.]


2. The basic Twiss / aka Courant Snyder parameters. 
--------------------------------------------------
Given as a table, for each lattice element, in phase advance beamline
order from the beginning of the beamline. 

Use cases: control room lattice diagnostics. 

Multiplicity: 1 such table for each modelling run of each beamline.

Columns:

i) - v) as above for the transfer matrix.

v) The Twiss parms themselves: Kinetic Energy (GeV), psi x, beta x (m),
alpha x (rad), eta x (m), eta x' (rad), psi y, beta y (m), alpha y
(rad), eta y (m), eta y' (rad).

vi) Slice Effective Length (m). Include this since only the model really
knows this, and 
not worth creating a different interface for it.

[Example in next draft]


3. The transfer matrix (6x6) of a single given lattice element. 
---------------------------------------------------------------
Given as the 36 floats of the 6x6 transfer matrix of the given lattice
element, in row major order.

Use cases: This is the basic interface for building bump software,
feedbacks etc. Eg this is the way you get the "R12" of a device.

Multiplicity: 1 such 6x6 table for each lattice element in each
beamline, computed by a model run. So, for instance, there would be 1
for a BPM; but there would be 3 for a single quad split into 2 slices
(the entry of the first slice, the exit of the first slice, and the exit
of the second slice). 

If no "B" parameter is given, then the transfer matrix returned should
be the R matrix from the the beamline start to the given lattice element
(or in case of circular machine, the C matrix from injection to given
element).

If a B parameter is given, the transfer matrix returned should be the R
matrix from the given device to B. 


B. BPM Orbit
============

Given as a table of the X and Y offsets from the calibrated center of
each beam monitor (BPMs and Toroids). Each value should be accompanied
by an RMS, giving the measured error. Valued 0 if only 1 reading was
made.

Note, this interface is independent of the question of whether all BPM
readings were taken on the same pulse, or whether the RMS is over
consecutive pulses/turns of ring etc. This same output interface can be
used.

Use cases: BPM orbit plots, orbit fitting, orbit correction.

Columns:

[columns i - iv as A.1 above]
i)    Ordinal position of the lattice slice described by this row (as
defined by phase advance)
ii)   EPICS device name, if there is one, of the lattice slice described
by this row
iii)  The MAD or otherwise offline modelling code name, if different to
ii.
iv)   S position (m) of the lattice slice described by this row
v)    X offset (mm) (null if Y only bpm)
vi)   Y offset (mm) (null if x only bpm)
vii)  TMIT (coulombs) 
viii) X rms (null if Y only bpm)
ix)   Y rms (null if x only bpm)
x)    BPM status (codified cause of most important severity or alarm). 

Note that a client might well merge the above table with a table of all
beamline devices by S (not included here), so that it can display an
orbit plot that includes beamline obstructions, stoppers etc.

For AIDA example see [2].


C. Magnet readback/control
==========================

Given as a table of magnet B fields [I'm not sure we can push B on
places that operate in KG/m etc. Also not sure all places integrate
field strength of different magnet types equally.]  

Use case: operational diagnostics. Orbit correction. 

[Columns i - iv as A.1 above]
i)    Ordinal position of the magnet by this row (as defined by phase
advance)
ii)   EPICS device name, if there is one, of the magnet. 
iii)  The MAD or otherwise offline modelling code name, if different to
ii.
iv)   S position of the magnet
v)    B field
      
B field may be the B "desired" (the control setpoint), B "actual", or B
"configured" (the last loaded configuration value, eg SCORE CON),
depending on an input parameter, eg (field=DES or field=CON). 

The above data type may be used for magnet readback or control.


D. Archive/History
==================

The archived data points for a given device.

Given as a table of timestamped values, containing a rectangular matrix
of 5 arrays: 

Columns:
i)   the values (double)
ii)  the times as strings "dd-mmm-yyyy hh:mm:ss" (matlab dateform '0',
suitable for datenum); 
iii) repeat count; 
iv)  the times as numeric timestamps in UNIX system time format
v)   the pulse id; 
vi)  the count (1 for a scalar and greater than 1 for a waveform-- count
may be used to index into the values array for waveforms to extract the
waveform values for each sample
vii) a flag indicating whether the times represent Daylight Savings
Time. 

For example see [3].


E. Klystron
===========
[To follow in next draft - for now see example in
http://www.slac.stanford.edu/grp/cd/soft/aida/slcKlysDpGuide.html]


Assumptions and Definitions.
============================

At each facility a naming convention will map EPICS PVs to beamline
devices (say the first 3 fields of the EPICS PV name identify the
device), and some other field identifies a property of the device. Eg In
PV name "XCOR:LI21:245:BDES," XCOR:LI21:245 identifies a device, and
BDES identifies a property.

Element. A modelled element in the MAD sense. A element may correspond
to an EPICS device, but not necessarily. Eg Mad Element "X245" may be
EPICS device "XCOR:LI21:245". However, there may well be elements that
have no corresponding EPICS device, eg "MARK4" might be a modelled
marker point, for which no EPICS PV is needed. *So, don't expect all
rows in a table of all the R-matrices of an accelerator to map to a PV,
and for those that do, don't expect only 1 row to match a single PV.  

References
==========
[1] Accelerator Independent Data Access (AIDA) [1],
http://www.slac.stanford.edu/grp/cd/soft/aida/. See in particular links
under the section "Individual Service Data Users Guides."

[2] AIDA SLC Bpm Data Provider Guide
http://www.slac.stanford.edu/grp/cd/soft/aida/slcBpmDpGuide.html

[3] AIDA EPICS Channel Archiver Data Provider Guide
http://www.slac.stanford.edu/grp/cd/soft/aida/epicsChannelArchiverDpGuid
e.html




------------------------------------------------------------------------
------
The Palm PDK Hot Apps Program offers developers who use the
Plug-In Development Kit to bring their C/C++ apps to Palm for a share 
of $1 Million in cash or HP Products. Visit us here for more details:
http://ad.doubleclick.net/clk;226879339;13503038;l?
http://clk.atdmt.com/CRS/go/247765532/direct/01/
_______________________________________________
Pvmanager-devel mailing list
Pvm...@li...
https://lists.sourceforge.net/lists/listinfo/pvmanager-devel

Re: [Pvmanager-devel] Partital Working Draft, Epics Scientific Service Interfaces

[Ralph L. <Ral...@be...>]

  On 29.07.2010 12:43, Dalesio, Leo wrote:
>
> Before going further with the proposal to filter, let's check the 
> performance is actually network bound.
> --- imagine I want to look at one year of temperature variation. 
> Computation over network bandwidth seems obvious. More importantly 
> though, I would like to introduce the archiving of statistical samples 
> - this would allow me to have my bpm fpga compute the statistical 
> information on turn by turn data that I could then archive or study 
> for errant behavoir.
>

Note that in this case - the BPM FPGA calculates the statistical values 
- those values would simply be the raw archived numbers.

Which is different from the proposal that had the statistics calculated 
by the archive retrieval daemon when sending binned data back to the client.
(Except that for your case - where the raw data are statistical values - 
the algorithm that calculates statistics on these when binning might 
have to be different.)

Ralph

ps. Please: send answers only to the mailing list, and configure your 
mail client to use the standard "> " quoting.

Re: [Pvmanager-devel] Partital Working Draft, Epics Scientific Service Interfaces

[Dalesio, L. <da...@bn...>]

Before going further with the proposal to filter, let's check the performance is actually network bound.
--- imagine I want to look at one year of temperature variation. Computation over network bandwidth seems obvious. More importantly though, I would like to introduce the archiving of statistical samples - this would allow me to have my bpm fpga compute the statistical information on turn by turn data that I could then archive or study for errant behavoir.

Re: [Pvmanager-devel] Partital Working Draft, Epics Scientific Service Interfaces

[Carcassi, G. <car...@bn...>]

>Before going further with the proposal to filter,
>let's check the performance is actually network bound.
Even if we stream all data from the archiver, it's helpful if these
structures are computed. For creating a plot, for example, one typically
needs to do that... if you are zooming in and out, for example, you will
typically _want_ all data to be on your client, but still create the
statistical information to be displayed... So, we need the data
structure definition, and I assume it's the VStatistical...

>>What about strings, enums, waveforms?
>Suggest not doing strings. Enums as ints.
You can't average enums as ints... Statistical information for Strings
and enums is histograms. Whether we want to support them or not I don't
know... For graphing purposes, I'd probably want to plot both values at
the same time slot.

Gabriele

Re: [Pvmanager-devel] Partital Working Draft, Epics Scientific Service Interfaces

[White, G. <gr...@sl...>]

Hi, comments inline


Sent from my iPad

On Jul 28, 2010, at 10:23 AM, "Ralph Lange" <Ral...@be...<mailto:Ral...@be...>> wrote:

On 28.07.2010 08:39, Dalesio, Leo wrote:

This sends the entire archive data. The server should be able to compress this into a statistical sample. I would propose:

for each sample
   sample start time - in unix gmt (should convert to ascii in the client side)

Should be our VTimestamp: POSIX time stamp plus the additional 32 int (that might be a pulse id or something completely different)
In the version I sent, the pulse Id has been extracted on the server to save clients having to do this. This is especially important for Matlab clients where the have to feed the timestamp into date to axis value converters. That's why we supply a clean unix ts.


   number of samples

Also needs the sample end time, or (alternatively) the sample duration


   first value
   last value
   mean

Yes, bob, when you wrote "end time", would that imply two consecutive set would have equal last and first value?

If you send a mean, or median filtered value, then the time stamp surely also must be interpolated?

Before going further with the proposal to filter, let's check the performance is actually network bound.

Median or average? both?


   high
   low
   first standard deviation

All this is for atomic numerical values only.

What about strings, enums, waveforms?
Suggest not doing strings. Enums as ints. Waveforms as arrays (we do this at slac as I described).
How is the metadata (enum choices, alarm status) represented?
Ints.

Ralph

Cheers
Greg

------------------------------------------------------------------------------
The Palm PDK Hot Apps Program offers developers who use the
Plug-In Development Kit to bring their C/C++ apps to Palm for a share
of $1 Million in cash or HP Products. Visit us here for more details:
http://p.sf.net/sfu/dev2dev-palm
_______________________________________________
Pvmanager-devel mailing list
Pvm...@li...<mailto:Pvm...@li...>
https://lists.sourceforge.net/lists/listinfo/pvmanager-devel