OpenSAF / Tickets / #191 osaf: Extended Name Type

Anders Bjornerstedt - 2013-05-14

Migrated from:
http://devel.opensaf.org/ticket/2475

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anders Bjornerstedt - 2013-05-14

A pair of wrapper functions for packing and unpacking long SaNameT values would also be
needed. This would be provided by he imma library. These functions would pack a string
shorter than 256 bytes in the regular way, but pack a longer string according to the
extended representation.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anders Widell - 2013-09-16

Milestone: future --> 4.4.FC
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anders Widell - 2013-09-16

I have done some prototyping on this - will publish code once I have cleaned it up a bit.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anders Bjornerstedt - 2013-09-16

Note that thios is a discussion topic ticket.
Actually doing something about SaNameT is much bigger than a change in IMM.

In the text for this discussion topic I outlined a solution that could be
ïmplemented in the IMM first, that is tranpsarently backwards compatible
for DNs shorter than 256 bytes, and where other services and OpenSAF
applications can be addapted later.

Once a proposal has been prototyped and agreed on, then separate enhancement
tickets should be created for the impacted OpensAF services and this
discussion topic could be closed. I doubt that the entire conversion
for OpenSAF could be accomplished in 4.4

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anders Bjornerstedt - 2013-09-17

status: accepted --> unassigned
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anders Bjornerstedt - 2013-10-17

status: unassigned --> assigned

assigned_to: Anders Bjornerstedt --> Anders Widell

Type: discussion --> enhancement

Component: imm --> osaf
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anders Widell - 2013-11-10

status: assigned --> accepted
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Hans Feldt - 2013-11-11

This ticket is a discussion topic. What does it mean to accept it?

I think there should be a new enhancement ticket once this ticket has been used to discuss and agree on the solution to implement.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anders Bjornerstedt - 2013-11-11

It was a discussion toppic, but I changed it to an enhancement some weeks ago.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anders Widell - 2013-11-19

Here is a prototype for long DN support in IMM. It is far from complete - for example I believe there are memory leaks in immutil.c, and thus, in all IMM tools. The patch only adapts the IMM service and tools. No other service has been changed, and thus no other service supports long DNs.

Adapting an application to support long DNs is fairly straight forward. You recompile the application with the preprocessor macro SA_LONGNAME_SOURCE defined, and fix all the resulting compilation errors. That's it.

gcc -DSA_LONGNAME_SOURCE myapp.c

You will get compilation errors at all places where the application is accessing the fiels inside the SaNameT type, or where is it using the SA_MAX_NAME_LENGTH constant. These places have to be fixed. The constant has been removed, so you will need to use e.g. dynamic memory allocation instead of using fixed-size buffers. And instead of accessing the fields inside SaNameT, you need to use the access function saAisNameLend() and saAisNameBorrow(). These functions are used to insert and retrieve a string pointer to/from an SaNameT object.

long_dn_prototype_v1.diff

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anders Widell - 2013-12-03

Milestone: 4.4.FC --> 4.5.FC
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anders Widell - 2014-02-07

I have re-based this patch so that it applies on OpenSAF 4.4.RC1. In addition, this new version contains (yet untested) modifications to NTF and SMF, to get a feeling for how difficult it is to adapt these services for long DNs.

long_dn_prototype_v2.diff

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Anders Bjornerstedt - 2014-02-07
  
  Good.
  
  I hope to also provide a first version of an SaStringT based IMM API
  (#643 https://sourceforge.net/p/opensaf/tickets/643)
  for review quite soon (as soon as we finish 4.4.0). This will be
  provided as incremental include files saImmOm_A_2_14.h
  and saImmOi_A_2_14.h and be available ofr imm handles initialized with
  A.2.14
  
  If and when that is approved in OpenSAF, then providing an
  implementation should not be too difficult.
  
  /AndersBj
  
  Anders Widell wrote:
  
  I have re-based this patch so that it applies on OpenSAF 4.4.RC1. In addition, this new version contains (yet untested) modifications to NTF and SMF, to get a feeling for how difficult it is to adapt these services for long DNs.
  
  Attachment: long_dn_prototype_v2.diff (232.3 kB; text/x-patch)
  
  ** [tickets:#191] IMM: SaNameT 256 byte size limit and RDN 64 byte size limit.**
  
  Status: accepted
  Created: Tue May 14, 2013 08:12 AM UTC by Anders Bjornerstedt
  Last Updated: Tue Dec 03, 2013 02:35 PM UTC
  Owner: Anders Widell
  
  The SaNameT type and in particular its limit in size of 256 bytes
  is an aspect of the SAF standard that was, shall we say, not so
  well conceived.
  
  The 256 byte limit on SaNameT implies the same limit on DN length.
  it is also the reason behind the 64 byte limit on RDN length, if the
  RDN attribute is defined on SaStringT.
  
  The SaNameT type is used more or less in the APIs of practically all
  SAF services. Thus an enhancement that would try to extend it
  length would not just be an enhancement on the IMM service,
  but all services.
  
  This discussion topic is intended to probe the possibility of
  increasing the size of SaNameT, or of replacing SaNameT with
  a type defined in the same way as SaStringT.
  There is still a point in having a DN type, that is semantically
  distinct from SaStringT, because a DN is in essence the IMM
  reference data-type.
  
  One possibility is to continue with SaNameT, but extend both
  its struct definition and the size limit.
  The "length" member is already defined as SaUint?16T,
  which means the length member is not the problem.
  
  Had the SAF standard been defined in C++ we could just create
  a subclass to the SaNameT class, but since we are restricted to
  the more primitive C language, one has to either create a
  completely new type SaNameT_2 or do some kind of "trick".
  
  Changing the type has the big problem that it impacts so many
  APIs and in a non backwards compatible way.
  
  So I propose some form of trick.
  
  For example, we can exploit the 16 bit size field, in a way that
  is both backwards compatible and forwards compatible.
  
  Assume that if the high order bit of the length feild is set,
  then it indicates that this is an extended SaNameT type.
  
  In this case, the value member will not contain the
  plain string. Instead it contains first a null terminated
  marker string such as "Long SaNameT..." (16 bytes long
  including the null termination).
  
  After this string there will be stored a pointer,
  which should be aligned on 64/32 bit architectures.
  That pointer is a pointer to char*, i.e. it is supposed
  to be a pointer to a null terminated string.
  
  The actual size of the string would not be stored in the length
  field. For one, it is not needed since we have a null
  terminated string. For another, it is prudent not to use the
  length field, except the high order bit, to guard against
  bugs and illegal use.
  
  This is backwards compatible in the sense that any old API
  that currently accepts SaNameT, but that has not been migrated
  to a version that understands the extended SaNameT, should
  reject the value as being too large. That is it should at least
  not just crash. The token string is also there so that naiive
  code that ignores the length field and just assumes a null
  terminated string in the old SaNameT value array, will find
  such a string. Again, it should not crash uncontrollably but
  at worst behave in a way that is not correct, but is quickly
  recognizable as to its cause.
  That "Long SaNameT..." string will be visible and recognizable
  for what it is and the bug in the old code can be fixed.
  
  Once an extended SaNameT type is supported in the the immsv,
  there is no reason to keep the 64 byte limit on RDN length.
  
  Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/opensaf/tickets/191/
  
  To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/
  
  Related
  
  Tickets: ~~#191~~
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anders Bjornerstedt - 2014-03-13

Clarification of the scope of this ticket.
This ticket covers only the introduction of the osaf library support
for tunneling long DNs through the SaNameT type.

Currently there is a quite big patch containing a prototype, illustrating
that the proposed solution is relatively easy to introduce on all OpenSAF
services. The goal with that prototype is to get acceptance for the
approach/mechanism.

Assuming that no serious problems or objects surface soon, a new smaller
patch containing only the osaf library support for the SaNameT mechanism
will be sent out, for formal review. If that review gets an ack then that
patch will be pushed.

After this new and separate enhancement tickets should be written per
service. The IMM service would then be among the first to add support
for this mechanism, hopefully in 4.5.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anders Bjornerstedt - 2014-03-13

These SaNameT overlaying functions need to be available also to
the applications using OpenSAF services.

I am not sure of the best way to provide that.
Maybe "samples" is the only option ?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anders Widell - 2014-03-17

That is a good question. In the prototype, I simply wrote them as inline functions in the header file, but that is probably not a good option for the final solution. I can see three alternatives:

Create a new library libSaAis (which currently doesn't exist)

Add them some existing library, e.g. libSaImmOm

Add them as samles, as you suggest. maybe to immutils?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anders Bjornerstedt - 2014-03-17

Actually samples or immutils will not work because we will be needing to
use these tunneling finctions inside new implementations of the
libSaImmaOm and libSaImmOi. This to support long DNs in the old IMM API.
Amd we cant have a dependency from these to immutils or samples.

Perhaps we can keep them as inline defined functions, say in saAis_B_5_14.h
??

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anders Widell - 2014-03-17

I am planning to add them to libopensaf_core, which can be used by the OpenSAF code itself. In order to make them accessible to applications, we can add wrapper functions in e.g. libSaAis or libSaImmOm, that call the functions in libopensaf_core. Or we can put a copy of them in immutils.

Inline functins is one possibility, but I put a question mark on the legal aspect. I am not fully sure what that would mean from a licensing perspective. Though it should be no different than the C++ standard library, which also has a lot of inline code. Need to check it first, though.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anders Bjornerstedt - 2014-03-17

Yes I suppose we could (perhaps even should) keep these SaNameT tunneling
primitives as a pure OpenSAF enhancement (not SAF).

This also means that the support for long DNs in the old SaNameT based APIs
will be an OpenSAF feature.

But supporting long DNs via the new SaStringT/SaConstrStringT API should be
kept compatible with being adopted by SAF in the future.
This distinction may be academic at this point in time, but I still think we
should keep the conceptual domains (and naming domains) clear.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

summary: IMM: SaNameT 256 byte size limit and RDN 64 byte size limit. --> osaf: Extended Name Type
Description has changed:

Diff:

--- old
+++ new
@@ -1,73 +1,61 @@
-The SaNameT type and in particular its limit in size of 256 bytes
-is an aspect of the SAF standard that was, shall we say, not so
-well conceived.
+Ticket [#643] and [#873] are for adding SaStringT-based APIs in IMM and NTF, respectively. This ticket is for implementing a tunneling API that can be used to tunnel NUL-terminated strings through the old SaNameT type. The tunneling API is defined below:

-The 256 byte limit on SaNameT implies the same limit on DN length.
-it is also the reason behind the 64 byte limit on RDN length, if the
-RDN attribute is defined on SaStringT.
+Extended Name Type
+==================
+The SaNameT type is deprecated will be replaced with string parameters in new SAF APIs. As an intermediate solution, the extended format of the SaNameT type can be used to pass string parameters to and from old SAF APIs as well, by tunneling them through the SaNameT type. To enable the extended SaNameT format, the application source code has to be compiled with the SA_EXTENDED_NAME_SOURCE preprocessor macro defined, and the environment variable SA_ENABLE_EXTENDED_NAMES must be set to the value 1 before the first call to any SAF API function.

-The SaNameT type is used more or less in the APIs of practically all
-SAF services. Thus an enhancement that would try to extend it
-length would not just be an enhancement on the IMM service,
-but all services.
+When the extended SaNameT format is enabled, the SA_MAX_NAME_LENGTH constant must not be used, and the application must treat the SaNameT type as opaque and not access any of its members directly. Instead, the saAisNameLend() and saAisNameBorrow() access functions shall be used. The SA_MAX_UNEXTENDED_NAME_LENGTH constant can be used to refer to the maximum string length that can be stored in the unextended SaNameT type.

-This discussion topic is intended to probe the possibility of
-increasing the size of SaNameT, or of replacing SaNameT with
-a type defined in the same way as SaStringT.
-There is still a point in having a DN type, that is semantically
-distinct from SaStringT, because a DN is in essence the IMM
-reference data-type.
+Definition
+----------

+    #ifdef SA_EXTENDED_NAME_SOURCE
+    
+    #define SA_MAX_UNEXTENDED_NAME_LENGTH 256
+    
+    #endif /* SA_EXTENDED_NAME_SOURCE */
+    
+saAisNameLend()
+===============
+Prototype
+---------
+    #ifdef SA_EXTENDED_NAME_SOURCE
+    
+    void
+    saAisNameLend(
+        SaConstStringT value,
+        SaNameT* name);
+    
+    #endif /* SA_EXTENDED_NAME_SOURCE */

-One possibility is to continue with SaNameT, but extend both
-its struct definition and the size limit.
-The "length" member is already defined as SaUint?16T,
-which means the length member is not the problem.
+Parameters
+----------
+value - [in] A pointer to a NUL-terminated string that will be tunneled through the SaNameT type.
+name - [out] A pointer to an SaNameT type to be used for tunneling.

-Had the SAF standard been defined in C++ we could just create
-a subclass to the SaNameT class, but since we are restricted to
-the more primitive C language, one has to either create a
-completely new type SaNameT_2 or do some kind of "trick".
+Description
+-----------
+Tunnel a NUL-terminated string through a SaNameT type. If length of the string is strictly less than SA_MAX_UNEXTENDED_NAME_LENGTH bytes, the contents of the string is copied into the SaNameT type and can be read in a backwards compatible way by legacy applications that do not support the extended SaNameT format. If length of the string is greater than or equal to SA_MAX_UNEXTENDED_NAME_LENGTH, no copying is performed. Instead, a reference to the original string is stored in the SaNameT type. In this case, it is therefore important that the original string is not modified or freed for as long as the SaNameT type may still used.

-Changing the type has the big problem that it impacts so many
-APIs and in a non backwards compatible way.
+saAisNameBorrow()
+=================
+Prototype
+---------

+    #ifdef SA_EXTENDED_NAME_SOURCE
+    
+    SaConstStringT
+    saAisNameBorrow(
+        const SaNameT* name);

-So I propose some form of trick.

+    #endif /* SA_EXTENDED_NAME_SOURCE */

-For example, we can exploit the 16 bit size field, in a way that
-is both backwards compatible and forwards compatible.
+Parameters
+----------
+name - [in]  A pointer to an SaNameT type that has been previously set using the saAisNameLend() function.

-Assume that if the high order bit of the length feild is set,
-then it indicates that this is an extended SaNameT type.
+Description
+-----------
+Retrieve a tunneled string from an SaNameT type. Before calling this function, the SaNameT stucture must have been initialized either by a call to the saAisNameLend() function or by being used as an output parameter of any other SAF API function. If the length of the returned string is strictly less than SA_MAX_UNEXTENDED_NAME_LENGTH bytes, the returned pointer points to a copy of the string stored inside the SaNameT type. Otherwise, the returned pointer is equal to the original string pointer that was passed as a parameter to the saAisNameLend() function.

-In this case, the value member will not contain the
-plain string. Instead it contains first a null terminated
-marker string such as "Long SaNameT..." (16 bytes long
-including the null termination).
-
-After this string there will be stored a pointer,
-which should be aligned on 64/32 bit architectures.
-That pointer is a pointer to char*, i.e. it is supposed
-to be a pointer to a null terminated string.
-
-The actual size of the string would not be stored in the length
-field. For one, it is not needed since we have a null
-terminated string. For another, it is prudent not to use the
-length field, except the high order bit, to guard against
-bugs and illegal use.
-
-This is backwards compatible in the sense that any old API
-that currently accepts SaNameT, but that has not been migrated
-to a version that understands the extended SaNameT, should
-reject the value as being too large. That is it should at least
-not just crash. The token string is also there so that naiive
-code that ignores the length field and just assumes a null
-terminated string in the old SaNameT value array, will find
-such a string. Again, it should not crash uncontrollably but
-at worst behave in a way that is not correct, but is quickly
-recognizable as to its cause.
-That "Long SaNameT..." string will be visible and recognizable
-for what it is and the bug in the old code can be fixed.
-
-Once an extended SaNameT type is supported in the the immsv,
-there is no reason to keep the 64 byte limit on RDN length.
-
+Return Values
+-------------
+A pointer to a NUL-terminated string, or a NULL pointer in case of a failure.

Anders Widell - 2014-05-02

Prototype re-based and updated to use the patches sent out for review.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anders Widell - 2014-05-02

status: accepted --> review
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Description has changed:

Diff:

--- old
+++ new
@@ -1,4 +1,11 @@
-Ticket [#643] and [#873] are for adding SaStringT-based APIs in IMM and NTF, respectively. This ticket is for implementing a tunneling API that can be used to tunnel NUL-terminated strings through the old SaNameT type. The tunneling API is defined below:
+This ticket is for implementing a tunneling API that can be used to tunnel
+NUL-terminated strings through the old SaNameT type.
+
+Ticket [#886] and [#873] are for adding support for long DN tunneling in IMM
+and NTF, respectively. Ticket [#643] is for adding a new SaStringT based API
+to the IMM that will also by default support long DNs.
+
+The tunneling API is defined below:

 Extended Name Type
 ==================

Anders Widell - 2014-06-09

Re-based the prototype to changeset f1f2eebd6019e56adc41fb69182d2678e4751150

long_dn_prototype_v5.diff

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

osaf: Extended Name Type

Milestone

Searches

Help

#191 osaf: Extended Name Type

Extended Name Type

Definition

saAisNameLend()

Prototype

Parameters

Description

saAisNameBorrow()

Prototype

Parameters

Description

Return Values

Related

Discussion

Related