agtk-devel — AGTK Developers Mailing List

[agtk-devel] AG API -- list return type

[Kazuaki M. <ma...@un...>]

Dear developers who use the AG API,

We are interested in hearing your opinions about the following possible
changes in the API.

In the current AG API, functions that return sets of ID's, names, etc. have
the return type of "string" instead of "list".  For example,
GetAnnotationSetByOffset returns a string that may look like
"AGSet1:AG1:Annotation1 AGSet1:AG1:Annotation2 ...".  This requires
additional processing on the application side.  We are thinking about
changing the API in a future version of AGLIB such that these functions
return lists instead of strings.

This change would affect the following types in ag.idl.

  typedef string AGIds;
  typedef string AnnotationIds;
  typedef string AnchorIds;
  typedef string SignalIds;
  typedef string FeatureNames;
  typedef string Features;

The new types would be something like the following.  These will be
lists (or arrays) in C++, Python, Java and Tcl.

  typedef sequence<AGId> AGIds;
  typedef sequence<AnnotationId> AnnotationIds;
  typedef sequence<AnchorId> AnchorIds;
  typedef sequence<SignalIds> string SignalIds;
  typedef sequence<FeatureNames> string FeatureNames;

This change in the API would require changes in applications if they are to
use the new version of AGLIB.  In order to have a painless change to the
newer API, we are thinking about adding a configuration(compile)-time
switch to choose the return types of the affected functions.

We'd welcome any options concerning 1) whether such changes should be made,
or 2) whether there are better ways to doing the changes.

Thank you,
-Kazuaki Maeda (LDC)

[agtk-devel] AG Library and UTF-8?

[Matthias T. <th...@ei...>]

Hello agtk developers,

I have just stumbled over a non-ASCII label which I added as a feature
value to an annotation graph via the tcl wrapper, and saved the
resulting AGSet in AIF Format.

When trying to read that file back, I get an error message like:

loading agset from file Test.xml...agf:Expected end of tag 'Feature'
Error in startup script: LoadErroragf:Expected end of tag 'Feature'
     while executing
"AG::load "AIF" $agSetFileName"

I checked the XML file, and apparently the characters are saved 'as-is',
which was the source of the error. I had a similar problem in a
different matter, and solved it by storing the data as UTF-8. That meant
converting all strings read to and from the XML document via the parser,
using helper functions like "iso2utf8 and utf82iso".

Is that something the AG Library user has to take care of, or should
that be implemented in the library itself?

Regards.
Matthias

Re: [agtk-devel] displaying AGs

[Mark L. <my...@un...>]

 >> Margit Gut writes:
 >> > Is there any program with which I can print an annotation graph as a
 >> > picture, like those in your documentation?  I would like to look,
 >> > what my graph looks like, if I print it out. It is difficult to read
 >> > the xml file and to imagine how it could look like.
 >
 >Margit,
 >
 >We don't have such a tool at this moment, but we just started working on a
 >program for graph visualization (CVS/AGAPPS-AGView).  It's going to use
 >either 1) Graphviz (dot) (http://www.research.att.com/sw/tools/graphviz/)
 >or 2) its own display widget using the Tk canvas widget.
 >
 >-Kazuaki Maeda
 >LDC

Let me add that if you are interested in graphics for a publication or
presentation, the graphviz package is quite flexible and relatively
easy to learn to use.

The general problem of laying out arbitrary graphs for vizualization
in a readable way is a very hard one, but if you have graphs of a
particular kind, you could probably write a quick perl (or similar)
script to mediate effectively between the XML transfer format and a
general graph-layout and rendering package such as graphviz.

   -Mark Liberman

Re: [agtk-devel] displaying AGs

[Kazuaki M. <ma...@un...>]

> Margit Gut writes:
> > Is there any program with which I can print an annotation graph as a
> > picture, like those in your documentation?  I would like to look,
> > what my graph looks like, if I print it out. It is difficult to read
> > the xml file and to imagine how it could look like.

Margit,

We don't have such a tool at this moment, but we just started working on a
program for graph visualization (CVS/AGAPPS-AGView).  It's going to use
either 1) Graphviz (dot) (http://www.research.att.com/sw/tools/graphviz/)
or 2) its own display widget using the Tk canvas widget.

-Kazuaki Maeda
LDC

[agtk-devel] displaying AGs

[Steven B. <sb...@un...>]

Margit Gut writes:
> Is there any program with which I can print an annotation graph as a
> picture, like those in your documentation?  I would like to look,
> what my graph looks like, if I print it out. It is difficult to read
> the xml file and to imagine how it could look like.

Re: [agtk-devel] lost anchor identifier at AG::splitAnnotation

[Matthias T. <th...@ei...>]

Hello Haejoong,

Haejoong Lee wrote:
 > Matthias,
 >
 > Thank you for reporting the bug!
 > It's now fixed for the next release.

Fine :) so I assume it's fixed in the CVS? I'm working with the CVS
because of the extended Tcl wrapper and the validation functions. By the
way, could you remove the debugging output from the validation
functions, especially Line 199 of Validation.cc floods my terminal ;)

 >>I have a similar problem when unsetting anchor offsets using
 >>AG::UnsetAnchorOffset, but no solution.
 >
 >
 > I'm not sure about the above problem.  It will be a great help
 > if you can give us a little more details.

Yes, certainly. I am building an AG where all anchors have an offset
first, and then try to unset all anchor offsets except for the start and
end anchor of the AG. It seems to work until the third or so AG of my
test data, then an exception occurs:

AGExceptionAG Exception caught when calling UnsetAnchorOffset:
Anchor not found!
     while executing
"AG::UnsetAnchorOffset $anchor"

I also noticed that before the exception occurs, [AG::GetAnchorSet
$agId] returns the anchor set in a different order than before, which is
the correct topological and chronological order.


Maybe this problem is related to the sorting of the anchors?
As I understood from the papers, an AG is a directed acyclic graph, so
that a topological sorting would be possible regardless of the anchor
offsets. But as I understand from the sources, the sorting of the
anchors is only done chronologically, i.e. if all anchors have an offset.

So apart from the problem above, wouldn't the topological sorting of the
anchors be a nice feature for the AG library?

 >>Maybe one of the developers could check the sources with respect to the
 >>Identifier management, in case more of these problems exist?
 >
 > I hope Xiaoyi can work on this when he gets back.

Ok.

 > Thank you.
 >
 > Haejoong

Thank you for your quick reply and fix :)

Regards.
Matthias

Re: [agtk-devel] lost anchor identifier at AG::splitAnnotation

[Haejoong L. <hae...@un...>]

Matthias,

Thank you for reporting the bug!
It's now fixed for the next release.

> I have a similar problem when unsetting anchor offsets using
> AG::UnsetAnchorOffset, but no solution.

I'm not sure about the above problem.  It will be a great help
if you can give us a little more details.

> Maybe one of the developers could check the sources with respect to the
> Identifier management, in case more of these problems exist?

I hope Xiaoyi can work on this when he gets back.

Thank you.

Haejoong

Re: [agtk-devel] lost anchor identifier at AG::splitAnnotation

[Kazuaki M. <ma...@un...>]

Dear Matthias,

Thank you very much for pointing out the bugs.  We will try to fix them as
soon as possible.  About the problem with UnsetAnchorOffset and your
suggestion about checking the sources with respect to the ID management, my
colleague Xiaoyi Ma at LDC, who is currently away, will probably be able to
work on these once he comes back.

Regards,
Kazuaki Maeda
Linguisitc Data Consortium

From: Matthias Thomae <th...@ei...>
Subject: [agtk-devel] lost anchor identifier at AG::splitAnnotation
Date: Mon, 05 Aug 2002 14:32:03 +0200

> Dear AG developers,
> 
> I encountered a problem using the AG::SplitAnnotation function from Tcl.
> When I try to reference the newly created anchor, e.g. by setting its
> offset, I get an exception with a message:
> 
> "<anchorId> is an invalid AnchorId"
> 
> I think this problem originates from the AG::splitAnnotation method in
> AG.cc, in that the anchor which is constructed is not added to the
> identifiers. After adding
> 
>    addAnchor(n);
> 
> to AG::splitAnnotation and recompiling, the problem disappears.
> 
> I have a similar problem when unsetting anchor offsets using
> AG::UnsetAnchorOffset, but no solution.
> 
> Maybe one of the developers could check the sources with respect to the
> Identifier management, in case more of these problems exist?
> 
> Regards.
> Matthias

[agtk-devel] lost anchor identifier at AG::splitAnnotation

[Matthias T. <th...@ei...>]

Dear AG developers,

I encountered a problem using the AG::SplitAnnotation function from Tcl.
When I try to reference the newly created anchor, e.g. by setting its
offset, I get an exception with a message:

"<anchorId> is an invalid AnchorId"

I think this problem originates from the AG::splitAnnotation method in
AG.cc, in that the anchor which is constructed is not added to the
identifiers. After adding

   addAnchor(n);

to AG::splitAnnotation and recompiling, the problem disappears.

I have a similar problem when unsetting anchor offsets using
AG::UnsetAnchorOffset, but no solution.

Maybe one of the developers could check the sources with respect to the
Identifier management, in case more of these problems exist?

Regards.
Matthias

[agtk-devel] Validating annotation graphs

[Steven B. <sb...@un...>]

I have recently added 8 validation functions to aglib, listed below.  The
purpose of these functions is to validate the content and structure of an
annotation graph.  Special purpose applications that load AG data from an
untrusted source may need to check the validity of the assumptions they
make about the data before operating on it.  This is an onerous task using
the existing API, but greatly simplified using the new functions.

Note that the functions are in the CVS tree
[http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/agtk/AGLIB/src/ag/Validation.cc?rev=1.5&content-type=text/vnd.viewcvs-markup],
but not yet available via the scripting language interfaces and not yet
included in any AGLIB distributions.  Once they are, their documentation
will show up in our online library documentation
[http://agtk.sourceforge.net/doc/api/].

The fundamental principle behind these functions is that they don't make a
"closed world assumption".  They assume that an annotation graph may
contain other layers of annotation that we don't care about.  This is
accomplished using types - the functions only work with annotations of the
specified type (and with the anchors these annotations are connected to).

For example, to check that an AG containing TIMIT data is well-formed,
we could call:

if (  checkAnchorOffsetTotal(ag, "phn")
   && checkFeatureExists(ag, "txt", "label")
   && checkFeatureExists(ag, "wrd", "label")
   && checkFeatureExists(ag, "phn", "label")
   && checkLinear(ag, "wrd")
   && checkLinear(ag, "phn")
   && checkSpan(ag, "txt", "wrd")
   && checkSpan(ag, "wrd", "phn") )
  # WELL-FORMED

Please see the function definitions below for a brief explanation of what
each one does.  Please let me know if you need any other low-level
validation functions.

(Note that some format conversion programs will need to check that an AG
contains nothing other than the validated structure, to ensure lossless
conversion.  An easy way to do this check is to get a list of all the
annotation types contained in the AG, and check that the list only contains
those types that the application knows about and has validated.)

Steven Bird

**********************************************************************

VALIDATION FUNCTIONS:

bool checkAnchorOffsetTotal(AG *ag, AnnotationType type);
Check that all anchors of annotations of a given type have an offset.

bool checkAnchorOffsetBounded(AG *ag, AnnotationType type);
Check that all anchors of annotations of a given type are bounded
by anchors that have an offset, following paths of this type.

bool checkFeatureExists(AG *ag, AnnotationType type, FeatureName featureName);
Check that all annotations of a given type have the specified feature.

bool checkFeatureIsAnnotationId(AG *ag, AnnotationType type, FeatureName featureName);
Check that all annotations of a given type have the specified feature and
that its value is a valid AnnotationId.

bool checkLinear(AG *ag, AnnotationType type);
Check that all annotations of a given type form a connected linear sequence.

bool checkConnected(AG *ag, AnnotationType type);
Check that all annotations of a given type form a connected subgraph.

bool checkCoextensive(AG *ag, AnnotationType type1, AnnotationType type2);
Check that all annotations of type type1 are coextensive with an
annotation of type type2, and vice versa (an existence not uniqueness test).

bool checkSpan(AG *ag, AnnotationType spanType, AnnotationType spannedType);
Check that all annotations of type spanType span annotations of
type spannedType and that all annotations of type spannedType are
spanned by annotations of type spanType.

**********************************************************************

Re: [agtk-devel] AGTK

[Haejoong L. <hae...@un...>]

Margit,

I guess you have built "AG-library" from the source package and
your compilation of aglib is not complete.

Please check if you have the following:

    aglib-1.0/ag_wrapper/python/.libs/ag_python.so

If not, it means you didn't enable ag_python option, which build
python ag module.  Please do the following:

    $ cd aglib-1.0
    $ make distclean
    $ ./configure --with-prefix=/dir/to/install \
                  --enable-ag_python \
                  --with-python=/dir/of/python/binary/python2.1 \
                  <put other options here, if any>
    $ make
    $ make install

If there is no problem in compiling, you have to be able to find the
followings:

    /dir/to/install/lib/libag.so
    /dir/to/install/lib/libagf.so
    /dir/to/install/lib/ag_python.so
    
Now, set your LD_LIBRARY_PATH as follows:

    $ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/dir/to/install/lib

    or (if you are using csh)

    % setenv LD_LIBRARY_PATH "$LD_LIBRARY_PATH":/dir/to/install/lib

That's it!!

Thanks,
Haejoong


On Tue, Apr 30, 2002 at 06:22:49PM +0200, Margit Gut wrote:
> HI, Kazuaki,
> 
> it's me again. I think I have more problems than I thought first.
> I tried to write the Test-program, you sent me. I think, the AG-library is
> everything that lies in
>                 algib-1.0/ag
> the
>                 import ag
> told me first, that it needs
>                 ag_python.
> I copied the ag_python.cc file in the directory ag
> the next message was
> object has no attribute CreateAgSet
> 
> I think I am doing something very wrong. I can imagine, that I have no
> problems any further when the demo program is running, because the
> documentation of the calls is very good. My problem, I think, is to
> understand, which modules play together so that the program is running. I am
> not really sure wether I need the .h and .cc files or the .py files.
> In a second try I wanted to run the ag_wrapper_test.py but I got the same
> message
>             object has no attribute CreateAgSet
> I think, I have not yet all resources I need, so I would be very grateful if
> you could help me once again
> 
> greetings and a lot of patience
> 
> Margit
> 
> 
> 
> _______________________________________________
> agtk-devel mailing list
> agt...@li...
> https://lists.sourceforge.net/lists/listinfo/agtk-devel

[agtk-devel] AGTK

[<Mar...@t-...>]

HI, Kazuaki,

it's me again. I think I have more problems than I thought first.
I tried to write the Test-program, you sent me. I think, the AG-library is
everything that lies in
                algib-1.0/ag
the
                import ag
told me first, that it needs
                ag_python.
I copied the ag_python.cc file in the directory ag
the next message was
object has no attribute CreateAgSet

I think I am doing something very wrong. I can imagine, that I have no
problems any further when the demo program is running, because the
documentation of the calls is very good. My problem, I think, is to
understand, which modules play together so that the program is running. I am
not really sure wether I need the .h and .cc files or the .py files.
In a second try I wanted to run the ag_wrapper_test.py but I got the same
message
            object has no attribute CreateAgSet
I think, I have not yet all resources I need, so I would be very grateful if
you could help me once again

greetings and a lot of patience

Margit

[agtk-devel] AGTK

[<Mar...@t-...>]

Hi,

many thanks for the response, Kazuaki Maeda. I will try this weekend, and I
hope I succeed.
It is much easier, to have an example which ist not a complete application
so I can understand, what I am doing.

greetings

Margit

Re: [agtk-devel] new on the list, I have a question

[Kazuaki M. <ma...@un...>]

Hello Margit,

Thank you very much for trying out the AG toolkit.  Assuming that you have
python, I've written a simple example of using the AG library from python.
The following python code uses the most basic functions in the AG library.
For the explanations of all available functions in the API, please refer to
the IDL API specification of the AG library:
http://www.ldc.upenn.edu/AG/doc/idl/ag-1.0.idl. Also, there are some more
examples (in C++, but the same things can be done in python) in the
demo subdirectory of the aglib distribution.

Regards,
-Kazuaki Maeda

# import the ag module
import ag

# create an AGSet, a timeline and an AG 
AGSetId = ag.CreateAGSet("sample")
timelineId = ag.CreateTimeline(AGSetId)
AGId = ag.CreateAG(AGSetId, timelineId)

# create three anchors
anchor1 = ag.CreateAnchor(AGId)
anchor2 = ag.CreateAnchor(AGId)
anchor3 = ag.CreateAnchor(AGId)

# create three annotations
annotation1 = ag.CreateAnnotation(AGId, anchor1, anchor2, "word")
annotation2 = ag.CreateAnnotation(AGId, anchor2, anchor3, "word")
annotation3 = ag.CreateAnnotation(AGId, anchor2, anchor3, "misc")

# set offsets of some of the anchors
ag.SetStartOffset(annotation1, 1.00)
ag.SetEndOffset(annotation1, 2.00)

# set features to an annotation
ag.SetFeature(annotation1, "English", "cat")
ag.SetFeature(annotation1, "Japanese", "neko")

# examples of accessing & printing information in an AG
print "The features associated with annotation1 are:",  
print ag.GetFeatureNames(annotation1)
print

print "The ID of the start anchor of annotation1 is:",
print ag.GetStartAnchor(annotation1)
print

print "The IDs of annotations starting with the end anchor of annotation1 are:"
print ag.GetOutgoingAnnotationSet(ag.GetEndAnchor(annotation1))
print

print "An AIF file for the annotation graph above is:"
print ag.toXML(AGId)


From: "Gut, Margit" <gu...@so...>
Subject: [agtk-devel] new on the list, I have a question
Date: Fri, 26 Apr 2002 14:04:21 +0200

> Hi, all 
> I am a student of computerlinguistics in Stuttgart Germany. Some days ago I
> met Annotation graph the first time. I try to learn python and I wish to ust
> annotation Graph. So can you tell me a very simple example how to use
> Annotation Graph? I tried to read the sources of tabletrans but it is too
> hard for a beginner. 
> I would be very happy, if there exists a more simple example just to
> understand how Annotation Graph works 
> many thanks to everyone 
> margit
> 
> 
> _______________________________________________
> agtk-devel mailing list
> agt...@li...
> https://lists.sourceforge.net/lists/listinfo/agtk-devel
> 

[agtk-devel] new on the list, I have a question

[Gut, M. <gu...@so...>]

Hi, all 
I am a student of computerlinguistics in Stuttgart Germany. Some days ago I
met Annotation graph the first time. I try to learn python and I wish to ust
annotation Graph. So can you tell me a very simple example how to use
Annotation Graph? I tried to read the sources of tabletrans but it is too
hard for a beginner. 
I would be very happy, if there exists a more simple example just to
understand how Annotation Graph works 
many thanks to everyone 
margit

Re: [agtk-devel] wrapper overhead

[Haejoong L. <hae...@un...>]

Hi Steve,

I've managed to directly wrap AGAPI.h.  I'm attaching corresponding
swig interface file (AGAPI.i).  Mapping tcl array to c++ map was
a little tricky.  Only one function hasn't been wrapped; I still
need to write a typemap from c++ list to tcl/python list, which is
required to wrap StoreSQLs() function.  Another remaining task is
to make the new wrapper use a namespace.

I measured the performance change.  The improvement looks
significant (see below).  Any suggestions or questions are more
than welcome.

Thanks,
Haejoong


Your result with the old wrapper:
> getting start times for 3 nodes
> [::AG_GetAnchorOffset [::AG_GetStartAnchor $entry]]: 10650 microseconds per  
> teration
> [::AG_GetStartAnchor $entry]: 6104 microseconds per iteration

Using the old wrapper:
getting start times for 3 nodes
[AG_GetAnchorOffset [AG_GetStartAnchor $entry]]: 12630 microseconds per iteration
[AG_GetStartAnchor $entry]: 7498 microseconds per iteration

Using the new wrapper:
getting start times for 3 nodes
[GetAnchorOffset [GetStartAnchor $entry]]: 274 microseconds per iteration
[GetStartAnchor $entry]: 232 microseconds per iteration

5 nodes:
getting start times for 5 nodes
[GetAnchorOffset [GetStartAnchor $entry]]: 431 microseconds per iteration
[GetStartAnchor $entry]: 351 microseconds per iteration

Re: [agtk-devel] wrapper overhead

[Steve C. <ste...@mq...>]

I looked at using SWIG to wrap AGAPI.h directly last night and got some of 
the way but not all. The problem I hit was mapping between tcl strings and 
the C++ stl string type.  I've since found a few resources on the net which 
might help with this, eg:

http://www-cad.eecs.berkeley.edu/~pinhong/scriptEDA/pyTypemapFAQ.html#38

discusses wrapping stl:string in python but is apparently a little out of 
data for swig 1.3.  I also found an example of doing this from someone on the 
swig mailing list which I've attached here along with my AGAPI.i file.  

I'll continue to fiddle with this if I get some time but feel free to take 
this and fiddle yourselves.  All input is welcome.

Steve

Re: [agtk-devel] wrapper overhead

[Haejoong L. <hae...@un...>]

You are a valuable user :)
You are totally correct, and we are well aware of the bottleneck.
One of our application project based on agtk is very serious about
the problem.  On the one hand, we are trying to reduce the number
of library calls by adding functions that can do many things by
one call.  On the other hand, we will definitely have to pursue
what you suggest.  I remember I gave up direct wrapping because
it didn't go smooth.  I guess it's time to try again.

Haejoong

On Thu, Mar 14, 2002 at 11:38:21AM +1100, Steve Cassidy wrote:
> Hi folks, Steve the whiner here again...
> 
> we've just hit some major inneficiencies in the tcl interface to AG, this 
> came when we tried to draw a real annotation in our new labeller and it took 
> 1000 seconds to complete 250 annotations.  The bottleneck turned out to be 
> calls to the AG routines to get anchor times, specifically:
> 
>  [::AG_GetAnchorOffset [::AG_GetStartAnchor $entry]]
> 
> 
> To get some figures I wrote a short tcl script which creates an annotation 
> (copied from ag_wrapper_test.tcl) and then times calls to the above fns. The 
> script is attached but the output is:
> 
>  tclsh timer.tcl
> getting start times for 3 nodes
> [::AG_GetAnchorOffset [::AG_GetStartAnchor $entry]]: 10650 microseconds per  
> teration
> [::AG_GetStartAnchor $entry]: 6104 microseconds per iteration
> 
> for comparison I wrote a similar script using Emu, accessing the same kind of 
> information (start/end times) which is stored in the same way (as a member 
> variable in a C++ class):
> 
> getting start times for 5 nodes
> 232 microseconds per iteration
> 
> This is a significant overhead which is going to make writing useable user 
> interfaces difficult if it is reflected accross the API as I suspect.  
> 
> I haven't compared these times to a raw C++ version but I'm hoping that that 
> would be significantly faster and that the bottleneck is in the wrapper code.
> 
> Looking at the code it seems that there is a lot of indirection in the 
> scripting wrapper that could probably be avoided. SWIG can take care of 
> converting things to strings if needs be.  I wonder if we shouldn't look at 
> how this is done and perhaps have another bash at the wrapper interface. 
> 
> I note that all the interfaces in AGAPI.h are in terms of strings so a simple 
> SWIG wrap of AGAPI.h might well work -- was this tried? 
> 
> We're working on another SWIG project here at the moment (wrapping the 
> Edinburgh speech tools) and so it might be worth our looking at this again at 
> the same time while we still understand SWIG! 
> 
> Comments welcomed.
> 
> Steve
>  
> # ag_wrapper_test.tcl
> # Haejoong Lee
> # Copyright (C) 2001 Linguistic Data Consortium, U Penn.
> # Web: http://www.ldc.upenn.edu/; Email: ld...@ld...
> # For license information, see the file `LICENSE' included
> # with the distribution.
> 
> 
> source ../ag_wrapper/tcl/ag.tcl
> load ../ag_wrapper/tcl/.libs/ag_tcl.so
> 
> set agsetId [AG_CreateAGSet "TIMIT"]
> set timelineId [AG_CreateTimeline $agsetId]
> AG_SetFeature $timelineId "length" "30 min"
> set signalIds [AG_CreateSignal $timelineId "test_uri" "test_mimeclass" "test_mimetype" "test_encoding" "test_unit" "test_track"]
> set agId [AG_CreateAG $agsetId $timelineId]
> 
> set anchor1 [AG_CreateAnchor $agId 10 "sec" $signalIds]
> set anchor2 [AG_CreateAnchor $agId 20 "sec" $signalIds]
> set anchor3 [AG_CreateAnchor $agId 30 "sec" $signalIds]
> set annotation1 [AG_CreateAnnotation $agId $anchor1 $anchor2 "timit"]
> set annotation2 [AG_CreateAnnotation $agId $anchor2 $anchor3 "timit"]
> set annotation3 [AG_CreateAnnotation $agId $anchor1 $anchor3 "timit"]
> AG_SetFeature $annotation1 "sentence" "It's raining."
> AG_SetFeature $annotation2 "sentence" "We'll see you around 8 o'clock."
> 
> 
> 
> puts "getting start times for [llength [::AG_GetAnnotationSeqByOffset $agId]] nodes"
> 
> puts -nonewline {[::AG_GetAnchorOffset [::AG_GetStartAnchor $entry]]: }
> puts [time {
>     foreach entry [::AG_GetAnnotationSeqByOffset $agId] {
> 	set x [::AG_GetAnchorOffset [::AG_GetStartAnchor $entry]]
>     }
> } 1000]
> 
> puts -nonewline {[::AG_GetStartAnchor $entry]: }
> puts [time {
>     foreach entry [::AG_GetAnnotationSeqByOffset $agId] {
> 	set x [::AG_GetStartAnchor $entry]
>     }
> } 1000]
> 
> 
> 
> 

> 
> emutemplate tpl demo
> 
> tpl hierarchy h msdjc001
> 
> puts "getting start times for [llength [h segments Phonetic]] nodes"
> 
> puts [time {
>     foreach s [h segments Phonetic] {
> 	set x [h seginfo $s times]
>     }
> } 1000]

[agtk-devel] wrapper overhead

[Steve C. <ste...@mq...>]

Hi folks, Steve the whiner here again...

we've just hit some major inneficiencies in the tcl interface to AG, this 
came when we tried to draw a real annotation in our new labeller and it took 
1000 seconds to complete 250 annotations.  The bottleneck turned out to be 
calls to the AG routines to get anchor times, specifically:

 [::AG_GetAnchorOffset [::AG_GetStartAnchor $entry]]


To get some figures I wrote a short tcl script which creates an annotation 
(copied from ag_wrapper_test.tcl) and then times calls to the above fns. The 
script is attached but the output is:

 tclsh timer.tcl
getting start times for 3 nodes
[::AG_GetAnchorOffset [::AG_GetStartAnchor $entry]]: 10650 microseconds per  
teration
[::AG_GetStartAnchor $entry]: 6104 microseconds per iteration

for comparison I wrote a similar script using Emu, accessing the same kind of 
information (start/end times) which is stored in the same way (as a member 
variable in a C++ class):

getting start times for 5 nodes
232 microseconds per iteration

This is a significant overhead which is going to make writing useable user 
interfaces difficult if it is reflected accross the API as I suspect.  

I haven't compared these times to a raw C++ version but I'm hoping that that 
would be significantly faster and that the bottleneck is in the wrapper code.

Looking at the code it seems that there is a lot of indirection in the 
scripting wrapper that could probably be avoided. SWIG can take care of 
converting things to strings if needs be.  I wonder if we shouldn't look at 
how this is done and perhaps have another bash at the wrapper interface. 

I note that all the interfaces in AGAPI.h are in terms of strings so a simple 
SWIG wrap of AGAPI.h might well work -- was this tried? 

We're working on another SWIG project here at the moment (wrapping the 
Edinburgh speech tools) and so it might be worth our looking at this again at 
the same time while we still understand SWIG! 

Comments welcomed.

Steve
 

Re: [agtk-devel] Problems building sources

[Steve C. <ste...@mq...>]

On Tue, 12 Mar 2002 18:20, you wrote:
>
>
> Okay.  It wasn't working.  How it wasn't working?  What was the
> configuration options you used?  What was the error message you had?
> Do you keep, by any chance, the config.log file that was created
> when it wasn't working?  How is your xercesc library installed; what
> is the location of xercesc include files and what is the location of
> xercesc libraries?  Would it be possible for you to give a procedure
> with which I can reproduce the error?

The problem was that although xerces_ver was building properly and when I ran 
it by hand it gave the right output (1_7_0) the variable XERCESC_VER wasn't 
getting set properly (it got xerces-c instead of xerces-c1_7_0).  I don't 
understand why this was happening since the script looks good and I'm sure 
has worked ok for others (and probably for me at other times).  

My xercesc is installed as normal in /usr/local, the test program you 
generate worked ok and gave the right output, the problem seems to be getting 
that output into the shell script variable.

I didn't have time to work on reproducing the error, as it was I spent most 
of yesterday getting xerces-c to compile cleanly (another configuration 
nightmare) on two different machines in order to get the new AG code to 
build. 

> Isn't it good thing to have a smart configuration script?  Can we
> first try to fix the problem instead of throwing it away without
> knowing the reason why it didn't work?
>

I wasn't trying to throw it away, I tried to wrap your test in an if..then 
but it produced a configure script that crashed, my shell scripting isn't the 
best so I thought I'd contribute what I had managed to get working in case it 
was useful.  If you can put the if..then around your test based on the 
--with-xerces-ver option then perhaps we'll have a workaround for cases when 
your test doesn't work.  

Steve

Re: [agtk-devel] Problems building sources

[Haejoong L. <hae...@un...>]

> > P.S.  Do you still have precision problem with AIF loader?  Please
> > let me know.
> 
> That's working ok now. 

That's great news.

> I managed to build AGLIB again after having problems with telling it where 
> xercesc was, for some reason your configure.in script which tries to find the 
> version of the library (a neat method btw which shouldn't be necessary) 
> wasn't working, even though the test program was ok and output the right 
> version number.  In the end I hacked configure.in to add a 
> --with-xercesc-version arg but wasn't able to modify the script to not run 
> your check if this arg was supplied (I tried but the configure script just 
> crashed). Here's the lines I added, perhaps you can work them in more 
> successfully:
> 
> 
> AC_ARG_WITH(xercesc_version,
>   [  --with-xercesc-version=N_N_N  version number of libxerces-c, eg 1_7_0],
>   XERCESC_VER=xerces-c$withval)

Okay.  It wasn't working.  How it wasn't working?  What was the
configuration options you used?  What was the error message you had?
Do you keep, by any chance, the config.log file that was created
when it wasn't working?  How is your xercesc library installed; what
is the location of xercesc include files and what is the location of
xercesc libraries?  Would it be possible for you to give a procedure
with which I can reproduce the error?

Isn't it good thing to have a smart configuration script?  Can we
first try to fix the problem instead of throwing it away without
knowing the reason why it didn't work?

> I also note that you need to make clean after reconfiguring etc. to ensure 
> that all libraries get updated. On my first build the ag_tcl.so library 
> didn't get rebuilt.  I guess that this is just good practice but could be the 
> cause of some errors.  I wonder if automake can force this?

I haven't found a way to do that.  Usually it's a good habit to make
things clean when you compile ANY source packages.  Sometimes "make
clean" is not enough.  I prefer "make distclean" when I'm not sure
about the behaviour of the configuration script of source packages.
It cleans even makefiles, so I need to configure the package again
after that.  That's a good way to avoid mysterious errors, although
it takes time.

Haejoong

Re: [agtk-devel] Problems building sources

[Steve C. <ste...@mq...>]

On Mon, 11 Mar 2002 14:23, Haejoong Lee wrote:

> P.S.  Do you still have precision problem with AIF loader?  Please
> let me know.

That's working ok now. 

I managed to build AGLIB again after having problems with telling it where 
xercesc was, for some reason your configure.in script which tries to find the 
version of the library (a neat method btw which shouldn't be necessary) 
wasn't working, even though the test program was ok and output the right 
version number.  In the end I hacked configure.in to add a 
--with-xercesc-version arg but wasn't able to modify the script to not run 
your check if this arg was supplied (I tried but the configure script just 
crashed). Here's the lines I added, perhaps you can work them in more 
successfully:


AC_ARG_WITH(xercesc_version,
  [  --with-xercesc-version=N_N_N  version number of libxerces-c, eg 1_7_0],
  XERCESC_VER=xerces-c$withval)

I also note that you need to make clean after reconfiguring etc. to ensure 
that all libraries get updated. On my first build the ag_tcl.so library 
didn't get rebuilt.  I guess that this is just good practice but could be the 
cause of some errors.  I wonder if automake can force this?

Steve

Re: [agtk-devel] Problems building sources

[Steve C. <ste...@mq...>]

On Mon, 11 Mar 2002 14:23, Haejoong Lee wrote:
> > I'm having repeated problems building from the CVS sources. The whole
> > build system seems very sensitive to changes and fails more often than
> > not after a cvs update 

> Sorry, but I don't understand what is wrong with the build system.
> I don't see what you don't like about the build system.  Could you
> be more specific?  Maybe you can just describe what you did and
> what went wrong.

Sorry I wasn't more informative before. The symptoms are varied but in 
general the build system seems very sensitive to changes. The last problem I 
had was related to the Makefile.in in regex not being regenerated. make 
clean, automake or anything else I could think of did nothing until I removed 
Makefile.in and ran automake. Then everything worked ok.  This was after a 
cvs update, not an edit of mine.  

> P.S.  Do you still have precision problem with AIF loader?  Please
> let me know.

I was trying to rebuild from the HEAD in order to test this but couldn't! 

Steve

Re: [agtk-devel] Problems building sources

[Haejoong L. <hae...@un...>]

> I'm having repeated problems building from the CVS sources. The whole build 
> system seems very sensitive to changes and fails more often than not after a 
> cvs update or hand edit when I want to recompile cleanly. Mainly this seems 
> to be to do with Makefile.in files needing regenerating etc.  
> 
> I'm not familiar enough with automake etc. to see easily what the answer is 
> but I hope that things could work better than this.
> 
> Sorry this is just a whinge at the moment, I'll try to be more constructive 
> if I get time to explore.

Sorry, but I don't understand what is wrong with the build system.
I don't see what you don't like about the build system.  Could you
be more specific?  Maybe you can just describe what you did and
what went wrong.

It is recommended that you not modify Makefile.in.  It's automatically 
generated by GNU automake.  You can play with Makefile.am or configure.in
files.  In that case, however, make sure that you have latest automake,
autoconf and maybe libtool packages installed.

If you want to learn more about autotools, there is a great reference
on the web:

    http://sources.redhat.com/autobook/

Thanks,
Haejoong

P.S.  Do you still have precision problem with AIF loader?  Please
let me know.

[agtk-devel] Problems building sources

[Steve C. <ste...@mq...>]

I'm having repeated problems building from the CVS sources. The whole build 
system seems very sensitive to changes and fails more often than not after a 
cvs update or hand edit when I want to recompile cleanly. Mainly this seems 
to be to do with Makefile.in files needing regenerating etc.  

I'm not familiar enough with automake etc. to see easily what the answer is 
but I hope that things could work better than this.

Sorry this is just a whinge at the moment, I'll try to be more constructive 
if I get time to explore.

Steve