hfst-info — Info to project members

[hfst-info] Macports tree for mac build

[Tommi A. P. <tom...@he...>]

The statically linked mac binaries are gone. The main support for mac
in future versions will be handled through MacPorts packages
distributed in a tarball, and hopefully, via macports main repo at some
point in the future (n.b. openfst is already in macports tree).

Re: [hfst-info] Lookup Limit Not Working?

[Tommi A. P. <tom...@he...>]

2012-06-30, Dougal Graham sanoi:

> I just updated to HFST 3.3.12 from 3.3.4. I've had this issue with
> both versions, was hoping 3.3.12 would fix it. Anyway, when I try to
> use the HFST
> 
>     lookup (const std::string &s, ssize_t limit=-1) const
> 
> method it performs the lookup fine, but it doesn't limit the results.
> An example call I might make would look like
> 
>     hfst::HfstOneLevelPaths *lookupResults =
> transducer->lookup(query, 5);
> 
> Is this a bug (known or unknown)? If so, where can I report it?

Sounds like a bug. This is from the --help message of tools:

Report bugs to <hfs...@he...> or directly to our bug tracker at:
<https://sourceforge.net/tracker/?atid=1061990&group_id=224521&func=browse>

if you know, also include the format of automata used. Thanks for the
report.

-- 
Tommi A. Pirinen, tietojenkäsittelijälukki sekä kieli-, puhe- ja
käännösteknologimestari <http://www.helsinki.fi/~tapirine/>

[hfst-info] Lookup Limit Not Working?

[Dougal G. <do...@gm...>]

Dear All,

I just updated to HFST 3.3.12 from 3.3.4. I've had this issue with
both versions, was hoping 3.3.12 would fix it. Anyway, when I try to
use the HFST

    lookup (const std::string &s, ssize_t limit=-1) const

method it performs the lookup fine, but it doesn't limit the results.
An example call I might make would look like

    hfst::HfstOneLevelPaths *lookupResults = transducer->lookup(query, 5);

Is this a bug (known or unknown)? If so, where can I report it?

Thanks, and take care,

Dougal Graham

Re: [hfst-info] Doing stuff after lookup (C++ API)

[Tommi A. P. <tom...@he...>]

2012-01-29, Dougal Graham sanoi:

> Of course, I realise that I am not checking for number of results, if
> there is a single result, only an HfstOneLevelPath object will be
> returned, right? So for working code I'd need to do a bit more
> checking for NULL results and so forth.

I think all typical situations will be covered without NULL handling,
one or zero results is not special in that way, for iterating through
STL objects. However, it always pays off to check for NULL result when
dealing with pointers, since that is what may happen on
unexpected errors (although I would hope HFST code never returns NULLs
in such situations).

> Also, can I ask what the reason for using a vector rather than a
> standard string is? (I'm sorry if this is an obvious question, but as
> I said, I'm new to C++)

That's a very good question; there are two reasons for using vector of
strings both of which are non-obvious. Firstly, since each arc of an
automata can contain string of multiple characters, certain very
unusual cases of overlapping strings can cause two theoretically very
different paths to have same strings as end result. Assume automaton A
with path of three transitions:

q_0 -c-> q_1 -a-> q_2 -t-> q_3

and automaton B with one transtion:

q_0 -cat-> q_3

The automata have very different structure, but lookup of cat would
result "cat" for both, to retain the distinction we store each arc as 
separate string in a vector.

Other, more pragmatic explanation is, that HFST interface was built
to replicate features of pre-existing tools, so printing each arc
separated by spaces after lookup is necessary.

-- 
Tommi A. Pirinen, tietojenkäsittelijälukki sekä kieli-, puhe- ja
käännösteknologimestari <http://www.helsinki.fi/~tapirine/>

Re: [hfst-info] Doing stuff after lookup (C++ API)

[Tommi A. P. <tom...@he...>]

2012-01-28, Dougal Graham sanoi:

> I hope this isn't documented somewhere, but I looked around for it and
> couldn't find anything. Just as a beginning I am trying to write a
> simple script that will read in a transducer, perform, a lookup and
> print out all the results.

Well, it's not directly documented I believe, but tracking down in API
is possible: from <http://hfst.sourceforge.net/hfst3/index.html> take
Classes -> HfstTransducer -> lookup() -> HfstOneLevelPaths ->
HfstOneLevelPath (and possibly <http://www.cplusplus.com/reference/>
for set, pair, vector and string)

> I can't figure out how to access the results from an HfstOneLevelPaths
> object. The documentation says it's a set of HfstOneLevelPath objects,
> so I tried to create an iterator:
> 
> 	results = t.lookup("blah");
> 	HfstOneLevelPaths::const_iterator pos;
> 	for (pos = results.begin(); pos != results.end(); ++pos) {
> 		std::cout << *pos << ' ';
> 	}
> 
> This gives me an error stating that "results" is a non-class type and
> has no "begin" method.
> 
> 	testCPP/main.cpp:50: error: request for member 'begin' in
> 'results', which is of non-class type 'hfst::HfstOneLevelPaths*'

This is g++ way of saying you are trying to use . operator on pointer
instead of ->, a very common mistake:

for (pos = results->begin(); 
	pos != results->end();
	++pos)
{}

> I couldn't find any examples of how to do this properly in the
> documentation, so could anyone elaborate on the best way to extract
> result strings from a lookup operation? Ideally, I'd just like a
> simple set of strings that I can print out/modify/etc at the end. Any
> examples for ways of dealing with lookups would be appreaciated!

As far as examples go,
<http://hfst.svn.sourceforge.net/viewvc/hfst/trunk/hfst3/tools/src/hfst-lookup.cc?revision=1895&view=markup>
should be the main place to look, since its code for our central lookup
tool, but it's currently littered with spaghetti code for i/o
formatting, so it may be very hard to follow.

-- 
Tommi A. Pirinen, tietojenkäsittelijälukki sekä kieli-, puhe- ja
käännösteknologimestari <http://www.helsinki.fi/~tapirine/>

[hfst-info] Doing stuff after lookup (C++ API)

[Dougal G. <do...@gm...>]

Hi All,

First I want to say that this is a great project, and has been very
useful. I was able to get all the advantages of XFST's syntax as well
as SFST's into one final transducer.

I hope this isn't documented somewhere, but I looked around for it and
couldn't find anything. Just as a beginning I am trying to write a
simple script that will read in a transducer, perform, a lookup and
print out all the results.

I can't figure out how to access the results from an HfstOneLevelPaths
object. The documentation says it's a set of HfstOneLevelPath objects,
so I tried to create an iterator:

	results = t.lookup("blah");
	HfstOneLevelPaths::const_iterator pos;
	for (pos = results.begin(); pos != results.end(); ++pos) {
		std::cout << *pos << ' ';
	}

This gives me an error stating that "results" is a non-class type and
has no "begin" method.

	testCPP/main.cpp:50: error: request for member 'begin' in 'results',
which is of non-class type 'hfst::HfstOneLevelPaths*'


I couldn't find any examples of how to do this properly in the
documentation, so could anyone elaborate on the best way to extract
result strings from a lookup operation? Ideally, I'd just like a
simple set of strings that I can print out/modify/etc at the end. Any
examples for ways of dealing with lookups would be appreaciated!

Also, I'm a bit of a C++ newbie so if that's the problem here, sorry!

Thank you very much,

Dougal Graham

Re: [hfst-info] Wrtiting files out so foma can read them in

[Tommi A. P. <tom...@he...>]

2011-09-20, Benson Margulies sanoi:

> hfst-fst2fst produces a file that has an hfst wrapper around the foma
> format. Is there a way to write out what foma can read in with 'load'?

You use the -b command line switch. This information can also be found
in:

 * my previous message
 * the --help message of hfst-fst2fst program
 * the man page of hfst-fst2fst
 * the wiki page of hfst-fst2fst
   <https://kitwiki.csc.fi/twiki/bin/view/KitWiki//HfstFst2Fst>

-- 
Tommi A. Pirinen, tietojenkäsittelijälukki sekä kieli-, puhe- ja
käännösteknologimestari <http://www.helsinki.fi/~tapirine/>

Re: [hfst-info] foma versus hfst and 'load'

[Tommi A. P. <tom...@he...>]

2011-09-20, Benson Margulies sanoi:

> In the following script, I suppose that I should not be surprised that
> foma can't read the default output of hfst. I'm stumped as to how to
> convert the output of hfst-invert, e.g., to the 'att' format (?) that
> foma consumes.

Foma format can be produced by hfst-fst2fst -f foma -b | gzip - (or
equivalent). AT&T tabulars can be produced with hfst-fst2txt for foma's
read att or load att.

-- 
Tommi A. Pirinen, tietojenkäsittelijälukki sekä kieli-, puhe- ja
käännösteknologimestari <http://www.helsinki.fi/~tapirine/>

[hfst-info] Wrtiting files out so foma can read them in

[Benson M. <bim...@gm...>]

hfst-fst2fst produces a file that has an hfst wrapper around the foma
format. Is there a way to write out what foma can read in with 'load'?

Re: [hfst-info] Setting flags?

[Tommi A. P. <tom...@he...>]

2011-09-20, Benson Margulies sanoi:

> set flag-is-epsilon ON
> load /Users/benson/x/hfst/ETROG-813/klex-hfst/data/rewrite_rules/rules.fst
> load /Users/benson/x/hfst/ETROG-813/klex-hfst/data/lexicon/all-lexicon.fst
> compose net
> save klex-rab-rom.fst
> 
> So I'm thinking I need to use hfst-compose, but then, how do I set
> the flag?

hfst-compose has flag-is-epsilon kind of functionality tied to -F, but
if you really need to compile xfst-heavy morphology you'd be better of
using foma. HFST has full support of foma transducers since foma when
foma is compiled in.
-- 
Tommi A. Pirinen, tietojenkäsittelijälukki sekä kieli-, puhe- ja
käännösteknologimestari <http://www.helsinki.fi/~tapirine/>

[hfst-info] foma versus hfst and 'load'

[Benson M. <bim...@gm...>]

In the following script, I suppose that I should not be surprised that
foma can't read the default output of hfst. I'm stumped as to how to
convert the output of hfst-invert, e.g., to the 'att' format (?) that
foma consumes.

load klex-rom-kor.inverted.fst
define net1
read regex net1 .o. P->p .o. T->t .o. S->s .o. H->h .o. L->l .o. R->l;
define net2
load /Users/benson/x/hfst/ETROG-813/klex-hfst/data/encoding/realroman.fst
push net2
save stack klex-rom-kor.net2.fst

[hfst-info] hfst-compose --inputN options don't seem to work

[Benson M. <bim...@gm...>]

hfst-compose --insert-missing-flags
--input1=/Users/benson/x/hfst/ETROG-813/klex-hfst/data/rewrite_rules/rules.fst
--input2=/Users/benson/x/hfst/ETROG-813/klex-hfst/data/lexicon/all-lexicon.fst
--output=klex-rab-rom.fst

hfst-compose: <stdin> is not a valid transducer file
make: *** [klex-rab-rom.fst] Error 1

[hfst-info] Setting flags?

[Benson M. <bim...@gm...>]

So, running this into hfst-xfst results in a foma error on the set.

set flag-is-epsilon ON
load /Users/benson/x/hfst/ETROG-813/klex-hfst/data/rewrite_rules/rules.fst
load /Users/benson/x/hfst/ETROG-813/klex-hfst/data/lexicon/all-lexicon.fst
compose net
save klex-rab-rom.fst

So I'm thinking I need to use hfst-compose, but then, how do I set the flag?

Re: [hfst-info] foma versus hfst-xfst

[Tommi A P. <tom...@he...>]

Benson Margulies wrote:
> I am puzzled by how saving work. If I add a 'save stack' command to
> the end of an xfst script, foma reads it and ?executes it?. So, when I
> run from hfst-xfst, how does it end up in the hfst format?

hfst-xfst greps the script and runs hfst-convert for all saved stacks. 
It's really simple but works just enough for all the trivial use cases.

[hfst-info] foma versus hfst-xfst

[Benson M. <bim...@gm...>]

I am puzzled by how saving work. If I add a 'save stack' command to
the end of an xfst script, foma reads it and ?executes it?. So, when I
run from hfst-xfst, how does it end up in the hfst format?

Re: [hfst-info] Hit syntax limitation, can anyone give me a hand?

[Tommi A. P. <tom...@he...>]

2011-09-19, Benson Margulies sanoi:

> %+ -> [..] || \%^CNT _ ;
> 
> and gets a 'syntax error at ||'.
> 
> Could some kind soul let me know what sort of problem I've got here?
> Or do I need to be pestering the foma people instead?

If it's foma that gives the error message there ,then it is most likely
missing support for that syntax in foma. I know for sure that
hfst-regexp2fst won't support that yet nor hfst-lexc's native parsers.

-- 
Tommi A. Pirinen, tietojenkäsittelijälukki sekä kieli-, puhe- ja
käännösteknologimestari <http://www.helsinki.fi/~tapirine/>

Re: [hfst-info] Building 3.2.0

[Tommi A. P. <tom...@he...>]

2011-09-19, Benson Margulies sanoi:

> I can build 3.2.0 with sfst 1.4.2, but not 1.4.6a or b.
> 
> In the hfst tree,
> ./libhfst/src/implementations/ConvertSfstTransducer.cc references
> SFST::NodeNumbering.
> 
> find and grep shows that in sfst 1.4.2, but not 1.4.6. And compiles of
> hfst fail complaining that they can't find NodeNumbering.
> 
> Are the web pages wrong, or is something else peculiar going on?

I fixed the description now, thanks for notifying.
-- 
Tommi A. Pirinen, tietojenkäsittelijälukki sekä kieli-, puhe- ja
käännösteknologimestari <http://www.helsinki.fi/~tapirine/>

[hfst-info] Hit syntax limitation, can anyone give me a hand?

[Benson M. <bim...@gm...>]

I'm looking at the KLEX fsts available from the LDC.

One file ends with:

%+ -> [..] || \%^CNT _ ;

and gets a 'syntax error at ||'.

Could some kind soul let me know what sort of problem I've got here?
Or do I need to be pestering the foma people instead?

[hfst-info] Building 3.2.0

[Benson M. <bim...@gm...>]

I can build 3.2.0 with sfst 1.4.2, but not 1.4.6a or b.

In the hfst tree,
./libhfst/src/implementations/ConvertSfstTransducer.cc references
SFST::NodeNumbering.

find and grep shows that in sfst 1.4.2, but not 1.4.6. And compiles of
hfst fail complaining that they can't find NodeNumbering.

Are the web pages wrong, or is something else peculiar going on?

Re: [hfst-info] Test

[Krister L. <kri...@he...>]

Krister Lindén skrev:
> This is a test that the mailing list is working.
>