doxygen-develop

[Doxygen-develop] new feature proposal: filter maps

[Francesco M. <f18...@ya...>]

Hi,
    I'd like to propose a new feature for Doxygen; if doxy devs find it 
interesting I'd be willing to provide a patch to implement it.

Using doxygen in a big project I started using doxygen grouping features 
and I must say they're very useful. However the groups created go in the 
HTML output under the "Modules" tab. Unfortunately this is very 
misleading in the context of that project.

I know doxygen can't know that I'd prefer "categories" as tab's label 
since I'm putting there groups of classes classified by category :)

However I think that it shouldn't be difficult to add to Doxygen a 
generic system which allows to override the default strings returned by 
the Translator-derived class selected for use.

I.e. we could add an option, say "OUTPUT_FILTER", which allows the user 
to specify a text file to use as "filter map".
Suppose I want in my project to have "Modules" label (returned by 
trModules() function) replaced by a "SomethingElse" label and  the text 
returned by trGeneratedAutomatically(const char*s) replaced by 
"Generated for " + s + " by Doxygen".
Then I could simply write in the "filter map":

trModules=SomethingElse
trGeneratedAutomatically=Generated for \1 by Doxygen

i.e. use argument substitution like it's done for ALIASes.

Doxygen modifications would consist in:
  1) creation of the control code for the new option
  2) in the setTranslator() function, if OUTPUT_FILTER option is 
non-empty, assign to theTranslator global the instance of a new 
TranslatorFilter (derived from Translator) chained to the "real" 
translator instance (e.g. TranslatorEnglish)
  3) write the TranslatorFilter; for each trXXX() function it should 
read the message from the filter map; if the message is there, just 
substitute \1, \2... and return the custom text; otherwise return 
m_realTranslator->trXXX();

Using a static "filter map" as I suggested means however that functions 
like trClass() which takes boolean to "decide" to return a plural form 
or not, and in general all trXXX() functions which involve "decision" 
could not be filtered in a smart way (i.e. doing decisions), since 
decision constructs would be an overkill for both the filter map parser 
and in general for this feature.

However the trXXX() functions which take such booleans are really few 
and the main things one may need to filter are argument-less trXXX() 
functions.

Dimitri, what do you think of this idea?
It should be an easy-to-implement and fairly robust solution, which 
avoids the user to install sed and write sed postprocessing scripts to 
fix the naming of some fixed labels in the output....

Thanks,
Francesco

Re: [Doxygen-develop] new feature proposal: filter maps

[Oleg B. <og...@gm...>]

Hi,

>  However I think that it shouldn't be difficult to add to Doxygen a
>  generic system which allows to override the default strings returned by
>  the Translator-derived class selected for use.
I would move "resourece strings" and localization completely to text
conf files. It worked very well in my past experience with Java and
Python.
And compilation time translator files take is huge.

>  I.e. we could add an option, say "OUTPUT_FILTER", which allows the user
>  to specify a text file to use as "filter map".
>  Suppose I want in my project to have "Modules" label (returned by
>  trModules() function) replaced by a "SomethingElse" label and  the text
>  returned by trGeneratedAutomatically(const char*s) replaced by
>  "Generated for " + s + " by Doxygen".
>  Then I could simply write in the "filter map":
>
>  trModules=SomethingElse
>  trGeneratedAutomatically=Generated for \1 by Doxygen
Yes, in Python I would use something like
module=module
generatedAutomatically=Generated for %{project-name} by Doxygen

So, you just need to tweak a resource (localization) file and pass it
to doxygen as a replacement to original.

But it is quite an effort to reimplement.

Regards,
Oleg

Re: [Doxygen-develop] new feature proposal: filter maps

[Francesco M. <f18...@ya...>]

Oleg Batrashev ha scritto:
> Hi,
> 
>>  However I think that it shouldn't be difficult to add to Doxygen a
>>  generic system which allows to override the default strings returned by
>>  the Translator-derived class selected for use.
> I would move "resourece strings" and localization completely to text
> conf files. It worked very well in my past experience with Java and
> Python.
however how do you take "decisions" in text conf files?
some functions take boolean arguments and output different strings 
dependending on it...
I was proposing something simpler only for project-specific filtering; 
changing all doxygen's Translator class implementations seems 
inefficient to me (doxygen would be slower since it needs to read the 
text conf file).

> And compilation time translator files take is huge.
I don't think this is a major problem... not many doxygen users compile 
it from sources I think (all major distro have it packaged) :)


>>  I.e. we could add an option, say "OUTPUT_FILTER", which allows the user
>>  to specify a text file to use as "filter map".
>>  Suppose I want in my project to have "Modules" label (returned by
>>  trModules() function) replaced by a "SomethingElse" label and  the text
>>  returned by trGeneratedAutomatically(const char*s) replaced by
>>  "Generated for " + s + " by Doxygen".
>>  Then I could simply write in the "filter map":
>>
>>  trModules=SomethingElse
>>  trGeneratedAutomatically=Generated for \1 by Doxygen
ops; here I wanted to write

trGeneratedAutomatically{1}=Generated for \1 by Doxygen

actually.

> Yes, in Python I would use something like
> module=module
> generatedAutomatically=Generated for %{project-name} by Doxygen
hmmm, I think reusing the same ALIASes syntax would be simpler and more 
intuitive for the user...

> So, you just need to tweak a resource (localization) file and pass it
> to doxygen as a replacement to original.
> 
> But it is quite an effort to reimplement.
in fact... my approach seems easier to me.

Thanks anyway for your suggestions :)

Francesco

[Doxygen-develop] new feature proposal: filter maps

[Oleg B. <og...@gm...>]

>  I was proposing something simpler only for project-specific filtering;
 >  changing all doxygen's Translator class implementations seems
 >  inefficient to me (doxygen would be slower since it needs to read the
 >  text conf file).
 You only need to read and parse one small conf file in the beginning.


 >
 >  > And compilation time translator files take is huge.
 >  I don't think this is a major problem... not many doxygen users compile
 >  it from sources I think (all major distro have it packaged) :)
 Agree, but a it is still a little annoying for me :)


 >  > But it is quite an effort to reimplement.
 >  in fact... my approach seems easier to me.
 Maybe it is really not that related to what you want. I've got this
 idea about translators about a year, so you pushed the button ;)



 >
 >  Thanks anyway for your suggestions :)

 Oleg

Re: [Doxygen-develop] new feature proposal: filter maps

[Francesco M. <f18...@ya...>]

Hi,
    I have some free time in these days and I'd like to implement this 
new feature if it's ok to do it as I described... however I still have 
got no clear comments in this sense...

Please tell me if a patch implementing this proposal would be accepted.

Thanks,
Francesco


Francesco Montorsi ha scritto:
> Hi,
>     I'd like to propose a new feature for Doxygen; if doxy devs find it 
> interesting I'd be willing to provide a patch to implement it.
> 
> Using doxygen in a big project I started using doxygen grouping features 
> and I must say they're very useful. However the groups created go in the 
> HTML output under the "Modules" tab. Unfortunately this is very 
> misleading in the context of that project.
> 
> I know doxygen can't know that I'd prefer "categories" as tab's label 
> since I'm putting there groups of classes classified by category :)
> 
> However I think that it shouldn't be difficult to add to Doxygen a 
> generic system which allows to override the default strings returned by 
> the Translator-derived class selected for use.
> 
> I.e. we could add an option, say "OUTPUT_FILTER", which allows the user 
> to specify a text file to use as "filter map".
> Suppose I want in my project to have "Modules" label (returned by 
> trModules() function) replaced by a "SomethingElse" label and  the text 
> returned by trGeneratedAutomatically(const char*s) replaced by 
> "Generated for " + s + " by Doxygen".
> Then I could simply write in the "filter map":
> 
> trModules=SomethingElse
> trGeneratedAutomatically=Generated for \1 by Doxygen
> 
> i.e. use argument substitution like it's done for ALIASes.
> 
> Doxygen modifications would consist in:
>   1) creation of the control code for the new option
>   2) in the setTranslator() function, if OUTPUT_FILTER option is 
> non-empty, assign to theTranslator global the instance of a new 
> TranslatorFilter (derived from Translator) chained to the "real" 
> translator instance (e.g. TranslatorEnglish)
>   3) write the TranslatorFilter; for each trXXX() function it should 
> read the message from the filter map; if the message is there, just 
> substitute \1, \2... and return the custom text; otherwise return 
> m_realTranslator->trXXX();
> 
> Using a static "filter map" as I suggested means however that functions 
> like trClass() which takes boolean to "decide" to return a plural form 
> or not, and in general all trXXX() functions which involve "decision" 
> could not be filtered in a smart way (i.e. doing decisions), since 
> decision constructs would be an overkill for both the filter map parser 
> and in general for this feature.
> 
> However the trXXX() functions which take such booleans are really few 
> and the main things one may need to filter are argument-less trXXX() 
> functions.
> 
> Dimitri, what do you think of this idea?
> It should be an easy-to-implement and fairly robust solution, which 
> avoids the user to install sed and write sed postprocessing scripts to 
> fix the naming of some fixed labels in the output....
> 
> Thanks,
> Francesco
> 
> 
> 
> 
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [Doxygen-develop] new feature proposal: filter maps

[Francesco M. <f18...@ya...>]

Hi,
    I have some free time in these days and I'd like to implement this 
new feature if it's ok to do it as I described... however I still have 
got no clear comments in this sense...

Please tell me if a patch implementing this proposal would be accepted.

Thanks,
Francesco


Francesco Montorsi ha scritto:
> Hi,
>     I'd like to propose a new feature for Doxygen; if doxy devs find it 
> interesting I'd be willing to provide a patch to implement it.
> 
> Using doxygen in a big project I started using doxygen grouping features 
> and I must say they're very useful. However the groups created go in the 
> HTML output under the "Modules" tab. Unfortunately this is very 
> misleading in the context of that project.
> 
> I know doxygen can't know that I'd prefer "categories" as tab's label 
> since I'm putting there groups of classes classified by category :)
> 
> However I think that it shouldn't be difficult to add to Doxygen a 
> generic system which allows to override the default strings returned by 
> the Translator-derived class selected for use.
> 
> I.e. we could add an option, say "OUTPUT_FILTER", which allows the user 
> to specify a text file to use as "filter map".
> Suppose I want in my project to have "Modules" label (returned by 
> trModules() function) replaced by a "SomethingElse" label and  the text 
> returned by trGeneratedAutomatically(const char*s) replaced by 
> "Generated for " + s + " by Doxygen".
> Then I could simply write in the "filter map":
> 
> trModules=SomethingElse
> trGeneratedAutomatically=Generated for \1 by Doxygen
> 
> i.e. use argument substitution like it's done for ALIASes.
> 
> Doxygen modifications would consist in:
>   1) creation of the control code for the new option
>   2) in the setTranslator() function, if OUTPUT_FILTER option is 
> non-empty, assign to theTranslator global the instance of a new 
> TranslatorFilter (derived from Translator) chained to the "real" 
> translator instance (e.g. TranslatorEnglish)
>   3) write the TranslatorFilter; for each trXXX() function it should 
> read the message from the filter map; if the message is there, just 
> substitute \1, \2... and return the custom text; otherwise return 
> m_realTranslator->trXXX();
> 
> Using a static "filter map" as I suggested means however that functions 
> like trClass() which takes boolean to "decide" to return a plural form 
> or not, and in general all trXXX() functions which involve "decision" 
> could not be filtered in a smart way (i.e. doing decisions), since 
> decision constructs would be an overkill for both the filter map parser 
> and in general for this feature.
> 
> However the trXXX() functions which take such booleans are really few 
> and the main things one may need to filter are argument-less trXXX() 
> functions.
> 
> Dimitri, what do you think of this idea?
> It should be an easy-to-implement and fairly robust solution, which 
> avoids the user to install sed and write sed postprocessing scripts to 
> fix the naming of some fixed labels in the output....
> 
> Thanks,
> Francesco
> 
> 
> 
> 
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [Doxygen-develop] new feature proposal: filter maps

[Dimitri V. H. <do...@gm...>]

Hi Francesco,

On 21 mrt 2008, at 10:59, Francesco Montorsi wrote:

> Hi,
>    I have some free time in these days and I'd like to implement this
> new feature if it's ok to do it as I described... however I still have
> got no clear comments in this sense...
>
> Please tell me if a patch implementing this proposal would be  
> accepted.
>
> Thanks,
> Francesco
>
>
> Francesco Montorsi ha scritto:
>> Hi,
>>    I'd like to propose a new feature for Doxygen; if doxy devs find  
>> it
>> interesting I'd be willing to provide a patch to implement it.
>>
>> Using doxygen in a big project I started using doxygen grouping  
>> features
>> and I must say they're very useful. However the groups created go  
>> in the
>> HTML output under the "Modules" tab. Unfortunately this is very
>> misleading in the context of that project.
>>
>> I know doxygen can't know that I'd prefer "categories" as tab's label
>> since I'm putting there groups of classes classified by category :)
>>
>> However I think that it shouldn't be difficult to add to Doxygen a
>> generic system which allows to override the default strings  
>> returned by
>> the Translator-derived class selected for use.
>>
>> I.e. we could add an option, say "OUTPUT_FILTER", which allows the  
>> user
>> to specify a text file to use as "filter map".
>> Suppose I want in my project to have "Modules" label (returned by
>> trModules() function) replaced by a "SomethingElse" label and  the  
>> text
>> returned by trGeneratedAutomatically(const char*s) replaced by
>> "Generated for " + s + " by Doxygen".
>> Then I could simply write in the "filter map":
>>
>> trModules=SomethingElse
>> trGeneratedAutomatically=Generated for \1 by Doxygen
>>
>> i.e. use argument substitution like it's done for ALIASes.
>>
>> Doxygen modifications would consist in:
>>  1) creation of the control code for the new option
>>  2) in the setTranslator() function, if OUTPUT_FILTER option is
>> non-empty, assign to theTranslator global the instance of a new
>> TranslatorFilter (derived from Translator) chained to the "real"
>> translator instance (e.g. TranslatorEnglish)
>>  3) write the TranslatorFilter; for each trXXX() function it should
>> read the message from the filter map; if the message is there, just
>> substitute \1, \2... and return the custom text; otherwise return
>> m_realTranslator->trXXX();
>>
>> Using a static "filter map" as I suggested means however that  
>> functions
>> like trClass() which takes boolean to "decide" to return a plural  
>> form
>> or not, and in general all trXXX() functions which involve "decision"
>> could not be filtered in a smart way (i.e. doing decisions), since
>> decision constructs would be an overkill for both the filter map  
>> parser
>> and in general for this feature.
>>
>> However the trXXX() functions which take such booleans are really few
>> and the main things one may need to filter are argument-less trXXX()
>> functions.
>>
>> Dimitri, what do you think of this idea?
>> It should be an easy-to-implement and fairly robust solution, which
>> avoids the user to install sed and write sed postprocessing scripts  
>> to
>> fix the naming of some fixed labels in the output....

It sounds a bit too much like a hack to me, so I'm not too  
enthusiastic about it.
A better solution would then be to allow translators to be plugins  
loaded at
startup. Either as a dynamically loadable library (but that is  
difficult to do in a portable way)
or as an XML file. One can even go one step further and implemented  
item 9 on the todo
list (http://www.doxygen.org/todo.html), i.e. a template file that  
defines the structure of the output.

Regards,
   Dimitri

Re: [Doxygen-develop] new feature proposal: filter maps

[Francesco M. <f18...@ya...>]

Dimitri Van Heesch ha scritto:
>>> Dimitri, what do you think of this idea?
>>> It should be an easy-to-implement and fairly robust solution, which
>>> avoids the user to install sed and write sed postprocessing scripts  
>>> to
>>> fix the naming of some fixed labels in the output....
> 
> It sounds a bit too much like a hack to me, so I'm not too  
> enthusiastic about it.
> A better solution would then be to allow translators to be plugins  
> loaded at startup. Either as a dynamically loadable library (but that is  
> difficult to do in a portable way)
this would do the job but as you say it would be impossible for the 
users to have a single DSO for all platforms where doxygen must be able 
to run (e.g. linuxes with different versions of libc, linux/win, etc).

Also it would be a very unconvenient way to change labels in the 
generated output: you'd have to write a C file, compile it, link it, 
test it...

> or as an XML file.
this is much more easier to do but would an XML translation file cope 
with trXXX() functions which take booleans/strings ? The latter can be 
just used as replacements for $1, $2 (or similar syntaxed) strings.
The first however needs a "decisional" syntax... which is not part of XML.

> One can even go one step further and implemented  
> item 9 on the todo
> list (http://www.doxygen.org/todo.html), i.e. a template file that  
> defines the structure of the output.
this would be the best (in the sense "more complete") solution.
It could even make possible to get XHTML output.

However I think that either we invent a new programming language ad-hoc 
for such template files (which would be quite complex and would produce 
probably a buggy implementation), or use some existing embeddable 
interpreter. Unfortunately embeddable interpreters are not very used: a 
Python interpreter is very difficult to embed AFAIK.
C++ interpreters exist (see e.g. UnderC as open source product) but are 
quite a niche category of software.

Something else which comes to my mind is that if Doxygen could output 
XHTML instead of HTML4, it would be easy to change "fixed labels" (and 
not only them!) doing an XSLT transformation on it...
Also, the XSLT solution over an XHTML output of doxygen would be 
possible only for the XHTML output itself: latex/rtf/etc would still 
have those "fixed labels".

In conclusion I think that in order to make the doxygen "fixed labels" 
more configurable two solutions are realistically possible:

  1) create a special Translator as I proposed

or

  2) make the HTML4 output XHTML compliant so that XSLT can be used on it


do I miss something?

Francesco

Re: [Doxygen-develop] new feature proposal: filter maps

[Francesco M. <f18...@ya...>]

Hi,

Oleg Batrashev ha scritto:
>>  do I miss something?
> I think yes, you do ;) I still push my solution which is close to XML one.
> 
> In translation file you could have one default section and specialized
> sections fot C, Fortran, etc which override default section strings
> (or use XML analogue)
> ------
>...
> ------
> 
> You will still have one Translator class but there are 2 possibilities
> 1) You create array of strings for every section. You leave booleans,
> string arguments and conditional statements as they are
it's still not clear to me how such a text-conf approach could cope with 
these booleans and conditional statements... take e.g. 
trCompoundReference() function:

     /*! used as the title of the HTML page of a class/struct/union */
     virtual QCString trCompoundReference(const char *clName,
                                     ClassDef::CompoundType compType,
                                     bool isTemplate)
     {
       QCString result=(QCString)clName;
       switch(compType)
       {
         case ClassDef::Class:      result+=" Class"; break;
         case ClassDef::Struct:     result+=" Struct"; break;
         case ClassDef::Union:      result+=" Union"; break;
         case ClassDef::Interface:  result+=" Interface"; break;
         case ClassDef::Protocol:   result+=" Protocol"; break;
         case ClassDef::Category:   result+=" Category"; break;
         case ClassDef::Exception:  result+=" Exception"; break;
       }
       if (isTemplate) result+=" Template";
       result+=" Reference";
       return result;
     }

how could you write it in a text-conf file?

Francesco

Re: [Doxygen-develop] new feature proposal: filter maps

[Oleg B. <og...@gm...>]

>  it's still not clear to me how such a text-conf approach could cope with
>  these booleans and conditional statements... take e.g.
>  trCompoundReference() function:
>
>      /*! used as the title of the HTML page of a class/struct/union */
>      virtual QCString trCompoundReference(const char *clName,
>                                      ClassDef::CompoundType compType,
>                                      bool isTemplate)
>      {
>        QCString result=(QCString)clName;
>        switch(compType)
>        {
>          case ClassDef::Class:      result+=" Class"; break;
>          case ClassDef::Struct:     result+=" Struct"; break;
>          case ClassDef::Union:      result+=" Union"; break;
>          case ClassDef::Interface:  result+=" Interface"; break;
>          case ClassDef::Protocol:   result+=" Protocol"; break;
>          case ClassDef::Category:   result+=" Category"; break;
>          case ClassDef::Exception:  result+=" Exception"; break;
>        }
>        if (isTemplate) result+=" Template";
>        result+=" Reference";
>        return result;
>      }
>
>  how could you write it in a text-conf file?
You leave all these conditionals there in the class, just add
tranlations for Class, Struct, Union, ... and Template.
Number of translate words does not have to be the same as number of tr
functions.

Oleg

Re: [Doxygen-develop] new feature proposal: filter maps

[Francesco M. <f18...@ya...>]

Oleg Batrashev ha scritto:
>>  >  how could you write it in a text-conf file?
>>  You leave all these conditionals there in the class, just add
>>  tranlations for Class, Struct, Union, ... and Template.
>>  Number of translate words does not have to be the same as number of tr
>>  functions.
> And you may probably have to keep them separate from just Class,
> Struct, ... translations because  languages other than english may use
> different forms. Although, as I heard i18n has the same problem.
I don't like very much the idea of breaking the labels in lots of 
separed words.

Consider also functions like:

     virtual QCString trMember(bool first_capital, bool singular)
     {
       QCString result((first_capital ? "Member" : "member"));
       if (!singular)  result+="s";
       return result;
     }

I can imagine that its implementation is different in other languages. 
And splitting the words to translate as "member" and "s" won't help :(

Plural form handling in GNU gettext is a bit complicated, in fact:

   http://www.gnu.org/software/gettext/manual/gettext.html#Plural-forms

Maybe Doxygen should simply use .po/.mo files after all...  both in this 
case and in the text-conf case however, we'd need to rewrite all 
translation files, which involves help from all translators to review 
tons of changes in the translation files... I'm not sure this is a good 
choice.

Francesco

Re: [Doxygen-develop] new feature proposal: filter maps

[Oleg B. <og...@gm...>]

>  I can imagine that its implementation is different in other languages.
>  And splitting the words to translate as "member" and "s" won't help :(
>
>  Plural form handling in GNU gettext is a bit complicated, in fact:
>
>    http://www.gnu.org/software/gettext/manual/gettext.html#Plural-forms
I give up, it is too cruel, I only deal with programming languages.
Maybe, it is really better to just create specialized Translator
class.

Oleg

Re: [Doxygen-develop] new feature proposal: filter maps

[Oleg B. <og...@gm...>]

>  In conclusion I think that in order to make the doxygen "fixed labels"
>  more configurable two solutions are realistically possible:
>
>   1) create a special Translator as I proposed
>
>  or
>
>   2) make the HTML4 output XHTML compliant so that XSLT can be used on it
>
>
>  do I miss something?
I think yes, you do ;) I still push my solution which is close to XML one.

In translation file you could have one default section and specialized
sections fot C, Fortran, etc which override default section strings
(or use XML analogue)
------
[DEFAULT]
trModules = Modules
trNamespaces = Namespaces
trHello = Hello \1

[FORTRAN]
trModules = Categories
trNamespaces = Modules

[C]
trHello = Hello C, \1 !
------

You will still have one Translator class but there are 2 possibilities
1) You create array of strings for every section. You leave booleans,
string arguments and conditional statements as they are but choose
different array if OPTIMIZE_FOR_X is set.
2) On doxygen startup, when you read translation file, you override
strings in the same, and only one, array.

Now, you only need to specify [USER] section (in user defined file)
which overrides definitions in all other sections.
Yes, and of course you need something like printf functionality for \1
arguments.

Need to look into other translator classes if they have more specialized code.

Oleg

Re: [Doxygen-develop] new feature proposal: filter maps

[Oleg B. <og...@gm...>]

>  >  how could you write it in a text-conf file?
>  You leave all these conditionals there in the class, just add
>  tranlations for Class, Struct, Union, ... and Template.
>  Number of translate words does not have to be the same as number of tr
>  functions.
And you may probably have to keep them separate from just Class,
Struct, ... translations because  languages other than english may use
different forms. Although, as I heard i18n has the same problem.

Oleg