#366 Export and import of segments

[Samuel Murray]

Problem:

Sometimes I can do stuff with my translation in some other program better than I can do it in OmegaT. Currently, doing so is a hack -- edit the project_save.tmx and use complex regular expressions to ensure I don't break anything.

One example is spell-checking. OmegaT has/will have Hunspell-capability, but not all languages have Hunspell-dictionaries, and sometimes the translator may have more faith in another spell checker than in the Hunspell-version for his language. One can spell-check the project_save.tmx file, but it is a hack.

Solution:

Allow the user to export the current state of the project to a plaintext file with each segment demarcated and numbered, so that he can do all sorts of editing on it in an outside program. When he is done, he can then import that file again.

When a user exports such a text, a warning should pop up reminding him that he can only import the text again if he had MADE NO CHANGES to the project between export time and import time.

Example:

The file would look like this:

[segment 001]
This is the contents of the first segment
[/segment 001]

[segment 002]
And this is the contents
of the second
segment
[/segment 002]

etc.

Issues/variations:

1. Only the target text of translated segments are exported.

2. Ideally, the user should be able to select which segments he wants to export, in case he uses this method a lot of times during the translation process and he doesn't want to redo a lot of stuff everytime. This can be done by asking the user for the starting and ending segment numbers in a dialog box, for example.

3. It should be possible for the tranlator to export multiple such files, then make changes to all of them, and then import them all again (one by one), as long as he didn't make any changes to the project in the mean time (eg added more source text files, or renamed some of them). This shouldn't be a problem because the user works on the target text, not the source text.

4. OpenOffice.org has a bug (or feature) whereby it automatically parses a file that looks like an XML file... so if a user were to open this exported file in OOo, there is a chance that OOo might interpret it and not open it as plain text. To combat this, it would be nice if OmegaT could create not only a plaintext expoort file but also an ODT file that contains the raw text of the export.

More uses:

1. If the user wants to do spell-checking or grammar checking in OOo or some other text editor.

2. If the user wants to make large-scale find/replace type of changes to his translation up to that point (eg if the client called in mid-project with terminology changes).

[Guido Leenders]

For what's worth, for exactly this reason we have an ant build file that handles all projects. Maybe it helps you a little further. It does an export using:

<xslt
      destdir="${translate.dir.projects}"
      style="${translate.dir.global.test}/translate-export.xsl"
      force="true"
      extension=".target4spellcheck.txt"
      filenameparameter="filename"
      filedirparameter="filedir"
      useImplicitFileset="false"
    >
      <fileset dir="${translate.dir.projects}">
<!-- These include orphan segments.        <include name="**/project_save.tmx" /> -->
        <include name="**/*-omegat.tmx" />
      </fileset>
      <factory name="net.sf.saxon.TransformerFactoryImpl" />
    </xslt>

And the XSL:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:fn="http://www.w3.org/2005/xpath-functions" >
<xsl:output method="text" omit-xml-declaration="yes" version="1.0" encoding="UTF-8"/>

<xsl:template match="//tu">

<!--
  Print sentences with the following replacements:

  * Remove all profile option names starting with bubs-.
  * Remove variable names with underscores in them.
  * Remove all resource references starting with {res:.
  * Replace all '-' by a space.
  * Replace all carriage returns by a space.
-->
<xsl:value-of
  select="replace(replace(replace(replace(replace(./tuv[2]/seg/text(), '[^ .,:;]*_[^ .,:;]*', ''), 'bubs-[^ ]*', ''), '\{res:[^}]*\}', ''), '-', ' '), '\\n', ' ')"
/><xsl:text>&#xa;&#xa;</xsl:text>

</xsl:template>

</xsl:stylesheet>

[Guido Leenders]

Export and import of (translated and untranslated) segments can also elevate the value of the many filters available for OmegaT to other translation platforms. In current use case, we have largely replaced OmegaT for translation work by Localazy.com due to more fine-grained control of the process. However, Localazy lacks segmentation completely. Back-and-forth import script are hard to write for instance when handling HTML.

[Jean-Christophe Helary]

You are probably aware of the various scripts that OmegaT distributes and that include an "export source and target to text files", among other things.

[st. pakebusch]

i have the same problem, i.e. i need to export certain filtered segments after search as csv and want to keep the newly added sorting order for the export - this is not available to plugins or scripts via existing api, so directly implementing the desired feature instead of incomplete workaround possibility seems indicated for my use case, so i am now working on the export part of this ticket.

import not covered (yet) - have to think about the approach, may be better to not have import into editor target segments directly, but use enforced translation via tmx. out of my scope, for now working only on export.

[Jean-Christophe Helary]

You are probably aware of Briac’s review plugin, that needs to be integrated into OmegaT, btw.
It exports selected project files to an ODT file with a table for editing. That same ODT file is used to import the modifications diirectly in memory.

There is a discussion about Briac’s code being used on more complex ODS files.

[st. pakebusch]

not yet, will look into that later, and my scope was really only simple export, not editing - solved my case already, opening PR https://github.com/omegat-org/omegat/pull/2336 there is a screenshot also.

maybe partly solves the initial reported use case also: use search + filter for translated or 'select' by writing special note into desired segments note field and search/filter for that, then export selected columns like seg-number, seg-id, source, target, author, date, .., adjust encoding+escaping settings for csv export, settings stored for next export, asks if overwrite existing file, - as i said this primarily solves my urgent problem, but seems also to solve half of this ticket

[Jean-Christophe Helary]

That’s the problem with your code. You solve "your case". I already told you that you could do that on your fork.

An "export" feature was already discussed here:

https://github.com/omegat-org/omegat/pull/2025

[st. pakebusch]

yes. you say the problem is, that i solve my problem. and that is correct. my problem is solved. then i try and share the solution, where others have reported similar problems. this is where i can indeed use my fork and have an internal integration version with all open PR's integrated on top of master. anyone can do that.

[st. pakebusch]

what i can and will not do is maintain and release my own parallel version of omegaT beyond where i work as a payed dayjob, this is where the public pull requests and my humble request of your kindly maintainance comes into play. please. and yes i try and comply with feedback, you may have noticed.

in response to critique from the previous cited PR 2025 where you say we need ONLY things when there is reported need of new functionality not attainable otherwise in whatever hacky plugin scripted CLI way (my interpretation of intent, may not 1000%ly overlap with your actual intent). which is the case here, as i explained above.

[st. pakebusch]

[…]

[Jean-Christophe Helary]

https://omegat.readthedocs.io/en/latest/04.ContributingOmegaT.html#start-a-conversation

The first step in contributing to OmegaT is to start a conversation on the development list.

That’s where you get feedback about the needs, the history, the way we implement things, etc.