#234 makewhatis awk complains about gnuplot-ja man page

[Brian De Wolf]

Hello, after some system updates we started seeing this error message from makewhatis:

awk: cmd. line:196: (FILENAME=- FNR=1287) warning: Invalid multibyte data detected. There may be a mismatch between your data and your locale.

I tracked it down to the gnuplot-ja man page and the errors go away if I remove it.

We are using LANG=en_US.UTF-8 on gentoo, with versions:

sci-visualization/gnuplot-5.0.1-r1
sys-apps/man-1.6g
sys-apps/gawk-4.1.4

[Ethan Merritt]

I'm lost. Of course you cannot expect to build foreign language documentation using an en_US locale. Even worse for non-UTF encondings like SJIS. But what exactly does this have to do with gnuplot, other than the fact that gnuplot is an example of a package that includes Japanese language documentation?

A quick look at the makewhatis man page suggests that the idea is to run it separately for each language it is expected to handle. E.g.
LANGUAGE=jp makewhatis -w

But I have not explored this myself.

[Paul Henson]

We don't expect to have Japanese language man pages 8-/.

Are you saying this is a distribution packaging issue, in that the Japanese man pages just shouldn't have been installed given that was not a valid locale for our system?

[Ethan Merritt]

I suppose so. I know very little about gentoo packaging policy (I thought the preferred option there was to build from source?). In general you can have multiple locales installed on a system even if only one of them is likely to be used in practice. The corresponding man pages may or may not be present. On my machines (Mageia) a "normal" install seems to bring in about 10-20 language-specific subdirectories of /usr/share/man/

To be fair, if you simply run "make install" from gnuplot source as downloaded from sf.net, both gnuplot.1 and gnuplot-ja.1 end up in /usr/local/share/man/man1. This may not be ideal but as you imply it is exactly the sort of thing that distro packaging is expected to tailor to their own preferred policy for where things go. The Mageia distro package puts the english man page in /usr/share/man/man1/gnuplot.1.xz and does not install the japanese one. I would have expected them to put the japanese man page in /usr/share/man/ja/man1/gnuplot.1.xz since the directory does exist, but apparently they didn't do that in packaging, I may suggest it to their packagers :-)

[Paul Henson]

Gentoo does allow you to compile from source as well as install precompiled binary packages. How either operation occurs is controlled by what is called an ebuild, which describes how the package should be compiled and installed. In Gentoo, it would be the responsibility of that config to deal with any internationalization issues as far as installing/not installing stuff based on the local environment.

It appears Gentoo does have locale specific man pages:

ls /usr/share/man/

bg cat8 de_DE fi fr_FR it.ISO8859-1 man0p man3 man6 nl pt ru.KOI8-R sv zh_CN
ca cs el fr hr it.UTF-8 man1 man3p man7 pl pt_BR ru.UTF-8 uk
cat1 da eo fr.ISO8859-1 hu ja man1p man4 man8 pl.ISO8859-2 ro sl vi
cat5 de es fr.UTF-8 it ko man2 man5 nb pl.UTF-8 ru sr

so presumably this file should be installed into the ja subdir. There are other man pages in the ja, hierarchy, although I can't say at this point it's because of Gentoo specific packaging or because the upstream package installed the man page into a ja subdir on its own and it was just copied as-is.