#243 Printing Regression Release_4_2_3b to Release_5_1_0. All text strings improperly located.

[David Fellows]

Printing to paper in Release_5_1_0 produces pages that have ALL text strings positioned incorrectly.

This is due to the position transformation in wlib/gtklib/print.c being done incorrectly.

A fix will be provided shortly. A test case .xtc is attached.

[Adam Richards]

Please note that the attached is output from PDF print on OSX (another use of gtklib/print.c) - which seems (unless I am missing something) to be perfectly aligned

[David Fellows]

That looks alright to me on screen and prints on my printer OK.
I just did a print to file using my version of 5.1.0. Viewing it on screen looks the same way as it looks on paper ie horrible. I did a print to file using my patched version looks correct on screen. The patched version prints (directly) on paper correctly.

[David Fellows]

Couldn't figure out how to add 2 attachements. Here is patched version. (incl patch for bug 242)
BTW the mercurial tag for what I am using as 5.1.0 is
Release_5_1_0 1481:43883c4baad0
I did an hg incoming on my repo just now and see that Martin has retagged things.
My last pull was Mar 5.

[Adam Richards]

That was at V5.2 BSF.

These are Landscape and Portrait from V5.1.0.

[Adam Richards]

So the code which is in question (only part between between PDF and non-PDF) in gtklib/cprint.c is this.

if (surface_type == CAIRO_SURFACE_TYPE_PDF ||
surface_type == CAIRO_SURFACE_TYPE_PS ||
surface_type == CAIRO_SURFACE_TYPE_SVG) {
double p_def=600;
cairo_surface_set_fallback_resolution (psPrint_d.curPrintSurface, p_def, p_def);
psPrint_d.dpi = p_def;
scale_adjust = 72/p_def;
} else {
psPrint_d.dpi = (double)gtk_print_settings_get_resolution(settings);
}

My guess is that the issue can be addressed by adjusting the scale adjust to be in proportion to dpi inside the else statement (which is for normal printing). That adjustment is clearly not needed in PS/PDF/SVG based on my experiments and files above. I say "guess" because on OSX the XQuartz printer is not working for some reason. That is why I didn't spot the issue in the first place when I was making PDF work to produce high-res files.

Let me know when you have fixed it and I'll regression test that PDF is working OK.

Adam

[David Fellows]

The problem does seem to be scale_adjust in print.c. The fix I settled on was instead of using
ascent = pango_font_metrics_get_ascent(metrics) / PANGO_SCALE * scale_adjust;
in the transformation was to use either
ascent = pango_font_metrics_get_ascent(metrics) / PANGO_SCALE;
or
ascent = pango_layout_get_baseline (layout) /PANGO_SCALE ;
Both functions seem to return the same number, the latter is easier to use.
The value returned seems to include the scale_adjust already. Applying it again makes ascent too small.

[Adam Richards]

Now I got my printing via XQuartz to work again - and I am getting pretty normal print outs with the text in the right places in landscape and portrait. Everything seems in position, sizes, angles and so forth.

So could this be a gtk level issue?

I'm at gtk2 @2.24.31_0

[David Fellows]

Maybe. I use Gentoo Linux. What I have installed is
x11-libs/gtk+-2.24.31-r1:2
x11-libs/gtk+-3.22.19:3
x11-libs/cairo-1.14.12:0
x11-libs/pango-1.40.14:0
There are no later versions available in Gentoo. I used ldd xtrkcad to verify that it was linking against gtk+2 and not 3. It is.

Hmm. I should check it is pulling in the right include files. I would think gcc would be complaining bitterly is it isn't.
I will do another hg pull today and see what has changed on the xtrkcad side.

[Adam Richards]

OK, I think the issue is that scale_adjust is not being set unless we are doing PDF/PS etc. It is a static set in print.c so we should set it to 1.0 in the non-PDF case - because thats the same as works for you. Because I have been doing the tests in the order PDF first then real print - I may be getting the value inintialized randomly.

Let me test that.

Adam

[Adam Richards]

So, actually I think its the other way around - if you run PDF first scale_adjust is set to something other than 1.0. (72/600 I think which is what is needed for PDF which is now 600 dpi). Then if you run print normally not to file, the scale_adjust is not reset to its base value 1.0 so everything is much smaller and shifted as though it is 600 dpi - which it isn't.

So when I was running print normal I was always restarting first, so that meant I never saw the bad combo.

Does that fit? Solution is to set scale_adjust back to 1.0 for normal print inside the else. That gets you the effect of no scale adjust.

Adam

[David Fellows]

Not sure. I have instrumented print.c to printout most of the variables around that transformation. I have never noticed scale_adjust to be anything but 0.12. I just repeated
using my modifications
start xtrkcad.
select print to file.
print
stop xtrkcad
start xtrcad
select in xtrackcad printer dialog Print to my specific printer (Samsung C410W)
print select my printer print
stop xtrkcad
start xtrkcad
select print in portable format
print select my printer print
In each case scale_adjust was 0.12
output looked good
printing to my specific printer the drawing was not positioned on the page quite right
but acceptable.
The positioning of the text relative to the baseline lines was "perfect" in all three cases.

Without looking it up I think my printer has 300dpi resolution.

I did spend quite a long time doing an hg bisect on this issue. This is hg's conclusion:
The first bad revision is:
changeset: 1387:0644d8e957a6
user: Adam Richards tynewydd962@yahoo.com
date: Sun Jan 28 03:13:09 2018 -0800
summary: Increase resolution GTK Cairo uses for Postscript/PS/SVG files. Use 600 dpi for drawing but then scale to 72 points per inch for vector output. That way when printing at a scale of 16, the output looks OK when zoomed to 800%, for example.

I haven't had time to look at the details of that changeset yet. It is bedtime for me now.

[Adam Richards]

David,
The code as of that change (which I wrote and included the lines quoted above) has only two possibilities for scale_adjust. The default (1.0) and the value set in the above code fragment if the surface_type is PDF, PS or SVG when it is set to be 72 (the assumed value of the code) divided by the dpi which is arbitrarly set to 600 - because that is the value being used in the bitmap that the print code is drawing to in order to save it to a high-resolution file. Pre my fix it was using 72dpi for PDF, which gave terrible results when magnified.

Therefore - I expect that you were actually seeing 0.12 - rather than 0.2 - since that is 72/600.

Now - in running tests with my setup and trace, I discovered that my printer is always seen as the PDF by GTK - that's the surface_type that gtk is always returning. So that would mean that we are always setting the fallback resolution and scale_adjust to 0.12 even for real printers - apparently my printer doesn't care, but it does have a postscript engine as well as a PCL emulator.

Therefore I have added a test for the printer name being the system defined "Print to File" to ensure we only increase the resolution if that has been chosen. I tried scale_adjust as 1.0 but that produced much too large output on my 300 dpi printer. So instead I set the scale_adjust for real printers using 72/native-printer-dpi - and that seemed to do the right thing for me.

I pushed the change to the default branch - can you test with that, to see if it works for your printer?

Adam

[David Fellows]

Yes, all I have ever seen for scale adjust is 0.12. I have checked - my printer is 600dpi so no different than pdf. I will give your change a try. Meanwhile - see below.

What I find peculiar is that presumably using the same code, data and settings (print to file) we get different results.

BTW your cornu stuff is great. Many thanks for that.

[David Fellows]

Attached is an hg bundle that provides the simplest fix to this bug for me on my system. There is some cleanup that could be one around it if it works for others. Should not affect the Windows platform.
I think that the problem arises from the fact that the scale_adjust gets applied to the cairo context via the Cairo_scale call around line 870 in print.c and is already applied to the value returned from pango_font_metric_get_ascent. But I don't understand why that would not also apply in the MacIntosh case. Adam, please test it and the patch for bug 252 also.

[Adam Richards]

David,
A .bundle file is a foreign concept outside UNIX - so can't read it. I have a suspicion that it may be a git file, perhaps, but we use Mercurial for the project.

Can you please just send over a source code print.c with the lines changed so I can diff it, or even a source code snippet including the chnaged lines?

Adam

[David Fellows]

A bundle is an object exported by mercurial containing changesets and the related meta-info. I don't think mercurial cares about the filename or extension. Just download the file to a convenient place. setup a convenient mercurial working repo, cd to it and do
hg unbundle pathtofile
hg update

Anyway, the change is trivial. Here is the mercurial diff of print.c against the Release_5_1_0 version:

diff -r 43883c4baad0 app/wlib/gtklib/print.c
--- a/app/wlib/gtklib/print.c   Sun Mar 04 16:05:03 2018 +0100
+++ b/app/wlib/gtklib/print.c   Tue Mar 13 13:15:14 2018 -0300
@@ -619,8 +619,7 @@
     metrics = pango_context_get_metrics(pcontext, desc,
                                         pango_context_get_language(pcontext));


-    ascent = pango_font_metrics_get_ascent(metrics) / PANGO_SCALE * scale_adjust ;
-
+    ascent = pango_font_metrics_get_ascent(metrics) / PANGO_SCALE ;
     cairo_identity_matrix(cr);

     cairo_translate(cr, x0 + ((ascent + (bBorder*scale_adjust)) * sin(-a * M_PI / 180.0))+((lBorder*scale_adjust)* cos(a * M_PI / 180.0)),

DaveF

[Adam Richards]

David,
I tried removing the scale from the ascent calc and it led to printing the text displaced vertically proportional to the font size on my real printer.

The use of the scale is because the pango value is a number of pango units that when multipled by the Pango_scale gives device pixels. And then we are scaling that into the dpi of the bitmap used to draw the vectors. Now I am wondering if the calc for non-PDF should be actual-device-dpi/surface-fallback-dpi. Then if they are identical you get scale =1.0,

Device is really only 72 "dpi" for postscript and PDF which is fancy postscript. Not sure about SVG but guess it is the same as Adobe spawned all of them ...

To quote stack overflow -

There is no default dpi in PostScript. User space is set up so that the co-ordinate system is 72 units to the inch, but that's not actually the same thing. If you want a different user space then you alter the CTM (Current Transformation Matrix). All co-ordinate system alterations rely on changes to a matrix, often the CTM.

In the case of images, the image operator maps the image into a 1 unit square area, which is then passed through the image matrix to render to the canvas, using the CTM. In the case of the example above the image is scaled by a factor of 52 in the x direction and 49 in the Y direction. If you change the scale or the image matrix, you will get different results.

The area covered by the image is device-independent, when you print it the user space objects are rendered into device space (using the device matrix). So if your printer is 600 dpi or 2400 dpi, the image will still cover the same area on the output page.

All you need to worry about is the area you want covered, you don't need to be concerned with the resolution.

Now for real printers... Your observation is that the scale is wrong jives with the idea that the device dpi is not universally 72. It is 96 or 300 or 600 or whatever - perhaps platform and certainly printer dependent.

I was thinking we should ask what the default fallback dpi is from gtk and use that as the denominator and divide by the device dpi to get the scale (if any). Like this -

else {

psPrint_d.dpi = (double)gtk_print_settings_get_resolution(settings);
 double p_def_x, p_def_y;
                       cairo_surface_get_fallback_resolution(psPrint_d.curPrintSurface,&p_def_x, &p_def_y);

scale_adjust = psPrint_d.dpi/p_def_x;
}

But I am getting from the cairo doc that this fallback_resolution value should have no effect for real devices...

The odd things are that -

1) My physical printer is seen as a PDF surface
2) And it needs scale set to 0.12 to work at the right DPI. Scale = 1.0 doesn't work.
3) Your cairo surface must also be seen as PDF or PS or SVG because scale was formerly being set to 0.12, but it shouldn't be set for you - you need scale=1.0;

So, logically your cairo bitmap dpi must = your printer dpi, while my cairo resolution must still be about 600/0.12 ~= 5000 dpi or it wouldn't work at scale 0.12 if the printer is seen by gtk as 600.

Now my printer driver does have a default 4800CQ quality setting as well as a 1200dpi ... So just perhaps....

I'll look into it further tomorrow.

Adam

[David Fellows]

So the displacement should be about 8 timesthe font size (1/0.12) . On my system all printing is spooled through CUPS - anything Xtrkcad produces is a pdf file.

[Adam Richards]

Aha!

https://wiki.linuxfoundation.org/openprinting/pdf_as_standard_print_job_format

The standard for GTK on Linux (and therefore MacOSX GTK) printing is ALWAYS PDF now (as of 2006). That explains why my macs always seem to use it even for native printers and why it basically works regardless of dpi by default and as long as we scale to 72 ppi with the fallback dpi. PDF always starts at 72ppi natively by definition. That is to say that 5.1.0 is working as GTK was designed. As noted in the linked article, some older Linux systems will need to be patched with compatible CUPS drivers and other parts -> I suggest if that is too onerous to achieve we need to have a compile-time option or environmental variable to force fallback to using non-standard printing.

[Adam Richards]

Decided that we needed run-time support and so I have added some logic to allow a user to set an environment variable of XTRKCADPRINTSCALE1 (any value will do like XTRKCADPRINTSCALE1=1). If it is set and the target is a real printer rather than a file, the dpi is force set to 72 (the PDF standard which was in place before I made changes) and the scale to 1.0 (i.e., no scaling). Tested and this works even for OSX - the output on things like solid ties goes back to looking bad because of the low resolution, but the text is all in place.

Now needs to be tried on Linux - hopefully as it restores things with the env set to where we were before the update it will be OK.

[David Fellows]

Sorry for the delay in responding, life has intervened from several directions this week.

I pulled from sourceforge , updated,
and built including changeset 1505 - Adam's test changes
Ran xtrkcad loading my testcase (and with serif selected as the font)
All my tests have used this.
Made test prints by selecting a print scale of 2 and the page in the lower left corner of the screen displayto print. This gives more detail to examine when comparing.
First use File ->Print setup to set US letter paper and format for "any Printer"
Test 1 was to print to file. The file is test1505-j.pdf
Test 2 Print to Claser in the print dialog
Test 3 Print-setup and select print to CLaser (specific printer)
Print and select Claser as destination.
Test 4 Disconnect USB cable from printer and repeat Test 3
Inspect spool file - it is a pdf.
Reconnect printer, page printed immediately.
Test 5 Use command line lpr test1505-j.pdf to print the pdf file.

NB All my printing is through CUPS spooling system. the printer driver is that supplied by Samsung --the Unified Linux Driver.

Results.

Overlaying the outputs of Test 3 and 4 and viewing at a light source. Shows no difference whatsover.

Overlaying the outputs of Test 2 and Test 3 and aligning the physical edges of the paper shows no differences. In both tests the MXg.. text is drawn so that the tops of the letters are right on the reference line.

Test 5 when printed shows all the boxes in the correct locations and reasonable located on the physical page. When overlaid on the test 3 page all the lines, etc., are in exactly the same places. All text strings are shifted about 1/32 inch in lower in the y direction. the MXg... text is located so the letter tops are about 1/32 inch below the reference line. This is subtly different to the Test 2 and 3 cases.
The engineering data text is also too low in all cases. The displacement is approximately 1/32 inches. It seems to be proportional to the font size. The pdf is attached.

During test 4 I verified that the print spool file for the page is indeed a pdf. In test 4 any position/printer specific data must have been obtained from the printer driver or the CUPS database.

In all cases the font sizes are "too big" ie. they are the actual font sizes as set in xtrkcad. In the sample pdf you showed at they beginning of this thread they are printed scaled by 0.75. For example the string XTrackCAD is spec'd as 30pt. In your pdf it is 22.5pt. When I print it is 30pt (and won't fit in the box provided for it).

One minor thought - Your original output.pdf file that I downloaded uses the genuine Helvetica and Times fonts. My pdfs use the TexGyre equivalents. I suppose it is remotely possible that the fonts are "defective" in some way.

I am very puzzled that from the same source code we should see different results. Perhaps it could be due to different compilers and/or optimizations producing different object code? My Cmake build parameters specify

CMAKE_BUILD_TYPE blank
CMAKE_C_FLAGS blank

I am using gcc-6.4.0. I assume that with no flags specified you get -O0 as the optimization level since that is the gcc default. Are you using the same or different optimization level?

When I get a chance, I will pull your latest changes and try them. I will also install Robert's deb package and install it on my Xubuntu netbook and see what it does.

[Adam Richards]

The issue now, I think, is about Pango and its rendering of fonts between different devices.
We are attempting to render a font onto a scale area duplicating what occurs on screen and what will be seen in a printout and so the difference between display (which can be, but crucially does not have to be), 96 DPI and PDF, which is defined to be always be equivalent to 72 DPI becomes important. Note that 72/96 = 0.75. That would explain the "use 0.75 but I dont know why" comment in the old code.

It is apparent that Pango on OSX seems to be using 72 DPI for display - which would explain why when rendered at 72 DPI for printing all is well for Mac. One imagines that Linux may be using 96 also - which may, or may not, be what it uses on Windows as well.

If we included a scale between the DPI (set by a compile time variable) and 72 DPI for the font size we could get around this issue or there may be a more run-time equivalent using pango absolute font size values instead of relative ones.

More research needed.

[David Fellows]

I downloaded Robert Heller's 5.1.0-1.i386.deb file to my xubuntu 17.04 system, installed it and ran my testcase. Printed to file. The result is the same as on my gentoo system. So the problem is not specific to my Gentoo system or to 64bit.
DF

[Adam Richards]

I can confirm that the text is being printed out at the correct size on OSX (that is the letters take up the same physical space between OSX display and OSX print - and that scaling the font down makes them too small). I printed out and used a ruler. The use of pango_description_set_size() is supposed to be device independent and ensure that this happens (all units are in pts not pixels). A pt is defined as 1/72 or an inch.

Pango does rely on the device drivers (both screen and printer) reporting DPI accurately since it needs to scale so that the pt size is the right number of pixels. And it also requires the font to report its size accurately as well. But a font with bad metrics would report the same when drawn to screen and printer, so that seems unlikely. Given we already had problems apparently with the DPI of the printer that led to scale not working, I would tend to think that more likely.

And indeed, there have been long-running problems with DPI and fonts in Linux because there seems to be no agreement on what size is "correct" and whether fonts specified in Gnome should be in px or in pt. Plus XServer apparently doesn't reliably report DPI either. For example, https://bugs.launchpad.net/ubuntu/+source/gnome-control-center/+bug/310353

Therefore, I have amended the env overrides to be these two

XTRKCADPRINTSCALE=scale_factor (a floating point ratio - often 1.0)
XTRKCADPRINTTEXTSCALE=text_factor (a floating point ratio - often 0.75)

Should we find these are used a lot, we can add them into the print dialog.

Adam