Can we get some view on the disk space and time requirements for building LilyPond documentation, and the resulting file sizes of the final PDFs, compared to without this patch?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
-
2016-06-06
Description has changed:
Diff:
Needs: -->
Patch: new --> needs_work
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
-
2016-06-06
Sorry, I wasn't paying attention, I think this fails make (but it cound be make check). It never got to Make doc, so it is one of those
/home/james/lilypond-git/build/out/share/lilypond/current/scm/backend-library.scm:154:7: In procedure copy-file in expression (copy-file from-name to-name):
/home/james/lilypond-git/build/out/share/lilypond/current/scm/backend-library.scm:154:7: No such file or directory
fatal error: Children (1) exited with errors.
--snip--
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
If I understand correctly,
"make doc" failure caused by ghostscript option "-dSAFER".
LilyPond with "-dgs-load-fonts" invokes a ghostscript with "-dNOSAFER".
I removed "-ds-load-fonts". In this case, LilyPond invokes a ghostscript with "-dSAFER".
Some snippets in input/regression/ use "\epsfile".
"\epsfile" with "-dSAFER" raises the problem.
So I'd like to propose new LilyPond option like "-dgs-safer".
It is different from "-dgs-load-fonts", it does not change font loading way.
Its default is #t except Windows. LilyPond invokes a ghostscript with "-dSAFER".
If it is #f, LilyPond invokes a ghostscript with "-dNOSAFER".
I'll make the patch.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'm ambiguous about this. In my opinion, if LilyPond is running PostScript as part of its PDF production process, things like gs-load-fonts and epsfile inclusion at runtime are perfectly valid means to increase efficient and reduce the size of intermediate PostScript files.
Where PostScript is the final format, fonts as well as epsfile and similar should be embedded in the file.
Also, we do have the problem that any use of \epsfile and other embedded user-specified PostScript content means that Ghostscript should only be run with -dSAFER. In other words: there is no actual use case for letting \epsfile work by reading an external file: we should really always embed the file directly into the PostScript output and only run the result with the equivalent of -dSAFER which rules out use of gs-load-fonts (unless we load all fonts before dropping file access privileges).
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The very issue is that you cannot embed OTCs or TTCs as PS resources, even if the
target format is PDF. In other words, two font formats that are fully valid in PDFs
don't work – and it is not trivial to the user to find out which fonts work and which
fonts don't work, especially on OS X that is going to use more and more OTCs even
for non-CJK scripts. Limited support for TTCs exist in the PS handling of GhostScript
using GhostScript extensions, but OTCs are really completely unsupported currently
on the PS side.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Unfortunately, my understanding was wrong.
The cause of this failure is not -dSAFER.
The cause is that \epsfile can't include the EPS that contain the byte of 0x00.
When you use -dgs-load-fonts for EPS building, the EPS does not contain the byte of 0x00.
However, when you do not use -dgs-load-fonts, the EPS contains the byte of 0x00.
I've compared it in my environment.
With --bigpdfs (Patch Set 1), the final PDFs file sizes become bigger.
So I propose the way that without both options, -dgs-load-fonts and --bigpdfs.
It is Patch Set 2.
---Original (with -dgs-load-fonts, without --bigpdfs)--- Patches:
master branch's
commit 37da55e4c857a21b017981eed0a28943e9fae119
with Issue 4890
Time requirements for building LilyPond documentation:
58m6.769s
/home/james/lilypond-git/build/out/share/lilypond/current/scm/backend-library.scm:154:7: In procedure copy-file in expression (copy-file from-name to-name):
/home/james/lilypond-git/build/out/share/lilypond/current/scm/backend-library.scm:154:7: No such file or directory
fatal error: Children (1) exited with errors.
summary: Replace -dgs-load-fonts' to--bigpdfs' in lilypond-book --> Remove -dgs-load-fonts in lilypond-book
Description has changed:
Diff:
--- old+++ new@@ -1,10 +1,4 @@-Replace `-dgs-load-fonts' to `--bigpdfs' in lilypond-book--Some fonts cannot use with `-dgs-load-fonts' option-(also `-dgs-load-lily-fonts' option).--`--bigpdfs' option is more suitable than `-dgs-load-fonts'-because the common font that is used by multiple snippets-in the document becomes integrated into one.+Some fonts cannot use with `-dgs-load-fonts` option+(also `-dgs-load-lily-fonts` option).
http://codereview.appspot.com/300280043
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Discussion:
http://lists.gnu.org/archive/html/lilypond-devel/2016-06/msg00029.html
Can we get some view on the disk space and time requirements for building LilyPond documentation, and the resulting file sizes of the final PDFs, compared to without this patch?
Diff:
Sorry, I wasn't paying attention, I think this fails make (but it cound be make check). It never got to Make doc, so it is one of those
--snip--
GNU LilyPond 2.19.43
Forking into jobs: (14404 14403 14402 14401 14400 14399 14398)
logfile lilypond-multi-run-1.log (exit 1):
1 #) ...) (ly:system cmd) ...)
169: 28 (if (not (= 1 anti-alias-factor)) (for-each (lambda # #) files))
170: 29 [for-each #<procedure #f="" (f)=""> ("./9e/lily-f2cfd744.png")]
In unknown file:
?: 30 [#<procedure #f="" (f)=""> "./9e/lily-f2cfd744.png"]
In /home/james/lilypond-git/build/out/share/lilypond/current/scm/ps-to-png.scm:
171: 31 [scale-down-image 2 "./9e/lily-f2cfd744.png"]
55: 32 (let (# # # ...) (ly:debug # file ...) ...)
73: 33* [copy-binary-file "./9e/lily-f2cfd744.png" "/tmp/lilypond-AhRTfM"]
In /home/james/lilypond-git/build/out/share/lilypond/current/scm/backend-library.scm:
138: 34 (if (eq? PLATFORM (quote windows)) (let (# #) (let loop # ...)) ...)
154: 35 [copy-file "./9e/lily-f2cfd744.png" "/tmp/lilypond-AhRTfM"]
/home/james/lilypond-git/build/out/share/lilypond/current/scm/backend-library.scm:154:7: In procedure copy-file in expression (copy-file from-name to-name):
/home/james/lilypond-git/build/out/share/lilypond/current/scm/backend-library.scm:154:7: No such file or directory
fatal error: Children (1) exited with errors.
--snip--
If I understand correctly,
"make doc" failure caused by ghostscript option "-dSAFER".
LilyPond with "-dgs-load-fonts" invokes a ghostscript with "-dNOSAFER".
I removed "-ds-load-fonts". In this case, LilyPond invokes a ghostscript with "-dSAFER".
Some snippets in input/regression/ use "\epsfile".
"\epsfile" with "-dSAFER" raises the problem.
It works with "-dgs-load-fonts".
But, in my humble opinion, both options "-dgs-load-fonts" and "-dgs-load-lily-fonts" should be deprecated.
http://lists.gnu.org/archive/html/lilypond-devel/2016-06/msg00056.html
So I'd like to propose new LilyPond option like "-dgs-safer".
It is different from "-dgs-load-fonts", it does not change font loading way.
Its default is #t except Windows. LilyPond invokes a ghostscript with "-dSAFER".
If it is #f, LilyPond invokes a ghostscript with "-dNOSAFER".
I'll make the patch.
I'm ambiguous about this. In my opinion, if LilyPond is running PostScript as part of its PDF production process, things like gs-load-fonts and epsfile inclusion at runtime are perfectly valid means to increase efficient and reduce the size of intermediate PostScript files.
Where PostScript is the final format, fonts as well as epsfile and similar should be embedded in the file.
Also, we do have the problem that any use of
\epsfileand other embedded user-specified PostScript content means that Ghostscript should only be run with -dSAFER. In other words: there is no actual use case for letting\epsfilework by reading an external file: we should really always embed the file directly into the PostScript output and only run the result with the equivalent of-dSAFERwhich rules out use ofgs-load-fonts(unless we load all fonts before dropping file access privileges).The very issue is that you cannot embed OTCs or TTCs as PS resources, even if the
target format is PDF. In other words, two font formats that are fully valid in PDFs
don't work – and it is not trivial to the user to find out which fonts work and which
fonts don't work, especially on OS X that is going to use more and more OTCs even
for non-CJK scripts. Limited support for TTCs exist in the PS handling of GhostScript
using GhostScript extensions, but OTCs are really completely unsupported currently
on the PS side.
Unfortunately, my understanding was wrong.
The cause of this failure is not
-dSAFER.The cause is that \epsfile can't include the EPS that contain the byte of 0x00.
When you use
-dgs-load-fontsfor EPS building, the EPS does not contain the byte of 0x00.However, when you do not use
-dgs-load-fonts, the EPS contains the byte of 0x00.I've created Issue 4890 for \epsfile issue.
https://sourceforge.net/p/testlilyissues/issues/4890/
Remove
-dgs-load-fontsin lilypond-bookhttp://codereview.appspot.com/300280043
This patch is required Issue 4890.
https://sourceforge.net/p/testlilyissues/issues/4890/
I've compared it in my environment.
With
--bigpdfs(Patch Set 1), the final PDFs file sizes become bigger.So I propose the way that without both options,
-dgs-load-fontsand--bigpdfs.It is Patch Set 2.
---Original (with
-dgs-load-fonts, without--bigpdfs)---Patches:
master branch's
commit 37da55e4c857a21b017981eed0a28943e9fae119
with Issue 4890
Time requirements for building LilyPond documentation:
58m6.769s
Disk size of building directory:
$ du -ms . 4648 .Resulting file sizes of the final PDFs:
---Patch Set 1 (without
-dgs-load-fonts, with--bigpdfs)---Patches:
master branch's
commit 37da55e4c857a21b017981eed0a28943e9fae119
with Issue 4890 + Issue 4882 Patch Set 1
Time requirements for building LilyPond documentation:
61m10.790s
Disk size of building directory:
$ du -ms . 14566 .Resulting file sizes of the final PDFs:
---Patch Set 2 (without
-dgs-load-fonts, without--bigpdfs)---Patches:
master branch's
commit 37da55e4c857a21b017981eed0a28943e9fae119
with Issue 4890 + Issue 4882 Patch Set 2
Time requirements for building LilyPond documentation:
58m45.157s
Disk size of building directory:
$ du -ms . 8500 .Resulting file sizes of the final PDFs:
This fails 'make check'
-snip-
169: 28 (if (not (= 1 anti-alias-factor)) (for-each (lambda # #) files))
170: 29 [for-each #<procedure #f="" (f)=""> ("./9e/lily-f2cfd744.png")]
In unknown file:
?: 30 [#<procedure #f="" (f)=""> "./9e/lily-f2cfd744.png"]
In /home/james/lilypond-git/build/out/share/lilypond/current/scm/ps-to-png.scm:
171: 31 [scale-down-image 2 "./9e/lily-f2cfd744.png"]
55: 32 (let (# # # ...) (ly:debug # file ...) ...)
73: 33* [copy-binary-file "./9e/lily-f2cfd744.png" "/tmp/lilypond-FQaqS6"]
In /home/james/lilypond-git/build/out/share/lilypond/current/scm/backend-library.scm:
138: 34 (if (eq? PLATFORM (quote windows)) (let (# #) (let loop # ...)) ...)
154: 35 [copy-file "./9e/lily-f2cfd744.png" "/tmp/lilypond-FQaqS6"]
/home/james/lilypond-git/build/out/share/lilypond/current/scm/backend-library.scm:154:7: In procedure copy-file in expression (copy-file from-name to-name):
/home/james/lilypond-git/build/out/share/lilypond/current/scm/backend-library.scm:154:7: No such file or directory
fatal error: Children (1) exited with errors.
--snip--
/9e/lily-f2cfd744.*
is the file generated from
$LILYPOND_GIT/Documentation/snippets/clip-systems.ly
This patch requires Issue 4890.
https://sourceforge.net/p/testlilyissues/issues/4890/
Did you use both patches?
https://codereview.appspot.com/300930043
https://codereview.appspot.com/300280043
-dgs-load-fonts' to--bigpdfs' in lilypond-book --> Remove-dgs-load-fontsin lilypond-bookDiff:
If we use -
dgs-load-lily-fonts, is it possible to omit-dinclude-eps-fonts? Because that may be involved with the disk space usage as well.In that case, intermediate PDF will be broken if you use a font that ghostscript can not find.
In my environment, ghostscript can not find TeX Gyre fonts.
Here is a sample file.
The following command builds intermediate EPS files and intermediate PDF files.
$ lilypond -dbackend=eps -dno-include-eps-fonts --ps --pdf aaa.lyThe generated
aaa-1.pdfis embedded wrong font (maybe Courier) as TeX Gyre.