trustedqsl-testing

[Trustedqsl-testing] OpenSSL 3.0.0

[Richard S. <hob...@gm...>]

I wanted to file this as a bug but every time I try to create an account on
mantis I get a 504 gateway error...

I'm the Fedora package maintainer and Fedora 36 will have OpenSSL 3.0.0.
Unfortunately, tqsl uses a function that's being removed:

ERR_GET_FUNC

I have a trivial patch to remove it:

Index: tqsl-2.5.9/src/openssl_cert.cpp
===================================================================
--- tqsl-2.5.9.orig/src/openssl_cert.cpp
+++ tqsl-2.5.9/src/openssl_cert.cpp
@@ -4285,7 +4285,7 @@ static int
 tqsl_ssl_error_is_nofile() {
  unsigned long l = ERR_peek_error();
  if (tQSL_Error == TQSL_OPENSSL_ERROR &&
- ERR_GET_LIB(l) == ERR_LIB_SYS && ERR_GET_FUNC(l) == SYS_F_FOPEN)
+ ERR_GET_LIB(l) == ERR_LIB_SYS == SYS_F_FOPEN)
  return 1;
  if (tQSL_Error == TQSL_SYSTEM_ERROR && tQSL_Errno == ENOENT)
  return 1;

I also have patches I maintain to clean up rpath/lib install location and
to provide a soversion for the tqsl library (required by Fedora).

Thanks,
Richard
KF5OIM

Re: [Trustedqsl-testing] OpenSSL 3.0.0

[Greg T. <gd...@le...>]

Richard Shaw <hob...@gm...> writes:

> Index: tqsl-2.5.9/src/openssl_cert.cpp
> ===================================================================
> --- tqsl-2.5.9.orig/src/openssl_cert.cpp
> +++ tqsl-2.5.9/src/openssl_cert.cpp
> @@ -4285,7 +4285,7 @@ static int
>  tqsl_ssl_error_is_nofile() {
>   unsigned long l = ERR_peek_error();
>   if (tQSL_Error == TQSL_OPENSSL_ERROR &&
> - ERR_GET_LIB(l) == ERR_LIB_SYS && ERR_GET_FUNC(l) == SYS_F_FOPEN)
> + ERR_GET_LIB(l) == ERR_LIB_SYS == SYS_F_FOPEN)
>   return 1;
>   if (tQSL_Error == TQSL_SYSTEM_ERROR && tQSL_Errno == ENOENT)
>   return 1;

Did you also meen to remove " == SYS_F_FOPEN"?

Is there really no semantics change from this?  (It seems sensible to
adjust, but I am not entirely following.

> I also have patches I maintain to clean up rpath/lib install location and
> to provide a soversion for the tqsl library (required by Fedora).

I'd be interested to see the rpath/lib patch.  pkgsrc norms are to have
RPATH of /usr/pkg/lib set in /usr/pkg/foo as /usr/pkg (or whatever) is
not in the default search path.  I know that various packaging systems
have very different rpath rules, and I think that leads to a need to be
able to configure the build for the varying environments.

73 de n1dam

Re: [Trustedqsl-testing] OpenSSL 3.0.0

[Richard S. <hob...@gm...>]

On Sun, Dec 5, 2021 at 8:56 AM Greg Troxel <gd...@le...> wrote:

>
> Richard Shaw <hob...@gm...> writes:
>
> > Index: tqsl-2.5.9/src/openssl_cert.cpp
> > ===================================================================
> > --- tqsl-2.5.9.orig/src/openssl_cert.cpp
> > +++ tqsl-2.5.9/src/openssl_cert.cpp
> > @@ -4285,7 +4285,7 @@ static int
> >  tqsl_ssl_error_is_nofile() {
> >   unsigned long l = ERR_peek_error();
> >   if (tQSL_Error == TQSL_OPENSSL_ERROR &&
> > - ERR_GET_LIB(l) == ERR_LIB_SYS && ERR_GET_FUNC(l) == SYS_F_FOPEN)
> > + ERR_GET_LIB(l) == ERR_LIB_SYS == SYS_F_FOPEN)
> >   return 1;
> >   if (tQSL_Error == TQSL_SYSTEM_ERROR && tQSL_Errno == ENOENT)
> >   return 1;
>
> Did you also meen to remove " == SYS_F_FOPEN"?
>
> Is there really no semantics change from this?  (It seems sensible to
> adjust, but I am not entirely following.
>

Should it be removed? I only really hack at C/C++ code.



> > I also have patches I maintain to clean up rpath/lib install location and
> > to provide a soversion for the tqsl library (required by Fedora).
>
> I'd be interested to see the rpath/lib patch.  pkgsrc norms are to have
> RPATH of /usr/pkg/lib set in /usr/pkg/foo as /usr/pkg (or whatever) is
> not in the default search path.  I know that various packaging systems
> have very different rpath rules, and I think that leads to a need to be
> able to configure the build for the varying environments.
>

As long as you're installing into system library locations (which we are)
they're already in the library search path so RPATH isn't needed, in fact
as a packager I have to remove RPATH in these situations.

So I guess maybe the first step would be to wrap some conditionals around
the current logic so it's only used when needed.

Here's my rpath/install patch:

https://src.fedoraproject.org/rpms/trustedqsl/blob/rawhide/f/tqsl-2.5-rpath.patch

The first part just roto-rooters the RPATH stuff from apps/CMakeLists.txt
since I don't need it. Probably not acceptable as is :)

The 2nd part should be more interesting to you. Here I replace all the
logic and replace it with the GNUInstallDirs module which will set all the
CMAKE_INSTALL_... variables correctly for Linux systems since RPM and DEB
based systems handle this very differently. This probably should be wrapped
in an "IF(LINUX)" conditional.

Thanks,
Richard

Re: [Trustedqsl-testing] OpenSSL 3.0.0

[Greg T. <gd...@le...>]

Richard Shaw <hob...@gm...> writes:

> On Sun, Dec 5, 2021 at 8:56 AM Greg Troxel <gd...@le...> wrote:
>
>>
>> Richard Shaw <hob...@gm...> writes:
>>
>> > Index: tqsl-2.5.9/src/openssl_cert.cpp
>> > ===================================================================
>> > --- tqsl-2.5.9.orig/src/openssl_cert.cpp
>> > +++ tqsl-2.5.9/src/openssl_cert.cpp
>> > @@ -4285,7 +4285,7 @@ static int
>> >  tqsl_ssl_error_is_nofile() {
>> >   unsigned long l = ERR_peek_error();
>> >   if (tQSL_Error == TQSL_OPENSSL_ERROR &&
>> > - ERR_GET_LIB(l) == ERR_LIB_SYS && ERR_GET_FUNC(l) == SYS_F_FOPEN)
>> > + ERR_GET_LIB(l) == ERR_LIB_SYS == SYS_F_FOPEN)
>> >   return 1;
>> >   if (tQSL_Error == TQSL_SYSTEM_ERROR && tQSL_Errno == ENOENT)
>> >   return 1;
>>
>> Did you also meen to remove " == SYS_F_FOPEN"?
>>
>> Is there really no semantics change from this?  (It seems sensible to
>> adjust, but I am not entirely following.
>
> Should it be removed? I only really hack at C/C++ code.

I am surprised it builds and I'm not sure what it does.

Basically

  && ERR_GET_FUNC(l) == SYS_F_FOPEN

is a logical and an equality test.   So dropping the entire test could
make sense and your + line would then be

>> > + ERR_GET_LIB(l) == ERR_LIB_SYS)

but I don't understand what this did, and how one should try to do the
same thing in the 3.0.0 world.

(It strikes me as aggressive to be moving to 3.0.0, but those are tough
calls, and surely beyond your control..)

>> > I also have patches I maintain to clean up rpath/lib install location and
>> > to provide a soversion for the tqsl library (required by Fedora).
>>
>> I'd be interested to see the rpath/lib patch.  pkgsrc norms are to have
>> RPATH of /usr/pkg/lib set in /usr/pkg/foo as /usr/pkg (or whatever) is
>> not in the default search path.  I know that various packaging systems
>> have very different rpath rules, and I think that leads to a need to be
>> able to configure the build for the varying environments.
>
> As long as you're installing into system library locations (which we are)
> they're already in the library search path so RPATH isn't needed, in fact
> as a packager I have to remove RPATH in these situations.

Yes, but we are installing not into the system library path and thus we
need to set RPATH.  This is the basic difference between systems that
have "base system" and then "packages" and systems that have "single
base system but many things are optional".  The first is the traditional
UNIX approach, and the second the GNU/Linux approach.

Even on GNU/Linux, if someone builds tqsl themselves with
prefix=/home/johndoe/HAM, then RPATH is needed and should happen.  So
it's really a special case that if libdir is /usr/lib then RPATH should
be skippped, and in general it should be included.

> So I guess maybe the first step would be to wrap some conditionals around
> the current logic so it's only used when needed.
>
> Here's my rpath/install patch:
>
> https://src.fedoraproject.org/rpms/trustedqsl/blob/rawhide/f/tqsl-2.5-rpath.patch
>
> The first part just roto-rooters the RPATH stuff from apps/CMakeLists.txt
> since I don't need it. Probably not acceptable as is :)

Indeed, that is not acceptable as it breaks many other systems.  My
impression is that CMake already has the concept of whether RPATH should
be set or not and there are cmake-time config arguments to say which way
it should be.  I would say aligning to that is the standard approach.

> The 2nd part should be more interesting to you. Here I replace all the
> logic and replace it with the GNUInstallDirs module which will set all the
> CMAKE_INSTALL_... variables correctly for Linux systems since RPM and DEB
> based systems handle this very differently. This probably should be wrapped
> in an "IF(LINUX)" conditional.

If it's only for GNU/Linux systems, yes, but GNUInstallDirs seems like
the right approach in general.  pkgsrc path norms are basically
inherited from autoconf and thus from the GNU coding standards, with
some adjustments to be similar to the BSD hier(7), but all of that
mostly is a common history from the 80s.

As for "libsuffix", my view is that that standard libdir is $prefix/lib,
and systems that use lib64 are unusual and it's ok to expect them to be
configured wtih cmake libsuffix=64 or whatever, but if there's a way to
query if the current system is like that and do it automatically, that's
ok.

Does the Fedora world use lib64 on x86_64, or just lib?  I have the
impression that lib64 happens on Debian.

NetBSD has no use of lib64.   Native libs are always in $prefix/lib.
Emulation libs are in /emul/foo/usr/lib for various other operating
systems foo,  and e.g. in /usr/lib/i386 on amd64 systems.  But there is
a "native-first" naming bias, and I sometimes run into configure scripts
that try to apply the lib64 idea to system where it doesn't belong.

Re: [Trustedqsl-testing] OpenSSL 3.0.0

[Richard S. <hob...@gm...>]

On Sun, Dec 5, 2021 at 9:35 AM Greg Troxel <gd...@le...> wrote:

>
> Richard Shaw <hob...@gm...> writes:
>
> > On Sun, Dec 5, 2021 at 8:56 AM Greg Troxel <gd...@le...> wrote:
> >
> >>
> >> Richard Shaw <hob...@gm...> writes:
> >>
> >> > Index: tqsl-2.5.9/src/openssl_cert.cpp
> >> > ===================================================================
> >> > --- tqsl-2.5.9.orig/src/openssl_cert.cpp
> >> > +++ tqsl-2.5.9/src/openssl_cert.cpp
> >> > @@ -4285,7 +4285,7 @@ static int
> >> >  tqsl_ssl_error_is_nofile() {
> >> >   unsigned long l = ERR_peek_error();
> >> >   if (tQSL_Error == TQSL_OPENSSL_ERROR &&
> >> > - ERR_GET_LIB(l) == ERR_LIB_SYS && ERR_GET_FUNC(l) == SYS_F_FOPEN)
> >> > + ERR_GET_LIB(l) == ERR_LIB_SYS == SYS_F_FOPEN)
> >> >   return 1;
> >> >   if (tQSL_Error == TQSL_SYSTEM_ERROR && tQSL_Errno == ENOENT)
> >> >   return 1;
> >>
> >> Did you also meen to remove " == SYS_F_FOPEN"?
> >>
> >> Is there really no semantics change from this?  (It seems sensible to
> >> adjust, but I am not entirely following.
> >
> > Should it be removed? I only really hack at C/C++ code.
>
> I am surprised it builds and I'm not sure what it does.
>
> Basically
>
>   && ERR_GET_FUNC(l) == SYS_F_FOPEN
>
> is a logical and an equality test.   So dropping the entire test could
> make sense and your + line would then be
>
> >> > + ERR_GET_LIB(l) == ERR_LIB_SYS)
>
> but I don't understand what this did, and how one should try to do the
> same thing in the 3.0.0 world.
>

Fixed in my patch, unfortunately I can't answer the last part...


(It strikes me as aggressive to be moving to 3.0.0, but those are tough
> calls, and surely beyond your control..)
>

Well Fedora tends to be leading edge... Latest kernels, latest GCC, latest
python (currently testing 3.11 alpha) etc.  Someone has to be first, right?

There's also some additional deprecated functions in OpenSSL 3  if you want
to get ahead of it.

https://kojipkgs.fedoraproject.org//packages/trustedqsl/2.5.9/1.fc36/data/logs/x86_64/build.log

Just search on "deprecated:"

Here's the porting guide:

https://www.openssl.org/docs/manmaster/man7/migration_guide.html


>> > I also have patches I maintain to clean up rpath/lib install location
> and
> >> > to provide a soversion for the tqsl library (required by Fedora).
> >>
> >> I'd be interested to see the rpath/lib patch.  pkgsrc norms are to have
> >> RPATH of /usr/pkg/lib set in /usr/pkg/foo as /usr/pkg (or whatever) is
> >> not in the default search path.  I know that various packaging systems
> >> have very different rpath rules, and I think that leads to a need to be
> >> able to configure the build for the varying environments.
> >
> > As long as you're installing into system library locations (which we are)
> > they're already in the library search path so RPATH isn't needed, in fact
> > as a packager I have to remove RPATH in these situations.
>
> Yes, but we are installing not into the system library path and thus we
> need to set RPATH.  This is the basic difference between systems that
> have "base system" and then "packages" and systems that have "single
> base system but many things are optional".  The first is the traditional
> UNIX approach, and the second the GNU/Linux approach.
>
> Even on GNU/Linux, if someone builds tqsl themselves with
> prefix=/home/johndoe/HAM, then RPATH is needed and should happen.  So
> it's really a special case that if libdir is /usr/lib then RPATH should
> be skippped, and in general it should be included.
>
> > So I guess maybe the first step would be to wrap some conditionals around
> > the current logic so it's only used when needed.
> >
> > Here's my rpath/install patch:
> >
> >
> https://src.fedoraproject.org/rpms/trustedqsl/blob/rawhide/f/tqsl-2.5-rpath.patch
> >
> > The first part just roto-rooters the RPATH stuff from apps/CMakeLists.txt
> > since I don't need it. Probably not acceptable as is :)
>
> Indeed, that is not acceptable as it breaks many other systems.  My
> impression is that CMake already has the concept of whether RPATH should
> be set or not and there are cmake-time config arguments to say which way
> it should be.  I would say aligning to that is the standard approach.
>

Yes, so I guess the question is why was that code once needed? Or are there
corner cases?


> The 2nd part should be more interesting to you. Here I replace all the
> > logic and replace it with the GNUInstallDirs module which will set all
> the
> > CMAKE_INSTALL_... variables correctly for Linux systems since RPM and DEB
> > based systems handle this very differently. This probably should be
> wrapped
> > in an "IF(LINUX)" conditional.
>
> If it's only for GNU/Linux systems, yes, but GNUInstallDirs seems like
> the right approach in general.  pkgsrc path norms are basically
> inherited from autoconf and thus from the GNU coding standards, with
> some adjustments to be similar to the BSD hier(7), but all of that
> mostly is a common history from the 80s.
>
> As for "libsuffix", my view is that that standard libdir is $prefix/lib,
> and systems that use lib64 are unusual and it's ok to expect them to be
> configured wtih cmake libsuffix=64 or whatever, but if there's a way to
> query if the current system is like that and do it automatically, that's
> ok.
>

As far as I know lib64 is used on all RPM based systems (certainly Fedora,
RHEL, OpenSUSE) that support multi-arch which allows use of 32-bit
libraries on 64-bit systems.


Does the Fedora world use lib64 on x86_64, or just lib?  I have the
> impression that lib64 happens on Debian.
>

No Debian uses a /lib/<multiarch-tuple> type arrangement.

https://cmake.org/cmake/help/latest/module/GNUInstallDirs.html

Thanks,
Richard
KF5OIM

Re: [Trustedqsl-testing] OpenSSL 3.0.0

[Rick M. <k1...@ar...>]

The function causing the problem here is "tqsl_ssl_error_is_nofile()".
It is intended to determine if an error in the SSL libraries was
caused because of a failure to open a file.

The "ERR_GET_LIB(l) == ERR_LIB_SYS" stanza means "did this error occur when
calling the system libraries?"
and the "ERR_GET_FUNC(l) == SYS_F_OPEN" means "did this error occur from an
fopen() call?"

If OpenSSL isn't going to return the function detail, then this entire
section should be deleted and fall back on the second check.
        if (tQSL_Error == TQSL_SYSTEM_ERROR && tQSL_Errno == ENOENT)
                return 1;

I'll map the first check out for OpenSSL > 3.0

As far as the libraries are concerned, I will be happy to take a fix that
works across all Linux distributions.  I think what's been happening here
is changes being made based on the last distribution to complain, which
isn't ideal.
73,
    -Rick


On Sun, Dec 5, 2021 at 11:37 AM Richard Shaw <hob...@gm...> wrote:

> On Sun, Dec 5, 2021 at 9:35 AM Greg Troxel <gd...@le...> wrote:
>
>>
>> Richard Shaw <hob...@gm...> writes:
>>
>> > On Sun, Dec 5, 2021 at 8:56 AM Greg Troxel <gd...@le...> wrote:
>> >
>> >>
>> >> Richard Shaw <hob...@gm...> writes:
>> >>
>> >> > Index: tqsl-2.5.9/src/openssl_cert.cpp
>> >> > ===================================================================
>> >> > --- tqsl-2.5.9.orig/src/openssl_cert.cpp
>> >> > +++ tqsl-2.5.9/src/openssl_cert.cpp
>> >> > @@ -4285,7 +4285,7 @@ static int
>> >> >  tqsl_ssl_error_is_nofile() {
>> >> >   unsigned long l = ERR_peek_error();
>> >> >   if (tQSL_Error == TQSL_OPENSSL_ERROR &&
>> >> > - ERR_GET_LIB(l) == ERR_LIB_SYS && ERR_GET_FUNC(l) == SYS_F_FOPEN)
>> >> > + ERR_GET_LIB(l) == ERR_LIB_SYS == SYS_F_FOPEN)
>> >> >   return 1;
>> >> >   if (tQSL_Error == TQSL_SYSTEM_ERROR && tQSL_Errno == ENOENT)
>> >> >   return 1;
>> >>
>> >> Did you also meen to remove " == SYS_F_FOPEN"?
>> >>
>> >> Is there really no semantics change from this?  (It seems sensible to
>> >> adjust, but I am not entirely following.
>> >
>> > Should it be removed? I only really hack at C/C++ code.
>>
>> I am surprised it builds and I'm not sure what it does.
>>
>> Basically
>>
>>   && ERR_GET_FUNC(l) == SYS_F_FOPEN
>>
>> is a logical and an equality test.   So dropping the entire test could
>> make sense and your + line would then be
>>
>> >> > + ERR_GET_LIB(l) == ERR_LIB_SYS)
>>
>> but I don't understand what this did, and how one should try to do the
>> same thing in the 3.0.0 world.
>>
>
> Fixed in my patch, unfortunately I can't answer the last part...
>
>
> (It strikes me as aggressive to be moving to 3.0.0, but those are tough
>> calls, and surely beyond your control..)
>>
>
> Well Fedora tends to be leading edge... Latest kernels, latest GCC, latest
> python (currently testing 3.11 alpha) etc.  Someone has to be first, right?
>
> There's also some additional deprecated functions in OpenSSL 3  if you
> want to get ahead of it.
>
>
> https://kojipkgs.fedoraproject.org//packages/trustedqsl/2.5.9/1.fc36/data/logs/x86_64/build.log
>
> Just search on "deprecated:"
>
> Here's the porting guide:
>
> https://www.openssl.org/docs/manmaster/man7/migration_guide.html
>
>
> >> > I also have patches I maintain to clean up rpath/lib install location
>> and
>> >> > to provide a soversion for the tqsl library (required by Fedora).
>> >>
>> >> I'd be interested to see the rpath/lib patch.  pkgsrc norms are to have
>> >> RPATH of /usr/pkg/lib set in /usr/pkg/foo as /usr/pkg (or whatever) is
>> >> not in the default search path.  I know that various packaging systems
>> >> have very different rpath rules, and I think that leads to a need to be
>> >> able to configure the build for the varying environments.
>> >
>> > As long as you're installing into system library locations (which we
>> are)
>> > they're already in the library search path so RPATH isn't needed, in
>> fact
>> > as a packager I have to remove RPATH in these situations.
>>
>> Yes, but we are installing not into the system library path and thus we
>> need to set RPATH.  This is the basic difference between systems that
>> have "base system" and then "packages" and systems that have "single
>> base system but many things are optional".  The first is the traditional
>> UNIX approach, and the second the GNU/Linux approach.
>>
>> Even on GNU/Linux, if someone builds tqsl themselves with
>> prefix=/home/johndoe/HAM, then RPATH is needed and should happen.  So
>> it's really a special case that if libdir is /usr/lib then RPATH should
>> be skippped, and in general it should be included.
>>
>> > So I guess maybe the first step would be to wrap some conditionals
>> around
>> > the current logic so it's only used when needed.
>> >
>> > Here's my rpath/install patch:
>> >
>> >
>> https://src.fedoraproject.org/rpms/trustedqsl/blob/rawhide/f/tqsl-2.5-rpath.patch
>> >
>> > The first part just roto-rooters the RPATH stuff from
>> apps/CMakeLists.txt
>> > since I don't need it. Probably not acceptable as is :)
>>
>> Indeed, that is not acceptable as it breaks many other systems.  My
>> impression is that CMake already has the concept of whether RPATH should
>> be set or not and there are cmake-time config arguments to say which way
>> it should be.  I would say aligning to that is the standard approach.
>>
>
> Yes, so I guess the question is why was that code once needed? Or are
> there corner cases?
>
>
> > The 2nd part should be more interesting to you. Here I replace all the
>> > logic and replace it with the GNUInstallDirs module which will set all
>> the
>> > CMAKE_INSTALL_... variables correctly for Linux systems since RPM and
>> DEB
>> > based systems handle this very differently. This probably should be
>> wrapped
>> > in an "IF(LINUX)" conditional.
>>
>> If it's only for GNU/Linux systems, yes, but GNUInstallDirs seems like
>> the right approach in general.  pkgsrc path norms are basically
>> inherited from autoconf and thus from the GNU coding standards, with
>> some adjustments to be similar to the BSD hier(7), but all of that
>> mostly is a common history from the 80s.
>>
>> As for "libsuffix", my view is that that standard libdir is $prefix/lib,
>> and systems that use lib64 are unusual and it's ok to expect them to be
>> configured wtih cmake libsuffix=64 or whatever, but if there's a way to
>> query if the current system is like that and do it automatically, that's
>> ok.
>>
>
> As far as I know lib64 is used on all RPM based systems (certainly Fedora,
> RHEL, OpenSUSE) that support multi-arch which allows use of 32-bit
> libraries on 64-bit systems.
>
>
> Does the Fedora world use lib64 on x86_64, or just lib?  I have the
>> impression that lib64 happens on Debian.
>>
>
> No Debian uses a /lib/<multiarch-tuple> type arrangement.
>
> https://cmake.org/cmake/help/latest/module/GNUInstallDirs.html
>
> Thanks,
> Richard
> KF5OIM
> _______________________________________________
> Trustedqsl-testing mailing list
> Tru...@li...
> https://lists.sourceforge.net/lists/listinfo/trustedqsl-testing
>


-- 
Rick Murphy, D.Sc., CISSP-ISSAP, K1MU/4, Annandale VA USA

Re: [Trustedqsl-testing] OpenSSL 3.0.0

[Greg T. <gd...@le...>]

Sounds good, except it isn't all Linux distributions, but all systems conforming to POSIX.  I'll have a look at the cmake stuff as I need to page it in anyway.

Is this supposed to build on windows?   That's the odd one out, not POSIX.

Re: [Trustedqsl-testing] OpenSSL 3.0.0

[Rick M. <k1...@ar...>]

Yes, this builds on MacOS, Windows. and Linux. It also builds on Debian,
Fedora, Ubuntu, .... building and installing is distinct from building to
conform to a distribution's policies.

For example, I stopped bundling a minimal trusted certificate store
(ca-bundle.crt) on Unix builds because some distro forbade that.

This issue with the RPATH setup in the CMakeLists.txt file is a patch that
was submitted to correct a problem with some distro that doesn't use
"/lib64" as I recall.  This is essentially me accepting changes that work
well for a single distribution and not working in a way to adapt to a wide
range of distro and distro ages, along with the fact that I'm not an expert
in how to build this portably.

I am happy to accept fixes, but not fixes that undo work that
supposedly corrects the issues already.
73,
    -Rick



On Sun, Dec 5, 2021 at 1:53 PM Greg Troxel <gd...@le...> wrote:

> Sounds good, except it isn't all Linux distributions, but all systems
> conforming to POSIX.  I'll have a look at the cmake stuff as I need to page
> it in anyway.
>
> Is this supposed to build on windows?   That's the odd one out, not POSIX.
>


-- 
Rick Murphy, D.Sc., CISSP-ISSAP, K1MU/4, Annandale VA USA

Re: [Trustedqsl-testing] OpenSSL 3.0.0

[Richard S. <hob...@gm...>]

On Sun, Dec 5, 2021 at 2:10 PM Rick Murphy <k1...@ar...> wrote:

> Yes, this builds on MacOS, Windows. and Linux. It also builds on Debian,
> Fedora, Ubuntu, .... building and installing is distinct from building to
> conform to a distribution's policies.
>
> For example, I stopped bundling a minimal trusted certificate store
> (ca-bundle.crt) on Unix builds because some distro forbade that.
>
> This issue with the RPATH setup in the CMakeLists.txt file is a patch that
> was submitted to correct a problem with some distro that doesn't use
> "/lib64" as I recall.  This is essentially me accepting changes that work
> well for a single distribution and not working in a way to adapt to a wide
> range of distro and distro ages, along with the fact that I'm not an expert
> in how to build this portably.
>

How long ago was it contributed? CMake handles RPATH pretty well these days
and should remove RPATH on install when installing into system locations,
or otherwise update it.

Thanks,
Richard
KF5OIM

>

Re: [Trustedqsl-testing] OpenSSL 3.0.0

[Rick M. <k1...@ar...>]

On Sun, Dec 5, 2021 at 3:16 PM Richard Shaw <hob...@gm...> wrote:

> On Sun, Dec 5, 2021 at 2:10 PM Rick Murphy <k1...@ar...> wrote:
>
>> Yes, this builds on MacOS, Windows. and Linux. It also builds on Debian,
>> Fedora, Ubuntu, .... building and installing is distinct from building to
>> conform to a distribution's policies.
>>
>> For example, I stopped bundling a minimal trusted certificate store
>> (ca-bundle.crt) on Unix builds because some distro forbade that.
>>
>> This issue with the RPATH setup in the CMakeLists.txt file is a patch
>> that was submitted to correct a problem with some distro that doesn't use
>> "/lib64" as I recall.  This is essentially me accepting changes that work
>> well for a single distribution and not working in a way to adapt to a wide
>> range of distro and distro ages, along with the fact that I'm not an expert
>> in how to build this portably.
>>
>
> How long ago was it contributed? CMake handles RPATH pretty well these
> days and should remove RPATH on install when installing into system
> locations, or otherwise update it.
>

A long while ago. These changes are in TQSL 2.1 - circa 2015. Perhaps
earlier.

I'm happy to accept help here with cleaning this up, as it seems over time
that this has become overcomplicated in a way that CMake is probably able
to do a much better job at, versus the current ad-hoc attempts.
73,
    -Rick
-- 
Rick Murphy, D.Sc., CISSP-ISSAP, K1MU/4, Annandale VA USA

Re: [Trustedqsl-testing] OpenSSL 3.0.0

[Richard S. <hob...@gm...>]

On Sun, Dec 5, 2021 at 3:23 PM Rick Murphy <k1...@ar...> wrote:

> On Sun, Dec 5, 2021 at 3:16 PM Richard Shaw <hob...@gm...> wrote:
>
>> On Sun, Dec 5, 2021 at 2:10 PM Rick Murphy <k1...@ar...> wrote:
>>
>>> Yes, this builds on MacOS, Windows. and Linux. It also builds on Debian,
>>> Fedora, Ubuntu, .... building and installing is distinct from building to
>>> conform to a distribution's policies.
>>>
>>> For example, I stopped bundling a minimal trusted certificate store
>>> (ca-bundle.crt) on Unix builds because some distro forbade that.
>>>
>>> This issue with the RPATH setup in the CMakeLists.txt file is a patch
>>> that was submitted to correct a problem with some distro that doesn't use
>>> "/lib64" as I recall.  This is essentially me accepting changes that work
>>> well for a single distribution and not working in a way to adapt to a wide
>>> range of distro and distro ages, along with the fact that I'm not an expert
>>> in how to build this portably.
>>>
>>
>> How long ago was it contributed? CMake handles RPATH pretty well these
>> days and should remove RPATH on install when installing into system
>> locations, or otherwise update it.
>>
>
> A long while ago. These changes are in TQSL 2.1 - circa 2015. Perhaps
> earlier.
>
> I'm happy to accept help here with cleaning this up, as it seems over time
> that this has become overcomplicated in a way that CMake is probably able
> to do a much better job at, versus the current ad-hoc attempts.
>

Sounds good to me, How do you propose we test? Obviously Fedora is OK and
by extension RHEL and derivatives as I build for Fedora EPEL as well.

I'm unfamiliar with how tqsl development is managed. Is there any infra
where CI is performed? Known group of "testers"?

Thanks,
Richard
KF5OIM

Re: [Trustedqsl-testing] OpenSSL 3.0.0

[Rick M. <k1...@ar...>]

On Wed, Dec 8, 2021 at 8:37 AM Richard Shaw <hob...@gm...> wrote:

> On Sun, Dec 5, 2021 at 3:23 PM Rick Murphy <k1...@ar...> wrote:
>
>> On Sun, Dec 5, 2021 at 3:16 PM Richard Shaw <hob...@gm...> wrote:
>>
>>> On Sun, Dec 5, 2021 at 2:10 PM Rick Murphy <k1...@ar...> wrote:
>>>
>>>> Yes, this builds on MacOS, Windows. and Linux. It also builds on
>>>> Debian, Fedora, Ubuntu, .... building and installing is distinct from
>>>> building to conform to a distribution's policies.
>>>>
>>>> For example, I stopped bundling a minimal trusted certificate store
>>>> (ca-bundle.crt) on Unix builds because some distro forbade that.
>>>>
>>>> This issue with the RPATH setup in the CMakeLists.txt file is a patch
>>>> that was submitted to correct a problem with some distro that doesn't use
>>>> "/lib64" as I recall.  This is essentially me accepting changes that work
>>>> well for a single distribution and not working in a way to adapt to a wide
>>>> range of distro and distro ages, along with the fact that I'm not an expert
>>>> in how to build this portably.
>>>>
>>>
>>> How long ago was it contributed? CMake handles RPATH pretty well these
>>> days and should remove RPATH on install when installing into system
>>> locations, or otherwise update it.
>>>
>>
>> A long while ago. These changes are in TQSL 2.1 - circa 2015. Perhaps
>> earlier.
>>
>> I'm happy to accept help here with cleaning this up, as it seems over
>> time that this has become overcomplicated in a way that CMake is probably
>> able to do a much better job at, versus the current ad-hoc attempts.
>>
>
> Sounds good to me, How do you propose we test? Obviously Fedora is OK and
> by extension RHEL and derivatives as I build for Fedora EPEL as well.
>

[Sorry, this draft apparently has been stalled for some time unsent,[

I accepted the patches here, removed the RPATH setting, removed the setting
for library path, and added an include for GNUInstallPaths, all of which I
assumed would make this a distribution responsibility.

Built a tar.gz and started down my list of VMs running various Linux
distributions.

The VERY FIRST TRY failed. CentOS 7, It installed libtqsllib.so into
/usr/local/lib/  and tqsl couldn't be activated because that library could
not be found (worked when moved to /lib64).

I'm still asking for help here.  People seem to be implying that including
GNUInstallPaths just fixes these library dependent issues, but I'm not sure
what to do when trying this runs right into an error.


> I'm unfamiliar with how tqsl development is managed. Is there any infra
> where CI is performed? Known group of "testers"?
>

That question is kind of interesting. There is one developer, who is me.
There's only one person who has ever provided me feedback on test releases
(that is the purpose of this mailing list.)
I have a fairly complete set of regression testing scripts for the command
line operations, but I have no experience with building test harnesses for
a multi-platform gui-driven UI.  Any help here would be quite welcome.

73,
    -Rick
-- 
Rick Murphy, D.Sc., CISSP-ISSAP, K1MU/4, Annandale VA USA

Re: [Trustedqsl-testing] OpenSSL 3.0.0

[Greg T. <gd...@le...>]

Rick Murphy <k1...@ar...> writes:

> [Sorry, this draft apparently has been stalled for some time unsent,[

no worries, everybody is busy

> I accepted the patches here, removed the RPATH setting, removed the setting
> for library path, and added an include for GNUInstallPaths, all of which I
> assumed would make this a distribution responsibility.

I tried to point out when I saw the patches that, while sensible in the
environment they were for, they would break most (in a general sense)
installs, working only for installs into the default prefix on systems
that don't use RPATH (which may be most installs on most computers by %,
but it's still narrow over the set of configurations).

> Built a tar.gz and started down my list of VMs running various Linux
> distributions.

Certainly you can test what you can test, but I tried to say earlier and
didn't seem to succeed: All the world is not Linux.  (Besides windows
and mac) tqsl should work on any system meeting POSIX and for which the
prereqs are available.  It has been working fine on NetBSD, for example,
perhaps with a bit of cmake patching, and I would expect it works on the
others BSDs and even on illumos, with minor effort. I have been
regularly testing and updating pkgsrc as releases happen for a number of
years, and I run tqsl on NetBSD for log upload.  I just don't say
anything because almost always it builds fine.

In addition, the earlier discussion has been focused on installs into
the default prefix of /usr.  While some packaging systems (typical on
GNU/Linux variants) do that, some don't, and users who build tqsl
themselves will almost always use a non-default prefix.  So dropping
RPATH setting always is basically wrong.  It is reasonable to omit RPATH
when the libdir is known to be a system libdir that is searched without
RPATH.

I'm glad to see you testing with /usr/local.  If /usr/local/lib64 is in
the default search path, though, I suggest you test with /usr/tqsl as
the prefix, or /usr/foo or something else.  tqsl should work for that
build too, and with the RPATH stuff taken out I bet it will not.

> The VERY FIRST TRY failed. CentOS 7, It installed libtqsllib.so into
> /usr/local/lib/  and tqsl couldn't be activated because that library could
> not be found (worked when moved to /lib64).

Presumably you will back out this change then.  You have encountered a
problem I expected from reading the patch.

> I'm still asking for help here.  People seem to be implying that including
> GNUInstallPaths just fixes these library dependent issues, but I'm not sure
> what to do when trying this runs right into an error.

I think the only reasonable path forward is to understand multiple
things and design a plan based on that.  I think it's

  have a way to find out if the system puts libs in /lib or /lib64.  As
  I understand it, lib64 is used on some but not all GNU/Linux systems
  on x86_64 computers, but not on other CPU types.  Or perhaps also on
  arm64 (and sparc64, to the extent anything still supports that).
  lib64 is never used on NetBSD; lib is always for native libs.  I would
  think Gnuinstallpaths would figure this out, or there'd be some other
  direct support in cmake, as every single program needs to accomodate
  this variation in where libs belong.  Perhaps, this can be conditional
  on the existence of /usr/lib64 if there isn't a better way.

  Set RPATH unless the configured libdir matches /usr/lib or /usr/lib64
  (if we found out that the system is of the lib64 persuasion).  Or
  perhaps "set RPATH unless prefix is /usr" is simpler and good enough,
  and on refletion I lean to it.

The first point is somewhat difficult, and the second should be fairly
easy.

Can someone comment in more detail on the lib64 convention on various
GNU/Linux systems?  Specifically:

  On systems with /usr/lib64, does the lib64 convention extend to
  arbitrary prefixes that a user builds software for?  E.g if I
  "./configure --prefix=/usr/foo" etc. should that instlal to
  /usr/foo/lib or /usr/foo/lib64?

  On typical systems, what directories are in the default linker search
  path, so that libs in them may be found without RPATH?  Is that
  uniform across GNU/Linux variants?

  Are there any norms and standard procedures for how to find out
  whether to use lib64 or not?  Please describe.

  Are there any norms and standard procedures for how to find out if a
  libdir is in the standard linker path and RPATH can be safely omitted?

  Are there norms for when RPATH should be omitted for libir in default
  path, and are they uniform over all GNU/Linux variants or varying?


For NetBSD the answers are

  always lib, never lib64

  /lib, /usr/lib is reliable.  Don't assume more.

  [don't ever do anything about lib64]

  It's ok to omit RPATH for /lib and /usr/lib, and best practice is to
  do so, but it's also irregular to build programs not provided with the
  operating system with --prefix=/usr, so this doesn't come up in
  practice.  It is never ok to drop RPATH otherwise.


Hope this helps; there is no easy way out.

73 de n1dam

Re: [Trustedqsl-testing] OpenSSL 3.0.0

[Rick M. <k1...@ar...>]

On Mon, Dec 20, 2021 at 9:32 AM Greg Troxel <gd...@le...> wrote:

>
> Rick Murphy <k1...@ar...> writes:
>
> > [Sorry, this draft apparently has been stalled for some time unsent,[
>
> no worries, everybody is busy
>
> > I accepted the patches here, removed the RPATH setting, removed the
> setting
> > for library path, and added an include for GNUInstallPaths, all of which
> I
> > assumed would make this a distribution responsibility.
>
> I tried to point out when I saw the patches that, while sensible in the
> environment they were for, they would break most (in a general sense)
> installs, working only for installs into the default prefix on systems
> that don't use RPATH (which may be most installs on most computers by %,
> but it's still narrow over the set of configurations).
>
> > Built a tar.gz and started down my list of VMs running various Linux
> > distributions.
>
> Certainly you can test what you can test, but I tried to say earlier and
> didn't seem to succeed: All the world is not Linux.  (Besides windows
> and mac) tqsl should work on any system meeting POSIX and for which the
> prereqs are available.  It has been working fine on NetBSD, for example,
> perhaps with a bit of cmake patching, and I would expect it works on the
> others BSDs and even on illumos, with minor effort. I have been
> regularly testing and updating pkgsrc as releases happen for a number of
> years, and I run tqsl on NetBSD for log upload.  I just don't say
> anything because almost always it builds fine.
>
> In addition, the earlier discussion has been focused on installs into
> the default prefix of /usr.  While some packaging systems (typical on
> GNU/Linux variants) do that, some don't, and users who build tqsl
> themselves will almost always use a non-default prefix.  So dropping
> RPATH setting always is basically wrong.  It is reasonable to omit RPATH
> when the libdir is known to be a system libdir that is searched without
> RPATH.
>
> I'm glad to see you testing with /usr/local.  If /usr/local/lib64 is in
> the default search path, though, I suggest you test with /usr/tqsl as
> the prefix, or /usr/foo or something else.  tqsl should work for that
> build too, and with the RPATH stuff taken out I bet it will not.
>
> > The VERY FIRST TRY failed. CentOS 7, It installed libtqsllib.so into
> > /usr/local/lib/  and tqsl couldn't be activated because that library
> could
> > not be found (worked when moved to /lib64).
>
> Presumably you will back out this change then.  You have encountered a
> problem I expected from reading the patch.
>
> > I'm still asking for help here.  People seem to be implying that
> including
> > GNUInstallPaths just fixes these library dependent issues, but I'm not
> sure
> > what to do when trying this runs right into an error.
>
> I think the only reasonable path forward is to understand multiple
> things and design a plan based on that.  I think it's
>
>   have a way to find out if the system puts libs in /lib or /lib64.  As
>   I understand it, lib64 is used on some but not all GNU/Linux systems
>   on x86_64 computers, but not on other CPU types.  Or perhaps also on
>   arm64 (and sparc64, to the extent anything still supports that).
>   lib64 is never used on NetBSD; lib is always for native libs.  I would
>   think Gnuinstallpaths would figure this out, or there'd be some other
>   direct support in cmake, as every single program needs to accomodate
>   this variation in where libs belong.  Perhaps, this can be conditional
>   on the existence of /usr/lib64 if there isn't a better way.
>
>   Set RPATH unless the configured libdir matches /usr/lib or /usr/lib64
>   (if we found out that the system is of the lib64 persuasion).  Or
>   perhaps "set RPATH unless prefix is /usr" is simpler and good enough,
>   and on refletion I lean to it.
>
> The first point is somewhat difficult, and the second should be fairly
> easy.
>
> Can someone comment in more detail on the lib64 convention on various
> GNU/Linux systems?  Specifically:
>
>   On systems with /usr/lib64, does the lib64 convention extend to
>   arbitrary prefixes that a user builds software for?  E.g if I
>   "./configure --prefix=/usr/foo" etc. should that instlal to
>   /usr/foo/lib or /usr/foo/lib64?
>
>   On typical systems, what directories are in the default linker search
>   path, so that libs in them may be found without RPATH?  Is that
>   uniform across GNU/Linux variants?
>
>   Are there any norms and standard procedures for how to find out
>   whether to use lib64 or not?  Please describe.
>
>   Are there any norms and standard procedures for how to find out if a
>   libdir is in the standard linker path and RPATH can be safely omitted?
>
>   Are there norms for when RPATH should be omitted for libir in default
>   path, and are they uniform over all GNU/Linux variants or varying?
>
>
> For NetBSD the answers are
>
>   always lib, never lib64
>
>   /lib, /usr/lib is reliable.  Don't assume more.
>
>   [don't ever do anything about lib64]
>
>   It's ok to omit RPATH for /lib and /usr/lib, and best practice is to
>   do so, but it's also irregular to build programs not provided with the
>   operating system with --prefix=/usr, so this doesn't come up in
>   practice.  It is never ok to drop RPATH otherwise.
>
>
> Hope this helps; there is no easy way out.
>
> 73 de n1dam
>

Thanks, Greg - appreciate the detail here.
This isn't a one-size-fits-all.   And yes, I'm quite aware of non-Linux
systems. If BSD builds are ever broken, I'd like to know.  I'm an OpenBSD
user but that's a firewall so no GUI.
73,
    -Rick
-- 
Rick Murphy, D.Sc., CISSP-ISSAP, K1MU/4, Annandale VA USA

Re: [Trustedqsl-testing] OpenSSL 3.0.0

[Richard S. <hob...@gm...>]

Ok, not quoting this time, too much to sort through :)

I installed a CentOS 7 VM and assuming you tried the modifications that are
in current master I see the problem, GNUInstallDirs isn't magic, the
install locations need to be updated as well in src/CMakeLists.txt

${CMAKE_INSTALL_LIBDIR}
${CMAKE_INSTALL_DATADIR}
${CMAKE_INSTALL_INCLUDEDIR}

respectively. I would have provided a git diff but I installed the VM to
command line only and apparently copy paste doesn't work with Gnome Boxes
in that instance, however, I can do so if helpful but it's a pretty trivial
change.

This would also allow the end user to override the install location and not
just the prefix if desired. With those changes I checked
src/cmake_install.cmake and lib64 was found as expected.

Thanks,
Richard

Re: [Trustedqsl-testing] OpenSSL 3.0.0

[Richard S. <hob...@gm...>]

Also, it's not necessary since CMake 2.8.12 seems to work fine, but I would
recommend installing the cmake3 package from Fedora EPEL for a MUCH more
modern version. Works the same except the binary is renamed to cmake3.

Thanks,
Richard