Filesystem in Userspace

On Wed, 15 Sep 2010, Raymes Khoury wrote:
> I'm trying to compile fuse with the gold linker.
> 
> I'm getting the following error:
> libtool: link: i686-pc-linux-gnu-gcc -shared  .libs/fuse.o
> .libs/fuse_kern_chan.o .libs/fuse_loop.o .libs/fuse_loop_mt.o
> .libs/fuse_lowlevel.o .libs/fuse_mt.o .libs/fuse_opt.o
> .libs/fuse_session.o .libs/fuse_signals.o .libs/cuse_lowlevel.o
> .libs/helper.o .libs/subdir.o .libs/iconv.o .libs/mount.o
> .libs/mount_util.o   -lrt -ldl -L/build/x86-generic/lib
> -L/build/x86-generic/usr/lib  -march=core2 -mtune=generic -mfpmath=sse
> -pthread -Wl,--version-script -Wl,./fuse_versionscript   -pthread
> -Wl,-soname -Wl,libfuse.so.2 -o .libs/libfuse.so.2.8.1
> /usr/libexec/gcc/i686-pc-linux-gnu/ld: warning: using 'FUSE_2.4' as
> version for 'fuse_chan_new' which is also named in version 'FUSE_2.6'
> in script
> /usr/libexec/gcc/i686-pc-linux-gnu/ld: error: .libs/fuse_session.o:
> multiple definition of 'fuse_chan_new'
> /usr/libexec/gcc/i686-pc-linux-gnu/ld: .libs/fuse_session.o: previous
> definition here
> /usr/libexec/gcc/i686-pc-linux-gnu/ld: error: symbol __fuse_exited has
> undefined version
> collect2: ld returned 1 exit status
> 
> I'm interested in the second error:
> /usr/libexec/gcc/i686-pc-linux-gnu/ld: error: symbol __fuse_exited has
> undefined version
> 
> This is caused by the following directive in fuse.c:
> FUSE_SYMVER(".symver fuse_exited,__fuse_exited@");
> Unfortunately it is not clear in GNU ld what the behavior should be
> when no version is specified for the symbol (or at least how not
> specifying a version can be useful) . What was the intention of
> including this directive (and several other similar directives)?

Symbol versioning in libfuse was introduced in fuse-2.2.  The first
public release of libfuse-2 was 2.1, which had symbols without
versions.

To allow compatibility with filesystems compiled against 2.1 these
unversioned symbols need to be provided in later versions of the
library.

Thanks,
Miklos

Thanks for the reply. I spoke with Ian Taylor (the author of gold) and
his response was as follows:

> I did some more testing, and it appears that when the dynamic linker
> sees a symbol with no version in an executable, it will link it to the
> symbol defined with the first version in the shared library.  So there
> is no need for the SYM@ syntax as far as I can see; the right thing to
> do to make tihs work is to put the original symbol in the first version
> listed in the version script.

So I have attached a patch that allows fuse-2.7.4 to link with gold.
Note that there are two problems addressed here:
 * Use of the symver directive without a version specified. This is
fixed by assigning these symbols to a version FUSE_UNVERSIONED which
appears first in the linker script.
 * If __FreeBSD__ is not defined, fuse_chan_new_compat24 is bound to
fuse_chan_new@... (in fuse_session.c), however the version script
also includes fuse_chan_new in FUSE_2.4 so two different versions of
the function are effectively bound and it is ambiguous which should be
used. GNU ld doesn't complain about this but gold does. This is fixed
by removing fuse_chan_new@... from the version script and
explicitly exporting it in fuse_session.c when __FreeBSD__ is defined.

Thanks,

Raymes

On Wed, Sep 22, 2010 at 6:06 AM, Miklos Szeredi <miklos@...> wrote:
> On Wed, 15 Sep 2010, Raymes Khoury wrote:
>> I'm trying to compile fuse with the gold linker.
>>
>> I'm getting the following error:
>> libtool: link: i686-pc-linux-gnu-gcc -shared  .libs/fuse.o
>> .libs/fuse_kern_chan.o .libs/fuse_loop.o .libs/fuse_loop_mt.o
>> .libs/fuse_lowlevel.o .libs/fuse_mt.o .libs/fuse_opt.o
>> .libs/fuse_session.o .libs/fuse_signals.o .libs/cuse_lowlevel.o
>> .libs/helper.o .libs/subdir.o .libs/iconv.o .libs/mount.o
>> .libs/mount_util.o   -lrt -ldl -L/build/x86-generic/lib
>> -L/build/x86-generic/usr/lib  -march=core2 -mtune=generic -mfpmath=sse
>> -pthread -Wl,--version-script -Wl,./fuse_versionscript   -pthread
>> -Wl,-soname -Wl,libfuse.so.2 -o .libs/libfuse.so.2.8.1
>> /usr/libexec/gcc/i686-pc-linux-gnu/ld: warning: using 'FUSE_2.4' as
>> version for 'fuse_chan_new' which is also named in version 'FUSE_2.6'
>> in script
>> /usr/libexec/gcc/i686-pc-linux-gnu/ld: error: .libs/fuse_session.o:
>> multiple definition of 'fuse_chan_new'
>> /usr/libexec/gcc/i686-pc-linux-gnu/ld: .libs/fuse_session.o: previous
>> definition here
>> /usr/libexec/gcc/i686-pc-linux-gnu/ld: error: symbol __fuse_exited has
>> undefined version
>> collect2: ld returned 1 exit status
>>
>> I'm interested in the second error:
>> /usr/libexec/gcc/i686-pc-linux-gnu/ld: error: symbol __fuse_exited has
>> undefined version
>>
>> This is caused by the following directive in fuse.c:
>> FUSE_SYMVER(".symver fuse_exited,__fuse_exited@");
>> Unfortunately it is not clear in GNU ld what the behavior should be
>> when no version is specified for the symbol (or at least how not
>> specifying a version can be useful) . What was the intention of
>> including this directive (and several other similar directives)?
>
> Symbol versioning in libfuse was introduced in fuse-2.2.  The first
> public release of libfuse-2 was 2.1, which had symbols without
> versions.
>
> To allow compatibility with filesystems compiled against 2.1 these
> unversioned symbols need to be provided in later versions of the
> library.
>
> Thanks,
> Miklos
>

On Tue, 28 Sep 2010, Raymes Khoury wrote:
> Thanks for the reply. I spoke with Ian Taylor (the author of gold) and
> his response was as follows:
> 
> > I did some more testing, and it appears that when the dynamic linker
> > sees a symbol with no version in an executable, it will link it to the
> > symbol defined with the first version in the shared library.  So there
> > is no need for the SYM@ syntax as far as I can see; the right thing to
> > do to make tihs work is to put the original symbol in the first version
> > listed in the version script.

This may be true, but it's completely undocumented.  At least I can't
find any trace of this in GNU ld manual, while the SYM@ syntax *is*
documented:

  `foo@' represents the symbol `foo' bound to the unspecified base
  version of the symbol

It is also possible to do that with the version script:

  Symbols defined in the library which aren't specifically bound to a
  version node are effectively bound to an unspecified base version of
  the library.

But this is impractical since it would mean having to specify every
non-exported symbol.

>  * If __FreeBSD__ is not defined, fuse_chan_new_compat24 is bound to
> fuse_chan_new@... (in fuse_session.c), however the version script
> also includes fuse_chan_new in FUSE_2.4 so two different versions of
> the function are effectively bound and it is ambiguous which should be
> used. GNU ld doesn't complain about this but gold does. This is fixed
> by removing fuse_chan_new@... from the version script and
> explicitly exporting it in fuse_session.c when __FreeBSD__ is defined.

Thanks, this is indeed a bug.

Miklos

Miklos Szeredi <miklos@...> writes:

> On Tue, 28 Sep 2010, Raymes Khoury wrote:
>> Thanks for the reply. I spoke with Ian Taylor (the author of gold) and
>> his response was as follows:
>> 
>> > I did some more testing, and it appears that when the dynamic linker
>> > sees a symbol with no version in an executable, it will link it to the
>> > symbol defined with the first version in the shared library.  So there
>> > is no need for the SYM@ syntax as far as I can see; the right thing to
>> > do to make tihs work is to put the original symbol in the first version
>> > listed in the version script.
>
> This may be true, but it's completely undocumented.  At least I can't
> find any trace of this in GNU ld manual, while the SYM@ syntax *is*
> documented:
>
>   `foo@' represents the symbol `foo' bound to the unspecified base
>   version of the symbol

The SYM@ syntax is documented but does not work, with either GNU ld or
gold.  It perhaps worked in the past, but it certainly hasn't worked in
a long time.  I can't personally take responsibility for the GNU ld
manual.

I agree that while attaching the symbol to the first version does work,
it is not documented.

The key program here is actually not the linker, it is the dynamic
linker.  That is the program which determines what will happen when an
executable with an unversioned symbol is linked against a shared library
with versioned symbols.  The glibc dynamic linker will arrange to link
the unversioned symbol in the executable with the symbol attached to the
first regular version, not the base version, in the shared library.
This appears to be an intentional choice in glibc, but I don't know if
there is any documentation of it.

Ian

On Tue, 28 Sep 2010, Ian Lance Taylor wrote:
> Miklos Szeredi <miklos@...> writes:
> 
> > On Tue, 28 Sep 2010, Raymes Khoury wrote:
> >> Thanks for the reply. I spoke with Ian Taylor (the author of gold) and
> >> his response was as follows:
> >> 
> >> > I did some more testing, and it appears that when the dynamic linker
> >> > sees a symbol with no version in an executable, it will link it to the
> >> > symbol defined with the first version in the shared library.  So there
> >> > is no need for the SYM@ syntax as far as I can see; the right thing to
> >> > do to make tihs work is to put the original symbol in the first version
> >> > listed in the version script.
> >
> > This may be true, but it's completely undocumented.  At least I can't
> > find any trace of this in GNU ld manual, while the SYM@ syntax *is*
> > documented:
> >
> >   `foo@' represents the symbol `foo' bound to the unspecified base
> >   version of the symbol
> 
> The SYM@ syntax is documented but does not work, with either GNU ld or
> gold.  It perhaps worked in the past, but it certainly hasn't worked in
> a long time.  I can't personally take responsibility for the GNU ld
> manual.

I'm not sure why you think it doesn't work.  I tried and it works as
expected.  If there exists a SYM@ then the dynamic linker will resolve
unversioned symbols to that, *not* the first symbol in the
versionscript, as you seem to have found.

Only if the base (SYM@) version is not found then will the first
version in the versionscript matching the symbol be used for
unversioned symbols.

> 
> I agree that while attaching the symbol to the first version does work,
> it is not documented.
> 
> The key program here is actually not the linker, it is the dynamic
> linker.  That is the program which determines what will happen when an
> executable with an unversioned symbol is linked against a shared library
> with versioned symbols.  The glibc dynamic linker will arrange to link
> the unversioned symbol in the executable with the symbol attached to the
> first regular version, not the base version, in the shared library.

Which version of the dynamic linker did you use?  If this is the case
then its a simple and plain bug since it used to work and still does
on my version (ld-2.10.1.so).

Thanks,
Miklos

On Wed, 29 Sep 2010, Ian Lance Taylor wrote:
> Miklos Szeredi <miklos@...> writes:
> 
> > I'm not sure why you think it doesn't work.  I tried and it works as
> > expected.  If there exists a SYM@ then the dynamic linker will resolve
> > unversioned symbols to that, *not* the first symbol in the
> > versionscript, as you seem to have found.
> >
> > Only if the base (SYM@) version is not found then will the first
> > version in the versionscript matching the symbol be used for
> > unversioned symbols.
> 
> That is interesting.  I was unable to make it work after several
> attempts.  Can you sketch out what you did, or even better provide a
> test case?  Which version of glibc are you using?

Essentially the same test case, hmm.

Well apparently what it comes down to is... the name of the function.
Try this on your test case:

  sed -i s/f1/t1/g *.c

So something is definitely wrong either in ld or ld.so.

Thanks,
Miklos

Miklos Szeredi <miklos@...> writes:

> On Wed, 29 Sep 2010, Ian Lance Taylor wrote:
>> 
>> That is interesting.  I was unable to make it work after several
>> attempts.  Can you sketch out what you did, or even better provide a
>> test case?  Which version of glibc are you using?
>
> Essentially the same test case, hmm.
>
> Well apparently what it comes down to is... the name of the function.
> Try this on your test case:
>
>   sed -i s/f1/t1/g *.c
>
> So something is definitely wrong either in ld or ld.so.

Cool.  It has to do with the order in which the symbols appear in the
dynamic hash table, which naturally depends on hash of the name.  I'm
going to have to see what I can come up with.

Ian

Miklos,

Ian tried to get some discussion going on the matter on the libc
mailing list (http://sourceware.org/ml/libc-alpha/2011-02/msg00154.html)
however there has been no response so far.

Given that you are currently relying on unpredictable behavior, do you
think it is worth removing the reference to the base version of the
symbol?

Thanks,
Raymes

On Wed, Sep 29, 2010 at 8:15 AM, Ian Lance Taylor <ian@...> wrote:
> Miklos Szeredi <miklos@...> writes:
>
>> On Wed, 29 Sep 2010, Ian Lance Taylor wrote:
>>>
>>> That is interesting.  I was unable to make it work after several
>>> attempts.  Can you sketch out what you did, or even better provide a
>>> test case?  Which version of glibc are you using?
>>
>> Essentially the same test case, hmm.
>>
>> Well apparently what it comes down to is... the name of the function.
>> Try this on your test case:
>>
>>   sed -i s/f1/t1/g *.c
>>
>> So something is definitely wrong either in ld or ld.so.
>
> Cool.  It has to do with the order in which the symbols appear in the
> dynamic hash table, which naturally depends on hash of the name.  I'm
> going to have to see what I can come up with.
>
> Ian
>

Raymes Khoury <raymes.khoury@...> writes:

> Ian tried to get some discussion going on the matter on the libc
> mailing list (http://sourceware.org/ml/libc-alpha/2011-02/msg00154.html)
> however there has been no response so far.
>
> Given that you are currently relying on unpredictable behavior, do you
> think it is worth removing the reference to the base version of the
> symbol?

With that I'd risk breaking backward compatibility, even if that
backward compatibility is only by accident.

I'll remove all such cruft from libfuse-3.0, which will be done RSN.

Thanks,
Miklos