elftoolchain-developers

[Elftoolchain-developers] Multiple archs and platforms.

[Chris J. <ch...@rt...>]

Hello,

I am developing a linker for the RTEMS project to support dynamic 
loading. I have just switched from the GPL libelf to your libelf after 
taking a look in the FreeBSD kernel. I am pleased to report there are no 
problems with the testing I have done. Our use of libelf is fairly basic.

You can find the RTEMS project at http://www.rtems.org/ and the linker's 
code is:

 
http://www.rtems.org/ftp/pub/rtems/people/chrisj/rtl/rtl-host-20111121.tar.bz2

To build install waf and then 'waf configure build'.

I plan to submit clean patches once I have the work completed.

I have copied in just the libelf and common directories. The code is 
built using waf and I can currently build on Linux (CentOS5), MacOS 
(Snow Leopard) and FreeBSD (v9). Windows is a work in progress and the 
reason for the post.

Some background. RTEMS supports around 12 architectures and all are ELF 
based. We need something to play with ELF files at a container level. 
The linker uses files, sections, and symbols and not much else. The low 
level target specific code is located in RTEMS and runs on the target 
hardware. So far I can handle SPARC, M68K and i386 ELF files on MacOS 
with the same executable. I do not expect problems in the other host 
operating systems.

Windows is a little more difficult to support. The couple of issues I 
have so far are:

1. Includes such as "#include <sys/errno.h>", and "#include 
<sys/cdefs.h>" are not supported. I have worked around this issue with a 
win32 specific include directory that maps to the standard headers in my 
application.

2. There is no mmap call. I see:

   LIBELF_F_RAWFILE_MALLOC | LIBELF_F_SPECIAL_FILE

could be used but I am not sure about a couple of things. Does this read 
the whole ELF file into memory ? Secondly, I really do not like the idea 
of __WIN32__ all over the place. I can support "HAVE_CONFIG_H" and 
"HAVE_MMAP" etc. How should I handle this ?

Any guidance or suggestions would be most welcome.

Chris

Re: [Elftoolchain-developers] Multiple archs and platforms.

[Joseph K. <jk...@us...>]

> I am developing a linker for the RTEMS project to support dynamic
> loading. ... I am pleased to report there are no problems with the
> testing I have done.

That's great to know.

> You can find the RTEMS project at http://www.rtems.org/ and the linker's
> code is: [snip]

I will take a look.

> Windows is a little more difficult to support. The couple of issues I
> have so far are:

> 1. Includes such as "#include <sys/errno.h>", and "#include
> <sys/cdefs.h>" are not supported. I have worked around this issue with a
> win32 specific include directory that maps to the standard headers in my
> application.

I lack clue about program development under Windows.

> 2. There is no mmap call. I see:
>
>   LIBELF_F_RAWFILE_MALLOC | LIBELF_F_SPECIAL_FILE
>
> could be used but I am not sure about a couple of things. Does this read
> the whole ELF file into memory ?

Yes, the RAWFILE_MALLOC case will read the whole ELF object into
memory.  The motivation for the code was to support reading of an ELF
object from device files ("/dev/ksyms" in {Free,Net}BSD) and from
streams (e.g., from sockets and pipes).  Such file descriptors usually
do not support mmap() or seek(), so ELF object needs to be read in
sequentially.

One optimization for the RAWFILE_MALLOC case could be to have libelf
read in portions of the ELF object "on demand".  That said, further
study would be needed to determine the tradeoffs involved---IIRC the
default layout puts the section header table at the very end of an ELF
object, so use cases that access sections could end up reading the
entire ELF object into memory, even if such an optmization were
implemented.

> Secondly, I really do not like the idea
> of __WIN32__ all over the place. I can support "HAVE_CONFIG_H" and
> "HAVE_MMAP" etc. How should I handle this ?

IMHO, an ELFTC_HAVE_MMAP configuration option would be the way to go.

FYI, I have created ticket #366 to track this item:

  http://sourceforge.net/apps/trac/elftoolchain/ticket/366.

Regards,
Koshy

Re: [Elftoolchain-developers] Multiple archs and platforms.

[Chris J. <ch...@rt...>]

On 23/11/11 4:18 PM, Joseph Koshy wrote:
>> I am developing a linker for the RTEMS project to support dynamic
>> loading. ... I am pleased to report there are no problems with the
>> testing I have done.
>
> That's great to know.
>

It is a nice package and seems to be working well. I plan to stress it a 
little more in the linker once I have the Windows port working.

>> You can find the RTEMS project at http://www.rtems.org/ and the linker's
>> code is: [snip]
>
> I will take a look.
>

Thanks.

>> Windows is a little more difficult to support. The couple of issues I
>> have so far are:
>
>> 1. Includes such as "#include<sys/errno.h>", and "#include
>> <sys/cdefs.h>" are not supported. I have worked around this issue with a
>> win32 specific include directory that maps to the standard headers in my
>> application.
>
> I lack clue about program development under Windows.
>

I am no expert, but given a need I will get something working. I have a 
need.

All the code compiles and builds under Windows with a small amount of 
help. This is a credit to the code base.

>> 2. There is no mmap call. I see:
>>
>>   LIBELF_F_RAWFILE_MALLOC | LIBELF_F_SPECIAL_FILE
>>
>> could be used but I am not sure about a couple of things. Does this read
>> the whole ELF file into memory ?
>
> Yes, the RAWFILE_MALLOC case will read the whole ELF object into
> memory.  The motivation for the code was to support reading of an ELF
> object from device files ("/dev/ksyms" in {Free,Net}BSD) and from
> streams (e.g., from sockets and pipes).  Such file descriptors usually
> do not support mmap() or seek(), so ELF object needs to be read in
> sequentially.
>
> One optimization for the RAWFILE_MALLOC case could be to have libelf
> read in portions of the ELF object "on demand".  That said, further
> study would be needed to determine the tradeoffs involved---IIRC the
> default layout puts the section header table at the very end of an ELF
> object, so use cases that access sections could end up reading the
> entire ELF object into memory, even if such an optmization were
> implemented.
>

I think we can avoid any need to mess with this code. See below.

>> Secondly, I really do not like the idea
>> of __WIN32__ all over the place. I can support "HAVE_CONFIG_H" and
>> "HAVE_MMAP" etc. How should I handle this ?
>
> IMHO, an ELFTC_HAVE_MMAP configuration option would be the way to go.
>

I played around and decided touching the code was not a great solution. 
Instead I have written a mmap/munmap pair of functions for Windows based 
on the Python code for the mmap module and I can read elf files without 
a problem on Windows. I do not know if the update code for writes works.

This means I can read an elf file created on MacOS using a m68k 
cross-compiler on Windows. I am happy with this. I have uploaded 
rtl-host-20111123.tar.bz2 with the Windows changes.

> FYI, I have created ticket #366 to track this item:
>
>    http://sourceforge.net/apps/trac/elftoolchain/ticket/366.
>

Thanks. Should I attach patches once I have them ?

Chris

Re: [Elftoolchain-developers] Multiple archs and platforms.

[Joseph K. <jk...@us...>]

> I played around and decided touching the code was not a great solution.
> Instead I have written a mmap/munmap pair of functions for Windows based on
> the Python code for the mmap module and I can read elf files without a
> problem on Windows. I do not know if the update code for writes works.

Just as a data point, the Minix team seems to have worked around the
lack of an mmap() system call in Minix by implementing a malloc() based
replacement:

  http://git.minix3.org/?p=minix.git;a=blob;f=lib/libelf/compat/mmap.c

Some of our tools also use mmap(); I need check whether we can use a
simple elf_begin() instead, for these cases.

Regards,
Koshy