elftoolchain-developers — Discussion on elftoolchain development.

Re: [Elftoolchain-developers] make options

[John T. <jo...@ba...>]

In message <CAA...@ma...>
          stephen Turner <ste...@gm...> wrote:

> Hi, first thanks for maintaining this project, It looks very promising and
> i am excited to be testing it out in my system soon.
> 
> Is there a reason elftoolchain has opted to not use gnu make? Is a part of
> the build process when using gnu make broken or unable to be executed
> properly?

In case autotools is an option for you (which will in the end create
a GNU make compatible makefile), have a look at autotools support in
ticket #321 : http://sourceforge.net/p/elftoolchain/tickets/321/#cf36.

Most probably that supplied patch is a bit outdated after 2+ years.

It is actually part of:

svn diff svn://svn.riscos.info/gccsdk/trunk/tools/elftoolchain/{upstream-trunk,current-trunk}

'upstream-trunk' follows sf.net svn trunk but I'm no longer regulary
syncing it.

FYI, the remaining parts in that diff are other fixes which have been sent
upstream but I'm not sure whether they are already accepted or not.

As said in above ticket comment, my main motivation was cross-compiler
support so pmake wasn't an option for me.

John.
-- 
John Tytgat                                                             BASS
Jo...@ba...

[Elftoolchain-developers] Error in section '.dynamic' to segment 'DYNAMIC' assignment

[Frank T. <fra...@ts...>]

Dear all,

I am facing an issue while adding a new section to an existing ELF
binary for the ARM platform.

For some reason the already contained dynamic section of the ELF binary
is somehow assigned to the wrong segment after updating the ELF binary
(elf_update).

If the command "readelf -l" is used on the unmodified ELF binary (shared
object), it results in the following output:

---------------------
readelf -l mylib.so

Header:
  Typ            Offset   VirtAdr    PhysAdr    DateiGr SpeiGr  Flg Ausr.
  EXIDX          0x127b3c 0x00127b3c 0x00127b3c 0x01490 0x01490 R   0x4
  PHDR           0x000034 0x00000034 0x00000034 0x00140 0x00140 R E 0x4
  INTERP         0x126e2c 0x00126e2c 0x00126e2c 0x00014 0x00014 R   0x4
  LOAD           0x000000 0x00000000 0x00000000 0x12c23c 0x12c23c R E 0x8000
  LOAD           0x12c6cc 0x001346cc 0x001346cc 0x02748 0x04ecc RW  0x8000
  DYNAMIC        0x12df28 0x00135f28 0x00135f28 0x000d8 0x000d8 RW  0x4
  NOTE           0x000174 0x00000174 0x00000174 0x00020 0x00020 R   0x4
  TLS            0x12c6cc 0x001346cc 0x001346cc 0x00008 0x0004c R   0x4
  GNU_STACK      0x000000 0x00000000 0x00000000 0x00000 0x00000 RW  0x4
  GNU_RELRO      0x12c6cc 0x001346cc 0x001346cc 0x01934 0x01934 R   0x1

...
   00     .ARM.exidx
   01
   02     .interp
   03     .note.ABI-tag .gnu.hash .dynsym .dynstr .gnu.version
.gnu.version_d .gnu.version_r .rel.dyn .rel.plt .plt .text
__libc_freeres_fn __libc_thread_freeres_fn .rodata .interp .ARM.extab
.ARM.exidx .eh_frame .hash
   04     .tdata __libc_subfreeres __libc_atexit
__libc_thread_subfreeres .data.rel.ro .dynamic .got .data .bss
   05     .dynamic
   06     .note.ABI-tag
   07     .tdata .tbss
   08
   09     .tdata __libc_subfreeres __libc_atexit
__libc_thread_subfreeres .data.rel.ro .dynamic

---------------------

After adding the new section and updating the ELF binary, the command
raises the warning '.dynamic section is not contained within the dynamic
segment' and the output is as follows:

---------------------
readelf -l mylib.so

Header:
  Typ            Offset   VirtAdr    PhysAdr    DateiGr SpeiGr  Flg Ausr.
  EXIDX          0x127b3c 0x00127b3c 0x00127b3c 0x01490 0x01490 R   0x4
  PHDR           0x000034 0x00000034 0x00000034 0x00140 0x00140 R E 0x4
  INTERP         0x126e2c 0x00126e2c 0x00126e2c 0x00014 0x00014 R   0x4
  LOAD           0x000000 0x00000000 0x00000000 0x12c23c 0x12c23c R E 0x8000
  LOAD           0x12c6cc 0x001346cc 0x001346cc 0x02748 0x04ecc RW  0x8000
  DYNAMIC        0x12df28 0x00135f28 0x00135f28 0x000d8 0x000d8 RW
0x4readelf: Warning: the .dynamic section is not contained within the
dynamic segment

  NOTE           0x000174 0x00000174 0x00000174 0x00020 0x00020 R   0x4
  TLS            0x12c6cc 0x001346cc 0x001346cc 0x00008 0x0004c R   0x4
  GNU_STACK      0x000000 0x00000000 0x00000000 0x00000 0x00000 RW  0x4
  GNU_RELRO      0x12c6cc 0x001346cc 0x001346cc 0x01934 0x01934 R   0x1

...
   00     .ARM.exidx
   01
   02     .interp
   03     .note.ABI-tag .gnu.hash .dynsym .dynstr .gnu.version
.gnu.version_d .gnu.version_r .rel.dyn .rel.plt .plt .text
__libc_freeres_fn __libc_thread_freeres_fn .rodata .interp .ARM.extab
.ARM.exidx .eh_frame .hash
   04     .dynamic .got .data .bss .ARM.attributes .shstrtab
   05
   06     .note.ABI-tag
   07     .tbss
   08
   09     .dynamic

---------------------

In contrast to the original file the ELF section '.dynamic', which
should be contained within segment no 5 (DYNAMIC), is contained within
segment no 4 (LOAD). Some other sections even seem to be mixed up or
missing compared to the original ELF binary.

As a result all programs using this library are crashing.

Any hints?

Regards,

Frank

Re: [Elftoolchain-developers] make options

[Ed M. <em...@fr...>]

On 11 February 2015 at 11:29, stephen Turner <ste...@gm...> wrote:
>
> Thanks,
> I planned on using pmake for the initial testing but would ultimately like
> to use gnumake. I figured if everything goes well I could see if I can
> generate a working makefile.

It's probably fairly straightforward to translate it to GNU makefile
syntax, but there is one downside: trying to maintain parallel build
systems is awkward. This is a problem in the LLVM family of projects,
for instance, which have up to three build systems: GNU make, cmake,
and Xcode project files. Almost every change that adds or removes
files breaks one of them. The elftoolchain build is certainly much
more straightforward than those projects, but I'm still not sure the
benefit of having GNU Makefiles would be worth that trouble.

> I will let you know how it goes, im looking at using it with pcc and musl-libc.

Sounds good, I'm interested in seeing how you get on.

Also, I have a fork of elftoolchain on Github at
https://github.com/emaste/elftoolchain. I am using this to collect
fixes for issues discovered as we work to incorporate elftoolchain
into FreeBSD, and there are about 20 fixes and improvements there that
have not yet made it to the upstream repository.

Re: [Elftoolchain-developers] make options

[stephen T. <ste...@gm...>]

> Various components of elftoolchain originated in FreeBSD and have used
> BSD make from the beginning.  That is, it's built using a BSD make for
> the same reason that much software originating in the GNU/Linux
> community uses GNU make.
>

> If your goal is just to build elftoolchain installing a bmake or pmake
> package should be straightforward.
>
> Best,
> Ed

Thanks,
I planned on using pmake for the initial testing but would ultimately like
to use gnumake. I figured if everything goes well I could see if I can
generate a working makefile.

I will let you know how it goes, im looking at using it with pcc and
musl-libc.

Re: [Elftoolchain-developers] make options

[Ed M. <em...@fr...>]

On 11 February 2015 at 10:27, stephen Turner <ste...@gm...> wrote:
>
> Is there a reason elftoolchain has opted to not use gnu make? Is a part of
> the build process when using gnu make broken or unable to be executed
> properly?

Thanks for your interest in elftoolchain!

Various components of elftoolchain originated in FreeBSD and have used
BSD make from the beginning.  That is, it's built using a BSD make for
the same reason that much software originating in the GNU/Linux
community uses GNU make.

If your goal is just to build elftoolchain installing a bmake or pmake
package should be straightforward.

Best,
Ed

[Elftoolchain-developers] make options

[stephen T. <ste...@gm...>]

Hi, first thanks for maintaining this project, It looks very promising and
i am excited to be testing it out in my system soon.

Is there a reason elftoolchain has opted to not use gnu make? Is a part of
the build process when using gnu make broken or unable to be executed
properly?

thanks,
Stephen

Re: [Elftoolchain-developers] Assigning e_machine value for a new architecture

[Ed M. <em...@fr...>]

On 19 January 2015 at 18:30, Chan, SiuChi <siu...@am...> wrote:
> Hi,
>
>  We are using ELF for a new machine architecture and we are wondering what’s
> the process of getting an e_machine value created for this new architecture?

Hi Siu Chi,

The discussion for new e_machine types usually happens on
http://groups.google.com/group/generic-abi it seems.

I've BCC'd John Wolfe who is the current maintainer of the list and I
hope can take care of your request.

-Ed

[Elftoolchain-developers] Assigning e_machine value for a new architecture

[Chan, S. <siu...@am...>]

Hi,

We are using ELF for a new machine architecture and we are wondering what's the process of getting an e_machine value created for this new architecture?

Thanks,
Siu Chi

[Elftoolchain-developers] Change DT_FEATURE_1 to DT_FEATURE

[henning p. <hen...@t-...>]

 DT_FEATURE_1 is changed to DT_FEATURE in binutils for years .

Re: [Elftoolchain-developers] dwarf_child() returns a value not in the options listed in the documentation

[Daniel W. <dan...@gm...>]

Note also that it is missing from the man page, per the subject line.

On Sat, Dec 20, 2014 at 2:50 AM, Joseph Koshy
<jko...@gm...> wrote:
>> I cannot find the definition of DW_DLE_NO_ENTRY:
>> find /home/dsw/elftoolchain-0.6.1 -type f -print0 | xargs -0 grep -nH DW_DLE_NO\
>> _ENTRY | grep define
>
> % grep DLE_NO_ENTRY libdwarf/*.h
> libdwarf/libdwarf.h: DW_DLE_NO_ENTRY, /* No entry. */
>
> Regards,
> Joseph Koshy

[Elftoolchain-developers] dwarf_child() returns a value not in the options listed in the documentation

[Daniel W. <dan...@gm...>]

man dwarf_child says:

RETURN VALUES
     These functions return the following values:

     [DW_DLV_OK]        The call succeeded.

     [DW_DLV_ERROR]     The requested operation failed.  Additional informa-
                        tion about the error encountered will be recorded in
                        argument err, if it is not NULL.

     [DW_DLV_NO_ENTRY]  For functions dwarf_child() and dwarf_siblingof(), the
                        descriptor denoted by argument die did not have a
                        child or sibling.  For function dwarf_offdie(), there
was no debugging information entry at the offset spec-
ified by argument offset.


However another possible value seems to be DW_DLE_NO_ENTRY.

I cannot find the definition of DW_DLE_NO_ENTRY:
find /home/dsw/elftoolchain-0.6.1 -type f -print0 | xargs -0 grep -nH DW_DLE_NO\
_ENTRY | grep define
Compilation exited abnormally with code 1 at Tue Dec 16 15:42:14

A similar search fails to find it as a value in an enum.

I am actually getting this return value, so this is a real point.

Daniel

[Elftoolchain-developers] elftoolchain-0.6.1 install does not seem to install libelf.h ? odd: OLDER release date?

[Daniel W. <dan...@gm...>]

The install does not seem to install libelf.h on 64-bit Ubuntu?  I've
done this before and it worked.  I cannot find how to add a ticket
into your issue tracker.

== Installed a while ago

on my 32-bit Ubuntu 14.04 box at the top of RELEASE-NOTES says
$Id: RELEASE-NOTES 2599 2012-09-25 06:25:51Z jkoshy
and after install has a /usr/include/: elfdefinitions.h, gelf.h,
libelftc.h, elf.h, libelf.h

== Installed very recently, ***NOTE*** the earlier date (?!)

This unpacks to "elftoolchain-0.6.1".
on my 64-bit Ubuntu 14.04 box at the top of RELEASE-NOTES says
$Id: RELEASE-NOTES 2592 2012-09-16 15:20:13Z jkoshy $
and after installing twice in /usr/include does not have a libelf.h
$ ls /usr/include/*elf*
/usr/include/elf.h

== Install seems to not complain that it failed

$ sudo pmake install
[sudo] password for dsw:
install ===> common
install ===> libelf
. . .
install ===> documentation
install ===> documentation/libelf-by-example
WARNING: make "install" in "libelf-by-example" skipped: missing tools.

$ echo $?
0

Yet no libelf.h:

$ ls /usr/include/libelf.h
ls: cannot access /usr/include/libelf.h: No such file or directory

[Elftoolchain-developers] Stripping all sections from a .o file

[Ed M. <em...@fr...>]

While running the binutils test cases against elftoolchain we
discovered that stripping all sections from an empty .o file results
in a strip segfault. I submitted ticket 463 for this.

I think it should be fine to add the default section names ("",
".symtab", ".strtab", ".shstrtab") to the shstrtab when it is
initialized, rather than upon the addition of the first non-default
entry. This change can be seen in FreeBSD's phabricator at
https://reviews.freebsd.org/D1341

This seems to be the most straightforward approach to me, and I
believe it should be fine since elfcopy and strip only work properly
if the input file has a section header and .shstrtab anyway. Does that
sound reasonable?

[Elftoolchain-developers] Elftoolchain in FreeBSD

[Ed M. <em...@fr...>]

FYI, I have added a build-time knob in the FreeBSD base system to use
the elftoolchain version of:

* addr2line
* elfcopy (strip)
* nm
* size
* strings

We still use the binutils versions of these tools:

* as
* ld
* objcopy
* objdump
* readelf

There are a few fixes in the FreeBSD tree and also in my github
elftoolchain mirror:

* Fix cross-endian ELF note file / memory conversion
https://svnweb.freebsd.org/base?view=revision&revision=275430
https://github.com/emaste/elftoolchain/commit/d12b90d78885c020c1e9669afab2b59c026e6734

* Build fixes due to static / stdint.h
https://svnweb.freebsd.org/base?view=revision&revision=275369
https://github.com/emaste/elftoolchain/commit/1aebda60b7929fd96e7673d61a2bcac9635678a3

* Correct elftoolchain strip(1) memory size calculation
https://svnweb.freebsd.org/base?view=revision&revision=275810
https://github.com/emaste/elftoolchain/commit/dd319dd1078e408a34560ad4169cbdf7b364cb29
https://github.com/emaste/elftoolchain/commit/1f934307a05e353578fce27eea0b0ced33307828

* Do not strip all when stripping an explicit symbol
https://svnweb.freebsd.org/base?view=revision&revision=275862
https://github.com/emaste/elftoolchain/commit/1c9a0290013ec4ac80f02b817d3faf4c9ce0fdde

-Ed

[Elftoolchain-developers] elftoolchain throws an assertion failure

[Daniel W. <dan...@gm...>]

if gelf_getphdr() is called on a relocatable ELF instead of an executable ELF.

libelf_phdr.c:81: _libelf_getphdr: Assertion `fsz > 0' failed.

Re: [Elftoolchain-developers] request for assistance with libdwarf attribute forms which have no API nor documentation on how obtain meaningful value from them

[Joseph K. <jk...@us...>]

dw> For some DWARF attribute forms I have been unable to get a meaningful
dw> value no matter how long I read your documentation or grep through
dw> your source.  (Note that below I omit error checking for readability.)

There are two other places you could look:

1. The original SGI/GPL libdwarf project.   Elftoolchain's libdwarf is a
    re-implementation of that API.

    http://www.prevanders.net/dwarf.html

2. The DWARF specification itself, available at http://www.dwarfstd.org/.

I'm also CC'ing Kai Wang -- the primary author of Elftoolchain's libdwarf.

Ticket #453 has been created to track documentation holes in libdwarf's
manual pages.

  [1]:  https://sourceforge.net/p/elftoolchain/tickets/453/

Regards,
Joseph Koshy

[Elftoolchain-developers] request for assistance with libdwarf attribute forms which have no API nor documentation on how obtain meaningful value from them

[Daniel W. <dan...@gm...>]

On Thu, Jul 24, 2014 at 10:15 AM, Joseph Koshy
<jk...@us...> wrote:
> dw> Libelf and Libdwarf are APIs to nowhere: they are full of functions
> dw> that tell you how to get a foo from a bar, but do not tell you how to
> dw> get a bar in the first place or what you can do with a foo once you
> dw> have one.

jk> As for DWARF(3), we have an open ticket (#23) to write a tutorial
jk> for the DWARF(3) API.   Please feel free to submit a patch; notes
jk> on some possible contents for the tutorial are here:
jk>
jk>    http://sourceforge.net/p/elftoolchain/wiki/LibDwarfTutorial/

For some DWARF attribute forms I have been unable to get a meaningful
value no matter how long I read your documentation or grep through
your source.  (Note that below I omit error checking for readability.)

>From a Dwarf_Die const die, I can get an attribute list:
    Dwarf_Attribute *attrbuf;
    Dwarf_Signed attrcount;
    Dwarf_Error dwarf_error;
    dwarf_attrlist(die, &attrbuf, &attrcount, &dwarf_error);

Iterating over the list I can get Dwarf_Attributes:
    Dwarf_Attribute const at = attrbuf[i];

>From each attribute I can get the code:
      Dwarf_Half at_code;
      dwarf_whatattr(at, &at_code, &dwarf_error);

I can get the attribute's name:
      char const *at_name;
      dwarf_get_AT_name(at_code, &at_name);

I can get the attribute's form:
      Dwarf_Half at_form;
      dwarf_whatform(at, &at_form, &dwarf_error);

And for many forms, I can get some sort of sensible value.  (Sometimes
there are even multiple APIs to do so: dwaf_attrval_flag() /
dward_formflag().)

      switch (at_form) {

      case DW_FORM_flag:
      case DW_FORM_flag_present: {
        Dwarf_Bool flag;
        // fix: dwarf_attrval_flag() seems redundant with
        // dwarf_formflag(); there are other such examples
        dwarf_attrval_flag(die, at_code, &flag, &dwarf_error);
        out << (flag ? "T" : "F") << std::endl;
        break;
      }

      case DW_FORM_string:
      case DW_FORM_strp: {
        char const *string;
        dwarf_attrval_string(die, at_code, &string, &dwarf_error);
        out << "'" << string << "'" << std::endl;
        break;
      }

      case DW_FORM_data1:
      case DW_FORM_data2:
      case DW_FORM_data4:
      case DW_FORM_data8:
      case DW_FORM_sdata: {
        Dwarf_Signed signed_int;
        dwarf_attrval_signed(die, at_code, &signed_int, &dwarf_error);
        out << (signed_int >= 0 ? "+" : "") << signed_int << std::endl;
        break;
      }

      case DW_FORM_udata: {
        Dwarf_Unsigned unsigned_int;
        dwarf_attrval_unsigned(die, at_code, &unsigned_int, &dwarf_error)
        out << unsigned_int << std::endl;
        break;
      }

      case DW_FORM_addr: {
        // from libdwarf.h:
        // typedef uint64_t Dwarf_Addr;
        Dwarf_Addr dwarf_addr;
        dwarf_formaddr(at, &dwarf_addr, &dwarf_error);
        out << dwarf_addr << std::endl;
        break;
      }

This one is quite tricky, as due to the DWARF DIE tree encoding: when
you recurse you have to remember to not look for the siblings of the
root as they are non-sensical when referred to in this context (the
symptom is that traversing the DIE tree leads to an infinite loop;
fun!):

      case DW_FORM_ref1:
      case DW_FORM_ref2:
      case DW_FORM_ref4:
      case DW_FORM_ref8:
      case DW_FORM_ref_udata: {
        Dwarf_Off ref_offset;
        dwarf_formref(at, &ref_offset, &dwarf_error);
        Dwarf_Die ref_die;
        dwarf_offdie(dwarf_debug, ref_offset, &ref_die, &dwarf_error);
        // NOTE: don't look for siblings here: dwarf nodes linked in
        // this way seem to exhibit phantom siblings due to the
        // dwarf encoding
        traversal.traverse(ref_die, depth+1, false/*look_for_siblings*/);
        break;
      }

But for some forms, even if I have an API to give me something, I have
no idea how to process the result in a meaningful way:

      case DW_FORM_block:
      case DW_FORM_block1:
      case DW_FORM_block2:
      case DW_FORM_block4: {
        // from libdwarf.h:
        // typedef struct {
        //      Dwarf_Unsigned  bl_len;
        //      Dwarf_Ptr       bl_data;
        // } Dwarf_Block;
        // typedef uint64_t Dwarf_Unsigned;
        // typedef void *Dwarf_Ptr;
        Dwarf_Block *block;
        dwarf_formblock(at, &block, &dwarf_error);
        // FIX: then what?
        break;
      }

      case DW_FORM_exprloc: {
        // from libdwarf.h:
        // typedef uint64_t Dwarf_Unsigned;
        // typedef void *Dwarf_Ptr;
        Dwarf_Unsigned exprloc_len;
        Dwarf_Ptr exprloc;
        dwarf_formexprloc(at, &exprloc_len, &exprloc, &dwarf_error);
        // FIX: then what?
        break;
      }

Note that the technique of case DW_FORM_ref1 above does not work here:

      case DW_FORM_ref_addr:
      case DW_FORM_sec_offset: {
        // from libdwarf.h:
        // typedef off_t Dwarf_Off;
        Dwarf_Off ref_offset;
        dwarf_global_formref(at, &ref_offset, &dwarf_error);
        // FIX: then what?
        break;
      }

Daniel

Re: [Elftoolchain-developers] Error adding section to existing ELF binary

[Joseph K. <jk...@us...>]

f.t> If the custom section is of type SHT_LOUSER, the current implementation
f.t> of elf_update() fails.

Thanks for the bug report; fixed in [r3080].

Regards,
Joseph Koshy

Re: [Elftoolchain-developers] a detailed example of libelf and libdwarf as "the APIs to nowhere" and two suggestions for improvement in their documentation

[Joseph K. <jk...@us...>]

dw> Here I elaborate on my point in calling libelf and libdwarf the APIs
dw> to nowhere using a detailed example.
...
dw> The p_offset field looks promising, but what is it an offset from
dw> *exactly*.  Do I really just seek from the start of the file?

The semantics of the p_offset field is documented in a number
of places.

1. In the elf(5) manual page (in *BSD).

% man 5 elf
...
   p_offset  This member holds the offset from the beginning of the
                  file at which the first byte of the segment resides.

2. Chapter 4 of the "Libelf by Example" tutorial, both in the text and
   in diagrams (figure 4.1)

3. Comments in the headers.


dw> Hmm, this is interesting.  I think your comments are off by one: read
dw> each comment and pair it carefully with the name of its field.  From
dw> common/elfdefinitions.h:

Fixed in r3077, thanks for the bug report.

dw> Libdwarf presents many more elaborate examples of the above
dw> situation, some of which I have yet to solve.

True.  We need a guide to the libdwarf API in the short term.

For the long term it may be worth investing effort in designing a
cleaner API---perhaps after the 1.0 release.

Regards,
Joseph Koshy

Re: [Elftoolchain-developers] Error adding section to existing ELF binary

[Frank T. <fra...@ts...>]

Dear all,

I tried version 0.6.1 of the elftoolchain (directly compiled from the
sourceforge repo) and the bug seems to be fixed.

But now I am facing another issue.

If the custom section is of type SHT_LOUSER, the current implementation
of elf_update() fails.

I tracked down the issue to the function _libelf_xlate_shtype() which
does not have a proper case statement for the type SHT_LOUSER. As this
function call fails, the elf_update() function aborts with an error
indicating an invalid section type.

I quick-fixed the issue in the function _libelf_xlate_shtype() by
removing the default-branch of the case statement and adding an addition
if-clause to return the correct ELF type (ELF_T_BYTE).

--------------------------------

switch (sht) {
...
case SHT_SUNW_verneed:	/* == SHT_GNU_verneed */
	return (ELF_T_VNEED);
case SHT_SUNW_versym:	/* == SHT_GNU_versym */
	return (ELF_T_HALF);
}

if (sht >= SHT_LOUSER && sht <= SHT_HIUSER) {
	return (ELF_T_BYTE);
}

return -1;
	
--------------------------------

I guess that an additional statement may be needed for SHT_LOPROC,
SHT_HIOS or similar type which are user-defined.

Regards,

Frank Thater



Am 24.07.2014 12:25, schrieb Frank Thater:
> Dear all,
> 
> I am trying to add a new section with an application specific content to
> an existing ELF file. Unfortunately the resulting ELF file is erroneous
> as the size/offset of the new section and the section type are not
> correctly set (see code snippet below).
> 
> "Readelf" indicates the following information (section 37 is the newly
> added section):
> 
> ...
> [36] .strtab           STRTAB          00000000 013ccc 000356 00      0
>   0  1
> [37]                   NULL            00000000 000000 8048154 8048168
>     0 65539  0
> 
> The type of the section (NULL) and the size/offset (should be 8 bytes
> starting at offset 0x14022) are wrong. I have manually verified the
> correct values within the data structures using a debugger. So I would
> guess that something gets broken during the final call to elf_update().
> 
> OS is Kubuntu 12.04 with package 'libelf1" installed (Version indicated
> is 0.152-1ubuntu).
> 
> I've read some old postings regarding the libelf support for ELF_C_RDWR
> which recommended to copy the sections to a new file due to a broken
> implementation. Are these bugs still present?
> 
> Or am I just missing something? Any hints?
> 
> 
> Frank
> 
> 
> --------------------------------------------------------------------------------------
> 
> 	int fd;
> 	Elf *elf_ref = NULL;
> 	Elf32_Ehdr *ehdr;
> 	Elf_Scn *elf_scn, *sig_scn;
> 	Elf_Data *elf_data;
> 	Elf32_Shdr *signature_scn_header;
> 	unsigned char sample_data[] = {0xca, 0xfe, 0xba, 0xbe, 0xca, 0xfe,
> 0xba, 0xbe};
> 
> 	if (argc != 2) {
> 		return -1;
> 	}
> 
> 	fd = open(argv[1], O_RDWR);
> 
> 	/* Protect from using a lower ELF version and initialize ELF library */
> 	if (elf_version(EV_CURRENT) == EV_NONE) {
> 		printf("ELF library init failed: %s\n", elf_errmsg(-1));
> 		close(fd);
> 		return -1;
> 	}
> 
> 	elf_ref = elf_begin(fd, ELF_C_RDWR, NULL);
> 
> 	if (elf_kind(elf_ref) != ELF_K_ELF) {
> 		printf("Program is not an ELF binary\n");
> 		close(fd);
> 		return -2;
> 	}
> 
> 	sig_scn = elf_newscn(elf_ref);
> 
> 	elf_data = elf_newdata(sig_scn);
> 	elf_data->d_align = 1;
> 	elf_data->d_off = 0LL ;
> 	elf_data->d_buf = sample_data ;
> 	elf_data->d_type = ELF_T_BYTE ;
> 	elf_data->d_size = sizeof(sample_data);
> 	elf_data->d_version = EV_CURRENT ;
> 
> 	signature_scn_header = elf32_getshdr(sig_scn);
> 	signature_scn_header->sh_name = 0;
> 	signature_scn_header->sh_type = SHT_LOUSER + 10;
> 
> 	if (elf_update(elf_ref , ELF_C_NULL ) < 0) {
> 		printf("ELF update failed: %s", elf_errmsg (-1));
> 		return -3;
> 	}
> 
> 	(void) elf_flagshdr(sig_scn, ELF_C_SET, ELF_F_DIRTY);
> 	(void) elf_flagscn(sig_scn, ELF_C_SET, ELF_F_DIRTY);
> 	(void) elf_flagdata(elf_data, ELF_C_SET, ELF_F_DIRTY);
> 	(void) elf_flagehdr(elf_ref, ELF_C_SET, ELF_F_DIRTY);
> 	(void) elf_flagelf(elf_ref, ELF_C_SET, ELF_F_DIRTY);
> 
> 	if (elf_update(elf_ref , ELF_C_WRITE ) < 0) {
> 		printf("ELF update failed: %s", elf_errmsg (-1));
> 		return -4;
> 	}
> 
> 	elf_end(elf_ref);
> 
> 	close(fd);
> 

[Elftoolchain-developers] a detailed example of libelf and libdwarf as "the APIs to nowhere" and two suggestions for improvement in their documentation

[Daniel W. <dan...@gm...>]

Here I elaborate on my point in calling libelf and libdwarf the APIs
to nowhere using a detailed example.

On Thu, Jul 24, 2014 at 1:21 PM, Daniel Wilkerson
<dan...@gm...> wrote:
> On Thu, Jul 24, 2014 at 10:15 AM, Joseph Koshy
> <jk...@us...> wrote:
>> dw> Libelf and Libdwarf are APIs to nowhere: they are full of functions
>> dw> that tell you how to get a foo from a bar, but do not tell you how to
>> dw> get a bar in the first place or what you can do with a foo once you
>> dw> have one.
>>
> jk> If you are looking for an overview of the ELF(3) API, you could
> jk> look at the tutorial "libelf by Example".  It is the first search result
> jk> when one searches for "libelf tutorial".
>
dw> Yes, libelf by example is pretty good for just getting started.
dw> Mostly my complaint is about dwarf, but there are still some
dw> interesting open questions about elf.

So here is an example of what I am talking about.  Libelf by Example,
chapter 4 "Examining the Program Header Table", you explain what a
"segment" is and then you show an example.  This example shows how to
use libelf to load an elf, get the number of entries in the program
header table (which seem to be another name for what one might call
"segment headers") using elf_getphdrnum(), and how to get the program
header itself (a GElf_Phdr ) using gelf_getphdr().

Ok, now that I have a GElf_Phdr phdr, what can I do with it?  Typing
man gelf_getphdr gives me a page which refers me to pages such as man
gelf(3) which helpfully explains:

     GElf_Phdr    A class-independent representation of an ELF Program
Header Table entry.

Ok, but what is the API I can use to do things with a GElf_Phdr ?!  As
far as I can tell from your docs, nothing.

So I started grepping through your source.  I find that a GElf_Phdr is
basically the same as an Elf64_Phdr:

gelf.h:46:typedef Elf64_Phdr    GElf_Phdr;      /* Program header */

Now back in man elf(5) we find lots of information on a Elf64_Phdr:

           typedef struct {
               uint32_t   p_type;
               uint32_t   p_flags;
               Elf64_Off  p_offset;
               Elf64_Addr p_vaddr;
               Elf64_Addr p_paddr;
               uint64_t   p_filesz;
               uint64_t   p_memsz;
               uint64_t   p_align;
           } Elf64_Phdr;

This is followed by extensive explanation on the meaning of each field
(which you also detail in Libelf by Example).  Ok, that's helpful: I
can find out all kinds of information about the segment.  But suppose
I wanted to actually load said segment into memory (if, say, its
p_type says it is loadable).  How do I get the data of the segment?

The p_offset field looks promising, but what is it an offset from
*exactly*.  Do I really just seek from the start of the file?  Even if
I try this simpler interpretation and it works on small examples, that
doesn't mean I'm really using it correctly and it will keep working.

Elf sections turned out to be more complex that that: there is a handy
API called elf_getdata() which returns successive data blocks, all
parts of a given section.  What is the program header equivalent of
elf_getdata() ?  Again, I find nothing when I look; if it is there, I
would argue that it is not easy to find.  So again, I start grepping
through your source.

Hmm, this is interesting.  I think your comments are off by one: read
each comment and pair it carefully with the name of its field.  From
common/elfdefinitions.h:

/* 64 bit PHDR entry. */
typedef struct {
   Elf64_Word      p_type;      /* Type of segment. */
   Elf64_Word      p_flags;     /* File offset to segment. */
   Elf64_Off       p_offset;    /* Virtual address in memory. */
   Elf64_Addr      p_vaddr;     /* Physical address (if relevant). */
   Elf64_Addr      p_paddr;     /* Size of segment in file. */
   Elf64_Xword     p_filesz;    /* Size of segment in memory. */
   Elf64_Xword     p_memsz;     /* Segment flags. */
   Elf64_Xword     p_align;     /* Alignment constraints. */
} Elf64_Phdr;

Ah, finally I find it in size/size.c:

static void
handle_core_note(Elf *elf, GElf_Ehdr *elfhdr, GElf_Phdr *phdr,
    char **cmd_line)
{
        size_t max_size;
        uint64_t raw_size;
        GElf_Off offset;
        static pid_t pid;
        uintptr_t ver;
        Elf32_Nhdr *nhdr, nhdr_l;
        static int reg_pseudo = 0, reg2_pseudo = 0, regxfp_pseudo = 0;
        char buf[BUF_SIZE], *data, *name;
        . . .
        data = elf_rawfile(elf, &max_size);
        offset = phdr->p_offset;
        while (data != NULL && offset < phdr->p_offset + phdr->p_filesz) {
                nhdr = (Elf32_Nhdr *)(uintptr_t)((char*)data + offset);

So the base of the p_offset is the return value of elf_rawfile() !  Yea!

But do note how obscure the answer is.  There is no manpage for
handle_core_note(); it's just an obscure internal function.

Note that I picked libelf as an example because you seem to think
Libelf by Example and the man pages are sufficient.  Libdwarf presents
many more elaborate examples of the above situation, some of which I
have yet to solve.

In sum, my complaint is not that you do not show me how to get
started, my complaint is that you do not show me how to get finished!
This kind of situation happens repeatedly in the libelf and libdwarf
APIs, which is how come I called them the APIs to nowhere.

Two suggestions:

(1) Elf is a container for information, so don't just show me how to
read the meta-data from the container, show me how to go *through* the
container to get at the contained data!  Libelf by example is great as
far as it goes; please go just a bit further and show actually getting
the contained data out.  I think in most cases this is just a couple
more API calls.

(2) More generally, show me how to locally walk the graph of nouns and
verbs. That is, the man pages would be far more helpful if they each
had two more paragraphs:

    (a) the APIs to get the *inputs* to the function in question, and

    (b) the APIs to operate on the *outputs* of the function in question.

Then I can connect APIs together and walk the graph from a Foo to a
get_Foo(Foo const *foo, Bar *bar) to a Bar, to a load_Bar(Bar *const,
void * data, int *length), etc.

Daniel

Re: [Elftoolchain-developers] Questions about building

[Daniel W. <dan...@gm...>]

On Thu, Jul 24, 2014 at 10:15 AM, Joseph Koshy
<jk...@us...> wrote:
> dw> Libelf and Libdwarf are APIs to nowhere: they are full of functions
> dw> that tell you how to get a foo from a bar, but do not tell you how to
> dw> get a bar in the first place or what you can do with a foo once you
> dw> have one.
>
> If you are looking for an overview of the ELF(3) API, you could
> look at the tutorial "libelf by Example".  It is the first search result
> when one searches for "libelf tutorial".

Yes, libelf by example is pretty good for just getting started.
Mostly my complaint is about dwarf, but there are still some
interesting open questions about elf.

> As for DWARF(3), we have an open ticket (#23) to write a tutorial
> for the DWARF(3) API.   Please feel free to submit a patch; notes
> on some possible contents for the tutorial are here:
>
>    http://sourceforge.net/p/elftoolchain/wiki/LibDwarfTutorial/
>
>> Note that when building on Linux, the instructions are wrong: just
>> using pmake will not work, you must invoke it as follows:
>>
>>     $ NOGCCERROR=1 pmake
>
> A simple 'pmake' should work on the supported versions of Ubuntu.
> The supported versions are listed in file "INSTALL" (currently 10.04LTS
> and 12.04LTS).

Well it did not work on a very recent Ubuntu install.  Further, it
took me quite a long time to find NOGCCERROR=1 as it is undocumented.

Daniel

Re: [Elftoolchain-developers] Error adding section to existing ELF binary

[Frank T. <fra...@ts...>]

Hi Joseph,

So the provided code snippet seems to be okay?

I am not sure which code base is used by the Ubuntu maintainers. Would it be a possible solution to try the version from the SourceForge repo?

Regards,

Frank Thater



On 24. Juli 2014 19:26:30 MESZ, Joseph Koshy <jk...@us...> wrote:
>f.t> OS is Kubuntu 12.04 with package 'libelf1" installed (Version
>indicated
>f.t> is 0.152-1ubuntu).
>
>Would this be the elfutils version of libelf from RedHat?    That is a
>different
>code base entirely.
>
>Regards,
>Joseph Koshy

Re: [Elftoolchain-developers] Error adding section to existing ELF binary

[Joseph K. <jk...@us...>]

f.t> OS is Kubuntu 12.04 with package 'libelf1" installed (Version indicated
f.t> is 0.152-1ubuntu).

Would this be the elfutils version of libelf from RedHat?    That is a different
code base entirely.

Regards,
Joseph Koshy

Re: [Elftoolchain-developers] Questions about building

[Joseph K. <jk...@us...>]

dw> Libelf and Libdwarf are APIs to nowhere: they are full of functions
dw> that tell you how to get a foo from a bar, but do not tell you how to
dw> get a bar in the first place or what you can do with a foo once you
dw> have one.

If you are looking for an overview of the ELF(3) API, you could
look at the tutorial "libelf by Example".  It is the first search result
when one searches for "libelf tutorial".

As for DWARF(3), we have an open ticket (#23) to write a tutorial
for the DWARF(3) API.   Please feel free to submit a patch; notes
on some possible contents for the tutorial are here:

   http://sourceforge.net/p/elftoolchain/wiki/LibDwarfTutorial/

> Note that when building on Linux, the instructions are wrong: just
> using pmake will not work, you must invoke it as follows:
>
>     $ NOGCCERROR=1 pmake

A simple 'pmake' should work on the supported versions of Ubuntu.
The supported versions are listed in file "INSTALL" (currently 10.04LTS
and 12.04LTS).

I will download Ubuntu 14.04 LTS this weekend to check.

Regards,
Joseph Koshy