elftoolchain-developers — Discussion on elftoolchain development.

[Elftoolchain-developers] The UQDS

[Joseph K. <jk...@us...>]

Hello All,

I recently came across a description of the way the Twisted (Python)
project develops their code and thought that it would be a good fit
for our project too:

 * http://divmod.org/trac/wiki/UltimateQualityDevelopmentSystem
 * http://shiny.thorne.id.au/2006/09/ultimate-quality-whats-that.html

In a nutshell, the core of the system is:

1) /trunk always builds and passes its tests.  This allows people to
  'svn cp trunk' and start work with the assurance that the code will
  be in a sane state.

2) Work happens on SVN branches; one SVN branch per
  (bug/feature/skunk-works) project.

  There would a ticket associated with every change.to trunk.  The
  ticket serves a "top level container" referencing everything needed
  to understand the change; e.g., links to wiki pages, lists of
  related tickets, lists of related source changes etc.

3) Work gets folded back into /trunk after review.  The main goal of
  the review is so that at least one more person has fully understood
  the proposed change.  Of course, bugs would also be caught by the
  review.

  Since code in branches is separate from /trunk, a developer need
  not wait for the reviewer and can start on another work item (i.e.,
  another branch) right away.

As a concrete example, let us say we want to create manual pages for
our tools and libraries.  We would:

* Create a ticket [#27 in this case].

* Create a branch with "svn cp trunk branches/27-manualpages"

 The convention being to name branches using the pattern
 TICKET#-SHORTDESCRIPTIVENAME, so that people can know at a glance
 where to go to find out more information.

* Do the work under branch "27-manualpages"; ticket #27 would track
 the state of this work.

* Once done and reviewed, the changes would be merged back to /trunk.
 The invariant maintained for /trunk is that it always is in a sane
 state.

* The work branch can then be "svn remove'd" to clean up the namespace
 under "branches/".

Thoughts?

Koshy

Re: [Elftoolchain-developers] Problem with elf_update after elf_flagshdr

[Joseph K. <jk...@us...>]

> I've uploaded an example to http://brabes.pochta.ru/elf_update_bug.tar.bz2 
> 
> Run it the program twice on the some.elf file and note that the sh_size changes as well as the sh_flags. The included log.txt shows my results from running it twice.
> 
> Hope that helps :)

Thank you very much.  Ticket #24 has been created to track the defect.

Koshy

Re: [Elftoolchain-developers] Problem with elf_update after elf_flagshdr

[Brabes <br...@po...>]

> > I am going through all sections with elf_nextscn() then getting the
> > header with elf32_getshdr(). For some sections I modify the sh_flags
> > member of the shdr, and then call elf_flagshdr on the scn.

> > When I call elf_update, though, the sh_size member of shdrs whose
> > flags I updated is set to zero!

> > Using gdb and a watchpoint, I found that the sh_size member is being
> > zeroed in _libelf_compute_section_extents(). I think the problem is
> > when _libelf_compute_section_extents sees a scn marked dirty, it
> > recomputes its size by adding up the Elf_Data d_size
> > values. However, because I haven't used elf_getdata on the scn, the
> > scn's s->s_data STAILQ is empty, and so the scn_size remains at
> > zero. This is backed up by that fact that if I do elf_getdata on the
> > scn before doing elf_flagshdr, the problem goes away. I also note
> > that the problem does not happen with the LGPL version of libelf.

> > I believe the solution would be to ensure that the Elf_Data
> > structures are loaded before calculating the section size in
> > _libelf_compute_section_extents().

> I'll look into this. It would *really* help if you could send over a
> short program that demonstrates the problem.

I've uploaded an example to http://brabes.pochta.ru/elf_update_bug.tar.bz2 

Run it the program twice on the some.elf file and note that the sh_size changes as well as the sh_flags. The included log.txt shows my results from running it twice.

Hope that helps :)

Re: [Elftoolchain-developers] Problem with elf_update after elf_flagshdr

[Joseph K. <jk...@us...>]

> I am going through all sections with elf_nextscn() then getting the
> header with elf32_getshdr(). For some sections I modify the sh_flags
> member of the shdr, and then call elf_flagshdr on the scn.

> When I call elf_update, though, the sh_size member of shdrs whose
> flags I updated is set to zero!

> Using gdb and a watchpoint, I found that the sh_size member is being
> zeroed in _libelf_compute_section_extents(). I think the problem is
> when _libelf_compute_section_extents sees a scn marked dirty, it
> recomputes its size by adding up the Elf_Data d_size
> values. However, because I haven't used elf_getdata on the scn, the
> scn's s->s_data STAILQ is empty, and so the scn_size remains at
> zero. This is backed up by that fact that if I do elf_getdata on the
> scn before doing elf_flagshdr, the problem goes away. I also note
> that the problem does not happen with the LGPL version of libelf.

> I believe the solution would be to ensure that the Elf_Data
> structures are loaded before calculating the section size in
> _libelf_compute_section_extents().

I'll look into this.  It would *really* help if you could send over a
short program that demonstrates the problem.

Thanks,
Koshy

[Elftoolchain-developers] Problem with elf_update after elf_flagshdr

[Brabes <br...@po...>]

Hello,

 

I am going through all sections with elf_nextscn() then getting the header with elf32_getshdr(). For some sections I modify the sh_flags member of the shdr, and then call elf_flagshdr on the scn.

 

When I call elf_update, though, the sh_size member of shdrs whose flags I updated is set to zero!

 

Using gdb and a watchpoint, I found that the sh_size member is being zeroed in _libelf_compute_section_extents(). I think the problem is when _libelf_compute_section_extents sees a scn marked dirty, it recomputes its size by adding up the Elf_Data d_size values. However, because I haven't used elf_getdata on the scn, the scn's s->s_data STAILQ is empty, and so the scn_size remains at zero. This is backed up by that fact that if I do elf_getdata on the scn before doing elf_flagshdr, the problem goes away. I also note that the problem does not happen with the LGPL version of libelf.

 

I believe the solution would be to ensure that the Elf_Data structures are loaded before calculating the section size in _libelf_compute_section_extents().

 

Thanks,

 

Re: [Elftoolchain-developers] deleting objects with libelf

[Kai W. <kai...@gm...>]

On Wed, Aug 12, 2009 at 05:29:04PM -0700, Chuck Tuffli wrote:
> Is it possible to delete objects (symbols, relocations, etc) using the
> current libelf API? I'm writing an application to remove a particular
> function call and all the relocation entries from an "executable"
> (really a kernel loadable module). Thanks!

It is doable with libelf API:

1. Use libelf API to read the kernel module's symbol table and relocation
   entries into internal symbol list and relocation list, respectively.
2. Search the symbol table for that particular function call and delete
   it from the internel symbol list. (Keep track of this symbol index)
3. Search the relocation list for the relocation entry refering to that
   deleted symbol and delete it.
4. Update other relocation entries since symbol numbering might have changed
   at this point.
5. Use libelf API write out the internal symbol list and relocation list
   and other unchanged section to a new object.

You may reuse some code from trunk/elfcopy.

-Kai

[Elftoolchain-developers] deleting objects with libelf

[Chuck T. <ct...@gm...>]

Is it possible to delete objects (symbols, relocations, etc) using the
current libelf API? I'm writing an application to remove a particular
function call and all the relocation entries from an "executable"
(really a kernel loadable module). Thanks!

---chuck

[Elftoolchain-developers] Website changes

[Joseph K. <jk...@us...>]

Hi,

Just a "heads-up" to let everyone know that we are migrating our
SF.Net website to Trac (a hosted application on SF).

The contents of the wiki and all tasks and tracker items that were in
SF.Net's Task Manager and Tracker have been moved to Trac.

Visiting http://elftoolchain.sourceforge.net/ should take you to our
Trac instance.

If you find any errors, feel free to correct them (or to let me know).

Regards,
Koshy

Re: [Elftoolchain-developers] addr2line

[Kai W. <kai...@gm...>]

On Sat, Jul 11, 2009 at 02:38:43PM +0900, Hyogeol Lee wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
> 
> Kai Wang :
> > On Sat, Jul 11, 2009 at 12:48:09AM +0200, Pawel Worach wrote:
> >> On Sat, Jul 11, 2009 at 00:38, Kai Wang<kai...@gm...> wrote:
> >>>> Hi again,
> >>>>
> >>>> Is anyone working on addr2line ? Is the necessary stuff for it in
> >>>> libelf already ?
> >>> Our addr2line will depend on libdwarf, while our libdwarf is still on
> >>> planning stage, so...
> >>>
> >> What is the status of nm/dwarf_line_number.c, it seems to have
> >> everything that addr2line would need :) Does it work?, nm -l does not
> >> show any line info.. /me investigates.
> >
> > I'm not sure about nm/dwarf_line_number.c, just did a little test and
> > nm -l doesn't work for me either.
> >
> > This needs more investigation. By the way nm/dwarf_line_number.c should
> > be integrated into libdwarf later.
> >
> > -Kai
> 
> Hi,
> 
> There was some bugs on nm/dwarf_line_number.c, and fixed now.
> 
> Would you test that ?

Yes it works now, thanks!

Re: [Elftoolchain-developers] addr2line

[Hyogeol L. <hyo...@gm...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Kai Wang :
> On Sat, Jul 11, 2009 at 12:48:09AM +0200, Pawel Worach wrote:
>> On Sat, Jul 11, 2009 at 00:38, Kai Wang<kai...@gm...> wrote:
>>>> Hi again,
>>>>
>>>> Is anyone working on addr2line ? Is the necessary stuff for it in
>>>> libelf already ?
>>> Our addr2line will depend on libdwarf, while our libdwarf is still on
>>> planning stage, so...
>>>
>> What is the status of nm/dwarf_line_number.c, it seems to have
>> everything that addr2line would need :) Does it work?, nm -l does not
>> show any line info.. /me investigates.
>
> I'm not sure about nm/dwarf_line_number.c, just did a little test and
> nm -l doesn't work for me either.
>
> This needs more investigation. By the way nm/dwarf_line_number.c should
> be integrated into libdwarf later.
>
> -Kai

Hi,

There was some bugs on nm/dwarf_line_number.c, and fixed now.

Would you test that ?

FYI, you need to compile test target with debug option, -g in GCC.


There is some limit on dwarf_line_number.c,

it does not print path if there is not 'include directory' information.

But GNU nm shows compiled path.


Regards,
- ----
Hyogeol Lee
hyo...@gm...
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.12 (FreeBSD)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEAREKAAYFAkpYJV8ACgkQ1D7/GiH6QSEdFQCdGPZgkMkoQjTHyyndI5JS5NKX
K1UAmgMI1CMF8jL1LjPfkX4tEp4g/Erz
=PQ3L
-----END PGP SIGNATURE-----

Re: [Elftoolchain-developers] addr2line

[Kai W. <kai...@gm...>]

On Sat, Jul 11, 2009 at 12:48:09AM +0200, Pawel Worach wrote:
> On Sat, Jul 11, 2009 at 00:38, Kai Wang<kai...@gm...> wrote:
> >> Hi again,
> >>
> >> Is anyone working on addr2line ? Is the necessary stuff for it in
> >> libelf already ?
> >
> > Our addr2line will depend on libdwarf, while our libdwarf is still on
> > planning stage, so...
> >
> 
> What is the status of nm/dwarf_line_number.c, it seems to have
> everything that addr2line would need :) Does it work?, nm -l does not
> show any line info.. /me investigates.

I'm not sure about nm/dwarf_line_number.c, just did a little test and
nm -l doesn't work for me either.

This needs more investigation. By the way nm/dwarf_line_number.c should
be integrated into libdwarf later.

-Kai

Re: [Elftoolchain-developers] addr2line

[Pawel W. <paw...@gm...>]

On Sat, Jul 11, 2009 at 00:38, Kai Wang<kai...@gm...> wrote:
>> Hi again,
>>
>> Is anyone working on addr2line ? Is the necessary stuff for it in
>> libelf already ?
>
> Our addr2line will depend on libdwarf, while our libdwarf is still on
> planning stage, so...
>

What is the status of nm/dwarf_line_number.c, it seems to have
everything that addr2line would need :) Does it work?, nm -l does not
show any line info.. /me investigates.

-- 
Pawel

Re: [Elftoolchain-developers] addr2line

[Kai W. <kai...@gm...>]

> Hi again,
> 
> Is anyone working on addr2line ? Is the necessary stuff for it in
> libelf already ?

Our addr2line will depend on libdwarf, while our libdwarf is still on
planning stage, so...

Re: [Elftoolchain-developers] [patch] readelf output fixes

[Kai W. <kai...@gm...>]

On Fri, Jul 10, 2009 at 09:18:38PM +0200, Pawel Worach wrote:
> Hi,
> 
> Here is patch to fix some case-typos and to reorder the output similar
> to gnu readelf.

Patch applied. Thanks!

> What is the future of readelf ? The wiki says merge with elfdump, does
> the include teaching elfdump the readelf output format ?

well.. at this point we will probably not merge them, however common
functions of elfdump/objdump/readelf might be factored out to libelftc.

-Kai

[Elftoolchain-developers] [patch] readelf output fixes

[Pawel W. <paw...@gm...>]

Hi,

Here is patch to fix some case-typos and to reorder the output similar
to gnu readelf.

What is the future of readelf ? The wiki says merge with elfdump, does
the include teaching elfdump the readelf output format ?

Index: readelf/readelf.c
===================================================================
--- readelf/readelf.c	(revision 419)
+++ readelf/readelf.c	(working copy)
@@ -528,7 +528,7 @@

 	switch (sbind) {
 	case STB_LOCAL: return "LOCAL";
-	case STB_GLOBAL: return "GLOBAl";
+	case STB_GLOBAL: return "GLOBAL";
 	case STB_WEAK: return "WEAK";
 	default:
 		if (sbind >= STB_LOOS && sbind <= STB_HIOS)
@@ -1347,7 +1347,7 @@
 			printf("  %-23s%-17s%-18s%s\n", "[Nr] Name", "Type",
 			    "Address", "Offset");
 			printf("       %-18s%-17s%-7s%-6s%-6s%s\n", "Size",
-			    "EntSize", "FLags", "Link", "Info", "Align");
+			    "EntSize", "Flags", "Link", "Info", "Align");
 		}
 	}
 	for (i = 0; (size_t)i < re->shnum; i++) {
@@ -1684,6 +1684,8 @@
 	}
 	if (d->d_size <= 0)
 		return;
+	printf("\nSymbol table (%s)", s->name);
+	printf(" contains %ju entries:\n", s->sz / s->entsize);
 	printf("%7s%9s%14s%5s%8s%6s%9s%5s\n", "Num:", "Value", "Size", "Type",
 	    "Bind", "Vis", "Ndx", "Name");
 	for (j = 0; (uint64_t)j < s->sz / s->entsize; j++) {
@@ -1782,7 +1784,7 @@
 		errx(EX_SOFTWARE, "calloc failed");
 	for (i = 0; (uint32_t)i < nbucket; i++)
 		c[bl[i]]++;
-	printf("Histogram for bucket list length (total of %u buckets):\n",
+	printf("\nHistogram for bucket list length (total of %u buckets):\n",
 	    nbucket);
 	printf(" Length\tNumber\t\t%% of total\tCoverage\n");
 	total = 0;
@@ -1853,6 +1855,26 @@
 		return;
 	}

+	vname[0] = "*local*";
+	vname[1] = "*global*";
+	if (vs_s) {
+		printf("\nVersion symbol section (%s):\n", vs_s->name);
+		if ((d = elf_getdata(vs_s->scn, NULL)) != NULL && d->d_size >
+		    0) {
+			j = d->d_size / sizeof(Elf32_Half);
+			vs = d->d_buf;
+			for (i = 0; i < j; i++) {
+				if ((i & 3) == 0) {
+					if (i > 0)
+						putchar('\n');
+					printf("  %03x", i);
+				}
+				printf(" %3d %-12s ", vs[i], vname[vs[i]]);
+			}
+			putchar('\n');
+		}
+	}
+
 	if (vd_s) {
 		printf("\nVersion definition section (%s):\n", vd_s->name);
 		if ((d = elf_getdata(vd_s->scn, NULL)) != NULL && d->d_size >
@@ -1914,26 +1936,6 @@
 			}
 		}
 	}
-
-	vname[0] = "*local*";
-	vname[1] = "*global*";
-	if (vs_s) {
-		printf("\nVersion symbol section (%s:)\n", vs_s->name);
-		if ((d = elf_getdata(vs_s->scn, NULL)) != NULL && d->d_size >
-		    0) {
-			j = d->d_size / sizeof(Elf32_Half);
-			vs = d->d_buf;
-			for (i = 0; i < j; i++) {
-				if ((i & 3) == 0) {
-					if (i > 0)
-						putchar('\n');
-					printf("  %03x", i);
-				}
-				printf(" %3d %-12s ", vs[i], vname[vs[i]]);
-			}
-			putchar('\n');
-		}
-	}
 #undef	SAVE_VERSION_NAME
 }

@@ -2083,10 +2085,10 @@

 	if (re->options & RE_H)
 		dump_ehdr(re);
+	if (re->options & RE_SS)
+		dump_shdr(re);
 	if (re->options & RE_L)
 		dump_phdr(re);
-	if (re->options & RE_SS)
-		dump_shdr(re);
 	if (re->options & RE_D)
 		dump_dynamic(re);
 	if (re->options & RE_R)


Regards
-- 
Pawel

Re: [Elftoolchain-developers] [patch] fix the build with clang

[Kai W. <kai...@gm...>]

On Tue, Jun 23, 2009 at 12:42:03PM +0200, Pawel Worach wrote:
> Hi,
> 
> Here is a patch to make the kit build cleanly with clang.

Patch committed. Thanks!

-Kai

[Elftoolchain-developers] [patch] fix the build with clang

[Pawel W. <paw...@gm...>]

Hi,

Here is a patch to make the kit build cleanly with clang.

Index: readelf/readelf.c
===================================================================
--- readelf/readelf.c	(revision 402)
+++ readelf/readelf.c	(working copy)
@@ -1296,7 +1296,7 @@
 	Elf_Data *d;
 	GElf_Sym sym;
 	const char *name;
-	int stab, j;
+	int elferr, stab, j;

 	s = &re->sl[i];
 	stab = s->link;
@@ -1354,7 +1354,7 @@
 	uint32_t	 nbucket, nchain;
 	uint32_t	*bucket, *chain;
 	uint32_t	*bl, *c, maxl, total;
-	int		 i, j;
+	int		 elferr, i, j;

 	/* TODO: Add support for .gnu.hash section. */

Index: size/size.c
===================================================================
--- size/size.c	(revision 402)
+++ size/size.c	(working copy)
@@ -834,7 +834,7 @@
 void
 usage()
 {
-	(void) fprintf(stderr, usagemsg);
+	(void) fprintf(stderr, "%s", usagemsg);
 	exit(EX_USAGE);
 }



Regards
-- 
Pawel

[Elftoolchain-developers] [Heads up] Source tree reorganization

[Joseph K. <jk...@us...>]

Hello Everyone,

I think I goofed on the design of the repository layout of our source
tree :).

First, I now see that the ELF(3) API isn't as "stable" as I thought
it was in the Unix world.  There are small incompatibilities between
the RH, GNU, Solaris implementations and ours that are being ironed
out.  See:

   http://elftoolchain.wiki.sourceforge.net/LibElfIncompatibilityRedHatElfUtils

These affect the tools using libelf.

Second, there are our own extensions to the API.  We also have new
libraries coming up such -lelftc.

Third, we need a place for common, OS dependent config information
that can be shared across tools.

Therefore, it makes to sense to keep all our tools together.

I am planning to move to move conventional source organization that
groups all elftoolchain code under one path: i.e., trunk/<each
projects>/ would simplify code.  In particular we could then share one
_config.h file across projects and it would, in general, become easier
to develop our libraries and tools together.

Please let me know if anyone has objections.  Apologies for the
repository churn.

Koshy

[Elftoolchain-developers] Fwd: Removing Debug Info from ELF

[Chengnian S. <che...@gm...>]

---------- Forwarded message ----------
From: Chengnian Sun <che...@gm...>
Date: Fri, Jun 12, 2009 at 2:00 PM
Subject: Re: [Elftoolchain-developers] Removing Debug Info from ELF
To: Joseph Koshy <jk...@us...>


In fact I am looking forward to a tool which can remove debug information
from ELF files.
Thanks for your information.

As strip(1), do you mean http://linux.die.net/man/1/strip ?

Thanks again.

Chengnian.


On Fri, Jun 12, 2009 at 1:24 PM, Joseph Koshy
<jk...@us...>wrote:

>
> > Dear All,
> > I would like to remove debug info from ELF.
> >
> > Could you give me some advice.
>
> It isn't clear from the question whether you are looking for
> information about the internals of ELF or about tools to remove debug
> information from ELF objects.
>
> If you are looking for tools, please see strip(1).
>
> Koshy
>



-- 
Best Regards.

Chengnian SUN.



-- 
Best Regards.

Chengnian SUN.

Re: [Elftoolchain-developers] Removing Debug Info from ELF

[Joseph K. <jk...@us...>]

> Dear All,
> I would like to remove debug info from ELF.
> 
> Could you give me some advice.

It isn't clear from the question whether you are looking for
information about the internals of ELF or about tools to remove debug
information from ELF objects.

If you are looking for tools, please see strip(1).

Koshy

Re: [Elftoolchain-developers] Removing Debug Info from ELF

[Kai W. <kai...@gm...>]

On Sun, Jun 07, 2009 at 01:07:43PM +0800, Chengnian Sun wrote:
> Dear All,
> I would like to remove debug info from ELF.
> 
> Could you give me some advice.

>From binutils/libbfd's point to of view: remove debug info from ELF objects
is to remove debugging sections and debugging symbols.

debugging sections are identified by their names:

	  ".debug*",
	  ".gnu.linkonce.wi.*",
          ".line*",
          ".stab*",

debugging symbols has symbol type STT_SECTION or STT_FILE.

-Kai

[Elftoolchain-developers] Removing Debug Info from ELF

[Chengnian S. <che...@gm...>]

Dear All,
I would like to remove debug info from ELF.

Could you give me some advice.


Thanks so much.


PS: there are two mailing lists, but I am not sure which one is what I
should post. Sorry for any convenience.



-- 
Best Regards.

Chengnian SUN.

[Elftoolchain-developers] New `elftc` library

[Joseph K. <jk...@us...>]

A small "HEADS-UP" to the list: I've started a "elftc" library under
$SVNROOT/projects/libelftc/.

This library would contain:
- code that is shared by multiple utilities in our toolset,
- convenience functions.

Koshy

Re: [Elftoolchain-developers] status of brandelf and elfdump

[Joseph K. <jk...@us...>]

> I have noticed that brandelf and elfdump aren't in repository yet,
> but they exist in a form of patch against FreeBSD-current.  Are
> there any plans to commit these utilities to the repository?

Yes.

Koshy

[Elftoolchain-developers] status of brandelf and elfdump

[Jan K. <lo...@ba...>]

Hi,

I have noticed that brandelf and elfdump aren't in repository
yet, but they exist in a form of patch against FreeBSD-current.
Are there any plans to commit these utilities to the
repository?

regards
Jan