#223 doesn't work packed .so(library) file in android.
Hello,
i really thanks to solve packed executable issue.
and i tried to pack my .so file, but upx shows an error "UnknownExecutableFormatException",
so i fixed file offset 0x10, 0x3(.so) -> 0x2(executable).
my test files
- main (no packed executable / call .so's function)
- libmymath.so (upx packed)
main.c includes the follow codes.
include "mymath.h"
include <stdio.h></stdio.h>
include <dlfcn.h></dlfcn.h>
include <string.h></string.h>
include <stdlib.h></stdlib.h>
int main(int argc, char argv)
{
void handle = NULL;
int (result)(int, int);
printf("one");
getchar();
handle = dlopen("./libmymath.so", RTLD_NOW);
if ( !handle )
{
printf("fail to dlopen, %s\n", dlerror());
return 0;
}
printf("two");
getchar();
result = dlsym(handle, "sum");
if ( dlerror() != NULL )
{
printf("fail to dlsym, %s\n", dlerror());
return 0;
}
printf("three");
getchar();
printf("10 + 20 = %d\n", result(10, 20));
dlclose(handle);
return 0;
}
mymath.c and mymath.h include the follow codes.
- mymath.h
include <math.h></math.h>
extern int sum(int n1, int n2);
- mymath.c
include "mymath.h"
int sum(int n1, int n2)
{
return n1+n2;
}
and many dummy codes(ex: __asm emit 0x90) in order to make big file size. (i got an error "file is too small")
But it occurs an error.
root@android:/data/local/tmp # ./main
./main
one
[1] + Stopped (signal) ./main
root@android:/data/local/tmp #
[1] + Bus error ./main
it just printed "one" and i guess dlopen() doesn't work.
and i debug the executable with gdb.
root@android:/data/local/tmp # ./gdb ./main
(gdb) r
Starting program: /data/local/tmp/main
warning: Unable to find dynamic linker breakpoint function.
GDB will be unable to debug shared library initializers
and track explicitly loaded dynamic code.
one
Program received signal SIGBUS, Bus error.
0xb0001458 in ?? () from /system/bin/linker
and strace results
root@android:/data/local/tmp # ./strace -p 14121
Process 14121 attached - interrupt to quit
read(0, "\n", 1024) = 1
stat64("/vendor/lib/./libmymath.so", 0xbed577c8) = -1 ENOENT (No such file or di
rectory)
stat64("/system/lib/./libmymath.so", 0xbed577c8) = -1 ENOENT (No such file or di
rectory)
stat64("/data/local/gcc/lib/./libmymath.so", {st_mode=S_IFREG|0666, st_size=4004
, ...}) = 0
open("/data/local/gcc/lib/./libmymath.so", O_RDONLY|O_LARGEFILE) = 3
lseek(3, 0, SEEK_SET) = 0
read(3, "\177ELF\1\1\1\3\0\0\0\0\0\0\0\0\2\0(\0\1\0\0\0(\3\0\000"..., 4096) = 40
04
lseek(3, -8, SEEK_END) = 3996
read(3, "P\0\0o\200\0\0\0", 8) = 8
mmap2(NULL, 45056, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40
0a1000
mmap2(0x400a1000, 3260, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0) = 0x40
0a1000
mprotect(0x400a1000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC) = 0
mmap2(0x400ab000, 3652, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x2) = 0
x400ab000
--- SIGBUS (Bus error) @ 0 (0) ---
open("/dev/log/main", O_WRONLY|O_LARGEFILE) = 4
writev(4, [{"\7", 1}, {"libc\0", 5}, {"Fatal signal 7 (SIGBUS) at 0x400"..., 47}
], 3) = 53
gettid() = 14121
socket(PF_FILE, SOCK_STREAM, 0) = 5
connect(5, {sa_family=AF_FILE, path=@android:debuggerd}, 20) = 0
write(5, ")7\0\0", 4) = 4
read(5, 0xbed57428, 1) = ? ERESTARTSYS (To be restarted)
--- SIGCONT (Continued) @ 0 (0) ---
read(5, ")", 1) = 1
close(5) = 0
sigaction(SIGBUS, {SIG_DFL}, {0xb0005a1d, [], SA_RESTART|SA_SIGINFO}, 0) = 0
rt_sigreturn(0) = 1074445892
--- SIGBUS (Bus error) @ 0 (0) ---
Process 14121 detached
no working in my phones.
I'll send more logs and informations which you want in order to solve this problm.
and I will send you my test phone if you want.
thanks.
Please show the output from "readelf --headers libmymath.so" (before upx packing, before "fixing" the byte at offset 0x10), where then upx complains "UnknownExecutableFormatException". If applying gzip to libmymath.so gives something small enough, then please upload that to an attachment, too.
UPX does not complain when I try to pack /lib/libm-2.11.3.so, where readelf shows:
For UPX to compress a shared library, then there must be a DT_INIT entry in the Dynamic section. This is created if there is a function called "_init" or when some other function is named in a command-line argument to the linker "-init=my_function", which is written as "-Wl,-init=my_function" on a gcc command line.
See the comment in p_lx_elf.cpp, and the code which follows.
thanks for the reply!
i checked "(INIT)" in dynamic section but it doesn't exist.
so i added,
mymath.h :
define init __attribute ((section (".text.init")))
void __init smp_init(void)
mymath.c :
void __init smp_init(void)
{
printf("init!!");
}
gcc -Wall -c mymath.c -fPIC
gcc -shared -o libmymath.so mymath.o
i made libmymath.so and check readelf --dynamic.
but there is no (INIT) section in .so file.
(sorry, i'm a newbie of linux/elf)
so i checked some .so (located at /system/lib) but i couldn't find (INIT) sections.
(just i found (INIT_ARRAY or INIT_ARRAYSZ)
i'll check more information in order to make init section,
and i'll re-post the result of testing.
I really appreciate your guide and help.
thanks
$ gcc -o mymain mymain.c -ldl ## here without -lmymath
$ ./mymain
one
mymath my_init ## after dlopen and after dlsym
two
three
10 + 20 = 30
$
it works perfectly in my phone!
thanks for the guide :)
ultimately, I'll make Android dynamic library(called "JNI or NDK") and try to pack it.
i made .so library with NDK cross compiler and it has init section, but there is no entry point(0x0) in header.
now i'm checking the Makefile, in order to check whether to set wrong compile option or not.
and i'm wondering about that entry point must have non zero If there is init section. (for calling decompress stub)
Last edit: sonh 2013-10-24
I tested the following two cases :
1) phone library
2) Android App Library (aka JNI)
of course, I set -Wl,-init=my_init -fPIC
so I traced Android App Prcoess, I catched an error.
mprotect(0x4a2e9000, 8192, PROT_READ|PROT_EXEC) = 0
mmap2(NULL, 7480, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0x2a) = 0x4d7bd000
syscall_983042(0x4d7bea8c, 0x4d7bed38, 0, 0x1afffffb, 0xffffffff, 0, 0x31, 0xf0002, 0xb00147b8, 0x4a2ee000, 0x4000, 0x4a2e9000, 0, 0xbebe8264, 0x4d7bec91, 0x4a2eae8a, 0x30, 0x4d7bea8c, 0, 0, 0xc764, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) = 0
mmap2(0x4a2e9000, 9091, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x4a2e9000
syscall_983042(0x4a2e9c60, 0x4a2eb383, 0, 0x4a2e9c60, 0, 0, 0x4a2e9000, 0xf0002, 0xb00147b8, 0x4a2ee000, 0x4000, 0x4a2e9000, 0, 0xbebe826c, 0x4d7beb08, 0x4d7bead0, 0x60000010, 0x4a2e9c60, 0, 0, 0xc764, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) = 0
syscall_983042(0x4a2e9000, 0x4a2eb383, 0, 0x5, 0x4d7bec59, 0x318, 0x4a2e9000, 0xf0002, 0xb00147b8, 0x4a2ee000, 0x4000, 0x4a2e9000, 0, 0xbebe8294, 0x4d7becd3, 0x4d7bed32, 0x30, 0x4a2e9000, 0, 0, 0xc764, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) = 0
mprotect(0x4a2e9000, 9091, PROT_READ|PROT_EXEC) = 0
munmap(0x4d7bd000, 7480) = 0
futex(0x1610c58, 0x81 / FUTEX_??? /, 2147483647) = 0
writev(3, [{"\5", 1}, {"dalvikvm\0", 9}, {"JNI WARNING: input is not valid "..., 72}], 3) = 82
writev(3, [{"\5", 1}, {"dalvikvm\0", 9}, {" string: \'...This is"..., 46}], 3) = 56
writev(3, [{"\5", 1}, {"dalvikvm\0", 9}, {" in Lcom/sklee/jnite"..., 91}], 3) = 101
i tested it at Eclipse Android Virtual Machine, and it shows
10-25 17:33:42.107: W/dalvikvm(32587): JNI WARNING: input is not valid Modified UTF-8: illegal start byte 0xfe
10-25 17:33:42.107: W/dalvikvm(32587): string: '...This is JNI test �.'
10-25 17:33:42.107: W/dalvikvm(32587): in Lcom/sklee/jnitest/HelloJNI;.getString:()Ljava/lang/String; (NewStringUTF)
I attached three files for strace log.
Plase check out this file.
Thanks always
Last edit: sonh 2013-10-25
/data/data/com.sklee.jnitest/lib/libhellojni.so is my library file.
it just contains the following code
include jni.h
include HelloJNI.h
include log.h
void my_init()
{
log("my_init");
}
JNIEXPORT jstring JNICALL Java_com_sklee_jnitest_HelloJNI_getString(JNIEnv env, jobject object)
{
log("function");
return (env)->NewStringUTF( env, "...This is JNI test :)" );
}
Tag Type Name/Value
0x00000003 (PLTGOT) 0x3fc8
0x00000002 (PLTRELSZ) 88 (bytes)
0x00000017 (JMPREL) 0xc08
0x00000014 (PLTREL) REL
0x00000011 (REL) 0xbc0
0x00000012 (RELSZ) 72 (bytes)
0x00000013 (RELENT) 8 (bytes)
0x6ffffffa (RELCOUNT) 6
0x00000006 (SYMTAB) 0x114
0x0000000b (SYMENT) 16 (bytes)
0x00000005 (STRTAB) 0x504
0x0000000a (STRSZ) 1314 (bytes)
0x00000004 (HASH) 0xa28
0x00000001 (NEEDED) Shared library: [liblog.so]
0x00000001 (NEEDED) Shared library: [libstdc++.so]
0x00000001 (NEEDED) Shared library: [libm.so]
0x00000001 (NEEDED) Shared library: [libc.so]
0x00000001 (NEEDED) Shared library: [libdl.so]
0x0000000e (SONAME) Library soname: [libhellojni.so]
0x0000000c (INIT) 0x1b51
0x0000001a (FINI_ARRAY) 0x3e94
0x0000001c (FINI_ARRAYSZ) 8 (bytes)
0x00000019 (INIT_ARRAY) 0x3e9c
0x0000001b (INIT_ARRAYSZ) 4 (bytes)
0x00000010 (SYMBOLIC) 0x0
0x0000001e (FLAGS) SYMBOLIC BIND_NOW
0x6ffffffb (FLAGS_1) Flags: NOW
0x00000000 (NULL) 0x0
When a shared library contains both a DT_INIT and a JNI_OnLoad, then which one gets called first when loaded by Java? Put a unique string in a call to fprintf(stderr, ...) or log(...) inside each of DT_INIT and JNI_OnLoad, then see which string appears first.
10-27 20:24:05.284: I/log(1060): [f:my_init][l:7]my_init log
10-27 20:24:05.315: I/log(1060): [f:JNI_OnLoad][l:24]OnLoad log
actually my_init() runs earlier than JNI_OnLoad()
in case of packed file, there is no my_init() and JNI_OnLoad() logs
10-27 20:27:03.724: D/dalvikvm(1153): Trying to load lib /data/app-lib/com.sklee.jnitest-1/libhellojni.so 0x416ef890
10-27 20:27:03.734: A/libc(1153): Fatal signal 11 (SIGSEGV) at 0x00000007 (code=1), thread 1153 (m.sklee.jnitest)
just it shows those errors.
Please let me know what to do for testing.
Thank you for determining that DT_INIT runs before JNI_OnLoad for a dynamically-loaded, non-compressed shared library. For the next step, I don't understand what happens when "in the case of packed file, there is no my_init() and JNI_OnLoad logs an error."
Earlier we had success with dlopen of a packed shared library, and saw that the DT_INIT my_init() was called. So what is different about "in the case of packed file, there is no my_init() and JNI_OnLoad logs an error"? Does the file (before packing by UPX) contain both JNI_OnLoad and a DT_INIT of my_init? Or does the file (before packing by UPX) contain only JNI_OnLoad, and does not contain any DT_INIT? UPX insists on a DT_INIT in order to proceed with packing, and in previous cases we saw that a library that does contain a DT_INIT could be packed then dlopen()ed successfully.
It would help me understand what is going on if you would please Attach an actual file that fails for this current case.
If that is not possible, then perhaps some experiments with gdb can give insight. When the main program is waiting for a read(), then attach the main program with gdb, and activate the gdb commands for debugging shared libraries. These are "catch load" and/or "set stop-on-solib-events 1". For "catch load" see https://sourceware.org/gdb/onlinedocs/gdb/Set-Catchpoints.html#Set-Catchpoints and for "set stop-on-solib-events 1" see https://sourceware.org/gdb/onlinedocs/gdb/Files.html. Then "continue" from gdb. gdb will stop when the dlopen has gotten far enough, but before running very much farther. Try to see if gdb can find my_init and JNI_OnLoad:
(gdb) print &my_init
(gdb) print &JNI_OnLoad
Please check out the attach file.
it contains library source and .apk / .so files
also it contains full android source. (very simple source)
you can check this source with eclipse(with android plugin)
(gdb) print &my_init
print &my_init
No symbol "my_init" in current context.
(gdb) continue
continue
Continuing.
///////----------------- I press the button at Phone UI
Program received signal SIGSEGV, Segmentation fault.
0xb0005492 in ?? () from /system/bin/linker
(gdb) print &my_init
print &my_init
No symbol "my_init" in current context.
(gdb)
cat /proc/pid/maps shows the following map
b0001000-b0009000 r-xp 00001000 b3:18 498 /system/bin/linker
b0009000-b000a000 rw-p 00009000 b3:18 498 /system/bin/linker
b000a000-b0016000 rw-p 00000000 00:00 0
befcb000-befec000 rw-p 00000000 00:00 0 [stack]
ffff0000-ffff1000 r-xp 00000000 00:00 0 [vectors]
root@android:/ #
Thank you for the attach.zip containing an actual example which fails. Using libhellojni.so I have found problems which require work in UPX. Due to my $DAYJOB it probably will take several days before more progress on UPX.
How do I use the source that you have provided for HelloJNI? I unzipped FullSource.zip and got directory HelloJNI. Then from developer.android.com/sdk I downloaded adt-bundle-linux-x86_64-20130917.zip, unzipped it, went to the eclipse directory, and ran ./eclipse. Then I File > Import an Android Project, then Browse and select the HelloJNI directory. But Project > Run says "An internal error occurred during: "Launching New_configuration (1)".
Path for project must have only one segment." and now I'm lost.
Some progress: a web search for that error message reveals that I must Run > Run configurations... > Android tab > Project > Browse then choose JNItest as the project name. Connecting my device, then Run or Debug, the device then displays "printHelloJNI With JNI" and the title bar has "JNITest". So I guess it works? Where is the log?
Please tell me the steps to run strace, starting as soon as possible during the execution of HelloJNI. Also, how can I single-step the app from eclipse?
I didn't run Debug mode and single-step.
You can see log functions at /HelloJNI/jni/HelloJNI.c
also you can see those logs at LogCat.
When you press the button at UI, you can see the attached file's log. (errors, in case of upx library)
Button function includes "LoadLibrary libhellojni.so", also you can see the source at HelloJNI.java
Steps
then you can see APP's UI.
Last edit: sonh 2013-11-04
Thank you for the attach.tar which contains strace.
In my adt-bundle-linux-x86_64-20130917 the "Run Configurations" command (above, first list of instructions, step 4) is under the Run menu, not the Project menu.
Apparently the way to use a upx-compressed library in place of the original libhellojni.so is to "adb push" the new library to /data/app-lib/com.sklee.jnitest*/libhellojni.so.
I was able to run your ptrace and see that the upx-compressed library failed. Now instead of attaching ptrace, I want to attach gdb so that I can look at process memory, plant breakpoints, and single step. I want to do something like what is discussed at http://www.kandroid.org/online-pdk/guide/debugging_gdb.html "Debugging with gdb". However, I see no arm-eabi-gdb in my adt bundle; how should I get arm-eabi-gdb?
[When I start the emulated environment, then sometimes I see "Starting emulator for AVD 'emulator-1'
WARNING: Data partition already in use. Changes will not persist!
WARNING: SD Card image already in use: /home/jreiser/.android/avd/emulator-1.avd/sdcard.img
WARNING: Cache partition already in use. Changes will not persist!"
I don't understand this, and I hope that it does not matter. Starting the emulator takes about 2 minutes on a 2.5GHz x86_64; is this normal?]
In the emulated Android environment, I found 'gdbserver' in /system/bin. "gdbserver my_host:1234 --attach <pid>" does run and waits for a connection. On my host x86_64 machine, then "adb forward tcp:1234 tcp:1234" gives no complaint. So I think I'm ready to connect to the gdbserver. Because I don't have arm-eabi-gdb, then I try "(gdb) target remote my-host:1234" using a gdb for ARM ("GNU gdb (GDB) 7.4.1-debian") from my sheevaplug ARM machine. But gdb complains "my-host🔢 No route to host." despite "ping my-host" works. Using the correct numeric IP4 instead of "my-host" also fails from the ARM gdb. Using port 5309 instead of 1234 also fails. (my-host is my x86_64 box which is running the android emulator, and because of "adb forward ..." then my-host supposedly is forwarding the tcp port to-and-from the emulated android environment.)
So, I think I'm stuck until I can figure out how to run gdb on the HelloJNI app, attaching to the app just before it loads the libhellojni.so. Of course another way would be to have a gdb itself, much like the strace that you sent.
Last edit: John Reiser 2013-11-09
I created a gdb for arm: yumdownloader --source gdb; rpm --install gdb*.src.rpm; cd SPECS; rpmbuild -ba gdb.spec; cd BUILD/gdb*; rm */config.cache; ./configure --target=arm-none-elf --with-system-readline --prefix=...; cp gdb/gdb gdb-arm. Then "adb forward my-host:1234 my-host:1234" and "gdbserver my-host:1234 --attach <pid>" and "(gdb) target remote localhost:1234". Symbols work via "(gdb) add-symbol-file libc.so 0x400339b8" where the address is 0x40027000 [find ELFMAG there] plus the .text address from "objdump -h libc.so | grep text". ["adb pull" files from /system/lib etc so that gdb-arm on my-host can read them.]
I also found a crash dump report: "logcat" in an adb shell; look for lines with "I/DEBUG". It even includes a short traceback and some symbols. This better than nothing!
Breakpoints in Thumb mode turn into an infinite loop, but Control-C interrupts. Working backwards in Thumb mode still is slow and hard.
i attached the app's process with local gdb (no gdb-server)
step
this is everything about the attaching process that i know.
and i didn't try it with gdb-server. i have read the lastest your post, and did you success the attaching process and debugging?
the emulator takes long time, in my pc(Intel I7 / 3.4G@QuadCore) - 1 min
Last edit: sonh 2013-11-11
Thank you for the gdb-static.
Yes, I was able to use the emulator's own gdbserver with my gdb-arm running on x86_64. I successfully attached to the process, continued, and caught the error. It is SIGSEGV from "strlen(7)": seven bytes beyond a NULL string, and of course there is no page mapped for page 0. strlen was called from __vfprintf, from vsnprintf, from __android_log_print. (That's from the traceback given by "logcat" on the emulator, looking at lines "I/DEBUG". Perhaps that was from Run > Debug, instead of from Run > Run.)
The problem now is that I want to go farther up the call chain. Perhaps gdb-static will let me. My gdb-arm somehow cannot do tracebacks. What is worse is that it cannot single-step Thumb code, and hitting a breakpoint in Thumb code turns into an infinite loop, instead of breaking. Control-C breaks the loop and permits examining registers and memory, but the only way to proceed is to remove the breakpoint and continue. It would be really nice if gdb-static can give tracebacks and single-step Thumb mode. I will test this in a few days.
There was a bug in the unfilter (last) stage of decompression of a shared library on ARM. The practical effect was data-specific for the characters "+;K[k{" near the end of the compressed segment. Get the fix from the source tree https://www.pysol.org:4443/hg/upx.hg .
Thank you for the gdb-static. When I tried, I still could not single-step Thumb code. I fell back to compiled-in breakpoints (.short 0xde01) in the UPX stub code, and hand patching "(gdb) x/xw $pc" and "(gdb) set *(short *)addr = 0xde01" and "(gdb) set *(short *)addr=0x46c0".
Last edit: John Reiser 2013-11-16