Menu
▾
▴
[icu-support] Problem using ICU Data Build Tool - library size is not reduced
|
From: Richard A. <ric...@go...> - 2019-09-19 11:18:53
|
I use the ICU library to add Regular Expression support to the SQLite database in an embedded system. The database strings only contain characters from the ASCII charset. As such, we are shipping a lot of ICU data which is never used by the code. I am trying to follow the instructions at https://github.com/unicode-org/icu/blob/master/docs/userguide/icu_data/buildtool.md to reduce the amount of data included in the build. I have created a JSON file (see below) to turn off some features, but the build is exactly the same size (same number of bytes) when I supply the configuration file. I am using ICU 64.2 release. I have verified that the configuration file is being processed because the configure script says: ------------------- Spawning Python to generate data/rules.mk... Note: Applying filters from /home/ra/Documents/work/LV-CAP/e666-LVCAP/trunk/environment/buildroot/buildroot-2018.02.9/../icu-data-config.json. Spawning Python to generate test/testdata/rules.mk... ------------------- I also tried running the buildtool separate from configure. The command given in the debuging advice does not work, but from the source/ directory of the distribution I can run: PYTHONPATH="data" python3 -m buildtool --mode gnumake --src_dir "data" --filter_file ../../../../icu-data-config.json and a set of rules are generated. These rules are only 1260 lines long compared to the unfiltered length of 1890 lines, so this seems to work. However when the build runs, it generates the same size data as without the filter, so the filter is not actually working. The bit of the make output where it builds the data/ directory looks like this: ------------------- /usr/bin/make[2]: Making `all' in `data' Makefile:254: warning: overriding recipe for target 'out/tmp/icudata.lst' ../data/rules.mk:1007: warning: ignoring old recipe for target 'out/tmp/icudata.lst' /usr/bin/make -f pkgdataMakefile /bin/sh ../mkinstalldirs ./out/build/icudt64l ./out/build/icudt64l/coll ./out/build/icudt64l/rbnf ./out/tmp ./out/tmp/coll ./out/tmp/locales ./out/tmp/rbnf rm -rf icupkg.inc mkdir ./out mkdir ./out/build mkdir ./out/build/icudt64l mkdir ./out/build/icudt64l/coll mkdir ./out/build/icudt64l/rbnf mkdir ./out/tmp mkdir ./out/tmp/coll mkdir ./out/tmp/locales mkdir ./out/tmp/rbnf echo timestamp > ./out/tmp/dirs.timestamp /bin/sh ../mkinstalldirs ./out/tmp ./out/build/icudt64l Unpacking ./in/icudt64l.dat and generating out/tmp/icudata.lst (list of data files) LD_LIBRARY_PATH=/home/ra/Documents/work/LV-CAP/e666-LVCAP/trunk/environment/buildroot/2403_ARM_outputs/build/host-icu-64-2/source/lib:/home/ra/Documents/work/LV-CAP/e666-LVCAP/trunk/environment/buildroot/2403_ARM_outputs/build/host-icu-64-2/source/stubdata:/home/ra/Documents/work/LV-CAP/e666-LVCAP/trunk/environment/buildroot/2403_ARM_outputs/build/host-icu-64-2/source/tools/ctestfw:$LD_LIBRARY_PATH /home/ra/Documents/work/LV-CAP/e666-LVCAP/trunk/environment/buildroot/2403_ARM_outputs/build/host-icu-64-2/source/bin/icupkg -d ./out/build/icudt64l --list -x \* ./in/icudt64l.dat -o out/tmp/icudata.lst echo timestamp > build-local LD_LIBRARY_PATH=/home/ra/Documents/work/LV-CAP/e666-LVCAP/trunk/environment/buildroot/2403_ARM_outputs/build/host-icu-64-2/source/stubdata:/home/ra/Documents/work/LV-CAP/e666-LVCAP/trunk/environment/buildroot/2403_ARM_outputs/build/host-icu-64-2/source/tools/ctestfw:/home/ra/Documents/work/LV-CAP/e666-LVCAP/trunk/environment/buildroot/2403_ARM_outputs/build/host-icu-64-2/source/lib:$LD_LIBRARY_PATH /home/ra/Documents/work/LV-CAP/e666-LVCAP/trunk/environment/buildroot/2403_ARM_outputs/build/host-icu-64-2/source/bin/pkgdata -O ../data/icupkg.inc -q -c -s /home/ra/Documents/work/LV-CAP/e666-LVCAP/trunk/environment/buildroot/2403_ARM_outputs/build/icu-64-2/source/data/out/build/icudt64l -d ../lib -e icudt64 -T ./out/tmp -p icudt64l -m dll -r 64.2 -L icudata ./out/tmp/icudata.lst pkgdata: /home/ra/Documents/work/LV-CAP/e666-LVCAP/trunk/environment/buildroot/2403_ARM_outputs/host/bin/arm-buildroot-linux-musleabihf-gcc -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_REENTRANT -DU_HAVE_ELF_H=1 -DU_HAVE_STRTOD_L=0 -DU_ATTRIBUTE_DEPRECATED= -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -std=c11 -pedantic -Wshadow -Wpointer-arith -Wmissing-prototypes -Wwrite-strings -c -I../common -I../common -DPIC -fPIC -o ./out/tmp/icudt64l_dat.o ./out/tmp/icudt64l_dat.S pkgdata: /home/ra/Documents/work/LV-CAP/e666-LVCAP/trunk/environment/buildroot/2403_ARM_outputs/host/bin/arm-buildroot-linux-musleabihf-gcc -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -std=c11 -pedantic -Wshadow -Wpointer-arith -Wmissing-prototypes -Wwrite-strings -shared -Wl,-Bsymbolic -o ../lib/libicudata.so.64.2 ./out/tmp/icudt64l_dat.o -Wl,-soname -Wl,libicudata.so.64 -Wl,-Bsymbolic pkgdata: cd ../lib/ && rm -f libicudata.so.64 && ln -s libicudata.so.64.2 libicudata.so.64 pkgdata: cd ../lib/ && rm -f libicudata.so && ln -s libicudata.so.64.2 libicudata.so echo timestamp > packagedata ------------------- The build process seems to be ignoring the new build rules, and using the pre-built data file anyway. I tried removing the in/ directory to force the matter, but that just failed to build at all. What should I do next to make my filter file take effect? Thanks, Richard Contents of file icu-data-config.json: ------------------- { "localeFilter": { "filterType": "locale", "includeChildren": false, "whitelist": [ "en" ] }, "featureFilters": { "confusables": { "filterType": "exclude" }, "brkitr_rules": { "filterType": "exclude" }, "brkitr_dictionaries": { "filterType": "exclude" }, "brkitr_tree": { "filterType": "exclude" }, "curr_supplemental": { "filterType": "exclude" }, "curr_tree": { "filterType": "exclude" }, "lang_tree": { "filterType": "exclude" }, "region_tree": { "filterType": "exclude" }, "zone_tree": { "filterType": "exclude" }, "translit": { "filterType": "exclude" }, "unit_tree": { "filterType": "exclude" } } } ------------------- |
