Re: [Rlib-devel] Postgre -> Postgres rename RPM SPEC fix, 64-bit breakage(?) in 1.3.6

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 454-5900

Bob Doan =EDrta:

>On Sat, 2005-12-17 at 16:46 +0100, Zoltan Boszormenyi wrote:
> =20
>
>>Hi,
>>
>>long time no see, erm, write? :-)
>>   =20
>>
>
>Hey!
>
> =20
>
>>I tried the latest CVS version of RLIB and I noticed two things.
>>
>>First, it didn't build by running rpmbuild because the rlib.spec.in was=
n't
>>updated when the postgre -> posgres renaming was done.
>>   =20
>>
>
>Applied thanks!
> =20
>

You're welcome. :-)

>>Second, it crashes on reports that worked earlier. I tried to track it =
down,
>>what I found out is that 1.3.5 works but 1.3.6 doesn't. I don't have a=20
>>32 bit
>>machine at hand, so I don't know whether it's just a 64-bit uncleanline=
ss or
>>something else. Still hunting...
>>   =20
>>
>
>Yea.. I changed things around quite a bit in order to support the "memo"
>fields.  Valgrind??  It's clean for me on 32 bit
>
>- Bob
> =20
>

I got this short log from valgrind-3.1.0:

****************************************************
=3D=3D25122=3D=3D Memcheck, a memory error detector.
=3D=3D25122=3D=3D Copyright (C) 2002-2005, and GNU GPL'd, by Julian Sewar=
d et al.
=3D=3D25122=3D=3D Using LibVEX rev 1471, a library for dynamic binary tra=
nslation.
=3D=3D25122=3D=3D Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LL=
P.
=3D=3D25122=3D=3D Using valgrind-3.1.0, a dynamic binary instrumentation =
framework.
=3D=3D25122=3D=3D Copyright (C) 2000-2005, and GNU GPL'd, by Julian Sewar=
d et al.
=3D=3D25122=3D=3D For more details, rerun with: -v
=3D=3D25122=3D=3D
1
2
3
4
5
rep
=3D=3D25122=3D=3D Invalid write of size 1
=3D=3D25122=3D=3D    at 0x4A1AED0: memset (mac_replace_strmem.c:464)
=3D=3D25122=3D=3D    by 0x4C4E608: rlib_charencoder_convert (charencoder.=
c:62)
=3D=3D25122=3D=3D    by 0x4C36707: rlib_resolve_field_value (resolution.c=
:81)
=3D=3D25122=3D=3D    by 0x4C3A6FB: rlib_operand_get_value (pcode.c:784)
=3D=3D25122=3D=3D    by 0x4C3A89A: execute_pcode (pcode.c:862)
=3D=3D25122=3D=3D    by 0x4C3D92C: rlib_pcode_operator_iif=20
(pcode_op_functions.c:1008)
=3D=3D25122=3D=3D    by 0x4C3A871: execute_pcode (pcode.c:866)
=3D=3D25122=3D=3D    by 0x4C3A981: rlib_execute_pcode (pcode.c:891)
=3D=3D25122=3D=3D    by 0x4C3FD32: rlib_process_expression_variables=20
(variables.c:123)
=3D=3D25122=3D=3D    by 0x4C376E0: rlib_resolve_report_fields (resolution=
.c:351)
=3D=3D25122=3D=3D    by 0x4C2E6CA: rlib_evaulate_single_report_variables=20
(reportgen.c:569)
=3D=3D25122=3D=3D    by 0x4C2ECA5: rlib_make_report (reportgen.c:614)
=3D=3D25122=3D=3D  Address 0x0 is not stack'd, malloc'd or (recently) fre=
e'd
** NUTS.. WE CRASHED
=3D=3D25122=3D=3D
=3D=3D25122=3D=3D ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 11=
 from 4)
=3D=3D25122=3D=3D malloc/free: in use at exit: 607,205 bytes in 839 block=
s.
=3D=3D25122=3D=3D malloc/free: 7,619 allocs, 6,780 frees, 13,266,761 byte=
s=20
allocated.
=3D=3D25122=3D=3D For counts of detected errors, rerun with: -v
=3D=3D25122=3D=3D searching for pointers to 839 not-freed blocks.
=3D=3D25122=3D=3D checked 1,497,592 bytes.
=3D=3D25122=3D=3D
=3D=3D25122=3D=3D LEAK SUMMARY:
=3D=3D25122=3D=3D    definitely lost: 5 bytes in 1 blocks.
=3D=3D25122=3D=3D      possibly lost: 0 bytes in 0 blocks.
=3D=3D25122=3D=3D    still reachable: 607,200 bytes in 838 blocks.
=3D=3D25122=3D=3D         suppressed: 0 bytes in 0 blocks.
=3D=3D25122=3D=3D Use --leak-check=3Dfull to see details of leaked memory=
.
****************************************************

And it seems to be caused by this difference between 1.3.5 and 1.3.6:

diff -urN rlib-1.3.5/libsrc/resolution.c rlib-1.3.6/libsrc/resolution.c

--- rlib-1.3.5/libsrc/resolution.c      2005-08-15 21:02:36.000000000 +02=
00
+++ rlib-1.3.6/libsrc/resolution.c      2005-11-01 16:53:40.000000000 +01=
00
@@ -59,9 +59,8 @@
 gchar * rlib_resolve_field_value(rlib *r, struct rlib_resultset_field=20
*rf) {
        struct input_filter *rs =3D INPUT(r, rf->resultset);
 #if !DISABLE_UTF8
-       gchar encoded_buf[MAXSTRLEN];
        gsize slen, elen;
-       gchar *ptr=3D encoded_buf;
+       gchar *ptr =3D NULL;
 #endif
        gchar *str;

@@ -79,8 +78,7 @@
        else {
                slen =3D strlen(str);
                elen =3D MAXSTRLEN;
-               rlib_charencoder_convert(rs->info.encoder, &str, &slen,=20
&ptr, &elen);
-               return g_strdup(encoded_buf);
+               return rlib_charencoder_convert(rs->info.encoder, &str,=20
&slen, &ptr, &elen);
        }
 #endif
 }

"ptr" is NULL and rlib_charencoder_convert does a memset() on that.
Deleting the memset() line wouldn't solve it, there are two other problem=
s
with this change in RLIB-1.3.6.

First, g_iconv() is only a wrapper over native iconv() and that doesn't=20
allocate
the output buffer for you, you have to pass an address of a pointer to an
already allocated buffer.

Second, rlib_charencoder_convert() returns a gint, the result of g_iconv(=
)
which is the number of a non-reversible conversions. Just like native=20
iconv().

Attached is patch which restores rlib_resolve_field_value() into a=20
working state
but not blindly to the original. I kept your intent of reducing stack usa=
ge.

After fixing it I found the same problem in=20
layout.c::rlib_layout_text_string(), too.
And every other call sites of rlib_encode_text() and=20
rlib_charencoder_convert()
should be audited, too.

And I got one double free() or free() without malloc() below.

****************************************************
=3D=3D12883=3D=3D Memcheck, a memory error detector.
=3D=3D12883=3D=3D Copyright (C) 2002-2005, and GNU GPL'd, by Julian Sewar=
d et al.
=3D=3D12883=3D=3D Using LibVEX rev 1471, a library for dynamic binary tra=
nslation.
=3D=3D12883=3D=3D Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LL=
P.
=3D=3D12883=3D=3D Using valgrind-3.1.0, a dynamic binary instrumentation =
framework.
=3D=3D12883=3D=3D Copyright (C) 2000-2005, and GNU GPL'd, by Julian Sewar=
d et al.
=3D=3D12883=3D=3D For more details, rerun with: -v
=3D=3D12883=3D=3D
1
2
3
4
5
rep
=3D=3D12883=3D=3D Invalid free() / delete / delete[]
=3D=3D12883=3D=3D    at 0x4A195DD: free (vg_replace_malloc.c:235)
=3D=3D12883=3D=3D    by 0x4C3A584: rlib_value_free (pcode.c:745)
=3D=3D12883=3D=3D    by 0x4C3D3B7: rlib_pcode_operator_val=20
(pcode_op_functions.c:885)
=3D=3D12883=3D=3D    by 0x4C3A941: execute_pcode (pcode.c:866)
=3D=3D12883=3D=3D    by 0x4C3D9FC: rlib_pcode_operator_iif=20
(pcode_op_functions.c:1010)
=3D=3D12883=3D=3D    by 0x4C3A941: execute_pcode (pcode.c:866)
=3D=3D12883=3D=3D    by 0x4C3AA51: rlib_execute_pcode (pcode.c:891)
=3D=3D12883=3D=3D    by 0x4C3FE02: rlib_process_expression_variables=20
(variables.c:123)
=3D=3D12883=3D=3D    by 0x4C377B0: rlib_resolve_report_fields (resolution=
.c:353)
=3D=3D12883=3D=3D    by 0x4C2E78A: rlib_evaulate_single_report_variables=20
(reportgen.c:568)
=3D=3D12883=3D=3D    by 0x4C2ED65: rlib_make_report (reportgen.c:613)
=3D=3D12883=3D=3D    by 0x4C359AB: rlib_execute (api.c:202)
=3D=3D12883=3D=3D  Address 0x8B3B2A1 is 1 bytes inside a block of size 1,=
024 alloc'd
=3D=3D12883=3D=3D    at 0x4A18A86: malloc (vg_replace_malloc.c:149)
=3D=3D12883=3D=3D    by 0x4C367B6: rlib_resolve_field_value (resolution.c=
:80)
=3D=3D12883=3D=3D    by 0x4C3A7CB: rlib_operand_get_value (pcode.c:784)
=3D=3D12883=3D=3D    by 0x4C3A96A: execute_pcode (pcode.c:862)
=3D=3D12883=3D=3D    by 0x4C3D9FC: rlib_pcode_operator_iif=20
(pcode_op_functions.c:1010)
=3D=3D12883=3D=3D    by 0x4C3A941: execute_pcode (pcode.c:866)
=3D=3D12883=3D=3D    by 0x4C3AA51: rlib_execute_pcode (pcode.c:891)
=3D=3D12883=3D=3D    by 0x4C3FE02: rlib_process_expression_variables=20
(variables.c:123)
=3D=3D12883=3D=3D    by 0x4C377B0: rlib_resolve_report_fields (resolution=
.c:353)
=3D=3D12883=3D=3D    by 0x4C2E78A: rlib_evaulate_single_report_variables=20
(reportgen.c:568)
=3D=3D12883=3D=3D    by 0x4C2ED65: rlib_make_report (reportgen.c:613)
=3D=3D12883=3D=3D    by 0x4C359AB: rlib_execute (api.c:202)
=3D=3D12883=3D=3D
=3D=3D12883=3D=3D Conditional jump or move depends on uninitialised value=
(s)
=3D=3D12883=3D=3D    at 0x4A1AEDD: memset (mac_replace_strmem.c:464)
=3D=3D12883=3D=3D    by 0x4C4E6D8: rlib_charencoder_convert (charencoder.=
c:62)
=3D=3D12883=3D=3D    by 0x4C2EF82: ??? (layout.c:122)
=3D=3D12883=3D=3D    by 0x4C2FB47: ??? (layout.c:384)
=3D=3D12883=3D=3D    by 0x4C317E2: ??? (layout.c:945)
=3D=3D12883=3D=3D    by 0x4C32A33: ??? (layout.c:1088)
=3D=3D12883=3D=3D    by 0x4C32BF7: rlib_layout_report_output (layout.c:11=
05)
=3D=3D12883=3D=3D    by 0x4C30058: rlib_layout_init_part_page (layout.c:1=
209)
=3D=3D12883=3D=3D    by 0x4C2EB57: rlib_make_report (reportgen.c:619)
=3D=3D12883=3D=3D    by 0x4C359AB: rlib_execute (api.c:202)
=3D=3D12883=3D=3D    by 0x4B1E214: report_szallito (in=20
/usr/lib64/libptgriport.so.1.0.0)=3D=3D12883=3D=3D    by 0x400A80: (withi=
n=20
/usr/bin/ptgriport)
=3D=3D12883=3D=3D
=3D=3D12883=3D=3D Invalid write of size 1
=3D=3D12883=3D=3D    at 0x4A1AED0: memset (mac_replace_strmem.c:464)
=3D=3D12883=3D=3D    by 0x4C4E6D8: rlib_charencoder_convert (charencoder.=
c:62)
=3D=3D12883=3D=3D    by 0x4C2EF82: ??? (layout.c:122)
=3D=3D12883=3D=3D    by 0x4C2FB47: ??? (layout.c:384)
=3D=3D12883=3D=3D    by 0x4C317E2: ??? (layout.c:945)
=3D=3D12883=3D=3D    by 0x4C32A33: ??? (layout.c:1088)
=3D=3D12883=3D=3D    by 0x4C32BF7: rlib_layout_report_output (layout.c:11=
05)
=3D=3D12883=3D=3D    by 0x4C30058: rlib_layout_init_part_page (layout.c:1=
209)
=3D=3D12883=3D=3D    by 0x4C2EB57: rlib_make_report (reportgen.c:619)
=3D=3D12883=3D=3D    by 0x4C359AB: rlib_execute (api.c:202)
=3D=3D12883=3D=3D    by 0x4B1E214: report_szallito (in=20
/usr/lib64/libptgriport.so.1.0.0)=3D=3D12883=3D=3D    by 0x400A80: (withi=
n=20
/usr/bin/ptgriport)
=3D=3D12883=3D=3D  Address 0x8B7CCA0 is 0 bytes after a block of size 1,0=
24 alloc'd
=3D=3D12883=3D=3D    at 0x4A18A86: malloc (vg_replace_malloc.c:149)
=3D=3D12883=3D=3D    by 0x3B52A2CAAE: g_malloc (in=20
/usr/lib64/libglib-2.0.so.0.400.8)
=3D=3D12883=3D=3D    by 0x4C2FB32: ??? (layout.c:383)
=3D=3D12883=3D=3D    by 0x4C317E2: ??? (layout.c:945)
=3D=3D12883=3D=3D    by 0x4C32A33: ??? (layout.c:1088)
=3D=3D12883=3D=3D    by 0x4C32BF7: rlib_layout_report_output (layout.c:11=
05)
=3D=3D12883=3D=3D    by 0x4C30058: rlib_layout_init_part_page (layout.c:1=
209)
=3D=3D12883=3D=3D    by 0x4C2EB57: rlib_make_report (reportgen.c:619)
=3D=3D12883=3D=3D    by 0x4C359AB: rlib_execute (api.c:202)
=3D=3D12883=3D=3D    by 0x4B1E214: report_szallito (in=20
/usr/lib64/libptgriport.so.1.0.0)=3D=3D12883=3D=3D    by 0x400A80: (withi=
n=20
/usr/bin/ptgriport)
=3D=3D12883=3D=3D    by 0x4E874BA: __libc_start_main (in /lib64/tls/libc-=
2.3.6.so)
=3D=3D12883=3D=3D
=3D=3D12883=3D=3D Invalid write of size 1
=3D=3D12883=3D=3D    at 0x4A1AEDD: memset (mac_replace_strmem.c:464)
=3D=3D12883=3D=3D    by 0x4C4E6D8: rlib_charencoder_convert (charencoder.=
c:62)
=3D=3D12883=3D=3D    by 0x4C2EF82: ??? (layout.c:122)
=3D=3D12883=3D=3D    by 0x4C2FB47: ??? (layout.c:384)
=3D=3D12883=3D=3D    by 0x4C317E2: ??? (layout.c:945)
=3D=3D12883=3D=3D    by 0x4C32A33: ??? (layout.c:1088)
=3D=3D12883=3D=3D    by 0x4C32BF7: rlib_layout_report_output (layout.c:11=
05)
=3D=3D12883=3D=3D    by 0x4C30058: rlib_layout_init_part_page (layout.c:1=
209)
=3D=3D12883=3D=3D    by 0x4C2EB57: rlib_make_report (reportgen.c:619)
=3D=3D12883=3D=3D    by 0x4C359AB: rlib_execute (api.c:202)
=3D=3D12883=3D=3D    by 0x4B1E214: report_szallito (in=20
/usr/lib64/libptgriport.so.1.0.0)=3D=3D12883=3D=3D    by 0x400A80: (withi=
n=20
/usr/bin/ptgriport)
=3D=3D12883=3D=3D  Address 0x8B7CCA1 is 1 bytes after a block of size 1,0=
24 alloc'd
=3D=3D12883=3D=3D    at 0x4A18A86: malloc (vg_replace_malloc.c:149)
=3D=3D12883=3D=3D    by 0x3B52A2CAAE: g_malloc (in=20
/usr/lib64/libglib-2.0.so.0.400.8)
=3D=3D12883=3D=3D    by 0x4C2FB32: ??? (layout.c:383)
=3D=3D12883=3D=3D    by 0x4C317E2: ??? (layout.c:945)
=3D=3D12883=3D=3D    by 0x4C32A33: ??? (layout.c:1088)
=3D=3D12883=3D=3D    by 0x4C32BF7: rlib_layout_report_output (layout.c:11=
05)
=3D=3D12883=3D=3D    by 0x4C30058: rlib_layout_init_part_page (layout.c:1=
209)
=3D=3D12883=3D=3D    by 0x4C2EB57: rlib_make_report (reportgen.c:619)
=3D=3D12883=3D=3D    by 0x4C359AB: rlib_execute (api.c:202)
=3D=3D12883=3D=3D    by 0x4B1E214: report_szallito (in=20
/usr/lib64/libptgriport.so.1.0.0)=3D=3D12883=3D=3D    by 0x400A80: (withi=
n=20
/usr/bin/ptgriport)
=3D=3D12883=3D=3D    by 0x4E874BA: __libc_start_main (in /lib64/tls/libc-=
2.3.6.so)
=3D=3D12883=3D=3D
=3D=3D12883=3D=3D More than 100000 total errors detected.  I'm not report=
ing any=20
more.
=3D=3D12883=3D=3D Final error counts will be inaccurate.  Go fix your pro=
gram!
=3D=3D12883=3D=3D Rerun with --error-limit=3Dno to disable this cutoff.  =
Note
=3D=3D12883=3D=3D that errors may occur in your program without prior war=
ning from
=3D=3D12883=3D=3D Valgrind, because errors are no longer being displayed.
=3D=3D12883=3D=3D

valgrind: m_mallocfree.c:170 (mk_plain_bszB): Assertion 'bszB !=3D 0' fai=
led.
=3D=3D12883=3D=3D    at 0x600110F8: report_and_quit (m_libcassert.c:122)
=3D=3D12883=3D=3D    by 0x60011364: vgPlain_assert_fail (m_libcassert.c:1=
85)
=3D=3D12883=3D=3D    by 0x6001A917: vgPlain_arena_malloc (m_mallocfree.c:=
170)
=3D=3D12883=3D=3D    by 0x600335C2: vgPlain_cli_malloc (replacemalloc_cor=
e.c:101)
=3D=3D12883=3D=3D    by 0x6000157C: vgMAC_malloc (mac_malloc_wrappers.c:1=
92)
=3D=3D12883=3D=3D    by 0x6003522E: do_client_request (scheduler.c:987)
=3D=3D12883=3D=3D    by 0x60034BBE: vgPlain_scheduler (scheduler.c:721)
=3D=3D12883=3D=3D    by 0x6004787E: thread_wrapper (syswrap-linux.c:86)
=3D=3D12883=3D=3D    by 0x60047975: run_a_thread_NORETURN (syswrap-linux.=
c:119)

sched status:
  running_tid=3D1

Thread 1: status =3D VgTs_Runnable
=3D=3D12883=3D=3D    at 0x4A18A86: malloc (vg_replace_malloc.c:149)
=3D=3D12883=3D=3D    by 0x4ECCA12: vasprintf (in /lib64/tls/libc-2.3.6.so=
)
=3D=3D12883=3D=3D    by 0x3B52A4A1EF: g_vasprintf (in=20
/usr/lib64/libglib-2.0.so.0.400.8)=3D=3D12883=3D=3D    by 0x3B52A3AF69:=20
g_strdup_vprintf (in /usr/lib64/libglib-2.0.so.0.400.8)
=3D=3D12883=3D=3D    by 0x4C3845F: rlogit (util.c:195)
=3D=3D12883=3D=3D    by 0x4C384A4: ??? (util.c:90)
=3D=3D12883=3D=3D    by 0x4E9940F: (within /lib64/tls/libc-2.3.6.so)
=3D=3D12883=3D=3D    by 0x4C4E6D8: rlib_charencoder_convert (charencoder.=
c:62)
=3D=3D12883=3D=3D    by 0x4C2EF82: ??? (layout.c:122)
=3D=3D12883=3D=3D    by 0x4C2FB47: ??? (layout.c:384)
=3D=3D12883=3D=3D    by 0x4C317E2: ??? (layout.c:945)
=3D=3D12883=3D=3D    by 0x4C32A33: ??? (layout.c:1088)
=3D=3D12883=3D=3D    by 0x4C32BF7: rlib_layout_report_output (layout.c:11=
05)
=3D=3D12883=3D=3D    by 0x4C30058: rlib_layout_init_part_page (layout.c:1=
209)
=3D=3D12883=3D=3D    by 0x4C2EB57: rlib_make_report (reportgen.c:619)
=3D=3D12883=3D=3D    by 0x4C359AB: rlib_execute (api.c:202)
=3D=3D12883=3D=3D    by 0x4B1E214: report_szallito (in=20
/usr/lib64/libptgriport.so.1.0.0)=3D=3D12883=3D=3D    by 0x400A80: (withi=
n=20
/usr/bin/ptgriport)
=3D=3D12883=3D=3D    by 0x4E874BA: __libc_start_main (in /lib64/tls/libc-=
2.3.6.so)


Note: see also the FAQ.txt in the source distribution.
It contains workarounds to several common problems.

If that doesn't help, please report this bug to: www.valgrind.org

In the bug report, send all the above text, the valgrind
version, and what Linux distro you are using.  Thanks.
****************************************************

Best regards,
Zolt=E1n B=F6sz=F6rm=E9nyi

