rlib-devel — RLIB Development Mailing List

Re: [Rlib-devel] Re: Output parameters?

[Zoltan B. <zb...@du...>]

Bob Doan =C3=ADrta:
> It's actually=20
>=20
> rlib_add_parameter
>=20
> and reference them as "m.xxxxxx"
>=20
> The API is kinda confusing here.
>=20
> - Bob

Thanks, it worked.

Best regards,
Zolt=C3=A1n B=C3=B6sz=C3=B6rm=C3=A9nyi

[Rlib-devel] Report attribute naming

[William K. V. <wk...@us...>]

Hello,
	Just was observing that for most attributes the caps word
convention was followed, i.e.:
fontSize
topMargin
leftMargin
paperType

These two exceptions crept in however:
detail_columns
column_pad

Is it worth making them consistent?  I'll do it and update the docs
if that is desired.

Cheers,
William.

Re: [Rlib-devel] Re: Buglet in detail layout on first page

[William K. V. <wk...@us...>]

Hello,
Just an update. I added the check for the size of an image to the
get_output_size routine, added a logo to the page footer, and now
am seeing a double compensation for the size of the footer.

On Sun, 2005-06-12 at 11:51, William K. Volkman wrote:
<snip>
> 
> On Sun, 2005-06-12 at 09:17, Bob Doan wrote:
> > As of this moment things like this just won't work (And overwrites will
> > occur)
> > 
> > Perhaps the more correct thing to do execute hight changing properties
> > before we check to see if "it will fit"
> 
> Which is one reason why I put made the change to get_output_size,
> it is called as part of the "will fit" check (see second stack
> trace from GDB below).
> 
> > I could tackle this one if you don't feel comfortable doing it.  Let me
> > know.
> 
> I don't think that I'm familiar enough with the architecture to be in
> a position to judge the "best" way to handle this.  If you want to
> tackle it that is fine by me.  The change I made basically works in
> the static case albeit not in the most efficient manner and didn't
> add support for images (which is one change that I would make, logo
> goes in the page footer).

Take care,
William.

Re: [Rlib-devel] Re: Buglet in detail layout on first page

[William K. V. <wk...@us...>]

Hello Bob,

On Sun, 2005-06-12 at 09:17, Bob Doan wrote:
> Hym... This doesn't seem quite right because theoretically you could set
> the size of a HR to 0 to suppress it.

Yes, there appears to be two ways to suppress the horizontal line,
the xml attribute suppress and then just changing the height to 0.

> 
> As of this moment resolution.c would be the correct place to set it.
> 
Agreed, provided the intention was that these things are to be static.

> Another serious problem here is if people have expectations to change
> the height of things (font sizes, height, ect) dynamically.
> <HorizontalLine size="iif(r.detailcnt%2,5,100)" bgcolor="'white'"/>

This was the gist of my question, it appears that at some point a change
was made to enable these to be more dynamic however the change was not
carried all the way through.

> Same is true for images and lines.

I noted that lines have a suppress code which prompted one of my
questions, the fontSize attribute isn't dynamic at least.  Images
can't be suppressed however their height and width could change,
the current get_output_size routine doesn't even consider their
presence so would need that added.

> As of this moment things like this just won't work (And overwrites will
> occur)
> 
> Perhaps the more correct thing to do execute hight changing properties
> before we check to see if "it will fit"

Which is one reason why I put made the change to get_output_size,
it is called as part of the "will fit" check (see second stack
trace from GDB below).

> I could tackle this one if you don't feel comfortable doing it.  Let me
> know.

I don't think that I'm familiar enough with the architecture to be in
a position to judge the "best" way to handle this.  If you want to
tackle it that is fine by me.  The change I made basically works in
the static case albeit not in the most efficient manner and didn't
add support for images (which is one change that I would make, logo
goes in the page footer).

Take care,
William.

> On Sat, 2005-06-11 at 19:13 -0600, William K. Volkman wrote:
> > On Fri, 2005-06-10 at 19:58, William K. Volkman wrote:
> > > 	I just noticed an off-by-one issue with detail rows on
> > > the first page of a report.  It is overwriting the first part
> > > of the page footer before breaking.
> > 
> > Actually it is an off-by-break-header/footer-size problem.
> > One thing I've found is that the horizontal rule sizes aren't
> > being calculated until late in the game.
> > 
> > resolution.c:rlib_hr_resolve_pcode does this: rhl->size = 0;
> > and set's up size_code
> > 
> > reportgen.c:get_output_size uses this: RLIB_GET_LINE(rhl->size);
> > 
> > reportgen.c:rlib_make_report calls rlib_layout_init_part_page
> > which is going to call get_outputs_size (which calls get_output_size)
> > to determine the part->bottom_size however it does this before
> > any calls to rlib_layout_report_output (which calls
> > rlib_layout_report_outputs_across_pages, who then calls
> > rlib_layout_report_output_array, who will finally perform:
> > 
> > rlib_execute_as_float(r, rhl->size_code, &f)
> > rhl->size = f;
> > 
> > The next path that gets confused is (the break is on rhl->size being
> > 0):
> > 
> > #0  get_output_size (r=0x8049d60, roa=0x8079f80) at reportgen.c:220
> > #1  0x40042e88 in get_outputs_size (r=0x8049d60, e=0x807a9d8, page=0) at
> > reportgen.c:233
> > #2  0x400543d7 in rlib_handle_break_headers (r=0x8049d60,
> > part=0x807b880, report=0x80729a0) at breaks.c:133
> > #3  0x40043baa in rlib_layout_report (r=0x8049d60, part=0x807b880,
> > report=0x80729a0, left_margin_offset=0, top_margin_offset=0) at
> > reportgen.c:559
> > #4  0x400440d2 in rlib_layout_part_td (r=0x8049d60, part=0x807b880,
> > part_deviations=0x804a7a0, page_number=1, position_top=0,
> > rrp=0xbfffcef8) at reportgen.c:668
> > #5  0x40044331 in rlib_layout_part_tr (r=0x8049d60, part=0x807b880) at
> > reportgen.c:715
> > #6  0x400446ec in rlib_make_report (r=0x8049d60) at reportgen.c:800
> > 
> > Then:
> > #0  get_output_size (r=0x8049d60, roa=0x8078570) at reportgen.c:220
> > #1  0x40042f68 in will_outputs_fit (r=0x8049d60, part=0x807b880,
> > report=0x80729a0, e=0x8078400, page=1) at reportgen.c:260
> > #2  0x40043056 in rlib_end_page_if_line_wont_fit (r=0x8049d60,
> > part=0x807b880, report=0x80729a0, e=0x8078400) at reportgen.c:278
> > #3  0x4005473e in rlib_break_all_below_in_reverse_order (r=0x8049d60,
> > part=0x807b880, report=0x80729a0, e=0x807a960) at breaks.c:204
> > #4  0x40054870 in rlib_handle_break_footers (r=0x8049d60,
> > part=0x807b880, report=0x80729a0) at breaks.c:264
> > #5  0x40043c52 in rlib_layout_report (r=0x8049d60, part=0x807b880,
> > report=0x80729a0, left_margin_offset=0, top_margin_offset=0) at
> > reportgen.c:587
> > #6  0x400440d2 in rlib_layout_part_td (r=0x8049d60, part=0x807b880,
> > part_deviations=0x804a7a0, page_number=1, position_top=0,
> > rrp=0xbfffcef8) at reportgen.c:668
> > #7  0x40044331 in rlib_layout_part_tr (r=0x8049d60, part=0x807b880) at
> > reportgen.c:715
> > #8  0x400446ec in rlib_make_report (r=0x8049d60) at reportgen.c:800
> > 
> > One solution would be to hack the get_output_size code to execute the
> > pcode for obtaining the size.  This however raises questions:
> > 
> > 1.  Should we always evaluate the pcode or only if rhl->size is 0?
> > (if not then perhaps resolution should setup rhl->size)
> > 2.  Should the check for the size of the text line take into account
> > the is_not_suppressed check?
> > 
> > Experimental patch attached, it fixes my reports, YMMV.

[Rlib-devel] Re: Output parameters?

[Bob D. <bd...@si...>]

It's actually 

rlib_add_parameter

and reference them as "m.xxxxxx"

The API is kinda confusing here.

- Bob

On Sun, 2005-06-12 at 17:41 +0200, Zoltan Boszormenyi wrote:
> Hi,
> 
> how can I pass some strings to my reports?
> I saw rlib_set_output_parameter() in the documentation,
> both the C and PHP API says: SEE OUTPUT PARAMETERS.
> But this section isn't in the documentation.
> 
> So, if I would like to display a date interval in a subtitle-like
> line, I use
> 
> rlib_set_output_parameter(r, "start_date", mystartdate);
> rlib_set_output_parameter(r, "end_date", mystartdate);
> 
> but how can I use them in the report XML?
> r.start_date? Not an internal variable.
> v.start_date? Not a variable defined in the XML.
> 
> Best regards,
> Zoltán Böszörményi
-- 
Bob Doan <bd...@si...>

[Rlib-devel] Output parameters?

[Zoltan B. <zb...@du...>]

Hi,

how can I pass some strings to my reports?
I saw rlib_set_output_parameter() in the documentation,
both the C and PHP API says: SEE OUTPUT PARAMETERS.
But this section isn't in the documentation.

So, if I would like to display a date interval in a subtitle-like
line, I use

rlib_set_output_parameter(r, "start_date", mystartdate);
rlib_set_output_parameter(r, "end_date", mystartdate);

but how can I use them in the report XML?
r.start_date? Not an internal variable.
v.start_date? Not a variable defined in the XML.

Best regards,
Zolt=E1n B=F6sz=F6rm=E9nyi

Re: [Rlib-devel] Re: Buglet in detail layout on first page

[Bob D. <bd...@si...>]

Hym... This doesn't seem quite right because theoretically you could set
the size of a HR to 0 to suppress it.

As of this moment resolution.c would be the correct place to set it.

But:

Another serious problem here is if people have expectations to change
the height of things (font sizes, height, ect) dynamically.

like 

<HorizontalLine size="iif(r.detailcnt%2,5,100)" bgcolor="'white'"/>

Same is true for images and lines.

As of this moment things like this just won't work (And overwrites will
occur)

Perhaps the more correct thing to do execute hight changing properties
before we check to see if "it will fit"

I could tackle this one if you don't feel comfortable doing it.  Let me
know.

- bob

On Sat, 2005-06-11 at 19:13 -0600, William K. Volkman wrote:
> On Fri, 2005-06-10 at 19:58, William K. Volkman wrote:
> > 	I just noticed an off-by-one issue with detail rows on
> > the first page of a report.  It is overwriting the first part
> > of the page footer before breaking.
> 
> Actually it is an off-by-break-header/footer-size problem.
> One thing I've found is that the horizontal rule sizes aren't
> being calculated until late in the game.
> 
> resolution.c:rlib_hr_resolve_pcode does this: rhl->size = 0;
> and set's up size_code
> 
> reportgen.c:get_output_size uses this: RLIB_GET_LINE(rhl->size);
> 
> reportgen.c:rlib_make_report calls rlib_layout_init_part_page
> which is going to call get_outputs_size (which calls get_output_size)
> to determine the part->bottom_size however it does this before
> any calls to rlib_layout_report_output (which calls
> rlib_layout_report_outputs_across_pages, who then calls
> rlib_layout_report_output_array, who will finally perform:
> 
> rlib_execute_as_float(r, rhl->size_code, &f)
> rhl->size = f;
> 
> The next path that gets confused is (the break is on rhl->size being
> 0):
> 
> #0  get_output_size (r=0x8049d60, roa=0x8079f80) at reportgen.c:220
> #1  0x40042e88 in get_outputs_size (r=0x8049d60, e=0x807a9d8, page=0) at
> reportgen.c:233
> #2  0x400543d7 in rlib_handle_break_headers (r=0x8049d60,
> part=0x807b880, report=0x80729a0) at breaks.c:133
> #3  0x40043baa in rlib_layout_report (r=0x8049d60, part=0x807b880,
> report=0x80729a0, left_margin_offset=0, top_margin_offset=0) at
> reportgen.c:559
> #4  0x400440d2 in rlib_layout_part_td (r=0x8049d60, part=0x807b880,
> part_deviations=0x804a7a0, page_number=1, position_top=0,
> rrp=0xbfffcef8) at reportgen.c:668
> #5  0x40044331 in rlib_layout_part_tr (r=0x8049d60, part=0x807b880) at
> reportgen.c:715
> #6  0x400446ec in rlib_make_report (r=0x8049d60) at reportgen.c:800
> 
> Then:
> #0  get_output_size (r=0x8049d60, roa=0x8078570) at reportgen.c:220
> #1  0x40042f68 in will_outputs_fit (r=0x8049d60, part=0x807b880,
> report=0x80729a0, e=0x8078400, page=1) at reportgen.c:260
> #2  0x40043056 in rlib_end_page_if_line_wont_fit (r=0x8049d60,
> part=0x807b880, report=0x80729a0, e=0x8078400) at reportgen.c:278
> #3  0x4005473e in rlib_break_all_below_in_reverse_order (r=0x8049d60,
> part=0x807b880, report=0x80729a0, e=0x807a960) at breaks.c:204
> #4  0x40054870 in rlib_handle_break_footers (r=0x8049d60,
> part=0x807b880, report=0x80729a0) at breaks.c:264
> #5  0x40043c52 in rlib_layout_report (r=0x8049d60, part=0x807b880,
> report=0x80729a0, left_margin_offset=0, top_margin_offset=0) at
> reportgen.c:587
> #6  0x400440d2 in rlib_layout_part_td (r=0x8049d60, part=0x807b880,
> part_deviations=0x804a7a0, page_number=1, position_top=0,
> rrp=0xbfffcef8) at reportgen.c:668
> #7  0x40044331 in rlib_layout_part_tr (r=0x8049d60, part=0x807b880) at
> reportgen.c:715
> #8  0x400446ec in rlib_make_report (r=0x8049d60) at reportgen.c:800
> 
> One solution would be to hack the get_output_size code to execute the
> pcode for obtaining the size.  This however raises questions:
> 
> 1.  Should we always evaluate the pcode or only if rhl->size is 0?
> (if not then perhaps resolution should setup rhl->size)
> 2.  Should the check for the size of the text line take into account
> the is_not_suppressed check?
> 
> Experimental patch attached, it fixes my reports, YMMV.
> 
> Cheers,
> William.
> 
> 
> 
> 
> 
-- 
Bob Doan <bd...@si...>

[Rlib-devel] Re: Buglet in detail layout on first page

[William K. V. <wk...@us...>]

On Fri, 2005-06-10 at 19:58, William K. Volkman wrote:
> 	I just noticed an off-by-one issue with detail rows on
> the first page of a report.  It is overwriting the first part
> of the page footer before breaking.

Actually it is an off-by-break-header/footer-size problem.
One thing I've found is that the horizontal rule sizes aren't
being calculated until late in the game.

resolution.c:rlib_hr_resolve_pcode does this: rhl->size = 0;
and set's up size_code

reportgen.c:get_output_size uses this: RLIB_GET_LINE(rhl->size);

reportgen.c:rlib_make_report calls rlib_layout_init_part_page
which is going to call get_outputs_size (which calls get_output_size)
to determine the part->bottom_size however it does this before
any calls to rlib_layout_report_output (which calls
rlib_layout_report_outputs_across_pages, who then calls
rlib_layout_report_output_array, who will finally perform:

rlib_execute_as_float(r, rhl->size_code, &f)
rhl->size = f;

The next path that gets confused is (the break is on rhl->size being
0):

#0  get_output_size (r=0x8049d60, roa=0x8079f80) at reportgen.c:220
#1  0x40042e88 in get_outputs_size (r=0x8049d60, e=0x807a9d8, page=0) at
reportgen.c:233
#2  0x400543d7 in rlib_handle_break_headers (r=0x8049d60,
part=0x807b880, report=0x80729a0) at breaks.c:133
#3  0x40043baa in rlib_layout_report (r=0x8049d60, part=0x807b880,
report=0x80729a0, left_margin_offset=0, top_margin_offset=0) at
reportgen.c:559
#4  0x400440d2 in rlib_layout_part_td (r=0x8049d60, part=0x807b880,
part_deviations=0x804a7a0, page_number=1, position_top=0,
rrp=0xbfffcef8) at reportgen.c:668
#5  0x40044331 in rlib_layout_part_tr (r=0x8049d60, part=0x807b880) at
reportgen.c:715
#6  0x400446ec in rlib_make_report (r=0x8049d60) at reportgen.c:800

Then:
#0  get_output_size (r=0x8049d60, roa=0x8078570) at reportgen.c:220
#1  0x40042f68 in will_outputs_fit (r=0x8049d60, part=0x807b880,
report=0x80729a0, e=0x8078400, page=1) at reportgen.c:260
#2  0x40043056 in rlib_end_page_if_line_wont_fit (r=0x8049d60,
part=0x807b880, report=0x80729a0, e=0x8078400) at reportgen.c:278
#3  0x4005473e in rlib_break_all_below_in_reverse_order (r=0x8049d60,
part=0x807b880, report=0x80729a0, e=0x807a960) at breaks.c:204
#4  0x40054870 in rlib_handle_break_footers (r=0x8049d60,
part=0x807b880, report=0x80729a0) at breaks.c:264
#5  0x40043c52 in rlib_layout_report (r=0x8049d60, part=0x807b880,
report=0x80729a0, left_margin_offset=0, top_margin_offset=0) at
reportgen.c:587
#6  0x400440d2 in rlib_layout_part_td (r=0x8049d60, part=0x807b880,
part_deviations=0x804a7a0, page_number=1, position_top=0,
rrp=0xbfffcef8) at reportgen.c:668
#7  0x40044331 in rlib_layout_part_tr (r=0x8049d60, part=0x807b880) at
reportgen.c:715
#8  0x400446ec in rlib_make_report (r=0x8049d60) at reportgen.c:800

One solution would be to hack the get_output_size code to execute the
pcode for obtaining the size.  This however raises questions:

1.  Should we always evaluate the pcode or only if rhl->size is 0?
(if not then perhaps resolution should setup rhl->size)
2.  Should the check for the size of the text line take into account
the is_not_suppressed check?

Experimental patch attached, it fixes my reports, YMMV.

Cheers,
William.

[Rlib-devel] Buglet in detail layout on first page

[William K. V. <wk...@us...>]

Hello,
	I just noticed an off-by-one issue with detail rows on
the first page of a report.  It is overwriting the first part
of the page footer before breaking.

I may get time to look into it later this weekend but I've got
a lot of other fires to fix first.

Cheers and happy coding,

William.

Re: [Rlib-devel] Patch to postgres.c input filter

[Bob D. <bd...@si...>]

Patch looks good.  I don't use postgres but I know a lot of rlib users
do.  I wrote the postgres input filter over a year ago (and checked it
mostly worked) and from then on tried not to break it ;)

I'll trust postgres still works and apply the patch

- Bob


On Fri, 2005-06-10 at 18:21 -0600, William K. Volkman wrote:
> Hello,
> 	Attached find a patch to the postgre.c input filter
> which cures the evilness that occurs when a query fails.
> I debated with myself about just using an empty result set
> however that would keep you from seeing the error message.
> I don't use MySQL however I've installed it and as soon
> as I get some tables created I'll check out that input
> filter.
> 
> Comments and criticism welcome.
> 
> Cheers,
> William.
> 
-- 
Bob Doan <bd...@si...>

[Rlib-devel] Re: Patch to mysql.c input filter

[William K. V. <wk...@us...>]

Clicked too fast, now you'll find the patch.  Sorry :-(

On Fri, 2005-06-10 at 19:31, William K. Volkman wrote:
> Hello,
>       Attached find a patch to the mysql.c input filter
> which should cure the occasional "** NUTS.. WE CRASHED" message
> if a query fails.  Similar to the previous patch for
> postgres.c
> 
> Comments and criticism welcome.

[Rlib-devel] Patch to mysql.c input filter

[William K. V. <wk...@us...>]

Hello,
      Attached find a patch to the mysql.c input filter
which should cure the occasional "** NUTS.. WE CRASHED" message
if a query fails.  Similar to the previous patch for
postgres.c

Comments and criticism welcome.

Cheers,
William.

[Rlib-devel] Patch to postgres.c input filter

[William K. V. <wk...@us...>]

Hello,
	Attached find a patch to the postgre.c input filter
which cures the evilness that occurs when a query fails.
I debated with myself about just using an empty result set
however that would keep you from seeing the error message.
I don't use MySQL however I've installed it and as soon
as I get some tables created I'll check out that input
filter.

Comments and criticism welcome.

Cheers,
William.

Re: [Rlib-devel] Memory leaks

[Shannon W. <we...@ro...>]

Hi William,

William K. Volkman wrote:
> Hello Shannon,
> 	Help me out a bit if you will by giving me a little more
> information, unless your only interested in Bob looking into it.
> 

Nope, I'm not picky about who solves my problems :)


> On Mon, 2005-06-06 at 07:47, Shannon Weyrick wrote:
> 
>>It's been hard for me to track down because running it in gdb (apache 
>>single process) isn't giving much information and I don't have a way to 
>>reproduce it without apache. I'm willing to try any suggestions you guys 
>>have.
>>
>>You say the #1 cause is a bad query/datasouce -- why should that ever
>>actually segfault, as opposed to issuing an error message and exiting 
>>cleanly?
> 
> 
> What language are you using (PHP)?
> Which data source(s) do you use?
> I've checked out fields, literals, variables, and images.  Do you use
> graphs?  Callback functions?
> 
> 

PHP4 and MySQL. I use all of the above except images, but I believe I 
can duplicate the segfault with even a small, basic report.

Unfortunately I'm in the middle of crunch time on a project that doesn't 
use rlib so it's bit tough to work on this at the moment. When I get a 
chance I'll try to scrape together a little more info and maybe even a 
reproduction script/report. I also have yet to try it on multiple 
systems, I'm not sure I can duplicate it on anything but my own devel 
machine.

Thanks,
Shannon

Re: [Rlib-devel] Memory leaks

[William K. V. <wk...@us...>]

Hello Shannon,
	Help me out a bit if you will by giving me a little more
information, unless your only interested in Bob looking into it.

On Mon, 2005-06-06 at 07:47, Shannon Weyrick wrote:
> It's been hard for me to track down because running it in gdb (apache 
> single process) isn't giving much information and I don't have a way to 
> reproduce it without apache. I'm willing to try any suggestions you guys 
> have.
> 
> You say the #1 cause is a bad query/datasouce -- why should that ever
> actually segfault, as opposed to issuing an error message and exiting 
> cleanly?

What language are you using (PHP)?
Which data source(s) do you use?
I've checked out fields, literals, variables, and images.  Do you use
graphs?  Callback functions?


Thanks,
William.

Re: [Rlib-devel] Memory leaks

[William K. V. <wk...@us...>]

Hello Bob,

On Mon, 2005-06-06 at 08:30, Bob Doan wrote:
> Patch applied!
> 
> >   One left however is in rpdf.c,
> > the call to rpdf_text is leaving allocations on the slist:
> > 
> > ==14555== 13627 bytes in 32 blocks are still reachable in loss record 28
> > of 32
> > ==14555==    at 0x1B904A80: malloc (vg_replace_malloc.c:131)
> > ==14555==    by 0xAA7A56: g_malloc (in /usr/lib/libglib-2.0.so.0.400.8)
> > ==14555==    by 0xAA8C03: g_mem_chunk_alloc (in
> > /usr/lib/libglib-2.0.so.0.400.8)
> > ==14555==    by 0xAB675C: (within /usr/lib/libglib-2.0.so.0.400.8)
> > ==14555==    by 0xAB5CB1: g_slist_append (in
> > /usr/lib/libglib-2.0.so.0.400.8)
> > ==14555==    by 0x1B945607: rpdf_stream_append (rpdf.c:142)
<snip>
> > Since I'm not a glib guru (actually I've not coded in C since 1997)
> > I'm not certain what's going wrong here, I added the call
> > to g_slist_free in rpdf_free however it didn't fix the problem.
> 
> I see in your new patch you freed the page_contents in RPDF which looks
> like the right thing to do to fix the problem.  I'm going to go w/ that
> patch you have now since it fixes a lot of things.  I'll poke it a bit
> further and see why valgrind still complains.  GLIB doesn't always sys
> free on a glib free call and valgrind doesn't know about this so its
> harder to find mem leaks.. let me poke @ it.

Actually reading the glib docs closely you hit on it:

g_slist_free ()
void        g_slist_free                    (GSList *list);
Frees all of the memory used by a GSList. The freed elements are added
to the GListAllocator free list.

I had also seen this entry which I already figured out could not be
fixed:
==17376== 40800 bytes in 2 blocks are still reachable in loss record 30
of 32
==17376==    at 0x1B904A80: malloc (vg_replace_malloc.c:131)
==17376==    by 0x73795B: __gconv_open (in /lib/tls/libc-2.3.3.so)
==17376==    by 0x7374E4: iconv_open (in /lib/tls/libc-2.3.3.so)
==17376==    by 0xA8D835: (within /usr/lib/libglib-2.0.so.0.400.8)
==17376==    by 0xA8D8E2: g_iconv_open (in
/usr/lib/libglib-2.0.so.0.400.8)
==17376==    by 0xA8DEEE: (within /usr/lib/libglib-2.0.so.0.400.8)
==17376==    by 0xA8E1C0: g_convert (in /usr/lib/libglib-2.0.so.0.400.8)
==17376==    by 0xA8EC81: g_locale_from_utf8 (in
/usr/lib/libglib-2.0.so.0.400.8)
==17376==    by 0xA94EEE: g_date_strftime (in
/usr/lib/libglib-2.0.so.0.400.8)
==17376==    by 0x1BD7FFC0: rlib_datetime_format_date (datetime.c:119)

Since it is now the normal work week my time becomes a bit more limited.
Hopefully everything goes well.

Take care,
William.

Re: [Rlib-devel] Memory leaks

[Bob D. <bd...@si...>]

Patch applied!

>   One left however is in rpdf.c,
> the call to rpdf_text is leaving allocations on the slist:
> 
> ==14555== 13627 bytes in 32 blocks are still reachable in loss record 28
> of 32
> ==14555==    at 0x1B904A80: malloc (vg_replace_malloc.c:131)
> ==14555==    by 0xAA7A56: g_malloc (in /usr/lib/libglib-2.0.so.0.400.8)
> ==14555==    by 0xAA8C03: g_mem_chunk_alloc (in
> /usr/lib/libglib-2.0.so.0.400.8)
> ==14555==    by 0xAB675C: (within /usr/lib/libglib-2.0.so.0.400.8)
> ==14555==    by 0xAB5CB1: g_slist_append (in
> /usr/lib/libglib-2.0.so.0.400.8)
> ==14555==    by 0x1B945607: rpdf_stream_append (rpdf.c:142)
> ==14555==    by 0x1B94764B: rpdf_text (rpdf.c:798)
> ==14555==    by 0x1BD74545: pdf_print_text (pdf.c:113)
> ==14555==    by 0x1BD650EF: rlib_layout_text_string (layout.c:323)
> ==14555==    by 0x1BD66C9C: rlib_layout_report_output_array
> (layout.c:820)
> ==14555==    by 0x1BD672C7: rlib_layout_report_outputs_across_pages
> (layout.c:951)
> ==14555==    by 0x1BD67302: rlib_layout_report_output (layout.c:961)
> ==14555==    by 0x1BD63A73: rlib_layout_report (reportgen.c:571)
> ==14555==    by 0x1BD63F1D: rlib_layout_part_td (reportgen.c:668)
> ==14555==    by 0x1BD6416A: rlib_layout_part_tr (reportgen.c:715)
> ==14555==    by 0x1BD64503: rlib_make_report (reportgen.c:800)
> 
> Since I'm not a glib guru (actually I've not coded in C since 1997)
> I'm not certain what's going wrong here, I added the call
> to g_slist_free in rpdf_free however it didn't fix the problem.


Hym....

I see in your new patch you freed the page_contents in RPDF which looks
like the right thing to do to fix the problem.  I'm going to go w/ that
patch you have now since it fixes a lot of things.  I'll poke it a bit
further and see why valgrind still complains.  GLIB doesn't always sys
free on a glib free call and valgrind doesn't know about this so its
harder to find mem leaks.. let me poke @ it.

- bob

Re: [Rlib-devel] Memory leaks

[Bob D. <bd...@si...>]

> Well, the segfaults do only happen when it should be showing an error 
> message (ie, a failed sql query or an unknown pcode). But like I said, 
> at any one time, I can get at least 5 or 6 error messages out of it ok 
> (which show up in apache error log) before it starts segfaulting 
> (instead of showing error messages - it just says the apache child died 
> in the error log). When I restart apache, it works again for another 5 
> or 6. So something is starting out ok and getting corrupted along the way.
> 
> It's been hard for me to track down because running it in gdb (apache 
> single process) isn't giving much information and I don't have a way to 
> reproduce it without apache. I'm willing to try any suggestions you guys 
> have.


Are you getting a "*** NUTS WE CRASHED??" in the log

A few times I have tracked down a crash by going it to util.c

add a sleep(180)
to  myFaultHandler 

make, make install

then when it crashes quickly starting trying to grab the pids of apache
processes w/ gdb

gdb /usr/sbin/httpd xxxx
where xxx is one of the apache process ID's

once you find the pid that crashed but is in limbo issue a "bt" to see
whats going on.

This kinda sucks but it works.

> 
> You say the #1 cause is a bad query/datasouce -- why should that ever
> actually segfault, as opposed to issuing an error message and exiting 
> cleanly?


It does issue errors messages but you need to check the results of data
sources and queries before you call execute.  I think most of the
crashes have been fixed, there might be one left w/ data sources.

- bob

Re: [Rlib-devel] Memory leaks

[Shannon W. <we...@ro...>]

Bob Doan wrote:

> I'm not sure that not freeing something would cause a segfault...
> 

True, may be a stretch.

> I think the #1 cause for a segfault in RLIB is bad query/datasource and
> not checking the return codes...
> 

Well, the segfaults do only happen when it should be showing an error 
message (ie, a failed sql query or an unknown pcode). But like I said, 
at any one time, I can get at least 5 or 6 error messages out of it ok 
(which show up in apache error log) before it starts segfaulting 
(instead of showing error messages - it just says the apache child died 
in the error log). When I restart apache, it works again for another 5 
or 6. So something is starting out ok and getting corrupted along the way.

It's been hard for me to track down because running it in gdb (apache 
single process) isn't giving much information and I don't have a way to 
reproduce it without apache. I'm willing to try any suggestions you guys 
have.

You say the #1 cause is a bad query/datasouce -- why should that ever
actually segfault, as opposed to issuing an error message and exiting 
cleanly?

Thanks,
Shannon

Re: [Rlib-devel] Memory leaks

[William K. V. <wk...@us...>]

On Sun, 2005-06-05 at 18:28, Bob Doan wrote:
> Patch looks good.  But there are two problems.
> 
> In resolution.c the strduping needs to stay.  There is no guarantee that
> the pointer returned from data sources is going to stay where its at and
> it since RLIB manipulates some of that data it would be rude to change
> it on the datasource.
> 
> But there is a serious leak there.  Put the strduping back but in
> pcode.c line 738:
> 
> Change:
> 
> return rlib_value_new(rval, RLIB_VALUE_STRING, FALSE,
> rlib_resolve_field_value(r, o->value));
> 
> To:
> 
> return rlib_value_new(rval, RLIB_VALUE_STRING, TRUE,
> rlib_resolve_field_value(r, o->value));
> 
> So it knows to free the string when the RVAL is freed.

Did that and verified that it still doesn't leak.

> Second is rlib_value_dup_contents removal of the strdup.
> 
> There is going to be a double free if it's not duped
> 
> Follow "color" in layout.c:
> 
> extra_data[i].rval_color = line_rval_color; // <--- Really a memset here
> rlib_value_dup_contents(&extra_data[i].rval_color);
> .
> .
> .
> rlib_value_free(&line_rval_color);

I had run test cases, and just verified again that the example case
doesn't fail, even for color (from a previous message):

Well I commented it out, ran some test cases like:
<Line bgcolor="'0xefefef'">
 <literal bgcolor="'0xffffee'" width="95" align="right">Total:</literal>
 <field value="v.subtotal" format="'!$%(n'" width="11" align="right"/>
</Line>

And did not see any problems, no references to freed memory or things
like that.  That said however you are the designer so know what you
intended so I tried your other correction.

> .
> and later the extra data is freed.
> 
> The correct thing to do is probably is set the RVAL to free:
> 
>  struct rlib_value * rlib_value_dup_contents(struct rlib_value *rval) {
>  	if(rval->type == RLIB_VALUE_STRING) {
>  		rval->string_value = g_strdup(rval->string_value);
> 		rval->free = TRUE;
> 	}
>  	return rval;
> }
> 
Tried this and it also worked, the leaking was fixed, this is now
the patch.

> How did you track these down?  Typically I use valgrind to find these
> sorta things.

I first noticed the leaking from the python side and it's garbage
collector.  It's a check I always do when I find out SWIG has
been used (every time I find major memory leaks, some of them
even SWIG related :-)  I then perused the code and, due to having
just poked my nose into a python-libxml2 interface, spotted the
usage of GetProp without the frees.  The leaking was better however
still there from python, code reading found the resolve_fields case.
Today I bit the bullet so to speak and downloaded and fired up
valgrind, generated supressions for python, and verified and
fixed almost all remaining issues.  One left however is in rpdf.c,
the call to rpdf_text is leaving allocations on the slist:

==14555== 13627 bytes in 32 blocks are still reachable in loss record 28
of 32
==14555==    at 0x1B904A80: malloc (vg_replace_malloc.c:131)
==14555==    by 0xAA7A56: g_malloc (in /usr/lib/libglib-2.0.so.0.400.8)
==14555==    by 0xAA8C03: g_mem_chunk_alloc (in
/usr/lib/libglib-2.0.so.0.400.8)
==14555==    by 0xAB675C: (within /usr/lib/libglib-2.0.so.0.400.8)
==14555==    by 0xAB5CB1: g_slist_append (in
/usr/lib/libglib-2.0.so.0.400.8)
==14555==    by 0x1B945607: rpdf_stream_append (rpdf.c:142)
==14555==    by 0x1B94764B: rpdf_text (rpdf.c:798)
==14555==    by 0x1BD74545: pdf_print_text (pdf.c:113)
==14555==    by 0x1BD650EF: rlib_layout_text_string (layout.c:323)
==14555==    by 0x1BD66C9C: rlib_layout_report_output_array
(layout.c:820)
==14555==    by 0x1BD672C7: rlib_layout_report_outputs_across_pages
(layout.c:951)
==14555==    by 0x1BD67302: rlib_layout_report_output (layout.c:961)
==14555==    by 0x1BD63A73: rlib_layout_report (reportgen.c:571)
==14555==    by 0x1BD63F1D: rlib_layout_part_td (reportgen.c:668)
==14555==    by 0x1BD6416A: rlib_layout_part_tr (reportgen.c:715)
==14555==    by 0x1BD64503: rlib_make_report (reportgen.c:800)

Since I'm not a glib guru (actually I've not coded in C since 1997)
I'm not certain what's going wrong here, I added the call
to g_slist_free in rpdf_free however it didn't fix the problem.

Suggestions welcome, otherwise I'll come back to it after I get
the python interface re-done.

btw.  Updated patch with your corrections attached.

Cheers,
William.

Re: [Rlib-devel] Memory leaks

[Bob D. <bd...@si...>]

Hi,

Patch looks good.  But there are two problems.

In resolution.c the strduping needs to stay.  There is no guarantee that
the pointer returned from data sources is going to stay where its at and
it since RLIB manipulates some of that data it would be rude to change
it on the datasource.

But there is a serious leak there.  Put the strduping back but in
pcode.c line 738:

Change:

return rlib_value_new(rval, RLIB_VALUE_STRING, FALSE,
rlib_resolve_field_value(r, o->value));

To:

return rlib_value_new(rval, RLIB_VALUE_STRING, TRUE,
rlib_resolve_field_value(r, o->value));

So it knows to free the string when the RVAL is freed.



Second is rlib_value_dup_contents removal of the strdup.

There is going to be a double free if it's not duped

Follow "color" in layout.c:

extra_data[i].rval_color = line_rval_color; // <--- Really a memset here
rlib_value_dup_contents(&extra_data[i].rval_color);
.
.
.
rlib_value_free(&line_rval_color);
.
.
.
.
and later the extra data is freed.

The correct thing to do is probably is set the RVAL to free:

 struct rlib_value * rlib_value_dup_contents(struct rlib_value *rval) {
 	if(rval->type == RLIB_VALUE_STRING) {
 		rval->string_value = g_strdup(rval->string_value);
		rval->free = TRUE;
	}
 	return rval;
}



Other then that great work!

How did you track these down?  Typically I use valgrind to find these
sorta things.

- bob

Re: [Rlib-devel] Design purpose of rlib_value_dup_contents?

[William K. V. <wk...@us...>]

Hello Bob,
On Sun, 2005-06-05 at 15:19, Bob Doan wrote:
> It's only used in rlib_layout_execute_pcodes_for_line
> 
> It's used for things on a line (image, literal, text) that don't have a
> specific property that a line done.  Like if a line had a bgcolor of
> 'red' but the field did not specify the bgcolor so the field would
> inherit the value of the line
> 
> It's duped because the value is manipulated later in the code, then
> freed.

Well I commented it out, ran some test cases like:
<Line bgcolor="'0xefefef'">
 <literal bgcolor="'0xffffee'" width="95" align="right">Total:</literal>
 <field value="v.subtotal" format="'!$%(n'" width="11" align="right"/>
</Line>

And did not see any problems, no references to freed memory or things
like that.

> 
> Most of the time it is kinda untouched and this is the one place in the
> RLIB code ref counting RVALS would be more efficient instead of
> duplicating.  However moving to this would be a little work and odds are
> things would break.  It might be a little late in the release cycle to
> tackle this one.  
> 
> But it you want to fix this and not break anything go for it ;)
> 
> - bob
> 
> On Sun, 2005-06-05 at 14:13 -0600, William K. Volkman wrote:
> > On the surface this code looks like it's designed to leak,
> > I can however imagine a use case so I thought I better ask.
> > 
> > struct rlib_value * rlib_value_dup_contents(struct rlib_value *rval) {
> > 	if(rval->type == RLIB_VALUE_STRING)
> > 		rval->string_value = g_strdup(rval->string_value);
> > 	return rval;
> > }
> > 
> > I'm going to comment it out and see what happens.
> > 
> > https://lists.sourceforge.net/lists/listinfo/rlib-devel

Re: [Rlib-devel] Memory leaks

[William K. V. <wk...@us...>]

Hello,
	I've finished making changes, except for a leaklet in
rpdf.c (not sure what's going on there).  There were a couple
of places that I was not certain what the design intent was
so I left code in "#if 0"ed out, if that's unacceptable let me
know.  It still runs my smallish report just fine, YMMV.  If
it breaks anything let me know.

On Sun, 2005-06-05 at 11:11, Bob Doan wrote:
> To fix this double free probably in rlib_layout_part_tr check 
> "report->is_the_only_report" and don't call it again if it is TRUE
> 
> Thanks for pointing these out.
> 
> Will you be able to send in a patch fixing all of this?

Attached.

Cheers,
William.

Re: [Rlib-devel] Design purpose of rlib_value_dup_contents?

[Bob D. <bd...@si...>]

It's only used in rlib_layout_execute_pcodes_for_line

It's used for things on a line (image, literal, text) that don't have a
specific property that a line done.  Like if a line had a bgcolor of
'red' but the field did not specify the bgcolor so the field would
inherit the value of the line

It's duped because the value is manipulated later in the code, then
freed.  

Most of the time it is kinda untouched and this is the one place in the
RLIB code ref counting RVALS would be more efficient instead of
duplicating.  However moving to this would be a little work and odds are
things would break.  It might be a little late in the release cycle to
tackle this one.  

But it you want to fix this and not break anything go for it ;)

- bob

On Sun, 2005-06-05 at 14:13 -0600, William K. Volkman wrote:
> On the surface this code looks like it's designed to leak,
> I can however imagine a use case so I thought I better ask.
> 
> struct rlib_value * rlib_value_dup_contents(struct rlib_value *rval) {
> 	if(rval->type == RLIB_VALUE_STRING)
> 		rval->string_value = g_strdup(rval->string_value);
> 	return rval;
> }
> 
> I'm going to comment it out and see what happens.
> 
> Thanks for any insight.
> 
> Cheers,
> William.
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: NEC IT Guy Games.  How far can you shotput
> a projector? How fast can you ride your desk chair down the office luge track?
> If you want to score the big prize, get to know the little guy.  
> Play to win an NEC 61" plasma display: http://www.necitguy.com/?r=20
> _______________________________________________
> Rlib-devel mailing list
> Rli...@li...
> https://lists.sourceforge.net/lists/listinfo/rlib-devel
-- 
Bob Doan <bd...@si...>

[Rlib-devel] Design purpose of rlib_value_dup_contents?

[William K. V. <wk...@us...>]

On the surface this code looks like it's designed to leak,
I can however imagine a use case so I thought I better ask.

struct rlib_value * rlib_value_dup_contents(struct rlib_value *rval) {
	if(rval->type == RLIB_VALUE_STRING)
		rval->string_value = g_strdup(rval->string_value);
	return rval;
}

I'm going to comment it out and see what happens.

Thanks for any insight.

Cheers,
William.