#691 HeidiSQL runs out of memory displaying database table content

[Anonymous]

Originally created by: lutz.thi...@gtempaccount.com

What steps will reproduce the problem?
1. select a database on database tree
2. double click on a database table (in "Database: ..." tab) to switch to
"Table: " tab to see table definition
3. Click on "Data" tab to preview data

What is the expected output? What do you see instead?
HeidiSQL runs out of memory for a 20MB table (500.000 rows, MyISAM) and
crashes.

What version of the product are you using? On what operating system?
version: 1691; WinXP & W2K

[Anonymous]

Originally posted by: rosenfie...@gmail.com

If you want this fixed in a hurry, the easiest way is to let at least one person
from the development team have temporary access to the tables in question.

You can attach data to this case, or I can send you a link to an upload-only FTP
site.

Alternatively, you could install Delphi (trial available).  There's a very easy
getting started readme for building HeidiSQL, and that way you'll be able to pause
the program yourself and see where in the source code it's looping.  (Report that
here...)

Labels: -Priority-Medium Severity-Broken
Status: Accepted

[Anonymous]

Originally posted by: rosenfie...@gmail.com

(Even a SHOW CREATE TABLE would be nice.)

No reply for 5 days, closing due to lack of feedback.

Status: NeedInfo

[Anonymous]

Originally posted by: lutz.thi...@gtempaccount.com

Closing an issue after five days is very fast - I spend a week on vacation...

I'm sorry, but because of security and IP reasons I don't have the permission to
provide a database table (or other data) to verify this problem. Furthermore, I have
no change to install Delphi here. Maybe yo could have an eye on this problem. I'm
sure that it will arise again by other users...

[Anonymous]

Originally posted by: lutz.thi...@gtempaccount.com

I run into the mentioned problem again (I tried to view data of a table with rows:
>2.500.000.000 and size: ~120GB). But this time HeidiSQL reported an error (see
attached screen shot). Maybe this error message could help.
Thanks % regards,

Lutz

[Anonymous]

Originally posted by: rosenfie...@gmail.com

I actually have no clue how to map a memory address to a source code line with
Delphi, so it's less help than you'd think.

[Anonymous]

Originally posted by: lutz.thi...@gtempaccount.com

I found an additional indication - maybe pointing to the root cause...
My PC has 2GB RAM. If I try to open tables greater than 2GB, HeidiSQL crashes. This
is reproducible.
Depending on your memory configuration, I could send you a neutral sample table, but
I don't want to waste upload bandwidth on your ftp server...  ;-)

Regards,

Lutz

[Anonymous]

Originally posted by: rosenfie...@gmail.com

What size is the sample table if you gzip it?

If it's sufficiently small in compressed form, you could just attach it to this case
or email it.

[Anonymous]

Originally posted by: rosenfie...@gmail.com

You can also upload to ftp.heidisql.com.
User: upload
Pass: upload
Folder: /upload/

You have write-only access, so you cannot see your data once you have uploaded it.

[Anonymous]

Originally posted by: lutz.thi...@gtempaccount.com

Thanks you very much!
I'm going to prepare a neutral database table. If finished I'll upload it.

Regards,

Lutz

[Anonymous]

Originally posted by: rosenfie...@gmail.com

If you end up creating a tool to fill the table, then it would be quite cool if that
tool could read the table layout from information_schema for any table and be used
to fill in random data.  Could be used by other users to create garbage-ized test
cases too.

[Anonymous]

Originally posted by: lutz.thi...@gtempaccount.com

Upload of file "HeidiSQLbug_issue691.sql.gz" has been finished.
I didn't used a script to create this file. Therefore, I can't provide a tool to fill
tables with random data. If I'll have some time left, I'll try to code such a script
and provide it to the community.
Regarding this bug:
Assuming you have a PC with 2GB RAM, the provided table should lead to a HeidiSQL
crash. If you monitor you system memory, you'll see that HeidiSQL tries to allocate
approximately the same size of memory as you table consumes. (BTW: This is also the
case for smaller tables.)
I'm not sure if this is intended. If so, in my opinion it is not a good idea to
allocate all available memory, because this lowers the system performance of other
application running on the PC.
Regards,

Lutz

[Anonymous]

Originally posted by: a...@anse.de

Thanks a lot for your action!

Are you sure the file upload was succesfull? (It's not there yet)

[Anonymous]

Originally posted by: lutz.thi...@gtempaccount.com

No, I'm not sure. I got an 421-error.
Because I don't have read permissions, I couldn't check if upload was successful. I
fear that our firewall has blocked this upload. I'll try to upload the file from
home. Unfortunately my private PC crashed last days - but I'll find a PC and a DSL
connection. Give me two more days...

Lutz

[Anonymous]

Originally posted by: lutz.thi...@gtempaccount.com

Uploaded file should be available now...

[Anonymous]

Originally posted by: rosenfie...@gmail.com

(No comment was entered for this change.)

Status: Accepted

[Anonymous]

Originally posted by: rosenfie...@gmail.com

Loading to test server...

In the mean time, just to clarify, is this still the steps you took:
> 1. select a database on database tree
> 2. double click on a database table to switch to "Table: " tab
> 3. Click on "Data" tab to preview data

or did you do more this time?

(For example, did you browse through the entire data set while in the 'data' tab?)

[Anonymous]

Originally posted by: lutz.thi...@gtempaccount.com

No, I didn't more than described (no browsing etc.).

[Anonymous]

Originally posted by: rosenfie...@gmail.com

Ok.  The places I can imagine a memory problem is:
- initializing internal data array
- initializing grid gui (virtualtree) information array
- browsing too much in the grid (data is never disposed of while in grid)

Markers have been added to the code, so in [r1795] you can see exactly where the
problem is.  In dbgview.exe, it should go like this:

  HeidiSQL: mem: clearing browse data.
  HeidiSQL: mem: initializing browse columns.
  HeidiSQL: mem: browse column initialization complete.
  HeidiSQL: mem: initializing browse rows (internal data).
  HeidiSQL: mem: initializing browse rows (grid).
  HeidiSQL: mem: browse row initialization complete.

If at any point it stops and consumes too much memory, we'll know exactly where it
is.

Please try this and report back.
Thanks!

[Anonymous]

Originally posted by: rosenfie...@gmail.com

This is what I see (which is not necessarily the same as what you see) using the
test table provided.

[Anonymous]

Originally posted by: lutz.thi...@gtempaccount.com

Hi Albert,
I'm sorry, but I can't help you debugging using dbgview.exe.
Please find attached a screen shot of my task manager. It shows that HeidiSQL ties my
PC for minutes and consume all available memory. But HeidiSQl (1795) don't crashes
anymore... :-)
Regards,

Lutz

[Anonymous]

Originally posted by: rosenfie...@gmail.com

> I'm sorry, but I can't help you debugging using dbgview.exe.

Ok, no problem.  Since I'm not sure we're seeing the same issue, I'll close this
case (and open my own for the one I'm seeing).

Status: NeedInfo

[Anonymous]

Originally posted by: serge...@pisem.net

>I actually have no clue how to map a memory address to a source code line with
>Delphi, so it's less help than you'd think.
View\Debug Windows\Modules - find the method responsible for AV
View\Debug Windows\CPU - address of the code line in that method

[Anonymous]

Originally posted by: rosenfie...@gmail.com

Ah, exactly.  You're a genius.

The thing that you need to know when using the 'Modules' window is that it's
completely empty (and thus useless) until you actually start the program.  Only then
will the relevant procedure names and addresses be filled into the otherwise void
window.

The CPU window didn't seem useful at first, but now that you mention it, there is in
fact the 'go to address' function!  The program has to be running and the process
suspended (in the 'paused' state) before the window will show.  Right-click the CPU
window and there you have the 'go to address' menu item.  Enter addresses in Pascal
notation, for example "$65C6DB".  Make sure that the 'mixed source' right-click menu
item is selected (shortcut: CTRL-X), then browse upwards in the disassembly view and
at some point a source code file and a line number will be visible among the
disassembly lines.

Nice!

[Anonymous]

Originally posted by: rosenfie...@gmail.com

For this particular issue, the crash was in line 4758 of childwin.pas:
  for i := 0 to Length(Cols) - 1 do

Which I guess is due to the internal data structures being initialized after the
grid's data structures, and the grid's data structures crashing the initialization
procedure.  Now it's done in reverse (internal stuff first), so this bug should be
gone.