|
From: Ayodele T. <em...@st...> - 2004-01-07 00:55:37
|
Because of the memory requirements of my skin, I would like to do a preprocessing step where I look at all of the data addresses produced and dump a file containing addresses that are read only. Then I want to read those addresses in and use them. Currently, I have one skin that does the preprocessing and another than does the main processing. However, when I re-execute the code with the different skin, the memory addresses move and they no longer match up. Is there a way to force the application to run a second time within the same skin? (i.e. make two passes). I think then the addresses would be stable between the two runs. Ayodele -- --------------------------------------------------------------- Ayodele Bolaji Thomas "Joy in the Morning" Ph.D. Candidate Electrical Engineering Computer Systems Laboratory Stanford University Ayo...@st... Support our troops. Support our country. Pray for Peace. \o/ "We succeed, not because of Affirmative Action, but in spite of the conditions that make it necessary" (ABE '98) "War may sometimes be a necessary evil. But no matter how necessary, it is always an evil, never a good. We will not learn to live together in peace by killing each other's children." Jimmy Carter '02 --------------------------------------------------------------- |
|
From: Nicholas N. <nj...@ca...> - 2004-01-07 12:10:51
|
On Tue, 6 Jan 2004, Ayodele Thomas wrote: > Because of the memory requirements of my skin, I would like to do a > preprocessing step where I look at all of the data addresses produced and > dump a file containing addresses that are read only. Then I > want to read those addresses in and use them. Currently, I > have one skin that does the preprocessing and another than does > the main processing. Does the second step have to be done as a Valgrind skin -- could it be done with a standalone program? Or, can you do the pre-processing and the main processing at the same time? That seems a better way to do things. > However, when I re-execute the code with the > different skin, the memory addresses move and they no longer match up. I think this is because the different skins have different sizes and thus cause memory to be laid out in a different way. When developing Cachegrind, I found that the hit/miss counts would change slightly whenever I made the slightest change to the skin code, for exactly this reason. > Is there a way to force the application to run a second time within the > same skin? (i.e. make two passes). I think then the addresses would be > stable between the two runs. Not that I'm aware of. N |
|
From: Ayodele T. <em...@st...> - 2004-01-07 16:43:58
|
My initial step is designed to identify data addresses that are read only so that they can be excluded from the main analysis performed by the second step. Therefore, the second step is the critical step and must be performed in Valgrind. I can't think of a way that both steps can be performed at the same time because the goal is to determine which data can be ignored - but that cannot be determined until the entire application has run because I am locating dependencies. The initial step is designed to reduce the memory requirements because I must keep track of a tremendous amount of state in the main pass and have issues running out of memory and crashing, even with the 2GB of memory on my machine. Ayodele > > On Tue, 6 Jan 2004, Ayodele Thomas wrote: > > > Because of the memory requirements of my skin, I would like to do a > > preprocessing step where I look at all of the data addresses produced and > > dump a file containing addresses that are read only. Then I > > want to read those addresses in and use them. Currently, I > > have one skin that does the preprocessing and another than does > > the main processing. > > Does the second step have to be done as a Valgrind skin -- could it be > done with a standalone program? Or, can you do the pre-processing and the > main processing at the same time? That seems a better way to do things. > > > However, when I re-execute the code with the > > different skin, the memory addresses move and they no longer match up. > > I think this is because the different skins have different sizes and thus > cause memory to be laid out in a different way. When developing > Cachegrind, I found that the hit/miss counts would change slightly > whenever I made the slightest change to the skin code, for exactly this > reason. > > > Is there a way to force the application to run a second time within the > > same skin? (i.e. make two passes). I think then the addresses would be > > stable between the two runs. > > Not that I'm aware of. > > N > -- --------------------------------------------------------------- Ayodele Bolaji Thomas "Joy in the Morning" Ph.D. Candidate Electrical Engineering Computer Systems Laboratory Stanford University Ayo...@st... Support our troops. Support our country. Pray for Peace. \o/ "We succeed, not because of Affirmative Action, but in spite of the conditions that make it necessary" (ABE '98) "War may sometimes be a necessary evil. But no matter how necessary, it is always an evil, never a good. We will not learn to live together in peace by killing each other's children." Jimmy Carter '02 --------------------------------------------------------------- |
|
From: Nicholas N. <nj...@ca...> - 2004-01-07 16:51:30
|
On Wed, 7 Jan 2004, Ayodele Thomas wrote: > My initial step is designed to identify data addresses that are read only > so that they can be excluded from the main analysis performed by the > second step. Therefore, the second step is the critical step and must be > performed in Valgrind. I can't think of a way that both steps can be > performed at the same time because the goal is to determine which data can > be ignored - but that cannot be determined until the entire application > has run because I am locating dependencies. So you can't tell which addresses are read only until the program has finished? Ok, I think I understand. > The initial step is designed to reduce the memory requirements because I must > keep track of a tremendous amount of state in the main pass and have > issues running out of memory and crashing, even with the 2GB of memory on > my machine. What about if you put both passes into the same skin, and then switch between the pre-processing and main processing with a command-line arg? That way, you will hopefully get the same memory addresses for both runs. However, even if it does work, it sounds fragile. N |
|
From: Ayodele T. <em...@st...> - 2004-01-07 17:48:43
|
I tried running the same skin twice on the same executable (with no changes), and even that gave different results for the data memory addresses. I will have to figure out another way to reduce my memory requirements - maybe dumping state to a file occasionally and then only reading it back when needed. Ayo > > On Wed, 7 Jan 2004, Ayodele Thomas wrote: > > > My initial step is designed to identify data addresses that are read only > > so that they can be excluded from the main analysis performed by the > > second step. Therefore, the second step is the critical step and must be > > performed in Valgrind. I can't think of a way that both steps can be > > performed at the same time because the goal is to determine which data can > > be ignored - but that cannot be determined until the entire application > > has run because I am locating dependencies. > > So you can't tell which addresses are read only until the program has > finished? Ok, I think I understand. > > > The initial step is designed to reduce the memory requirements because I must > > keep track of a tremendous amount of state in the main pass and have > > issues running out of memory and crashing, even with the 2GB of memory on > > my machine. > > What about if you put both passes into the same skin, and then switch > between the pre-processing and main processing with a command-line arg? > That way, you will hopefully get the same memory addresses for both runs. > However, even if it does work, it sounds fragile. > > N > -- --------------------------------------------------------------- Ayodele Bolaji Thomas "Joy in the Morning" Ph.D. Candidate Electrical Engineering Computer Systems Laboratory Stanford University Ayo...@st... Support our troops. Support our country. Pray for Peace. \o/ "We succeed, not because of Affirmative Action, but in spite of the conditions that make it necessary" (ABE '98) "War may sometimes be a necessary evil. But no matter how necessary, it is always an evil, never a good. We will not learn to live together in peace by killing each other's children." Jimmy Carter '02 --------------------------------------------------------------- |
|
From: Ayodele T. <em...@st...> - 2004-01-10 08:11:02
|
I am having serious issues running out of memory. Valgrind claims that it
cannot allocate 1M of memory (below), but when I look at "top" I see
almost 1GB of free memory, no swap space being used and no memory shared.
VG_(get_memory_from_mmap): request for 1048576 bytes failed.
VG_(get_memory_from_mmap): 2144292771 bytes already allocated.
Valgrind seems to think it has run out of memory although the machine
seems to believe that it has ample memory. Any suggestions on how to fix
this problem? It is very frustrating. I am not using the latest release.
The following is a dump of "top", just before it dies.
Ayo
00:07:55 up 2 days, 9:28, 13 users, load average: 1.70, 0.97, 0.78
89 processes: 86 sleeping, 3 running, 0 zombie, 0 stopped
CPU states: 99.0% user 1.0% system 0.0% nice 0.0% iowait 0.0%
idle
Mem: 2064560k av, 1091960k used, 972600k free, 0k shrd, 192148k
buff
939968k actv, 31616k in_d, 45976k in_c
Swap: 4233116k av, 0k used, 4233116k free 467696k cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
959 ayodele 25 0 246M 246M 556 R 99.3 12.2 3:31 0 pegwit
---------------------------------------------------------------
Ayodele Bolaji Thomas "Joy in the Morning"
Ph.D. Candidate Electrical Engineering
Computer Systems Laboratory
Stanford University
Ayo...@st...
---------------------------------------------------------------
|
|
From: Tom H. <th...@cy...> - 2004-01-10 08:14:34
|
In message <200...@ca...>
"Ayodele Thomas" <em...@st...> wrote:
> Valgrind seems to think it has run out of memory although the machine
> seems to believe that it has ample memory. Any suggestions on how to fix
> this problem? It is very frustrating. I am not using the latest release.
Well if you're not using the latest release, what version are you using?
Tom
--
Tom Hughes (th...@cy...)
Software Engineer, Cyberscience Corporation
http://www.cyberscience.com/
|
|
From: Dirk M. <dm...@gm...> - 2004-01-10 12:40:04
|
On Saturday 10 January 2004 09:11, Ayodele Thomas wrote: > VG_(get_memory_from_mmap): request for 1048576 bytes failed. > VG_(get_memory_from_mmap): 2144292771 bytes already allocated. on intel x86 archictures, which valgrind runs on, a single process cannot allocate more than 2GB of RAM. Even though you have 20GB of swap, as pointers have only 32bit size, you can't create a bigger process image. reduce your working set size and try again. Dirk |