|
From: Maurice v. S. <ma...@bl...> - 2013-01-02 19:08:56
|
Hi, First of all I want all the valgrind developers to know what an amazing tool valgrind is and that we have used it many times to fix those hard to debug problems. Now for my question: is it possible to go beyond the 32G memory allocation limit? Thanks and happy new year. ][ Maurice van Swaaij ][ |
|
From: Julian S. <js...@ac...> - 2013-01-02 22:00:21
|
On Wednesday, January 02, 2013, Maurice van Swaaij wrote:
> Now for my question: is it possible to go beyond the 32G memory allocation
> limit?
The patch below allows it to use up to 64G, that is, it doubles the
available memory. If someone would test it properly, it might get
committed; but so far we have had (AFAIK) no definitive "it works"
feedback.
J
Index: memcheck/mc_main.c
===================================================================
--- memcheck/mc_main.c (revision 13179)
+++ memcheck/mc_main.c (working copy)
@@ -167,10 +167,10 @@
#else
-/* Just handle the first 32G fast and the rest via auxiliary
+/* Just handle the first 64G fast and the rest via auxiliary
primaries. If you change this, Memcheck will assert at startup.
See the definition of UNALIGNED_OR_HIGH for extensive comments. */
-# define N_PRIMARY_BITS 19
+# define N_PRIMARY_BITS 20
#endif
@@ -6524,11 +6524,11 @@
tl_assert(sizeof(Addr) == 8);
tl_assert(sizeof(UWord) == 8);
tl_assert(sizeof(Word) == 8);
- tl_assert(MAX_PRIMARY_ADDRESS == 0x7FFFFFFFFULL);
- tl_assert(MASK(1) == 0xFFFFFFF800000000ULL);
- tl_assert(MASK(2) == 0xFFFFFFF800000001ULL);
- tl_assert(MASK(4) == 0xFFFFFFF800000003ULL);
- tl_assert(MASK(8) == 0xFFFFFFF800000007ULL);
+ tl_assert(MAX_PRIMARY_ADDRESS == 0xFFFFFFFFFULL);
+ tl_assert(MASK(1) == 0xFFFFFFF000000000ULL);
+ tl_assert(MASK(2) == 0xFFFFFFF000000001ULL);
+ tl_assert(MASK(4) == 0xFFFFFFF000000003ULL);
+ tl_assert(MASK(8) == 0xFFFFFFF000000007ULL);
# endif
}
Index: coregrind/m_aspacemgr/aspacemgr-linux.c
===================================================================
--- coregrind/m_aspacemgr/aspacemgr-linux.c (revision 13179)
+++ coregrind/m_aspacemgr/aspacemgr-linux.c (working copy)
@@ -1649,7 +1649,7 @@
aspacem_minAddr = (Addr) 0x04000000; // 64M
# if VG_WORDSIZE == 8
- aspacem_maxAddr = (Addr)0x800000000 - 1; // 32G
+ aspacem_maxAddr = (Addr)0x1000000000ULL - 1; // 64G
# ifdef ENABLE_INNER
{ Addr cse = VG_PGROUNDDN( sp_at_startup ) - 1;
if (aspacem_maxAddr > cse)
|
|
From: Maurice v. S. <ma...@bl...> - 2013-01-02 22:34:11
|
Thanks, I'll try it out with 3.8.1 and let you know if it works.
][ Maurice van Swaaij ][
----- Original Message -----
On Wednesday, January 02, 2013, Maurice van Swaaij wrote:
> Now for my question: is it possible to go beyond the 32G memory allocation
> limit?
The patch below allows it to use up to 64G, that is, it doubles the
available memory. If someone would test it properly, it might get
committed; but so far we have had (AFAIK) no definitive "it works"
feedback.
J
Index: memcheck/mc_main.c
===================================================================
--- memcheck/mc_main.c (revision 13179)
+++ memcheck/mc_main.c (working copy)
@@ -167,10 +167,10 @@
#else
-/* Just handle the first 32G fast and the rest via auxiliary
+/* Just handle the first 64G fast and the rest via auxiliary
primaries. If you change this, Memcheck will assert at startup.
See the definition of UNALIGNED_OR_HIGH for extensive comments. */
-# define N_PRIMARY_BITS 19
+# define N_PRIMARY_BITS 20
#endif
@@ -6524,11 +6524,11 @@
tl_assert(sizeof(Addr) == 8);
tl_assert(sizeof(UWord) == 8);
tl_assert(sizeof(Word) == 8);
- tl_assert(MAX_PRIMARY_ADDRESS == 0x7FFFFFFFFULL);
- tl_assert(MASK(1) == 0xFFFFFFF800000000ULL);
- tl_assert(MASK(2) == 0xFFFFFFF800000001ULL);
- tl_assert(MASK(4) == 0xFFFFFFF800000003ULL);
- tl_assert(MASK(8) == 0xFFFFFFF800000007ULL);
+ tl_assert(MAX_PRIMARY_ADDRESS == 0xFFFFFFFFFULL);
+ tl_assert(MASK(1) == 0xFFFFFFF000000000ULL);
+ tl_assert(MASK(2) == 0xFFFFFFF000000001ULL);
+ tl_assert(MASK(4) == 0xFFFFFFF000000003ULL);
+ tl_assert(MASK(8) == 0xFFFFFFF000000007ULL);
# endif
}
Index: coregrind/m_aspacemgr/aspacemgr-linux.c
===================================================================
--- coregrind/m_aspacemgr/aspacemgr-linux.c (revision 13179)
+++ coregrind/m_aspacemgr/aspacemgr-linux.c (working copy)
@@ -1649,7 +1649,7 @@
aspacem_minAddr = (Addr) 0x04000000; // 64M
# if VG_WORDSIZE == 8
- aspacem_maxAddr = (Addr)0x800000000 - 1; // 32G
+ aspacem_maxAddr = (Addr)0x1000000000ULL - 1; // 64G
# ifdef ENABLE_INNER
{ Addr cse = VG_PGROUNDDN( sp_at_startup ) - 1;
if (aspacem_maxAddr > cse)
|
|
From: Maurice v. S. <ma...@bl...> - 2013-01-09 17:29:01
|
Hi Julian,
I have been running valgrind 3.8.1 with your patch but for various reasons (unrelated to valgrind) I haven't been able to keep that large job that needs more than 32G running long enough to arrive at a significant memory use. I'll let you know when it does.
On a smaller job (using 18G) I do still get these messages:
==15201== Warning: set address range perms: large range [0x247cd5040, 0x267cd5040) (undefined)
==15201== Warning: set address range perms: large range [0x247cd5028, 0x267cd5058) (noaccess)
but I understand they are not interfering with valgrind's operation.
Thanks again for your help.
][ Maurice van Swaaij ][
----- Original Message -----
Thanks, I'll try it out with 3.8.1 and let you know if it works.
][ Maurice van Swaaij ][
----- Original Message -----
On Wednesday, January 02, 2013, Maurice van Swaaij wrote:
> Now for my question: is it possible to go beyond the 32G memory allocation
> limit?
The patch below allows it to use up to 64G, that is, it doubles the
available memory. If someone would test it properly, it might get
committed; but so far we have had (AFAIK) no definitive "it works"
feedback.
J
Index: memcheck/mc_main.c
===================================================================
--- memcheck/mc_main.c (revision 13179)
+++ memcheck/mc_main.c (working copy)
@@ -167,10 +167,10 @@
#else
-/* Just handle the first 32G fast and the rest via auxiliary
+/* Just handle the first 64G fast and the rest via auxiliary
primaries. If you change this, Memcheck will assert at startup.
See the definition of UNALIGNED_OR_HIGH for extensive comments. */
-# define N_PRIMARY_BITS 19
+# define N_PRIMARY_BITS 20
#endif
@@ -6524,11 +6524,11 @@
tl_assert(sizeof(Addr) == 8);
tl_assert(sizeof(UWord) == 8);
tl_assert(sizeof(Word) == 8);
- tl_assert(MAX_PRIMARY_ADDRESS == 0x7FFFFFFFFULL);
- tl_assert(MASK(1) == 0xFFFFFFF800000000ULL);
- tl_assert(MASK(2) == 0xFFFFFFF800000001ULL);
- tl_assert(MASK(4) == 0xFFFFFFF800000003ULL);
- tl_assert(MASK(8) == 0xFFFFFFF800000007ULL);
+ tl_assert(MAX_PRIMARY_ADDRESS == 0xFFFFFFFFFULL);
+ tl_assert(MASK(1) == 0xFFFFFFF000000000ULL);
+ tl_assert(MASK(2) == 0xFFFFFFF000000001ULL);
+ tl_assert(MASK(4) == 0xFFFFFFF000000003ULL);
+ tl_assert(MASK(8) == 0xFFFFFFF000000007ULL);
# endif
}
Index: coregrind/m_aspacemgr/aspacemgr-linux.c
===================================================================
--- coregrind/m_aspacemgr/aspacemgr-linux.c (revision 13179)
+++ coregrind/m_aspacemgr/aspacemgr-linux.c (working copy)
@@ -1649,7 +1649,7 @@
aspacem_minAddr = (Addr) 0x04000000; // 64M
# if VG_WORDSIZE == 8
- aspacem_maxAddr = (Addr)0x800000000 - 1; // 32G
+ aspacem_maxAddr = (Addr)0x1000000000ULL - 1; // 64G
# ifdef ENABLE_INNER
{ Addr cse = VG_PGROUNDDN( sp_at_startup ) - 1;
if (aspacem_maxAddr > cse)
|
|
From: Maurice v. S. <ma...@bl...> - 2013-01-24 18:48:29
|
Hi Julian,
I have run valgrind 3.8.1 with your patch successfully on a job that neared 64G in memory allocation.
Thanks again.
][ Maurice van Swaaij ][
----- Original Message -----
Hi Julian,
I have been running valgrind 3.8.1 with your patch but for various reasons (unrelated to valgrind) I haven't been able to keep that large job that needs more than 32G running long enough to arrive at a significant memory use. I'll let you know when it does.
On a smaller job (using 18G) I do still get these messages:
==15201== Warning: set address range perms: large range [0x247cd5040, 0x267cd5040) (undefined)
==15201== Warning: set address range perms: large range [0x247cd5028, 0x267cd5058) (noaccess)
but I understand they are not interfering with valgrind's operation.
Thanks again for your help.
][ Maurice van Swaaij ][
----- Original Message -----
Thanks, I'll try it out with 3.8.1 and let you know if it works.
][ Maurice van Swaaij ][
----- Original Message -----
On Wednesday, January 02, 2013, Maurice van Swaaij wrote:
> Now for my question: is it possible to go beyond the 32G memory allocation
> limit?
The patch below allows it to use up to 64G, that is, it doubles the
available memory. If someone would test it properly, it might get
committed; but so far we have had (AFAIK) no definitive "it works"
feedback.
J
Index: memcheck/mc_main.c
===================================================================
--- memcheck/mc_main.c (revision 13179)
+++ memcheck/mc_main.c (working copy)
@@ -167,10 +167,10 @@
#else
-/* Just handle the first 32G fast and the rest via auxiliary
+/* Just handle the first 64G fast and the rest via auxiliary
primaries. If you change this, Memcheck will assert at startup.
See the definition of UNALIGNED_OR_HIGH for extensive comments. */
-# define N_PRIMARY_BITS 19
+# define N_PRIMARY_BITS 20
#endif
@@ -6524,11 +6524,11 @@
tl_assert(sizeof(Addr) == 8);
tl_assert(sizeof(UWord) == 8);
tl_assert(sizeof(Word) == 8);
- tl_assert(MAX_PRIMARY_ADDRESS == 0x7FFFFFFFFULL);
- tl_assert(MASK(1) == 0xFFFFFFF800000000ULL);
- tl_assert(MASK(2) == 0xFFFFFFF800000001ULL);
- tl_assert(MASK(4) == 0xFFFFFFF800000003ULL);
- tl_assert(MASK(8) == 0xFFFFFFF800000007ULL);
+ tl_assert(MAX_PRIMARY_ADDRESS == 0xFFFFFFFFFULL);
+ tl_assert(MASK(1) == 0xFFFFFFF000000000ULL);
+ tl_assert(MASK(2) == 0xFFFFFFF000000001ULL);
+ tl_assert(MASK(4) == 0xFFFFFFF000000003ULL);
+ tl_assert(MASK(8) == 0xFFFFFFF000000007ULL);
# endif
}
Index: coregrind/m_aspacemgr/aspacemgr-linux.c
===================================================================
--- coregrind/m_aspacemgr/aspacemgr-linux.c (revision 13179)
+++ coregrind/m_aspacemgr/aspacemgr-linux.c (working copy)
@@ -1649,7 +1649,7 @@
aspacem_minAddr = (Addr) 0x04000000; // 64M
# if VG_WORDSIZE == 8
- aspacem_maxAddr = (Addr)0x800000000 - 1; // 32G
+ aspacem_maxAddr = (Addr)0x1000000000ULL - 1; // 64G
# ifdef ENABLE_INNER
{ Addr cse = VG_PGROUNDDN( sp_at_startup ) - 1;
if (aspacem_maxAddr > cse)
------------------------------------------------------------------------------
Master Java SE, Java EE, Eclipse, Spring, Hibernate, JavaScript, jQuery
and much more. Keep your Java skills current with LearnJavaNow -
200+ hours of step-by-step video tutorials by Java experts.
SALE $49.99 this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122612
_______________________________________________
Valgrind-users mailing list
Val...@li...
https://lists.sourceforge.net/lists/listinfo/valgrind-users
|
|
From: Maurice v. S. <ma...@bl...> - 2013-01-25 23:38:55
|
Hi,
it looks like even 64G is not enough. Is it possible to go even higher or is 64G the limit?
Our biggest machines have 256G, is it possible to go that high?
Thanks.
][ Maurice van Swaaij ][
----- Original Message -----
Hi Julian,
I have run valgrind 3.8.1 with your patch successfully on a job that neared 64G in memory allocation.
Thanks again.
][ Maurice van Swaaij ][
----- Original Message -----
Hi Julian,
I have been running valgrind 3.8.1 with your patch but for various reasons (unrelated to valgrind) I haven't been able to keep that large job that needs more than 32G running long enough to arrive at a significant memory use. I'll let you know when it does.
On a smaller job (using 18G) I do still get these messages:
==15201== Warning: set address range perms: large range [0x247cd5040, 0x267cd5040) (undefined)
==15201== Warning: set address range perms: large range [0x247cd5028, 0x267cd5058) (noaccess)
but I understand they are not interfering with valgrind's operation.
Thanks again for your help.
][ Maurice van Swaaij ][
----- Original Message -----
Thanks, I'll try it out with 3.8.1 and let you know if it works.
][ Maurice van Swaaij ][
----- Original Message -----
On Wednesday, January 02, 2013, Maurice van Swaaij wrote:
> Now for my question: is it possible to go beyond the 32G memory allocation
> limit?
The patch below allows it to use up to 64G, that is, it doubles the
available memory. If someone would test it properly, it might get
committed; but so far we have had (AFAIK) no definitive "it works"
feedback.
J
Index: memcheck/mc_main.c
===================================================================
--- memcheck/mc_main.c (revision 13179)
+++ memcheck/mc_main.c (working copy)
@@ -167,10 +167,10 @@
#else
-/* Just handle the first 32G fast and the rest via auxiliary
+/* Just handle the first 64G fast and the rest via auxiliary
primaries. If you change this, Memcheck will assert at startup.
See the definition of UNALIGNED_OR_HIGH for extensive comments. */
-# define N_PRIMARY_BITS 19
+# define N_PRIMARY_BITS 20
#endif
@@ -6524,11 +6524,11 @@
tl_assert(sizeof(Addr) == 8);
tl_assert(sizeof(UWord) == 8);
tl_assert(sizeof(Word) == 8);
- tl_assert(MAX_PRIMARY_ADDRESS == 0x7FFFFFFFFULL);
- tl_assert(MASK(1) == 0xFFFFFFF800000000ULL);
- tl_assert(MASK(2) == 0xFFFFFFF800000001ULL);
- tl_assert(MASK(4) == 0xFFFFFFF800000003ULL);
- tl_assert(MASK(8) == 0xFFFFFFF800000007ULL);
+ tl_assert(MAX_PRIMARY_ADDRESS == 0xFFFFFFFFFULL);
+ tl_assert(MASK(1) == 0xFFFFFFF000000000ULL);
+ tl_assert(MASK(2) == 0xFFFFFFF000000001ULL);
+ tl_assert(MASK(4) == 0xFFFFFFF000000003ULL);
+ tl_assert(MASK(8) == 0xFFFFFFF000000007ULL);
# endif
}
Index: coregrind/m_aspacemgr/aspacemgr-linux.c
===================================================================
--- coregrind/m_aspacemgr/aspacemgr-linux.c (revision 13179)
+++ coregrind/m_aspacemgr/aspacemgr-linux.c (working copy)
@@ -1649,7 +1649,7 @@
aspacem_minAddr = (Addr) 0x04000000; // 64M
# if VG_WORDSIZE == 8
- aspacem_maxAddr = (Addr)0x800000000 - 1; // 32G
+ aspacem_maxAddr = (Addr)0x1000000000ULL - 1; // 64G
# ifdef ENABLE_INNER
{ Addr cse = VG_PGROUNDDN( sp_at_startup ) - 1;
if (aspacem_maxAddr > cse)
------------------------------------------------------------------------------
Master Java SE, Java EE, Eclipse, Spring, Hibernate, JavaScript, jQuery
and much more. Keep your Java skills current with LearnJavaNow -
200+ hours of step-by-step video tutorials by Java experts.
SALE $49.99 this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122612
_______________________________________________
Valgrind-users mailing list
Val...@li...
https://lists.sourceforge.net/lists/listinfo/valgrind-users
|
|
From: Julian S. <js...@ac...> - 2013-01-28 15:24:22
|
On 01/26/2013 12:38 AM, Maurice van Swaaij wrote: > it looks like even 64G is not enough. Is it possible to go even higher or is 64G the limit? > Our biggest machines have 256G, is it possible to go that high? There is no specific limit. If you look at the current patch, you'll find it multiplies a constant by 2 and moves some 1/0 bit boundaries in some constants one bit leftwards. If you construct a new patch which does that once more, you should be able to get to 128GB, and twice more, to 256GB, etc. J |
|
From: Julian S. <js...@ac...> - 2013-01-28 15:21:50
|
On 01/24/2013 07:48 PM, Maurice van Swaaij wrote: > I have run valgrind 3.8.1 with your patch successfully on a job that neared 64G in memory allocation. Thanks for the check. I'll commit the patch on the trunk. J |
|
From: Julian S. <js...@ac...> - 2013-01-30 08:25:58
|
The patch was committed as r13278, so the trunk can by default now do up to 64GB. J On 01/24/2013 07:48 PM, Maurice van Swaaij wrote: > Hi Julian, > > I have run valgrind 3.8.1 with your patch successfully on a job that neared 64G in memory allocation. > [...] |
|
From: John R. <jr...@bi...> - 2013-01-02 22:18:32
|
> Now for my question: is it possible to go beyond the 32G memory allocation limit? Did you try the obvious extension of the patch which enabled enlarging from 16G(user) to 32G(user)? See Julian's post of 12/14/2012: [Valgrind-developers] Cannot create large memory map on 64-bit linux -- |