user-mode-linux-user

Re: [uml-user] impact on erlang?

[Riccardo M. <ric...@gm...>]

Hi Matthias,

On Tue, Dec 13, 2011 at 20:21, Matthias Rieber <sup...@su...> wrote:
> Am 13.12.2011 16:13, schrieb richard -rw- weinberger:
>> I'll try that. What should I grep for in the strace log?
>>
>> For anything which can cause your problem.
>> E.g. Failing system calls which should not fail...
>>
>> We've had problems with java and glibc where glibc used x86 specific
>> futex operations which are not supported on UML...
>>
> I traced a larger part and it seems that erlang causes many futex calls
> on the hostsystem and none on the uml kernel.
>

I think you should run the strace *in* the UML, not on the host.

Best regards,
Riccardo

Re: [uml-user] impact on erlang?

[Matthias R. <sup...@su...>]

Hi,

Am 13.12.2011 16:13, schrieb richard -rw- weinberger:
> I'll try that. What should I grep for in the strace log?
>
> For anything which can cause your problem.
> E.g. Failing system calls which should not fail...
>
> We've had problems with java and glibc where glibc used x86 specific
> futex operations which are not supported on UML...
>
I traced a larger part and it seems that erlang causes many futex calls 
on the hostsystem and none on the uml kernel.

Matthias

Re: [uml-user] impact on erlang?

[Matthias R. <sup...@su...>]

Hi,

On 13.12.2011 16:13, richard -rw- weinberger wrote:
> On Tue, Dec 13, 2011 at 2:55 PM, Matthias Rieber <sup...@su...> wrote:
>>> Maybe Erlang makes use of a system call which is not available on UML...
>>> You can try strace...
>>
>> I'll try that. What should I grep for in the strace log?
>>
> 
> For anything which can cause your problem.
> E.g. Failing system calls which should not fail...
> 
> We've had problems with java and glibc where glibc used x86 specific
> futex operations which are not supported on UML...

running strace (stracer -s 16000 -f -p <erlang process>):

---UML---
{{EPOLLIN, {u32=244, u64=4954239656334983412}}}, 256, 5891) = 1
clock_gettime(CLOCK_MONOTONIC, {452384, 184995846}) = 0
recvfrom(244, "\27\3\1\0
\6\362\333>F\36\273[OL\350{\325\0073\301S^\250\334!3\272<6\t\2505\367>\246\264\27\3\1\0000z\367\217\371\257\323\237\244\237\205&?JR}\224\271\201\355k\341\0163\n\24\6\233\220\t\334\225_q\357\34\246k\375\365\342A\220\307\251\254\2134;",
1460, 0, NULL, NULL) = 90
getpeername(244, {sa_family=AF_INET, sin_port=htons(56873),
sin_addr=inet_addr("<ip>")}, [16]) = 0
clock_gettime(CLOCK_MONOTONIC, {452384, 188543846}) = 0
clock_gettime(CLOCK_MONOTONIC, {452384, 189644846}) = 0
clock_gettime(CLOCK_MONOTONIC, {452384, 190721846}) = 0
clock_gettime(CLOCK_MONOTONIC, {452384, 191717846}) = 0
clock_gettime(CLOCK_MONOTONIC, {452384, 192725846}) = 0
writev(244, [{NULL, 0}, {"\27\3\1\0
f\364^\233\275\316\336@\333\215\20\204\212\353\26\\\311_\5r.J\20\265\305qQ\243a\265\7\336\27\3\1\0000\367c\235=\236\17b\357H\301\36\254B\362~\205\3&\10\261\212y\355\36\324\270\4m\374\245\231\365E>OWC\360\355\334\316\361s\357X\340\3267",
90}], 2) = 90
clock_gettime(CLOCK_MONOTONIC, {452384, 195375846}) = 0
clock_gettime(CLOCK_MONOTONIC, {452384, 196278846}) = 0
epoll_ctl(3, EPOLL_CTL_DEL, 244, {EPOLLIN, {u32=244,
u64=4954239656334983412}}) = 0
epoll_ctl(3, EPOLL_CTL_ADD, 244, {EPOLLIN, {u32=244,
u64=4954239656334983412}}) = 0
epoll_wait(3, {{EPOLLIN, {u32=244, u64=4954239656334983412}}}, 256, 63) = 1
clock_gettime(CLOCK_MONOTONIC, {452384, 204040846}) = 0
recvfrom(244, "\27\3\1\0
9\227\3\240\0009\272\363\233\224\317\244\365\344\207\"~3\t.usg\24ka\254W\313:\301$\27\3\1\0000\242\3316\345\312[j\227\0\32\301w;\34;\210T@\235\202\16\366#\346\3640\261\261\215\223RD\335\373\253'l=\374\266p\201\n\26\31\377!\220",
1460, 0, NULL, NULL) = 90
getpeername(244, {sa_family=AF_INET, sin_port=htons(56873),
sin_addr=inet_addr("<ip>")}, [16]) = 0
clock_gettime(CLOCK_MONOTONIC, {452384, 207843846}) = 0
clock_gettime(CLOCK_MONOTONIC, {452384, 208831846}) = 0
clock_gettime(CLOCK_MONOTONIC, {452384, 209830846}) = 0
clock_gettime(CLOCK_MONOTONIC, {452384, 210909846}) = 0
clock_gettime(CLOCK_MONOTONIC, {452384, 211799846}) = 0
writev(244, [{NULL, 0}, {"\27\3\1\0
\276\214\3162\353\21\367\3567?Hu(`}\36\274\342\345c\351\313\5\3038\302\247X=\23\3137\27\3\1\0000\3406\265k\306\303rA\177\313\nC]o\332\314\212\1M\203\r\5\26\366|\311@9i\\f\267\t\247\344\344\235\5\23\341\5\2411\356Xf\337\331",
90}], 2) = 90
clock_gettime(CLOCK_MONOTONIC, {452384, 214603846}) = 0
clock_gettime(CLOCK_MONOTONIC, {452384, 215550846}) = 0
epoll_ctl(3, EPOLL_CTL_DEL, 244, {EPOLLIN, {u32=244,
u64=4954239656334983412}}) = 0
epoll_ctl(3, EPOLL_CTL_ADD, 244, {EPOLLIN, {u32=244,
u64=4954239656334983412}}) = 0
epoll_wait(3, {}, 256, 44)              = 0
clock_gettime(CLOCK_MONOTONIC, {452384, 243013848}) = 0
clock_gettime(CLOCK_MONOTONIC, {452384, 244243848}) = 0
epoll_wait(3, {}, 256, 15)              = 0
clock_gettime(CLOCK_MONOTONIC, {452384, 266889850}) = 0
clock_gettime(CLOCK_MONOTONIC, {452384, 268038850}) = 0
clock_gettime(CLOCK_MONOTONIC, {452384, 269136850}) = 0
epoll_wait(3, {}, 256, 241)             = 0
clock_gettime(CLOCK_MONOTONIC, {452384, 513838874}) = 0
clock_gettime(CLOCK_MONOTONIC, {452384, 514661874}) = 0
clock_gettime(CLOCK_MONOTONIC, {452384, 515497874}) = 0

---HOST Kernel---
epoll_wait(3, {}, 256, 1)               = 0
epoll_wait(3, {}, 256, 2984)            = 0
epoll_wait(3, {}, 256, 1)               = 0
epoll_wait(3, {{EPOLLIN, {u32=256, u64=256}}}, 256, 9997) = 1
recvfrom(256, "\27\3\1\0 \0241\v\215b\1x\33n?\1\10
\341\37~\333t\31\32pL\vE\16k\225\0108\235\215@\27\3\1\0000\375\275
?\310`p\253\t`cy\2248\271\22\251\33\323\326\372\f[Q\351Ks\213\3052\25\366\177\267b\233'<s\371\341\fI\324>\353t\3",
1460, 0, NULL, NULL) = 90
getpeername(256, {sa_family=AF_INET, sin_port=htons(56910),
sin_addr=inet_addr("<ip>")}, [16]) = 0
epoll_ctl(3, EPOLL_CTL_DEL, 256, {EPOLLIN, {u32=256, u64=256}}) = 0
epoll_ctl(3, EPOLL_CTL_ADD, 256, {EPOLLIN, {u32=256, u64=256}}) = 0
epoll_wait(3, {}, 256, 0)               = 0
epoll_wait(3, {}, 256, 0)               = 0
writev(256, [{NULL, 0}, {"\27\3\1\0
}\267Ug\354\21F\224\362\242\21\271`9\212\327d\257\20\370\306\325\6\16\206\265U\4\374\343u\210\27\3\1\0000\261\222\270g\324\26600&y[\227\5r\302Qw\331\335;d\332\211\235\217\360\376\240\276
$7\352\237\242[\322{\320\352\333\323c\366N\201{\34", 90}], 2) = 90
writev(256, [{NULL, 0}, {"\27\3\1\0
2\340K\220\264<\307\376\246Z\311\254\374
\313\3454m\245\241T\347HM\250\214a\237c\201\326\n\27\3\1\000031`X\0\275G\17%\335\200=\316\272\274\351\276\34\3352\267\316~\355\37\262\240\341uCe\307\337\262\270\260\350]\334\353\307\363\367\302?\255\276\375",
90}], 2) = 90
writev(270, [{NULL, 0}, {"<outgoing jabber messages>", 457}], 2) = 457
writev(274, [{NULL, 0}, {"<outgoing jabber messages>", 467}], 2) = 467
writev(256, [{NULL, 0}, {"\27\3\1\0
!\201}\376\5\246\202\352\26\r%#k\312\342x\251s`\325\0061\3573\321\221\211\10V\370\351T\27\3\1\00006\242\253B\330W
\3225MwZC\t\356\17!\276\231[\270\36\213\347\237\0\206-\373nc\34\20z\316\370\362N\27\2\33g\261\17\302^xa",
90}], 2) = 90
epoll_wait(3, {{EPOLLIN, {u32=256, u64=256}}}, 256, 5936) = 1
recvfrom(256, "\27\3\1\0
\20\7\320sQ\344D\20DAF\265+\211j=\245`GD\342]S\275\252\234{\217\277ao\212\27\3\1\0000\203\367\255q\364\330\266\254\336\356B\10\222\304gxC|\257\304\371l7p\354\33\3122\263SD\2>\200q\252\r\253\3552\2\205\27A~\371\237O",
1460, 0, NULL, NULL) = 90
getpeername(256, {sa_family=AF_INET, sin_port=htons(56910),
sin_addr=inet_addr("<ip>")}, [16]) = 0
epoll_ctl(3, EPOLL_CTL_DEL, 256, {EPOLLIN, {u32=256, u64=256}}) = 0
epoll_ctl(3, EPOLL_CTL_ADD, 256, {EPOLLIN, {u32=256, u64=256}}) = 0
epoll_wait(3, {}, 256, 0)               = 0
mmap(NULL, 4825088, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x7f8bbbc44000
epoll_wait(3, {}, 256, 0)               = 0
writev(256, [{NULL, 0}, {"\27\3\1\0 ^t\16\344P\203\337m\246x\350\314i
E\313q\344j7\363\37K\23\241\200g*~B\273d\27\3\1\0000\246kj\304\21P\253\rK}\22Z\343{1\371\235I2\224JP\336ds\232Nw[M\237\n(\336\251\343\2*\37u\332\312\376R\201=\204z",
90}], 2) = 90
writev(256, [{NULL, 0}, {"\27\3\1\0
\\`$G\241\242/`\315\21L\337\341F\367sZ\342\314\351f\272:\252B\273\246OsH\234\240\27\3\1\0000&mH\3\236v\4\227-|bD=\237\357\306[a+\375\344\3469S\375\375\236x\370\232\255q6\243\265\303\216\237\376\363>O\244R8\5\377^",
90}], 2) = 90
writev(270, [{NULL, 0}, {"<outgoing jabber messages>", 418}], 2) = 418
writev(275, [{NULL, 0}, {"<outgoing jabber messages>", 457}], 2) = 457
writev(275, [{NULL, 0}, {"<outgoing jabber messages>, 458}], 2) = 458
writev(275, [{NULL, 0}, {"<outgoing jabber messages>", 419}], 2) = 419
writev(274, [{NULL, 0}, {"<outgoing jabber messages>", 428}], 2) = 428
epoll_wait(3, {}, 256, 0)               = 0
writev(256, [{NULL, 0}, {"\27\3\1\0
;\24^\5\246\367Q\256\337\225D%2\372B\247\271\fb=\333\312\0\254\205\264
\n\4P\10,\27\3\1\0000\350\236'\23\266\313KN\241]\25\5j\332-u\312X\fS\355o\216\356W\203b\222\20nc\2008\2735\236B|J\0278\22\264\235\n\250\207\r",
90}], 2) = 90

The only big difference I can see are the many
clock_gettime(CLOCK_MONOTONIC, ...).

Regards,
Matthias

Re: [uml-user] impact on erlang?

[richard -r. w. <ric...@gm...>]

On Tue, Dec 13, 2011 at 2:55 PM, Matthias Rieber <sup...@su...> wrote:
> Hello,
>
> On 13.12.2011 13:54, richard -rw- weinberger wrote:
>> On Tue, Dec 13, 2011 at 1:10 PM, Matthias Rieber <sup...@su...> wrote:
>>> Hi,
>>>
>>> I've a strange problem with ejabberd(a jabber server written in Erlang)
>>> running under the user mode linux kernel. Almost everything works as
>>> expected but a certain function, pubsub/pep, doesn't work. When I run
>>> the very same installation in a chroot environment on the host, it
>>> behaves correctly. The UML Kernel is a vanilla 2.6.32.49. I've tried
>>> different systems 32bit/64bit, they show the same behaviour.
>>
>> Please define "doesn't work".
>
> hard to describe. The jabber server receives a xml stanza, it returns a
> confirmation, that it has been processed correctly. The stanza should
> lead to an information storage in the mnesia database, which didn't
> happen, when I run ejabberd in UML.
>
>>> Due to the lack of any Erlang knowledge I can't check why the jabber
>>> server doesn't work. Are there know issues with Erlang? Any ideas to
>>> debug it without digging in the server?
>>
>> Maybe Erlang makes use of a system call which is not available on UML...
>> You can try strace...
>
> I'll try that. What should I grep for in the strace log?
>

For anything which can cause your problem.
E.g. Failing system calls which should not fail...

We've had problems with java and glibc where glibc used x86 specific
futex operations which are not supported on UML...

-- 
Thanks,
//richard

Re: [uml-user] impact on erlang?

[Matthias R. <sup...@su...>]

Hello,

On 13.12.2011 13:54, richard -rw- weinberger wrote:
> On Tue, Dec 13, 2011 at 1:10 PM, Matthias Rieber <sup...@su...> wrote:
>> Hi,
>>
>> I've a strange problem with ejabberd(a jabber server written in Erlang)
>> running under the user mode linux kernel. Almost everything works as
>> expected but a certain function, pubsub/pep, doesn't work. When I run
>> the very same installation in a chroot environment on the host, it
>> behaves correctly. The UML Kernel is a vanilla 2.6.32.49. I've tried
>> different systems 32bit/64bit, they show the same behaviour.
> 
> Please define "doesn't work".

hard to describe. The jabber server receives a xml stanza, it returns a
confirmation, that it has been processed correctly. The stanza should
lead to an information storage in the mnesia database, which didn't
happen, when I run ejabberd in UML.

>> Due to the lack of any Erlang knowledge I can't check why the jabber
>> server doesn't work. Are there know issues with Erlang? Any ideas to
>> debug it without digging in the server?
> 
> Maybe Erlang makes use of a system call which is not available on UML...
> You can try strace...

I'll try that. What should I grep for in the strace log?

Matthias

Re: [uml-user] impact on erlang?

[richard -r. w. <ric...@gm...>]

On Tue, Dec 13, 2011 at 1:10 PM, Matthias Rieber <sup...@su...> wrote:
> Hi,
>
> I've a strange problem with ejabberd(a jabber server written in Erlang)
> running under the user mode linux kernel. Almost everything works as
> expected but a certain function, pubsub/pep, doesn't work. When I run
> the very same installation in a chroot environment on the host, it
> behaves correctly. The UML Kernel is a vanilla 2.6.32.49. I've tried
> different systems 32bit/64bit, they show the same behaviour.

Please define "doesn't work".

> Due to the lack of any Erlang knowledge I can't check why the jabber
> server doesn't work. Are there know issues with Erlang? Any ideas to
> debug it without digging in the server?

Maybe Erlang makes use of a system call which is not available on UML...
You can try strace...

-- 
Thanks,
//richard

[uml-user] impact on erlang?

[Matthias R. <sup...@su...>]

Hi,

I've a strange problem with ejabberd(a jabber server written in Erlang)
running under the user mode linux kernel. Almost everything works as
expected but a certain function, pubsub/pep, doesn't work. When I run
the very same installation in a chroot environment on the host, it
behaves correctly. The UML Kernel is a vanilla 2.6.32.49. I've tried
different systems 32bit/64bit, they show the same behaviour.

Due to the lack of any Erlang knowledge I can't check why the jabber
server doesn't work. Are there know issues with Erlang? Any ideas to
debug it without digging in the server?

Regards,
Matthias

Re: [uml-user] The max instance of uml is limited by memory and disk size ?

[<cl...@cl...>]

I do not know exactly how much shm is taken in that case, just try
to launch the uml it an see the percent taken from shm.
UML has to have shm, this is all I know.


> cl...@cl... wrote:
>>
>> Hello,
>> For UML machines, you have to look carefully at the /dev/shm value with
>> df, if the used up /dev/shm gets too high, new UML cannot be launched.
>
> Excuse a newbie question, but if I understand correctly your statement,
> then the following:
>
> host@:~# df -h
> Filesystem      Size  Used Avail Use% Mounted on
> /dev/root       183G   15G  159G   9% /
> tmpfs           119M     0  119M   0% /dev/shm
>
> implies that the maximum memory a single UML instance can have is 119MB?
> As in:
>
> ./linux ubda=root_fs mem=119MB
>
> Thank you for your time,
> -Stathis
>
>>
>> To look at the value you have:
>>   df -h /dev/shm/
>>
>> To change it to 2 giga:
>>   mount -t tmpfs shmfs -o size=2g /dev/shm
>>
>> Look again:
>>    df -h /dev/shm/
>>
>>
>>
>>
>>
>>>
>>> Hi all,
>>> when i run 60 umls in the same host with "mem=128M""free -m" says
>>> almost
>>> no free memory left.
>>> if i run more, all umls got killed.
>>> Now if i run uml with "mem=64M", was trying to run 100 umls, but still
>>> got
>>> terminated.Why ?
>>> (our goal is to run 200 umls in each host, 4GB memory each host.)(Also
>>> disk size is big enough, each rootfs is 0.1 G, 100 of umls is 10G)
>>> Thanks !
>>>
>>>  		 	   		  ------------------------------------------------------------------------------
>>> Learn Windows Azure Live!  Tuesday, Dec 13, 2011
>>> Microsoft is holding a special Learn Windows Azure training event for
>>> developers. It will provide a great way to learn Windows Azure and what
>>> it
>>> provides. You can attend the event by watching it streamed LIVE online.
>>> Learn more at
>>> http://p.sf.net/sfu/ms-windowsazure_______________________________________________
>>> User-mode-linux-user mailing list
>>> Use...@li...
>>> https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Learn Windows Azure Live!  Tuesday, Dec 13, 2011
>> Microsoft is holding a special Learn Windows Azure training event for
>> developers. It will provide a great way to learn Windows Azure and what
>> it
>> provides. You can attend the event by watching it streamed LIVE online.
>> Learn more at http://p.sf.net/sfu/ms-windowsazure
>> _______________________________________________
>> User-mode-linux-user mailing list
>> Use...@li...
>> https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user
>>
>
>
>
> ------------------------------------------------------------------------------
> Learn Windows Azure Live!  Tuesday, Dec 13, 2011
> Microsoft is holding a special Learn Windows Azure training event for
> developers. It will provide a great way to learn Windows Azure and what it
> provides. You can attend the event by watching it streamed LIVE online.
> Learn more at http://p.sf.net/sfu/ms-windowsazure
> _______________________________________________
> User-mode-linux-user mailing list
> Use...@li...
> https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user
>

Re: [uml-user] The max instance of uml is limited by memory and disk size ?

[<cl...@cl...>]

I do not remember when I first had this problem, but I made cloonix, a
virtual network and I have always tried to have many machines in one PC,
uml, kvm or lxc machines in a cloonix network.

The uml need shared memory because the processes inside a vm are
processes of the host and they have to "see" the same memory, this
shared memory is the virtual machine's, I think.

If you set mem=500 or mem=32 it will probably take the same amount
of memory in both cases (which is just the minimum needed to run).

Try big mallocs inside your uml to see where it is taken, I suppose
it will come from shared memory.


>
>
> Thanks for the info, that's a great help.
> BTW,
> 1. when i change "mem=128M" to "mem=64M", seems it didn't help, still uses
> the same amount of /dev/shm.    why ?
> 2. Where did you learn the /dev/shm info for uml ?
> Have a nice day !
>
>> Date: Sun, 11 Dec 2011 03:20:44 -0600
>> Subject: Re: [uml-user] The max instance of uml is limited by memory and
>> disk size ?
>> From: cl...@cl...
>> To: ni...@ho...
>> CC: use...@li...
>>
>>
>> Hello,
>> For UML machines, you have to look carefully at the /dev/shm value with
>> df, if the used up /dev/shm gets too high, new UML cannot be launched.
>>
>> To look at the value you have:
>>   df -h /dev/shm/
>>
>> To change it to 2 giga:
>>   mount -t tmpfs shmfs -o size=2g /dev/shm
>>
>> Look again:
>>    df -h /dev/shm/
>>
>>
>>
>>
>>
>> >
>> > Hi all,
>> > when i run 60 umls in the same host with "mem=128M""free -m" says
>> almost
>> > no free memory left.
>> > if i run more, all umls got killed.
>> > Now if i run uml with "mem=64M", was trying to run 100 umls, but still
>> got
>> > terminated.Why ?
>> > (our goal is to run 200 umls in each host, 4GB memory each host.)(Also
>> > disk size is big enough, each rootfs is 0.1 G, 100 of umls is 10G)
>> > Thanks !
>> >
>> >  		 	   		  ------------------------------------------------------------------------------
>> > Learn Windows Azure Live!  Tuesday, Dec 13, 2011
>> > Microsoft is holding a special Learn Windows Azure training event for
>> > developers. It will provide a great way to learn Windows Azure and
>> what it
>> > provides. You can attend the event by watching it streamed LIVE
>> online.
>> > Learn more at
>> > http://p.sf.net/sfu/ms-windowsazure_______________________________________________
>> > User-mode-linux-user mailing list
>> > Use...@li...
>> > https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user
>> >
>>
>

Re: [uml-user] The max instance of uml is limited by memory and disk size ?

[ning ji <ni...@ho...>]

Thanks for the info, that's a great help.
BTW,
1. when i change "mem=128M" to "mem=64M", seems it didn't help, still uses the same amount of /dev/shm.    why ?
2. Where did you learn the /dev/shm info for uml ? 
Have a nice day !

> Date: Sun, 11 Dec 2011 03:20:44 -0600
> Subject: Re: [uml-user] The max instance of uml is limited by memory and disk size ?
> From: cl...@cl...
> To: ni...@ho...
> CC: use...@li...
> 
> 
> Hello,
> For UML machines, you have to look carefully at the /dev/shm value with
> df, if the used up /dev/shm gets too high, new UML cannot be launched.
> 
> To look at the value you have:
>   df -h /dev/shm/
> 
> To change it to 2 giga:
>   mount -t tmpfs shmfs -o size=2g /dev/shm
> 
> Look again:
>    df -h /dev/shm/
> 
> 
> 
> 
> 
> >
> > Hi all,
> > when i run 60 umls in the same host with "mem=128M""free -m" says almost
> > no free memory left.
> > if i run more, all umls got killed.
> > Now if i run uml with "mem=64M", was trying to run 100 umls, but still got
> > terminated.Why ?
> > (our goal is to run 200 umls in each host, 4GB memory each host.)(Also
> > disk size is big enough, each rootfs is 0.1 G, 100 of umls is 10G)
> > Thanks !
> >
> >  		 	   		  ------------------------------------------------------------------------------
> > Learn Windows Azure Live!  Tuesday, Dec 13, 2011
> > Microsoft is holding a special Learn Windows Azure training event for
> > developers. It will provide a great way to learn Windows Azure and what it
> > provides. You can attend the event by watching it streamed LIVE online.
> > Learn more at
> > http://p.sf.net/sfu/ms-windowsazure_______________________________________________
> > User-mode-linux-user mailing list
> > Use...@li...
> > https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user
> >
> 
 		 	   		  

Re: [uml-user] The max instance of uml is limited by memory and disk size ?

[<ro...@mm...>]

cl...@cl... wrote:
>
> Hello,
> For UML machines, you have to look carefully at the /dev/shm value with
> df, if the used up /dev/shm gets too high, new UML cannot be launched.

Excuse a newbie question, but if I understand correctly your statement,
then the following:

host@:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root       183G   15G  159G   9% /
tmpfs           119M     0  119M   0% /dev/shm

implies that the maximum memory a single UML instance can have is 119MB?
As in:

./linux ubda=root_fs mem=119MB

Thank you for your time,
-Stathis

>
> To look at the value you have:
>   df -h /dev/shm/
>
> To change it to 2 giga:
>   mount -t tmpfs shmfs -o size=2g /dev/shm
>
> Look again:
>    df -h /dev/shm/
>
>
>
>
>
>>
>> Hi all,
>> when i run 60 umls in the same host with "mem=128M""free -m" says almost
>> no free memory left.
>> if i run more, all umls got killed.
>> Now if i run uml with "mem=64M", was trying to run 100 umls, but still
>> got
>> terminated.Why ?
>> (our goal is to run 200 umls in each host, 4GB memory each host.)(Also
>> disk size is big enough, each rootfs is 0.1 G, 100 of umls is 10G)
>> Thanks !
>>
>>  		 	   		  ------------------------------------------------------------------------------
>> Learn Windows Azure Live!  Tuesday, Dec 13, 2011
>> Microsoft is holding a special Learn Windows Azure training event for
>> developers. It will provide a great way to learn Windows Azure and what
>> it
>> provides. You can attend the event by watching it streamed LIVE online.
>> Learn more at
>> http://p.sf.net/sfu/ms-windowsazure_______________________________________________
>> User-mode-linux-user mailing list
>> Use...@li...
>> https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user
>>
>
>
> ------------------------------------------------------------------------------
> Learn Windows Azure Live!  Tuesday, Dec 13, 2011
> Microsoft is holding a special Learn Windows Azure training event for
> developers. It will provide a great way to learn Windows Azure and what it
> provides. You can attend the event by watching it streamed LIVE online.
> Learn more at http://p.sf.net/sfu/ms-windowsazure
> _______________________________________________
> User-mode-linux-user mailing list
> Use...@li...
> https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user
>

Re: [uml-user] The max instance of uml is limited by memory and disk size ?

[<cl...@cl...>]

Hello,
For UML machines, you have to look carefully at the /dev/shm value with
df, if the used up /dev/shm gets too high, new UML cannot be launched.

To look at the value you have:
  df -h /dev/shm/

To change it to 2 giga:
  mount -t tmpfs shmfs -o size=2g /dev/shm

Look again:
   df -h /dev/shm/





>
> Hi all,
> when i run 60 umls in the same host with "mem=128M""free -m" says almost
> no free memory left.
> if i run more, all umls got killed.
> Now if i run uml with "mem=64M", was trying to run 100 umls, but still got
> terminated.Why ?
> (our goal is to run 200 umls in each host, 4GB memory each host.)(Also
> disk size is big enough, each rootfs is 0.1 G, 100 of umls is 10G)
> Thanks !
>
>  		 	   		  ------------------------------------------------------------------------------
> Learn Windows Azure Live!  Tuesday, Dec 13, 2011
> Microsoft is holding a special Learn Windows Azure training event for
> developers. It will provide a great way to learn Windows Azure and what it
> provides. You can attend the event by watching it streamed LIVE online.
> Learn more at
> http://p.sf.net/sfu/ms-windowsazure_______________________________________________
> User-mode-linux-user mailing list
> Use...@li...
> https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user
>

[uml-user] The max instance of uml is limited by memory and disk size ?

[ning ji <ni...@ho...>]

Hi all,
when i run 60 umls in the same host with "mem=128M""free -m" says almost no free memory left.
if i run more, all umls got killed.
Now if i run uml with "mem=64M", was trying to run 100 umls, but still got terminated.Why ?
(our goal is to run 200 umls in each host, 4GB memory each host.)(Also disk size is big enough, each rootfs is 0.1 G, 100 of umls is 10G)
Thanks !

 		 	   		  

Re: [uml-user] slow ubd performance

[richard -r. w. <ric...@gm...>]

On Thu, Dec 8, 2011 at 12:38 PM, Riccardo Murri
<ric...@gm...> wrote:
> Hi *,
>
> the file-backed `ubd` devices in my UML instances are terribly slow,
> and generate a lot of "iowait" load on the host.  It looks like as if
> filesystems mounted from `ubd` devices were using the `sync` option.
>
> This happens with kernels downloaded from uml.devloop.org.uk, as well
> with kernels that I compiled myself (but using basically the same
> config as the devloop ones); I tried 2.6.38.x and 2.6.39.x (for some
> random values of "x").
>
> This does *not* happen with Debian/Ubuntu's kernel 2.6.32.5, which
> unfortunately exhibit a filesystem corruption bug.
>
> The host is in any case Ubuntu "natty", running 2.6.39-3-generic.
>
> Is this expected?

Not really ;)

The ubd got only a few patches since 2.6.32, bisecting should be an easy task.
Commit 4752690 fixes the data corruption issue.

[rw@inhelltoy tip]$ git log --oneline v2.6.32.. arch/um/drivers/ubd_*
8535639 um: fix ubd cow size
c506c0e um: take ubd_user.h to its users...
0a9e70b um: kill shared/mem_kern.h
dbddc51 um: don't include kern.h unless it's needed
078073a um: -include user.h for USER_OBJ, trim includes
22e6500 um: Replace deprecated spinlock initialization
5704e44 Merge branch 'config' of
git://git.kernel.org/pub/scm/linux/kernel/git/arnd/bkl
9a181c5 uml: kill big kernel lock
4752690 ubd: fix incorrect sector handling during request restart
6e9624b block: push down BKL into .open and .release
5a0e3ad include cleanup: Update gfp.h and slab.h includes to prepare
for breaking implicit slab.h inclusion from percpu.h
8a78362 block: Consolidate phys_segment and hw_segment limits
086fa5f block: Rename blk_queue_max_sectors to blk_queue_max_hw_sectors
6613c5e uml: convert to seq_file/proc_fops

-- 
Thanks,
//richard

[uml-user] slow ubd performance

[Riccardo M. <ric...@gm...>]

Hi *,

the file-backed `ubd` devices in my UML instances are terribly slow,
and generate a lot of "iowait" load on the host.  It looks like as if
filesystems mounted from `ubd` devices were using the `sync` option.

This happens with kernels downloaded from uml.devloop.org.uk, as well
with kernels that I compiled myself (but using basically the same
config as the devloop ones); I tried 2.6.38.x and 2.6.39.x (for some
random values of "x").

This does *not* happen with Debian/Ubuntu's kernel 2.6.32.5, which
unfortunately exhibit a filesystem corruption bug.

The host is in any case Ubuntu "natty", running 2.6.39-3-generic.

Is this expected?

Thanks,
Riccardo

Re: [uml-user] UML, threads and memory usage

[<cl...@cl...>]

Sorry! I was referring to the SKAS3/4, I forgot the number.
So are the SKAS3/4 still alive ?
I have never tried them, I was waiting for them to get into the mainline.
The normal uml machine is already very good, but if it can be even better,
it would be good news.


> On Tue, Dec 6, 2011 at 11:40 PM,  <cl...@cl...> wrote:
>>
>> I did not read everything carefully but I thought that with the skas0,
>> I would see only one uml linux process in the host for the whole uml
>> machine, and also that it would not take the /dev/shm memory anymore.
>
> No.
> Maybe your are referring to SKAS3/4?
>
>> Sometimes, with some weak PC or a bad /dev/shm config, if you put too
>> many
>> machines, the /dev/shm memory reaches its limit and new machines crash.
>>
>> Do I have the skas0 with mainline kernel 3.1.1 and without any more
>> options at uml launch than before, I mean is it the default value?
>>
>
> SKAS0 is default.
>
> --
> Thanks,
> //richard
>

Re: [uml-user] UML, threads and memory usage

[<cl...@cl...>]

I did not read everything carefully but I thought that with the skas0,
I would see only one uml linux process in the host for the whole uml
machine, and also that it would not take the /dev/shm memory anymore.

Sometimes, with some weak PC or a bad /dev/shm config, if you put too many
machines, the /dev/shm memory reaches its limit and new machines crash.

Do I have the skas0 with mainline kernel 3.1.1 and without any more
options at uml launch than before, I mean is it the default value?






> On Tue, Dec 6, 2011 at 10:31 PM,  <cl...@cl...> wrote:
>>
>> Is there still a chance that the skas0 patch will end up in the
>> mainline?
>>
>
> It is already in mainline.
>
> --
> Thanks,
> //richard
>

Re: [uml-user] UML, threads and memory usage

[richard -r. w. <ric...@gm...>]

On Tue, Dec 6, 2011 at 11:40 PM,  <cl...@cl...> wrote:
>
> I did not read everything carefully but I thought that with the skas0,
> I would see only one uml linux process in the host for the whole uml
> machine, and also that it would not take the /dev/shm memory anymore.

No.
Maybe your are referring to SKAS3/4?

> Sometimes, with some weak PC or a bad /dev/shm config, if you put too many
> machines, the /dev/shm memory reaches its limit and new machines crash.
>
> Do I have the skas0 with mainline kernel 3.1.1 and without any more
> options at uml launch than before, I mean is it the default value?
>

SKAS0 is default.

-- 
Thanks,
//richard

Re: [uml-user] UML, threads and memory usage

[Riccardo M. <ric...@gm...>]

Hi Jeff, Richard,

many thanks for your explanations!  I think I got it now...

One more question:

On Tue, Dec 6, 2011 at 21:49, Jeff Dike <jd...@ad...> wrote:
> On Tue, Dec 06, 2011 at 07:48:40PM +0100, Riccardo Murri wrote:
>> In addition, *every* syscall generates a SIGTRAP to the UML kernel
>> process, which handles it.  The advantage of SKAS0 over TT is that
>> memory management syscalls allow the separation of kernel and process
>> address space, but every other syscall needs to be handled exactly as
>> in TT: e.g., open() needs to map paths using the UML filesystem, etc.
>> Right?
>
> A little off the rails here - in TT mode, there is one address space
> in which userspace runs, on every context switch, that address space
> needs to be completely remapped in order to become the memory of the
> switched-in process.
>

Does this mean that in TT mode all UML "guest processes" are really
threads of a single host process?  i.e., they are created with
clone(CLONE_VM|...) so they literally share any single page of memory?

(So the it's the job of the UML kernel to mprotect() all the pages
upon every in-UML context switch?)

Best regards,
Riccardo

Re: [uml-user] UML, threads and memory usage

[richard -r. w. <ric...@gm...>]

On Tue, Dec 6, 2011 at 10:31 PM,  <cl...@cl...> wrote:
>
> Is there still a chance that the skas0 patch will end up in the mainline?
>

It is already in mainline.

-- 
Thanks,
//richard

Re: [uml-user] UML, threads and memory usage

[<cl...@cl...>]

Is there still a chance that the skas0 patch will end up in the mainline?




> On Tue, Dec 06, 2011 at 07:48:40PM +0100, Riccardo Murri wrote:
>> Sorry again for resurrecting an old thread, but I each time I look
>> into this issue I realize that I haven't quite understood the details...
>
> You basically have it all right.
>
>> In addition, *every* syscall generates a SIGTRAP to the UML kernel
>> process, which handles it.  The advantage of SKAS0 over TT is that
>> memory management syscalls allow the separation of kernel and process
>> address space, but every other syscall needs to be handled exactly as
>> in TT: e.g., open() needs to map paths using the UML filesystem, etc.
>> Right?
>
> A little off the rails here - in TT mode, there is one address space
> in which userspace runs, on every context switch, that address space
> needs to be completely remapped in order to become the memory of the
> switched-in process.
>
> In SKAS, every UML process has a host address space, and UML process
> context switching is done by the host, at hardware speed.
>
>> Now a final question: according to the above `ps` output, the shared
>> memory among UML processes is ~13GB each.  If the above is correct,
>> only the UML kernel process should have large shared memory.  Is this
>> due to `top` misreporting shared memory occupation? (CentOS 5.x w/
>> stock kernel) Or could it be rather a feature of the program that was
>> running in the UML? (a data-intensive scientific application)
>
> I can't find any documentation of the exact meaning of SHR, but I'd
> guess that it's looking at MAP_SHARED pages, which for a UML process,
> is everything.  No utilities are good at accounting for shared
> memory.  If you just add up the numbers, you end up far away from
> reality.
>
> 				Jeff
>
> ------------------------------------------------------------------------------
> Cloud Services Checklist: Pricing and Packaging Optimization
> This white paper is intended to serve as a reference, checklist and point
> of
> discussion for anyone considering optimizing the pricing and packaging
> model
> of a cloud services business. Read Now!
> http://www.accelacomm.com/jaw/sfnl/114/51491232/
> _______________________________________________
> User-mode-linux-user mailing list
> Use...@li...
> https://lists.sourceforge.net/lists/listinfo/user-mode-linux-user
>

Re: [uml-user] UML, threads and memory usage

[Jeff D. <jd...@ad...>]

On Tue, Dec 06, 2011 at 07:48:40PM +0100, Riccardo Murri wrote:
> Sorry again for resurrecting an old thread, but I each time I look
> into this issue I realize that I haven't quite understood the details...

You basically have it all right.

> In addition, *every* syscall generates a SIGTRAP to the UML kernel
> process, which handles it.  The advantage of SKAS0 over TT is that
> memory management syscalls allow the separation of kernel and process
> address space, but every other syscall needs to be handled exactly as
> in TT: e.g., open() needs to map paths using the UML filesystem, etc.
> Right?

A little off the rails here - in TT mode, there is one address space
in which userspace runs, on every context switch, that address space
needs to be completely remapped in order to become the memory of the
switched-in process.

In SKAS, every UML process has a host address space, and UML process
context switching is done by the host, at hardware speed.

> Now a final question: according to the above `ps` output, the shared
> memory among UML processes is ~13GB each.  If the above is correct,
> only the UML kernel process should have large shared memory.  Is this
> due to `top` misreporting shared memory occupation? (CentOS 5.x w/
> stock kernel) Or could it be rather a feature of the program that was
> running in the UML? (a data-intensive scientific application)

I can't find any documentation of the exact meaning of SHR, but I'd
guess that it's looking at MAP_SHARED pages, which for a UML process,
is everything.  No utilities are good at accounting for shared
memory.  If you just add up the numbers, you end up far away from
reality.

				Jeff

Re: [uml-user] UML, threads and memory usage

[richard -r. w. <ric...@gm...>]

Hi,

On Tue, Dec 6, 2011 at 7:48 PM, Riccardo Murri <ric...@gm...> wrote:
>
> If I got it right:
>
>  - The UML kernel runs in its own process (hence kernel space
>    separation, enforced by the host kernel), which is the parent of
>    all the UML processes (one per guest process).

The separation is enforced by memory mappings and mprotect().
A strict separation via processes would make UML very slow.
(Yes, even more slower :P)

>  - The UML kernel process ptrace()'s its child processes. (just like
>    in TT mode, right?)

Yep.
UML is a system call emulator.
Thus, to the guest processes UML looks like a real kernel...

>  - Two extra memory pages are allocated per child process, which
>    are to communicate with the kernel process during syscalls.

Yes.
This is where the black magic happens.
UML installs hooks into the guest processes such that they cannot
remove or modify
memory mappings.

> Since these pages need to be shared among two host processes (the
> UML kernel and its child), they are allocated through mmap()
> backed by a temporary file.
>
> Actually, I guess that the whole UML memory is allocated as mmap()'ed
> pages from a temporary file: the UML kernel creates a file the size of the
> requested memory, and when it has to satisfy a memory allocation it
> just mmap()'s a page from that file.  Correct?

Correct. Using this technique the kernel is able to share only some parts with
other guest processes.

> In addition, *every* syscall generates a SIGTRAP to the UML kernel
> process, which handles it.  The advantage of SKAS0 over TT is that
> memory management syscalls allow the separation of kernel and process
> address space, but every other syscall needs to be handled exactly as
> in TT: e.g., open() needs to map paths using the UML filesystem, etc.
> Right?

Correct.
As I said, UML is a system call emulator. It uses ptrace() to get notified
of every executed system call and emulates it.
IOW UML is a ptrace() based Linux sandbox...

> Now a final question: according to the above `ps` output, the shared
> memory among UML processes is ~13GB each.  If the above is correct,
> only the UML kernel process should have large shared memory.  Is this
> due to `top` misreporting shared memory occupation? (CentOS 5.x w/
> stock kernel) Or could it be rather a feature of the program that was
> running in the UML? (a data-intensive scientific application)

If your UML instance has 512MiB all UML processes (kernel and guest) use
together 512MiB.
For tools like "top" it looks like as each process would use 512MiB.
"top" cannot know that these processes are threads (constructed using
clone()) and share all memory.
This would only work if UML would use pthreads.
Using clone() you can create nearly any kind of (unportable) threads.

I don't know whether it's possible to implement SKAS0 using pthreads.

-- 
Thanks,
//richard

Re: [uml-user] UML, threads and memory usage

[Riccardo M. <ric...@gm...>]

Hello,

Sorry again for resurrecting an old thread, but I each time I look
into this issue I realize that I haven't quite understood the details...

On Thu, Oct 13, 2011 at 03:35, Jeff Dike <jd...@ad...> wrote:
> On Thu, Oct 13, 2011 at 01:37:24AM +0200, Riccardo Murri wrote:
>> - however, UML "threads" do share a large portion of the memory, as
>>   can be seen from this "ps" output:
>>
>>       PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>>       6467 admin     15   0 32.0g  13g  13g S  0.0 27.7   0:00.00 kernel64-3.0.4
>>       6466 admin     16   0 32.0g  13g  13g S  0.0 27.7   0:00.15 kernel64-3.0.4
>>       6465 admin     22   0 32.0g  13g  13g S  0.0 27.7   0:00.00 kernel64-3.0.4
>>       6458 admin     15   0 32.0g  13g  13g S 39.2 27.7  37:00.04 kernel64-3.0.4
>>       7437 admin     15   0 12.0g  12g  12g T 52.9 25.6  70:54.39 kernel64-3.0.4
>
> UML uses separate address spaces for its processes, thus
> they don't look like threads to anything else, but the bulk of the
> memory (the UML kernel) in those address spaces is shared.
>

I couldn't find many explanations of the SKAS0 mode, and the source
code needs to much kernel-fu for me, so I'm trying to understand it
from Jeff Dike's original announcement:



If I got it right:

  - The UML kernel runs in its own process (hence kernel space
    separation, enforced by the host kernel), which is the parent of
    all the UML processes (one per guest process).

  - The UML kernel process ptrace()'s its child processes. (just like
    in TT mode, right?)

  - Two extra memory pages are allocated per child process, which
    are to communicate with the kernel process during syscalls.

Since these pages need to be shared among two host processes (the
UML kernel and its child), they are allocated through mmap()
backed by a temporary file.

Actually, I guess that the whole UML memory is allocated as mmap()'ed
pages from a temporary file: the UML kernel creates a file the size of the
requested memory, and when it has to satisfy a memory allocation it
just mmap()'s a page from that file.  Correct?

In addition, *every* syscall generates a SIGTRAP to the UML kernel
process, which handles it.  The advantage of SKAS0 over TT is that
memory management syscalls allow the separation of kernel and process
address space, but every other syscall needs to be handled exactly as
in TT: e.g., open() needs to map paths using the UML filesystem, etc.
Right?

Now a final question: according to the above `ps` output, the shared
memory among UML processes is ~13GB each.  If the above is correct,
only the UML kernel process should have large shared memory.  Is this
due to `top` misreporting shared memory occupation? (CentOS 5.x w/
stock kernel) Or could it be rather a feature of the program that was
running in the UML? (a data-intensive scientific application)

Thank you very much for any explanation.

Best regards,
Riccardo

Re: [uml-user] sparse files as root images

[<ro...@mm...>]

richard -rw- weinberger wrote:
> Hi,
>
> On Mon, Dec 5, 2011 at 7:20 PM,  <ro...@mm...> wrote:
>> dd if=/dev/zero of=$OUTFS bs=1024 count=1 seek=$((20*1024*1024-1))
>
> Does it work if the filesystem is not sparse?

After all, it doesn't seem this to be the problem.

Today I started from scratch in a different machine having:
OS: Slackware.12.2
RAM: 512MB

I compiled kernel.3.1.2 for ARCH=um, built a 5GB sparse file as an image
and a 1GB non-sparse for swap.
Installed Slackware.12.2 into the 5GB sparse file.
Executed 'make modules_install INSTALL_MOD_PATH=/mnt/sparse-file ARCH=um'.
Installed fstab thus:
----
/dev/ubdb        swap             swap        defaults         0   0
/dev/ubda        /                ext3        defaults         1   1
#/dev/cdrom      /mnt/cdrom       auto        noauto,owner,ro  0   0
#/dev/fd0         /mnt/floppy      auto        noauto,owner     0   0
devpts           /dev/pts         devpts      gid=5,mode=620   0   0
proc             /proc            proc        defaults         0   0
tmpfs            /dev/shm         tmpfs       defaults         0   0
---
and fired UML with:

./linux ubda=uml_fs ubdb=uml_fs_swap mem=xxxM

trying 64, 128, 384, 448 and 512 as values for mem.

No problems whatsoever. I even started mysql and played around for a while.

I realize this is a different configuration than the one I was having
issues with. Is it possible though that my fstab is somehow not 'correct'?

Anyway, I won't have access to the machine that gave troubles until
Monday, so I can't be more specific.

Thank you for your interest,
-Stathis

>
>> but I'm having kernel panics (not syncing) whenever I try to run various
>> commands inside UML.
>
> Please post the complete panic.
>
>> So far I got kernel panics with top and updatedb.
>>
>> Please consider that I am able to boot into UML. The trouble begins when
>> I'm trying to actually use the beast.
>>
>> Background:
>> kernel.3.1.4, built with defconfig ARCH=um
>> host OS = Slackware.13.37
>> Guest OS = Slackware.13.37
>> Host RAM=256MB
>> Guest RAM= mem=128M
>
> What is the exact command line?
>
> --
> Thanks,
> //richard
>