Linux Containers

Dear Patrick,

As I understand /dev/null isn't writable in your container. That's definitely a wrong configuration.

Please check, that there is a real device node for  /dev/null  (and others) in your container and you have it (and others) in the lxc device access control list (lxc.cgroup.devices.allow = c 1:3 rw)

Note that -- depending on the linux flavor in your LXC container -- you might have to populate /dev by your own, because it's not reasonable to run udev or something like this inside a container.

Greetings

Guido

There is indeed a node at /dev/null.  The configuration that I base my containers off of is as follows:

--------------------------------------------------------------------------------------------

lxc.tty = 4
lxc.pts = 1024
lxc.rootfs = /lxc/debian_squeeze_template/rootfs
lxc.cgroup.devices.deny = a
lxc.network.type = veth
lxc.network.link = br0
#lxc.network.veth.pair = 
lxc.network.ipv4 = 192.168.80.100
# /dev/null and zero
lxc.cgroup.devices.allow = c 1:3 rwm
lxc.cgroup.devices.allow = c 1:5 rwm
# consoles
lxc.cgroup.devices.allow = c 5:1 rwm
lxc.cgroup.devices.allow = c 5:0 rwm
lxc.cgroup.devices.allow = c 4:0 rwm
lxc.cgroup.devices.allow = c 4:1 rwm
# /dev/{,u}random
lxc.cgroup.devices.allow = c 1:9 rwm
lxc.cgroup.devices.allow = c 1:8 rwm
lxc.cgroup.devices.allow = c 136:* rwm
lxc.cgroup.devices.allow = c 5:2 rwm
# rtc
lxc.cgroup.devices.allow = c 254:0 rwm

# mounts point
lxc.mount.entry=proc /lxc/debian_squeeze_template//rootfs/proc proc nodev,noexec,nosuid 0 0
lxc.mount.entry=sysfs /lxc/debian_squeeze_template//rootfs/sys sysfs defaults  0 0

-------------------------------------------------------------------------------------------------------

Everything stays the same from container to container except for the IP and the rootfs.  I think I have it set up right?  Am I missing it?

Pat



----- Original Message -----
From: "Guido Jäkel" <G.Jaekel@...>
To: "Patrick Kevin McCaffrey" <pkm@...>, "lxc-users" <lxc-users@...>
Sent: Tuesday, December 20, 2011 2:06:49 AM
Subject: Re: [Lxc-users] PostgreSQL - "sh: cannot create /dev/null: Permission denied" - LXC Issue?

Dear Patrick,

As I understand /dev/null isn't writable in your container. That's definitely a wrong configuration.

Please check, that there is a real device node for  /dev/null  (and others) in your container and you have it (and others) in the lxc device access control list (lxc.cgroup.devices.allow = c 1:3 rw)

Note that -- depending on the linux flavor in your LXC container -- you might have to populate /dev by your own, because it's not reasonable to run udev or something like this inside a container.

Greetings

Guido

You need a name as in:
lxc.utsname = vm0

I have a wiki you can check & follow:
http://wiki.robotamer.com/linux/lxc

On Tue, 20 Dec 2011, Patrick Kevin McCaffrey wrote:

> I'm running into this issue when trying to set up a PostgreSQL server 
> inside one of my containers.  The Postgre mailing list seems suspect of 
> my LXC setup, so I thought I'd see if anyone has any input.  The outline 
> of my problem is below.  I've got Postgre installed/configured, but I 
> can't run the "initdb" command as seen below...

> Subject: Re: [GENERAL] New User: PostgreSQL Setup - "The Program 'postgress' is needed by initdb but was not found in the same directory..."
>
> Patrick Kevin McCaffrey <pkm@...> writes:
>>> I'm following the instructions that come with the source, and am stuck on this line:
>>> /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data
>>> When I run it, I get the following:
>>> sh: cannot create /dev/null: Permission denied

If the other replies aren't helping, I'm wondering if your running your 
container under NFS?

I've been doing this recently and having issues with NFSv3 & 4, but 2 is 
OK. I think it's something to do with ACSs possibly being enabled, but 
I've not had time to check further - as it works with good old v2 and 
that's OK for me, for now...

Gordon

OK I found my problem was due this file:
/etc/udev/rules.d/70-persistent-net.rules

Once removed, everything was fine....dhcp is working by default on
container start.

Martin

2011/12/22 Martin Konečný <martin.konecny@...>

> Hi,
>
> I've been looking into this for a few days without any success. When I
> create an LXC container using lxc-ubuntu shell script, then my containers
> can successfully obtain an IP address via DHCP.
>
> However when I create a container using lxc-sshd, then I need to specify
> the IP address manually for it to acquire an IP. I think I know why this is
> happening (SSHD container has its own init script which probably forgoes
> all network service initialization). I'd really like my SSHD container to
> acquire an IP dynamically.
>
> Things I've tried so far:
>
> 1) Running "dhclient"
> 2) Running ifup eth0
>
> I'm running Ubuntu 11.10 operating system, and here is the contents of
> /etc/network/interfaces (container's point of view)
>
> auto lo
>> iface lo inet loopback
>> auto eth0
>> iface eth0 inet dhcp
>
>
>
> What can I do to start the network initialization? I'm learning all I can
> about Linux network configuration right now, but a nudge in the right
> direction would be appreciated.
>
> Martin
>
> Martin Konecny
> Software Developer, Sourcefabric
> martin.konecny@...
>
> 720 Bathurst St. Suite 203
> M5S 2R4, Toronto, ON, Canada
> +1 (416) 892-8420 (Cell)
> Skype: martin.konecny15
>
> http://www.sourcefabric.org
> http://www.twitter.com/Sourcefabric
>
>


-- 
Open source radio in the cloud. Get yours now! ---> http://airtime.pro

Martin Konecny
Software Developer, Sourcefabric
martin.konecny@...

720 Bathurst St. Suite 203
M5S 2R4, Toronto, ON, Canada
+1 (416) 892-8420 (Cell)
Skype: martin.konecny15

http://www.sourcefabric.org
http://www.twitter.com/Sourcefabric

I am sorry, the problem was on my side - did a syntax mistake in the
lxc.conf.
Disregard my previous message and continue the good job you are doing! :)

--
Arie



On Sun, Dec 18, 2011 at 18:19, Arie Skliarouk <skliarie@...> wrote:

> What could be worse than a cgroup is not deleted by lxc-destroy? Why,
> inability to create a cgroup using lxc-create!
>
> Seriously, the host machine can not start vservers anymore. This is after
> one of the cgroups got stuck in the unremovable state.
>
> With these issues it becomes harder and harder for me to justify LXC to my
> boss...
>
> --
> Arie
>
>
> On Tue, Dec 13, 2011 at 00:01, Serge Hallyn <serge.hallyn@...:
>
>> Quoting Gordon Henderson (gordon@...):
>> > On Thu, 8 Dec 2011, Arie Skliarouk wrote:
>> >
>> > > When I tried to restart the vserver, it did not came up. Long story
>> short,
>> > > I found that lxc-destroy did not destroy the cgroup of the same name
>> as the
>> > > server. The cgroup remains visible in the /sys/fs/cgroup/cpu/master
>> > > directory. The tasks file is empty though.
>> >
>> > And just now, I've had the same thing happen - a container failed to
>> > start and it left it's body in /cgroup - with empy tasks.
>>
>> The patch I sent out on Friday should help handle that more gracefully -
>> it moves the cgroup out of the way so a new container can start.  You'll
>> need to clean the old one up by hand if you care to, though lxc could
>> easily provide a tool to clean it up (and move and analyze any tasks left
>> running in the cgroup, though I suspect in most cases there are none).
>>
>> -serge
>>
>>
>> ------------------------------------------------------------------------------
>> Learn Windows Azure Live!  Tuesday, Dec 13, 2011
>> Microsoft is holding a special Learn Windows Azure training event for
>> developers. It will provide a great way to learn Windows Azure and what it
>> provides. You can attend the event by watching it streamed LIVE online.
>> Learn more at http://p.sf.net/sfu/ms-windowsazure
>> _______________________________________________
>> Lxc-users mailing list
>> Lxc-users@...
>> https://lists.sourceforge.net/lists/listinfo/lxc-users
>>
>
>

Hi Arie,

Glad to hear that you could fix this!
Otherwise, it can happen sometimes that a cgroup is stuck with 
unkillable processes.
It can be annoying because you can't remove the cgroup, so if you want 
to start a new one with the same name, you might think you have to reboot.
If that happens, just try to terminate the other processes running in 
the cgroup, rename it ("mv /cgroup/mylittlecontainer /cgroup/broken") 
and restart it.
This saved me a few reboots already :-)

Cheers,

On 12/18/2011 10:24 AM, Arie Skliarouk wrote:
> I am sorry, the problem was on my side - did a syntax mistake in the 
> lxc.conf.
> Disregard my previous message and continue the good job you are doing! :)
>
> --
> Arie
>
>
>
> On Sun, Dec 18, 2011 at 18:19, Arie Skliarouk <skliarie@... 
> <mailto:skliarie@...>> wrote:
>
>     What could be worse than a cgroup is not deleted by lxc-destroy?
>     Why, inability to create a cgroup using lxc-create!
>
>     Seriously, the host machine can not start vservers anymore. This
>     is after one of the cgroups got stuck in the unremovable state.
>
>     With these issues it becomes harder and harder for me to justify
>     LXC to my boss...
>
>     --
>     Arie
>
>
>     On Tue, Dec 13, 2011 at 00:01, Serge Hallyn
>     <serge.hallyn@... <mailto:serge.hallyn@...>>
>     wrote:
>
>         Quoting Gordon Henderson (gordon@...
>         <mailto:gordon@...>):
>         > On Thu, 8 Dec 2011, Arie Skliarouk wrote:
>         >
>         > > When I tried to restart the vserver, it did not came up.
>         Long story short,
>         > > I found that lxc-destroy did not destroy the cgroup of the
>         same name as the
>         > > server. The cgroup remains visible in the
>         /sys/fs/cgroup/cpu/master
>         > > directory. The tasks file is empty though.
>         >
>         > And just now, I've had the same thing happen - a container
>         failed to
>         > start and it left it's body in /cgroup - with empy tasks.
>
>         The patch I sent out on Friday should help handle that more
>         gracefully -
>         it moves the cgroup out of the way so a new container can
>         start.  You'll
>         need to clean the old one up by hand if you care to, though
>         lxc could
>         easily provide a tool to clean it up (and move and analyze any
>         tasks left
>         running in the cgroup, though I suspect in most cases there
>         are none).
>
>         -serge
>
>         ------------------------------------------------------------------------------
>         Learn Windows Azure Live!  Tuesday, Dec 13, 2011
>         Microsoft is holding a special Learn Windows Azure training
>         event for
>         developers. It will provide a great way to learn Windows Azure
>         and what it
>         provides. You can attend the event by watching it streamed
>         LIVE online.
>         Learn more at http://p.sf.net/sfu/ms-windowsazure
>         _______________________________________________
>         Lxc-users mailing list
>         Lxc-users@...
>         <mailto:Lxc-users@...>
>         https://lists.sourceforge.net/lists/listinfo/lxc-users
>
>
>
>
>
> ------------------------------------------------------------------------------
> Learn Windows Azure Live!  Tuesday, Dec 13, 2011
> Microsoft is holding a special Learn Windows Azure training event for
> developers. It will provide a great way to learn Windows Azure and what it
> provides. You can attend the event by watching it streamed LIVE online.
> Learn more at http://p.sf.net/sfu/ms-windowsazure
>
>
> _______________________________________________
> Lxc-users mailing list
> Lxc-users@...
> https://lists.sourceforge.net/lists/listinfo/lxc-users

On Dec 18, 2011 1:09 PM, "Jérôme Petazzoni" <jerome.petazzoni@...>
wrote:
>
> If that happens, just try to terminate the other processes running in the
cgroup, rename it ("mv /cgroup/mylittlecontainer /cgroup/broken") and
restart it.
> This saved me a few reboots already :-)

You should be able to remove an empty cgroup with `rmdir`.

--

C Anthony

On 12/18/2011 11:56 AM, C Anthony Risinger wrote:
>
> On Dec 18, 2011 1:09 PM, "Jérôme Petazzoni" 
> <jerome.petazzoni@... <mailto:jerome.petazzoni@...>> 
> wrote:
> >
> > If that happens, just try to terminate the other processes running 
> in the cgroup, rename it ("mv /cgroup/mylittlecontainer 
> /cgroup/broken") and restart it.
> > This saved me a few reboots already :-)
>
> You should be able to remove an empty cgroup with `rmdir`.
>

Sorry, I forgot to mention:
- if you can't remove the cgroup because it's not empty,
- if the process remaining in the cgroup cannot be killed (for instance, 
because it's in "uninterruptible sleep" state),
- if you can't move the said process in a different cgroup (with e.g. 
"echo $PID > /cgroup/othercgroup/tasks"),
- ... then you can still rename the old cgroup.

I don't have the /cgroup directory mounted. Somehow, the directory is
mounted automatically onto the /sys/fs/cgroup

*root@...:~# df | grep cgroup
cgroup                12368328         0  12368328   0% /sys/fs/cgroup
root@...:~# ls /sys/fs/cgroup/
blkio  cpu  cpuacct  cpuset  devices  freezer  memory  net_cls  perf_event*

Each subdirectory of the above contains directory per container with knobs
that are specific to the resource:

*root@...:~# ls /sys/fs/cgroup/cpu/dev
cgroup.clone_children  cgroup.procs      cpu.rt_runtime_us
notify_on_release
cgroup.event_control   cpu.rt_period_us  cpu.shares         tasks
root@...:~#*

Could well be this is because of the 3.0.0-12-server kernel. I don't see
how I can rename a stuck cgroup easily in this situation. Any advices?

BTW, I once had /cgroup mounted from fstab like this:

*none /cgroup cgroup defaults 0 0*

It grouped all settings into per-container directory nicely, but the server
failed to boot with that.

--
Arie




On Sun, Dec 18, 2011 at 22:23, Jérôme Petazzoni <
jerome.petazzoni@...> wrote:

>  On 12/18/2011 11:56 AM, C Anthony Risinger wrote:
>
> On Dec 18, 2011 1:09 PM, "Jérôme Petazzoni" <jerome.petazzoni@...>
> wrote:
> >
> > If that happens, just try to terminate the other processes running in
> the cgroup, rename it ("mv /cgroup/mylittlecontainer /cgroup/broken") and
> restart it.
> > This saved me a few reboots already :-)
>
> You should be able to remove an empty cgroup with `rmdir`.
>
>
> Sorry, I forgot to mention:
> - if you can't remove the cgroup because it's not empty,
> - if the process remaining in the cgroup cannot be killed (for instance,
> because it's in "uninterruptible sleep" state),
> - if you can't move the said process in a different cgroup (with e.g.
> "echo $PID > /cgroup/othercgroup/tasks"),
> - ... then you can still rename the old cgroup.
>
>
>

Quoting Arie Skliarouk (skliarie@...):
> I don't have the /cgroup directory mounted. Somehow, the directory is
> mounted automatically onto the /sys/fs/cgroup
> 
> *root@...:~# df | grep cgroup
> cgroup                12368328         0  12368328   0% /sys/fs/cgroup
> root@...:~# ls /sys/fs/cgroup/
> blkio  cpu  cpuacct  cpuset  devices  freezer  memory  net_cls  perf_event*
> 
> Each subdirectory of the above contains directory per container with knobs
> that are specific to the resource:
> 
> *root@...:~# ls /sys/fs/cgroup/cpu/dev
> cgroup.clone_children  cgroup.procs      cpu.rt_runtime_us
> notify_on_release
> cgroup.event_control   cpu.rt_period_us  cpu.shares         tasks
> root@...:~#*
> 
> Could well be this is because of the 3.0.0-12-server kernel. I don't see

No, userspace does the mounting.  i.e. in ubuntu the cgroup-lite or
cgroup-bin packages both do it.

> how I can rename a stuck cgroup easily in this situation. Any advices?

You can build an lxc with my patch (until Daniel has a chance to apply it),
but in the meantime you can make a script 'move_cgroup.sh' along the lines
of:

#!/bin/sh
if [ $# -lt 1 ]; then
	echo "Usage: $0 <cgroup-name>"
	echo " Moves the <cgroup-name> out of the way."
fi
g=$1

t=`mktemp -u cg.XXXXXXXX`
for d in /sys/fs/cgroup/*; do
	mv $d/$g $d/$g.$t
done

Note that doesn't clean anything up, so if there are hung tasks those will
still be around.  A script to list details of each task in the hung cgroup
would be pretty simple too, and useful - if you write one, you might send
it here for inclusion in lxc!

> BTW, I once had /cgroup mounted from fstab like this:
> 
> *none /cgroup cgroup defaults 0 0*
> 
> It grouped all settings into per-container directory nicely, but the server
> failed to boot with that.

Yes, once early userspace has mounted the /sys/fs/cgroup/*, that fstab
entry would cause trouble.  But if you remove the package doing the
cgroup mounting, you should be able to go back to using this fstab
entry.

-serge

On Sat 2011-12-17 (14:38), DTK wrote:
> I am trying to avoid having to install NTP in every container, so I
> came up with following idea.
> 
> Main server cron job
> 
>         * 0 *  * * /sbin/hwclock -w
> 
> Each container has following cron job
> 
>         10 0 *  * * /sbin/hwclock -s
> 
> However, the container can't access /dev/rtc and I am guessing that
> has something to do with my cgroup config line:
> 
>        lxc.cgroup.devices.allow = c 254:0 rwm # rtc
> 
> Anyway I can fix this? Or get the same result some other way?

Run ntpd on the server host.
time is a kernel issue and you have only one kernel running.

-- 
Ullrich Horlacher              Server- und Arbeitsplatzsysteme
Rechenzentrum                  E-Mail: horlacher@...
Universitaet Stuttgart         Tel:    ++49-711-685-65868
Allmandring 30                 Fax:    ++49-711-682357
70550 Stuttgart (Germany)      WWW:    http://www.rus.uni-stuttgart.de/
REF: <CAAzn7UxoS4r2UD2axsG-eZeXRRb_y8bhOJ4B+B6uD82gX=sDQg@...>

2011/12/18 DTK <gruessle@...>:
> I am trying to avoid having to install NTP in every container, so I
> came up with following idea.
>
> Main server cron job
>
>        * 0 *  * * /sbin/hwclock -w
>
> Each container has following cron job
>
>        10 0 *  * * /sbin/hwclock -s
>
> However, the container can't access /dev/rtc and I am guessing that
> has something to do with my cgroup config line:
>
>       lxc.cgroup.devices.allow = c 254:0 rwm # rtc
>
> Anyway I can fix this? Or get the same result some other way?
>
> ------------------------------------------------------------------------------
> Learn Windows Azure Live!  Tuesday, Dec 13, 2011
> Microsoft is holding a special Learn Windows Azure training event for
> developers. It will provide a great way to learn Windows Azure and what it
> provides. You can attend the event by watching it streamed LIVE online.
> Learn more at http://p.sf.net/sfu/ms-windowsazure
> _______________________________________________
> Lxc-users mailing list
> Lxc-users@...
> https://lists.sourceforge.net/lists/listinfo/lxc-users

IMHO if you just want to keep reading the same time on all the guests,
then you need to do nothing to achieve this, it's already there. Since
you are using one single setup of kernel and hardware. Running ntpd on
the guests is unnecessary.

--
Thanks,
Zhu Yanhai

Is there an official LXC F.A.Q.?  This question appears several 
times per week.

     Please see:

http://osdir.com/ml/lxc-chroot-linux-containers/2011-12/msg00029.html


--Derek

On 12/17/2011 03:24 PM, DTK wrote:
> Why is it that my server shell (ssh) freezes for about 5 minutes when
> I stop one container?
> It also doesn't allow for login with a new ssh connection.
> However the other containers seam to be unaffected, and run fine.
>
> Freaks me out every time.
>
> ------------------------------------------------------------------------------
> Learn Windows Azure Live!  Tuesday, Dec 13, 2011
> Microsoft is holding a special Learn Windows Azure training event for
> developers. It will provide a great way to learn Windows Azure and what it
> provides. You can attend the event by watching it streamed LIVE online.
> Learn more at http://p.sf.net/sfu/ms-windowsazure
> _______________________________________________
> Lxc-users mailing list
> Lxc-users@...
> https://lists.sourceforge.net/lists/listinfo/lxc-users