[Umlbuilder-devel] UML Builder 1.30-6 released

[Roger Binns <rogerb@ro...>]

UML Builder 1.30-6 is now available.  You should update if you
are a Redhat user, or if you compile your own UML kernels.

The Redhat issue is because they use a very old version of Python.
It thinks that files over 2GB in size aren't files!

Roger

[Umlbuilder-devel] Upcoming release

[Roger Binns <rogerb@ro...>]

I hope to release a new version within a week or so.  A notable
addition is the addition of a miniroot (old SunOS users will recognise
the term).  This is a small Linux runtime in an ext2 filesystem.
The miniroot is used to build initial ramdisks, install modules and
can even resize and change the type of filesystems.  (The latter is
done using brute force - make a new file, mount in UML with the old one
and cpio the contents across).

The 2GB limit on filesystems has also gone.  All files are created
by a tiny external binary that creates them of the right size or
the largest supported size of the filesystem.

Lots of other little fixes which I will list when doing the actual
release.

I have also been having two trains of thought which I'll investigate
further.  Any feedback is welcome.

The first is that since I already have the rpmindex.py files generated
on my system, I can include them with the UML Builder distribution and
so most people won't need to generate them themselves.  This can also
be taken further and allow for using the original distro CDs and
hence no need for copying the RPMs off first.

The second thought is to dump UML Builder completely and use an installer
that has been appropriated from a standard Linux distribution, and then
build
its config files.  Now that UML itself supports partitioning, most of the
installers should work with some hacking.  The only one I have looked at
so far is DrakX which is the Mandrake one.  See
http://cvs.mandrakesoft.com/cgi-bin/cvsweb.cgi/gi/
Unfortunately most of it is in Perl, and I have no intention of writing
any Perl code beyond a few lines now and then.

See also http://www.csh.rit.edu/~benjamin/log/e_clig.shtml

Roger

[Umlbuilder-devel] Feature Request: Setting exit code

[Roger Binns <rogerb@ro...>]

I would like to be able to set the exitcode of UML.  I
currently do it by scribbling into a file on hostfs.

I stop UML using reboot(RB_HALT_SYSTEM).

My suggestion is a /proc that allows it to be set.

Roger

Re: [Umlbuilder-devel] uml-builder rpm ordering

[Roger Binns <rogerb@ro...>]

> I noticed while getting Redhat7.3.p running for my local system that you
> process all the RPMS individually I have always found it much easier to
> process them as a group as this eliminates the circular dependency
problem.

Actually it doesn't.  Many many rpms are circularly dependent, and
the rpm binary cannot figure them out either.  It just leaves them
in the same order (but does reorder ones that aren't mutually dependent).
I tried really hard to get things working by letting rpm deal with it
all, and ran into so many problems that I gave up and did the ordering
myself.

> temp=""
> for rpm in rpms:
> temp=temp+" "+rpm
> runcommand("rpm -ivh "+temp)

The default value for maximum command line length on Linux is pitifully
small compared to other platforms (128KB IIRC).  Consequently the more
full featured installs will actually fall over since you won't be able
to specify all rpms in one command line.  It would need to be sorted
into smaller groups.

UML Builder also has the ability to do actions before and after particular
RPMs, as well as supplying different installation flags to some.  This turns
out necessary on some distros.

It all seems insane that UML Builder goes to these lengths, as well as
completely unnecessary.  However if you actually dig behind the scenes
in all distros, you will discover that they actually install the RPMs
one a time, usually come with a presorted ordering list that completely
ignores the dependencies stated in the RPMs themselves, and have many
installation error/warning messages printed.  Little wonder UML Builder
has to repeat that.

> I still have not figured out why the automated linux kernel boot fails on
my
> system but I get a segfault with no mm or a kernel panic with 2.4.18-28um

It is probably the initial ramdisk.  Delete the initrd-um file and the
control
script won't try to use it.

The next release of UML Builder includes my own code for making initial
ramdisks
and it actually works correctly in all the distros.

Roger

[Umlbuilder-devel] uml-builder rpm ordering

[McMechan, James <McMechanJW@na...>]

I noticed while getting Redhat7.3.p running for my local system that you
process all the RPMS individually I have always found it much easier to
process them as a group as this eliminates the circular dependency problem.
I am even worse a python ;) this was the first time I looked at a python
program.
I did my change by commenting out the entire pre-install-post loop & if
statements
I changed the /uml_install... script put into installfs to approximately

	temp=""
	for rpm in rpms:
		temp=temp+" "+rpm

	runcommand("rpm -ivh "+temp)
this worked much better as it did not fail and require reordering of the
RPMs
I still have not figured out why the automated linux kernel boot fails on my
system but I get a segfault with no mm or a kernel panic with 2.4.18-28um

[Umlbuilder-devel] Re: [umlbuilder-user] Re: UML Builder futures

[Roger Binns <rogerb@ro...>]

> Very cool, this is getting close to be an ISP-level UML toolkit.

Not quite.  I would imagine an ISP would need to do a lot more
work on the provisioning side.  They would also make their own
images, use the COW stuff to save disk space and all sorts of
other tricks.

> I suppose the way for the problem to go away is for UML to come out as part
> of a standard stable kernel, and then lean on the distros to provide UML
> install compatibility.

Actually UML has been getting better at faking being a real machine.
Check out the fakehd and fake_ide options in 
http://user-mode-linux.sourceforge.net/switches.html

> This thing just absolutely hovers madly in regal splendor over all those
> Windows serfs.

http://user-mode-linux.sourceforge.net/projects.html  (OS Ports section)
http://user-mode-linux.sourceforge.net/arch-port.html

Roger

[Umlbuilder-devel] Re: [umlbuilder-user] Re: UML Builder futures

[Steve Freitas <sflist@ih...>]

> That was kind of the intention behind the control script.  In a few
> more revisions it should have all the right things done.

Very cool, this is getting close to be an ISP-level UML toolkit.

> The cad doing a shutdown is important as well.  Currently there is no
> clean way to shutdown a UML session from outside of it.

Sure there is. Just do it with ssh. ;-)

> When UML "reboots", it reruns linux, which in this case is ls and everything
> stops.

Clever, if dirty.

> Doing the scan of the install logs is easy.  The time consuming bit is
> starting each UML session, logging in and check things out that way.
> (Currently at 12 versions, will be adding RedFlag soon, and want to do
> Redmond/Lycoris.  Go to http://www.distrowatch.com/ to see the size of
> the "problem".)

I suppose the way for the problem to go away is for UML to come out as part
of a standard stable kernel, and then lean on the distros to provide UML
install compatibility.

Maybe in 2004. :-)

> Possibly.  See "Package Selection" in
> http://umlbuilder.sourceforge.net/running.shtml

Ah yes, that'd be me having missed something. Thanks. I still believe an FTP
client is a reasonable tool for an internet-connected UML, though.

> He already does.  Look in /usr/lib/uml.  UML Builder automatically installs
> them for you during the initial install.  (Jeff you may want to remove
> modules-2.2.tar though :-)

Ah ha! Okay, I see it now. Thanks. I just originally tried "insmod nfs", and
assumed when it couldn't find it that the modules hadn't been copied in.

This thing just absolutely hovers madly in regal splendor over all those
Windows serfs.

Steve

[Umlbuilder-devel] Re: UML Builder futures

[Roger Binns <rogerb@ro...>]

> Good to see writing this thing hasn't cost you your sense of humor. :-)

I'm British so that would be 'humour'.  But then I do live in California ...

> And I had another idea in conjunction with these. If you've got the UML
> startup silent, and TTYs to TCP/IP, how about adding a SysV-type script that
> will fire up the UMLs on host bootup, and cleanly shut them down on host
> shutdown?

That was kind of the intention behind the control script.  In a few
more revisions it should have all the right things done.

> For this you'd need to set CAD to hit the right runlevel, of course.

The cad doing a shutdown is important as well.  Currently there is no
clean way to shutdown a UML session from outside of it.  I had a dirty
hack.  I would put a symlink named 'linux' alongside control that pointed 
at /bin/ls. Then do 'control cad'.  When UML "reboots", it reruns linux, 
which in this case is ls and everything stops.

Doing it properly is now higher on the todo list.

> Any way to automate this? Perhaps, if you're dumping the UML to a logfile,
> you could run a diff between the last time you ran the test and this time.
> Then hopefully you could scan for changes quickly.

Doing the scan of the install logs is easy.  The time consuming bit is
starting each UML session, logging in and check things out that way.
(Currently at 12 versions, will be adding RedFlag soon, and want to do
Redmond/Lycoris.  Go to http://www.distrowatch.com/ to see the size of
the "problem".)

> True. But in the interim, I'd think adding ftp is justifiable, no? As it
> stands, I don't really know any way to get "out" from a base UML install to
> install additional packages, without adding ftp or the like from outside. Am
> I missing something?

Possibly.  See "Package Selection" in
http://umlbuilder.sourceforge.net/running.shtml

> Are there any advantages of not re-using the host
> IP? Such as being able to actually ping the UML itself?

You should be able to ping the UML session anyway.  The
only thing you lose is the ability to ping the tap endpoint on the
host itself (and you could still that with ping -i).

The only other use for seperate IPs I can see is in iptables/ipchains
rules.  However you pick which host interface UML Builder uses, so you
can effectively make it look like the UML sessions are attached to
a specific real interface, and it will be filtered against the same
rules as other traffic on that interface.

> Yes, that'd be neat. Then I can easily add non-compiled modules. I suppose
> it'll require Jeff to start distributing the modules that are compiled with
> each new UML release, no?

He already does.  Look in /usr/lib/uml.  UML Builder automatically installs
them for you during the initial install.  (Jeff you may want to remove
modules-2.2.tar though :-)

Roger
--
Roger Binns   rogerb@...  http://www.tarantella.com
Business Development Manager, Tarantella
425 Encinal Street, Santa Cruz, CA 95060, USA
Tel 831 427 7367  Fax 831 427 5406

[Umlbuilder-devel] Re: UML Builder futures

[Steve Freitas <sflist@ih...>]

> I've noticed the same thing on real hardware.  The new Linux distributions
> are getting faster.  (I assume compiling against i586/i686 is helping
> amongst other things).

Yeah, I grabbed some stuff over 100baseT via FTP and actually got slightly
better speed under the UML than on the host. Since webserving is usually not
a highly CPU-intensive task, this tells me that UML is plenty fast enough,
at least for my types of needs.

However, this isn't in jailed mode, which will be a practical "must," as
soon as the speed problems with it are solved.

> Ideally I would like it to be come obsolete because UML would just run
> the native installers, and everything would "just work".  (Ok, so that
> would never happen :-)

Good to see writing this thing hasn't cost you your sense of humor. :-)

> I want to solve this a different way.  I would like there to be a file
> in /etc that lists what ip addresses a user can have, which uml_net
> enforces.

Yes! That's a killer way to solve it.

>> * Attach ttys to host's IP ports over sshd, so startup and running of UMLs
>> is effectively silent.
>> * In conjunction with the above, an addition to control script that directs
>> stdout to a logfile?

And I had another idea in conjunction with these. If you've got the UML
startup silent, and TTYs to TCP/IP, how about adding a SysV-type script that
will fire up the UMLs on host bootup, and cleanly shut them down on host
shutdown?

For this you'd need to set CAD to hit the right runlevel, of course.

>> * Ext3 support.
> 
> It actually already does.  Add 'ext3' to the 'supportedfs' list in the
> profile.

Cool! Thanks!

> I generally take the attitude that I only "support" what is in the standard
> UML RPM produced by Jeff.

That makes a lot of sense.

> (The testing is fun.  There is an automatic test generator script that grovels
> through my collection of rpms and does the installation.  I then have to dig
> through all the log files to ensure it went ok.  Finally I have to run each
> one and login, scrutinise start messages etc.  A single tweak in the main
> code then requires this whole process again, as some distributions are very
> temperamental about installation order, files/directories that already exist,
> contents of /proc etc).

Any way to automate this? Perhaps, if you're dumping the UML to a logfile,
you could run a diff between the last time you ran the test and this time.
Then hopefully you could scan for changes quickly.

>> * An ability to add "basic utils" to base install, like ftp, ncftp, wget,
>> lynx, lsof, emacs-nox, sshd, etc.
> 
> Everyone's definition of "basic utils" differs though.  I figure this will
> be covered by 505686.

True. But in the interim, I'd think adding ftp is justifiable, no? As it
stands, I don't really know any way to get "out" from a base UML install to
install additional packages, without adding ftp or the like from outside. Am
I missing something?

> It was a great relief when I originally worked on this since I really didn't
> want to allocate two IPs for UML session.

Yes, and it's a great relief to me. IPs ain't easy to get these days. I just
read the docs, and I see now that it mentions that you can re-use the host
IP if you're short on IPs. Are there any advantages of not re-using the host
IP? Such as being able to actually ping the UML itself?

> - Cope with upgrades of UML itself (install new modules, generate new
> initrd)

Yes, that'd be neat. Then I can easily add non-compiled modules. I suppose
it'll require Jeff to start distributing the modules that are compiled with
each new UML release, no?

> Longer term I want to make it complement UML features.  For example I
> anticipate that UML will eventually have the ability to save its memory
> out so that you can "suspend" a session.  UML Builder can provide a
> nice easy to use interface to that sort of thing.

That sounds cool. I imagine a Webmin interface for UMLs that provides a lot
of the same services. Like something that:

* Lets you run the same CLI command on all your UMLs
* Has a networking "wizard" that lets you configure common cases
* Works with Webmin's monitoring suite to make sure the UMLs stay online
* Can back them all up
* Can harvest logs and produce reports
* Etc...

Steve

[Umlbuilder-devel] UML Builder futures

[Roger Binns <rogerb@ro...>]

[I edited the order of your questions/points]

> I'm stunned,
> actually, by how fast the Redhat 7.2 installation runs, compared with that
> ancient Debian image from UML's SourceForge files page.

I've noticed the same thing on real hardware.  The new Linux distributions
are getting faster.  (I assume compiling against i586/i686 is helping
amongst other things).

> Roger, where are you looking at taking UMLBuilder? 

Ideally I would like it to be come obsolete because UML would just run
the native installers, and everything would "just work".  (Ok, so that
would never happen :-)

More detailed answer at the end.

> Some things I think would be cool are:
> 
> * Create persistent tun devices on host, so users running UML don't have the
> ability to grab IPs through setuid uml_helper.

I want to solve this a different way.  I would like there to be a file
in /etc that lists what ip addresses a user can have, which uml_net 
enforces.  This way minimal admin maintenance is required.  (It would
be in a similar style to tcp wrappers).

I am going to do this as a patch to UML soon (once I get back from a 
well earned holiday over the next two weeks).

> * Attach ttys to host's IP ports over sshd, so startup and running of UMLs
> is effectively silent.

Good idea.  FR# 543170

> * In conjunction with the above, an addition to control script that directs
> stdout to a logfile?

FR# 543171

> * Ext3 support.

It actually already does.  Add 'ext3' to the 'supportedfs' list in the profile.

I generally take the attitude that I only "support" what is in the standard
UML RPM produced by Jeff.  This is because any user who is capable of recompiling
UML to fit their needs is also capable of making their own highly customised
filesystems.

ext3 was only recently added to UML.  The testing effort is large for each
"feature" I turn on, so they lag a bit.  (I actually added ext3 a long time
ago, but didn't test it).

(The testing is fun.  There is an automatic test generator script that grovels
through my collection of rpms and does the installation.  I then have to dig
through all the log files to ensure it went ok.  Finally I have to run each
one and login, scrutinise start messages etc.  A single tweak in the main
code then requires this whole process again, as some distributions are very
temperamental about installation order, files/directories that already exist,
contents of /proc etc).

> * An ability to add "basic utils" to base install, like ftp, ncftp, wget,
> lynx, lsof, emacs-nox, sshd, etc.

Everyone's definition of "basic utils" differs though.  I figure this will
be covered by 505686.

> * Creation of a tmpfs tmp directory under the user's home directory for use
> by each UML.
> A couple of questions: I run my RH7.2 UML with mem=64M. I noticed that about
> that amount is taken up in /tmp now. How does that work? Do I need my tmpfs
> to be the size of all my UMLs' cumulative mem= parameters?

I believe Jeff is doing work in this area.  At the moment, memory is consumed
from tmpfs as you observed.

> Also, I noticed it sets the tun device to the same IP as the host. I thought
> I read it was supposed to be different from the host IP. Has that changed,
> or did I read wrong? And does this mean I can just make as many UMLs as I
> like, with them all pointed to that one tun device?

It works as I ship it.  The endpoint inside your UML session needs to be
different, but the endpoint on the host doesn't have to be unique.  The UML
docs show it being different.

It was a great relief when I originally worked on this since I really didn't
want to allocate two IPs for UML session.

> I've seen your ToDo list,
> obviously, but I'm curious what your long-term goals look like.

Short/medium term I want to make it do more of "the right thing".

- Improved user interface, including allowing changes after installation
  (eg changing the size and type of a filesystem, IP address etc)
- Make existing features of UML easier to use (for example "undoable"
  filesystems using cow)
- Cope with upgrades of UML itself (install new modules, generate new
  initrd)

Longer term I want to make it complement UML features.  For example I
anticipate that UML will eventually have the ability to save its memory
out so that you can "suspend" a session.  UML Builder can provide a
nice easy to use interface to that sort of thing.

Roger

[Umlbuilder-devel] ANN: UML Builder 1.2 released

[Roger Binns <rogerb@ro...>]

UML Builder 1.2 is now available from http://umlbuilder.sourceforge.net

UML Builder is a tool that helps you install various RPM based 
distributions for use with user mode linux.  It has both a command line
mode and super friendly gui mode.  Networking and X are also setup
giving you something very close the experience of the actual
distribution.

The list of distributions you can install is now:

   Redhat 6.2, 7.1, 7.2
   Mandrake 8.1, 8.2
   Conectiva 7.0
   Suse  7.2, 7.3
   TurboLinux 7.0
   Caldera 2.4, 3.1, 3.11

The changes over earlier versions include automatic detection of the
rpm index being out of date, using initial ramdisks, and many other
minor bug fixes and tweaks.

Roger

Re: [Umlbuilder-devel] UML Builder 1.0 out

[Roger Binns <rogerb@ro...>]

> The layout is really nice in the demo.  What will the Advanced UML section
> have?

Extra commandline arguments
Location of 'linux' binary if you don't want the default picked up from your path
Location of modules (currently hardwired to /usr/lib/uml/...)

I am also going to be adding a 'TTYs' tab similar in spirit to the filesystem one
so that it is easy to setup where they all go.  (See recent discussion in uml-user
for hard some people find to understand that).

The thing I am looking forward to the most is undoable filesystems which will be
available in the same interface at runtime.

The other thing I have been considering is also doing runtime changes that require
scripting a session.  For example, if a user decides to double the size of the
root filesystem, this can be done by starting up UML with the install root with
ubd0 pointing to oldfs and ubd1 pointing to new one, mkfs the new one and copy
the contents across.

Similarly if the user changes settings in the ttys, it would require starting up
with the install root and editing /etc/inittab.

This could be automated as above, or the instructions could just be printed out
and it left as an "exercise for the reader".

I'll probably go the whole hog ...

Roger

Re: [Umlbuilder-devel] UML Builder 1.0 out

[Jonathan Rosenbaum <mmac_swim@ya...>]

On Sat, 23 Feb 2002, Roger Binns wrote:

> I have also started work on a different user interface.  This is to
> allow the configuration of more settings, without annoying every user
> with the minute detail. 
> 
> A derived interface will also be used to allow changing many settings
> after you have created a guest instance. 
> 
> If you install 1.0, run umlbuilder_gui_v2demo to see what it will look
> like. 

The layout is really nice in the demo.  What will the Advanced UML section
have?

Jonathan

[Umlbuilder-devel] UML Builder 1.0 out

[Roger Binns <rogerb@ro...>]

I have released UML Builder 1.0.  There are no functional changes since 0.9-3
and it works well.

I have also started work on a different user interface.  This is to allow the
configuration of more settings, without annoying every user with the minute
detail.

A derived interface will also be used to allow changing many settings after
you have created a guest instance.

If you install 1.0, run umlbuilder_gui_v2demo to see what it will look like.

Roger

[Umlbuilder-devel] What I am working on next

[Roger Binns <rogerb@ro...>]

I am going to make UB create a file called settings in
the install directory.  This will contain all the settings
for the session and will be sourceable by the control script.

The control script will then effectively become read only
since you'll only ever need to change things in the settings
file.

The second step is a graphical control program which will
effectively edit the settings file.  (Power users can use
vi :-)

This will allow easy adding/removing of filesystems, having
undoable disks, resizing memory/swap etc.

Roger

[Umlbuilder-devel] Re: Reiserfs on /

[Roger Binns <rogerb@ro...>]

> I noticed that one of UML Builder's know problems was that reiserfs
> doesn't work on / and you are talking about the need to use an initrd to
> solve this.  

The reason is because the standard UML rpm that Jeff distributes
has reiserfs as a module.  The modules reside on the root filesystem
(under /lib/modules) and root can't be mounted.

If you build UML yourself, then you can include reiserfs as a builtin
so this situation won't arise.

UML Builder does in fact create an initrd.  The main problem is that
I don't know how to use this.  Some distributions (eg Caldera) don't
include the mkinitrd program either.

Roger

[Umlbuilder-devel] Reiserfs on /

[Jonathan Rosenbaum <mmac_swim@ya...>]

Roger,

I noticed that one of UML Builder's know problems was that reiserfs
doesn't work on / and you are talking about the need to use an initrd to
solve this.  There shouldn't be any need for this.  I've been successfully
creating reiserfs on root for a long time.  The only thing to watch out
for is that the fs is > than a certain size, I think it is 30M or
something, this is because it is a journaling fs and it requires some
space for that journal.  You may want experiment with gbootroot to
discover the required size, I think this is probably the cause of this
problem. 

Regards,


Jonathan