I'm pretty sure I've seen discussion of how to integrate g4l into an existing
Linux installation, but I'm not finding it now.
I have a 640Gb portable hard drive that I'd like to put Fedora on and
integrate several toolsets, including g4l, into it. I know I can put g4l on
its own partition as though it's a usb drive and hook it into grub.conf as a
boot option, but since I have several toolsets I want to integrate, it seems
cleaner to hack it into the main Fedora install. Or maybe not. Any pointers to
existing discussion of this?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The method that I use to run g4l with various versions is to make it an option on the grub boot menu. Simple option is using the 40_custom option, and then use grub2-mkconfig -o /boot/grub2/grub.cfg
Contents of the 40_custom file.
!/bin/sh
exec tail -n +3 $0
This file provides an easy way to add custom menu entries. Simply type the
menu entries you want to add after this comment. Be careful not to change
the 'exec tail' line above.
menuentry G4L {
linux /bz3x17.3 root=/dev/ram0 telnetd=yes
initrd /ramdisk.lzma
}
One then puts the kernel file (in this case the bz17.3 kernel, and then the g4l file system ramdisk.lzma. Then the grub menu will provide option to load the full g4l running from the ram that allows for backing up all partitions.
ftp://amd64gcc.dyndns.org/g4l0.48alpha/
Contains the latest versions that I am working on
ftp://amd64gcc.dyndns.org/g4l0.47alpha/
Is the files for the release version
ftp://amd64gcc.dyndns.org/g4l_kernels
Has the kernel files.
ftp://amd64gcc.dyndns.org/g4l_kernels/bz3x17.3
Is the current latest version
Recently, I've been also making flashes that can boot the g4l using grub4dos.
Have a USB 3.0 128GB flash on which I created a 100M Fat32 or NTFS partition, and then installed the files from one of the g4l-lite-fc20 files.
ftp://amd64gcc.dyndns.org/g4l0.48alpha/g4l-lite-fc20.7z
ftp://amd64gcc.dyndns.org/g4l0.48alpha/g4l-lite-fc20.exe
ftp://amd64gcc.dyndns.org/g4l0.48alpha/g4l-lite-fc20.zip
All the same files, just different compression.
The files are extracted to the 100M partition, and then use bootlace or the windows exe file to make the flash bootable.
Then created an ext4 partition with the rest of the flash for storing image files.
Can then boot from the flash, and do images from or to it.
In testing, I can restore a 160G Windows 7 image that uses about 40+G of space in about 8 minutes using USB 2 port, but takes just 4 1/2 minutes using the USB 3 port.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks, Mike. That's definitely cleaner than dedicating a disk partition. It
didn't occur to me that everything would be inside the initrd.
My original thought was to integrate g4l directly into the Fedora installation
so I could, for example, attach to a machine, boot off my drive, and run
ophcrack on the XP hashes, mount the volume(s) and zero out the free space,
and pull an image, all from one boot environment. Thinking about it further,
that may be too ambitious, anyway. I'd not only have to install anything
necessary for your script(s) to run, I'd have to devise a way to get necessary
drivers loaded for unpredictable hardware (as you have done so well).
I've said it before, but thanks again for g4l and your ongoing
support/maintenance of it. I use it in an enterprise environment at work, on
my farm of disparate machines at home, and for pro bono support work.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The g4l scripts can be run from fedora (or other linux) directly. I run the
cleandrive scripts from the fedora to clear things out. In the latest alphas,
I have a cleandrive8.dialog script that can mount and clear space.
As long as the support package are installed and the jetcat-mod program is
there. It should work, except for not being able to backup mounted partitions.
In my classroom, I put an ntfsclone image of the XP on a linux partition, and
have a script that can restore the XP in about 10 minutes. Even added an
option on the grub menu to do this, so a user can restore the XP by just
picking a grub option to restore and then reboot in about 10 minutes.
Note: with ntfsclone images, there is no real reason to clear free space.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
If you can hold my hand just a bit more, what support packages are needed?
Other than those that will be there in a basic Fedora 14 installation? IIRC,
you static compile everything, so I can just grab jetcat-mod off the iso, but
if you know the other packages offhand, you can save me crawling through a
bunch of "yum whatprovides". Thanks for the help.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The g4l uses a number of packages that may or may not already be installed as
default on Fedora or other distros.
The ones that are my own would be jetcat-mod. Note: There is a regular jetcat-
mod that works on 32 bit installs, but also a 64 bit version since the
libraries may not be there for the 32 bit one on 64 bit systems. I just copy
the 64 bit version to the 32 bit name on those machines.
In the past, I've found that lzop, dialog, ntfsclone, ncftp, udpcast may need
to be added. There is also aespipe, but only used if encrypting image.
I have a script on the cd called chkldd that I use to make sure I have all the
support libraries needed for the programs in each directory. It uses the ldd
program to check if anything for the programs is missing.
In the past ncftp needed to be added, but I found that it is not included at
least with my install of fedora 14.
Don't hesitate to ask for any help.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
1) Basic workstation install of Fedora 14 onto USB hard drive (had to use
advanced storage options to find the USB drive). I used a single 50Gb /
physical partition, and made a separate 500Gb /home for data.
With this as root I can launch g4l from a command line and use network mode
with localhost as the ftp server and do basic lzop images and restores. There
were too many quirks and presumptions in local mode to work with that, e.g. it
seems to want the images in the root level of an unmounted filesystem on a
physical partition. Have not tested nfts stuff, etc.
There are some quirks to getting the external USB drive booting consistently
on different platforms. Booting rescue mode off the F14 install CD 1 ON A
MACHINE WITH NO OTHER LINUX INSTALLATIONS, e.g. a Windows box, chrooting
/mnt/sysimage, and doing a "grub-install /dev/sda" seems to have made it
consistent (make sure /dev/sda is what you're chrooted on by looking at
"mount") . Don't know what Anaconda may have done differently that made its
boot work on some machines and not others.
If you have problems with some hardware booting from USB, you can use a PLOP
Boot Manager CD or diskette in conjunction with the USB drive. Google "plop
boot manager", download the zip, and use the iso or img from the cli folder.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
1) Basic workstation install of Fedora 14 onto USB hard drive (had to use
advanced storage options to find the USB drive). I used a single 50Gb /
physical partition, and made a separate 500Gb /home for data.
The 0.36 version is now based on Fedora 14, so it should only require the
actual g4l script (g4l30o9) and the jetcat-mod in one of the path directories.
The other files would be those not included in the install.
The second thing would be you would need the 64bit jetcat-mod if your using a
64bit version of Fedora or have the 32 bit libraries installed.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks for all the help, Mike, on this and previous issues.
I don't understand "The other files would be those not included in the
install.", though. In any case, I have something that works for walking up to
a machine and imaging it with the plus that I also have gparted, etc,
available in the same session. I'm looking now at configuring this to also
serve as a pxeboot server so I could walk into a lab or datacenter, boot one
machine from my drive, then pxe boot others and image them. Looks pretty
simple.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I use FC15 and would like to prepare a kickstart file which would install two
separate versions of g4l - 32 and 64bit - on a separate usb stick. After
downloading the development package I found the whole thing very confusing!
Could you enlighten me as to what packages do I need to add in my kickstart
file please? I do not wish to use any of the binaries present in that
"development" archive as these are tied up/compiled for a specific arch/kernel
versions and I don't want that really.
The kickstart file also gives me the possibility of configuring and executing
script actions and that is where I plan to add the g4l scripts at the end of
the installation process - I understand that this is the only essential part
of g4l - the rest are package dependencies, which can be compiled/downloaded
with most common distros currently available.
The end product would be a single usb stick capable of using g4l - both 32 and
64 bit (via grub-selected boot options) - depending on the target machine on
which g4l is to be executed. I can easily update that image by executing the
kickstart file when needed to do refresh the packages already installed.
I have been using version 0.26 up until now where I used to copy files3.tar.gz
on top of my installation image, but that was only for 32 bit and only for a
specific version(s) of the kernel. I see that much has changed since then!
Help would be appreciated! Thanks in advance!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Running g4l from grub or grub4dos or syslinux is generally just a matter of
adding the kernel and ramdisk.lzma files to the boot directory, and then
adding the lines to the grub.conf or menu.lst or syslinux.cfg.
On my Fedora 14 machines (both 32 and 64 bit) I just add the following to the
grub.conf.
The kernel is a 32 bit, but it runs fine on 64 bit machines, and the file
system has all the all the support files and libraries.
The files option was normally to add the g4l as a run option for other live
cds in which the g4l cd would not work, but I've generally found that the new
kernel builds work. They don't have the xwindows, but g4l doesn't use it.
As for 32 versus 64 bit, I don't think there would be much advantage in a 64
bit version running, since it is mostly io based. Might be some advantage in
the compression, but it is usually the io part that holds things back.
As far as the development kit, I copy the libraries that the programs require.
I have a script called chkfiles on my built machine that compares the files
currently in the g4l directories with those in the matching system
directories, and can then copy the newer versions when updates change them. To
test on the cd image that I didn't miss anything, I have another script called
chkldd, and it runs ldd on all files in the current director, and I check for
not found messages.
Using the kernel and ramdisk.lzma file works best as I see it, since it is all
running from ram, and can copy all images.
Also, one doesn't have to bother with the development kit.
Note: one can also add the run="options..." to the menu line to get it to
automate the process.
Example: In my lab, I have dual boot with Fedora 14 and XP, and XP can get
messed up in many ways, so wanted a quick way to restore it. Have an ntfsclone
image of the xp partition on /dev/sda6. The following lines allow for an user
to just select it, and it will restore the XP in about 12 minutes.
Loads the kernel and filesystem in ram, and then mounts /dev/sda6 and then
runs the reimagexp script that does and ntfsclone restore to partition
/dev/sda1 and then reboots.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I wish to prepare and build the entire g4l - both 32 and 64 bit versions -
from sources/rpms using kickstart. For that, I need to know what part forms
the g4l "package" (scripts etc) and what are its dependencies.
I am also building the kernel separately - in Fedora environment. So far from
reading the posts above, I figured that the following packages are required:
lzop: lzop
ncftp: ncftp ncftpget ncftpls ncftpput
fuse-sshfs: sshfs
udpcast: udp-receiver udp-sender
aespipe: aespipe
What is also missing, at least from your post dated 30 Jan 2011, is what forms
the "g4l" package itself as well as ifcheck2.sh and jetcat-mod. I figured the
jetcat-mod bit as it is in the contrib directory and I can build a separate
.spec file from this and create the rpm necessary for kickstart installation.
I also presume there must also be a dependency (not specified in any of the
posts above) on ntfs-3g package as well as g4l deals with ntfs partitions.
What I need to know is all the other stuff which forms "g4l" - stuff I can
package into a separate rpm (called g4l) and include it as part of the
kickstart installation process. In the "development" archive there are a lot
of scripts, particularly in the rootfs directory, which I am not certain
whether they form part of g4l or whether they are part of some obscure program
I haven't heard of.
I hope I've made my points a bit clearer this time.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I still don't see why you are trying to it that way???
I use a single kernel for both 32 and 64 bit Fedora 14 and 15 machines, but if
that is what you need.
You would have to basically check for all the programs in the following
directories to see if they exists on the machine.
/bin
/sbin
/usr/bin
/usr/sbin
/usr/local/bin
/usr/local/sbin
On my fedora 14 build machine, I have all programs available and can just run
the g4l script and it works.
Note: the /mnt/local directory does need to exist for many operations.
Some programs are not Fedora RPMs, but newer vresions of programs complied
from source.
I had created the filesx.gz stuff long ago to add to knoppix, since it booted
machines that the g4l kernels didn't.
Back then I had programs statically compiled, so they were large, but didn't
require the libraries.
I recall created a script about a year ago that used which to check for all
the files that were in any of the binary directories.
Ran this on a finnix livecd, since it had libraries that match for the most
part, knoppix had much older libraries. Most of the files were already there,
but a few were not, and those I included in the finnix3.gz file on the
amd64gcc.dyndns.org server.
I didn't include all the missing programs, but the important ones that were
not included.
Note: Running the g4l script from a mounted system doesn't allow for imaging
that disk or mounted partitions. Running it from the grub is all in ram, so it
is free to backup everything, and it does see LVM partitions as well.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I just made this script with the g4l0.39 that I am working on to do the check
I mentioned.
Used this to run script.
./g4lwhich >g4lwhich.out 2>g4lwhich.miss
The g4lwhich.miss shows the files that are not on the system.
I should also note that jetcatmod program needs to be included, which is my
own program for piping the data thru for the progress bar. The 32 and 64 bit
binaries are on the cd, since the clean scripts use them.
The g4lwhich script to check if the files are available.
which bash
which bicode.o3
which bkey
which busybox
which cat
which chgrp
which chkldd
which chmod
which chown
which cleandrive8
which cleandrive8.dialog
which comm
which cp
which date
which dd
which df
which diff
which dmesg
which dumpkmap
which echo
which efg4l
which egrep
which eject
which exg4l
which false
which fdflush
which fg4l
which fgrep
which ftpd
which fusermount
which g41
which g4l
which g4lmenu
which getopt
which grep
which gunzip
which gzip
which hostname
which iostat
which ipaddr
which ipcalc
which kill
which lblank7
which less
which ln
which login
which ls
which lzop
which lzopcat
which mdev
which mkdir
which mknod
which more
which mount
which mpstat
which mv
which nbd-client
which netstat
which ntfs-3g
which ntfs-3g.probe
which ntfs-3g.secaudit
which ntfs-3g.usermap
which ntfscat
which ntfsck
which ntfsclone.txt
which ntfscluster
which ntfscmp
which ntfsdecrypt
which ntfsdump_logfile
which ntfsfix
which ntfsinfo
which ntfsls
which ntfsmftalloc
which ntfsmount
which ntfsmove
which ntfsreloc
which ntfstruncate
which ntfswipe
which ntpclient
which openvt
which pidof
which ping
which powertop
which printenv
which ps
which pwd
which rm
which rmdir
which sed
which sh
which sleep
which startftpd
which strings
which stty
which sync
which tar
which tcpsvd
which telnetd
which touch
which true
which udhcpc
which umount
which uname
which unlzop
which which
which xg4l
which xprogress
which zexg4l
which zxg4l
which agetty
which blkid
which blockdev
which dosfsck
which e2fsck
which ethtool
which fdisk
which freeramdisk
which fsck
which fsck.cramfs
which fsck.ext2
which fsck.ext3
which fsck.ext4
which fsck.ext4dev
which fsck.jfs
which fsck.minix
which fsck.msdos
which fsck.vfat
which fsck.xfs
which halt
which hdparm
which hwclock
which ifconfig
which init
which insmod
which jfs_fsck
which jfs_fscklog
which klogd
which loadkmap
which losetup
which lsmod
which lvchange
which lvconvert
which lvcreate
which lvdisplay
which lvextend
which lvm
which lvmchange
which lvmconf
which lvmdiskscan
which lvmdump
which lvmsadc
which lvmsar
which lvreduce
which lvremove
which lvrename
which lvresize
which lvs
which lvscan
which makedevs
which mkdosfs
which mke2fs
which mkfs
which mkfs.cramfs
which mkfs.ext2
which mkfs.ext3
which mkfs.ext4
which mkfs.ext4dev
which mkfs.jfs
which mkfs.minix
which mkfs.msdos
which mkfs.vfat
which mkfs.xfs
which mkswap
which modprobe
which mount.cifs
which mount.fuse
which mount.nfs
which mount.nfs4
which mount.ntfs
which mount.ntfs-3g
which mount.ntfs-fuse
which nologin
which ntfsclone
which ntfscp
which ntfslabel
which ntfsresize
which ntfsundelete
which parted
which pivot_root
which poweroff
which pvchange
which pvck
which pvcreate
which pvdisplay
which pvmove
which pvremove
which pvresize
which pvs
which pvscan
which reboot
which resize2fs
which rmmod
which route
which rpcbind
which sfdisk
which swapoff
which swapon
which syslogd
which tune2fs
which umount.cifs
which umount.nfs
which umount.nfs4
which vgcfgbackup
which vgcfgrestore
which vgchange
which vgck
which vgconvert
which vgcreate
which vgdisplay
which vgexport
which vgextend
which vgimport
which vgimportclone
which vgmerge
which vgmknodes
which vgreduce
which vgremove
which vgrename
which vgs
which vgscan
which vgsplit
which watchdog
which arping
which awk
which base64
which basename
which bc
which beep
which bootchartd
which bunzip2
which bzcat
which bzip2
which chvt
which clear
which cmp
which cut
which ddrescue
which dd_rescue
which dd_rhelp
which deallocvt
which dialog
which dirname
which dnsd
which dnsdomainname
which dos2unix
which du
which env
which expr
which fdformat
which fgconsole
which find
which flock
which fold
which free
which gpg
which head
which hexdump
which hexedit
which hostid
which id
which inetd
which install
which ip
which ipcalc
which iplink
which iproute
which iptunnel
which jetcat-mod
which killall
which ldd
which lddlibc4
which length
which loadfont
which logger
which logname
which md5sum
which mesg
which mkfifo
which modinfo
which nano
which nc
which nslookup
which pmap
which printf
which raidautorun
which rdate
which renice
which reset
which rev
which scp
which setkeycodes
which setterm
which smemcap
which sort
which split
which ssh
which stat
which strace
which tail
which tee
which telnet
which test
which time
which top
which tr
which traceroute
which tty
which tune2fs.busybox
which uniq
which unix2dos
which unxz
which unzip
which uptime
which uudecode
which uuencode
which vi
which vim
which wc
which who
which whoami
which xargs
which xz
which xzcat
which yes
which chroot
which fbset
which gpm
which rpcinfo
which aespipe
which ftp
which ncftp
which ncftpget
which ncftpls
which ncftpput
which sshfs
which testdisk
which testdisk-6.13WIP
which fsarchiver
which fsarchiver7
which partimage
which partimaged
which udp-receiver
which udp-sender
A number of those files are little scripts that are not needs by g4l main
script.
On my system where the main g4l script works, I find that 56 files don't show
in the which, some are mount support files that are there, but which does see
as others.
which: no bicode.o3
which: no bkey
which: no chkldd
which: no cleandrive8
which: no cleandrive8.dialog
which: no dumpkmap
which: no efg4l
which: no exg4l
which: no fdflush
which: no fg4l
which: no ftpd
which: no g41
which: no g4l
which: no g4lmenu
which: no iostat
which: no ipaddr
which: no lblank7
which: no lzopcat
which: no mdev
which: no mpstat
which: no nbd-client
which: no ntfsclone.txt
which: no ntfsreloc
which: no ntpclient
which: no powertop
which: no startftpd
which: no tcpsvd
which: no telnetd
which: no udhcpc
which: no unlzop
which: no xg4l
which: no xprogress
which: no zexg4l
which: no zxg4l
which: no freeramdisk
which: no fsck.minix
which: no klogd
which: no loadkmap
which: no makedevs
which: no mkfs.minix
which: no mount.ntfs
which: no mount.ntfs-3g
which: no mount.ntfs-fuse
which: no syslogd in
which: no umount.cifs
which: no watchdog
which: no bootchartd
which: no dnsd
which: no inetd
which: no iplink
which: no iproute
which: no length
which: no loadfont
which: no raidautorun
which: no smemcap
which: no tune2fs.busybox
I was thinking of making a setup option similar to memtest, which it would
just put the kernel file, ramdisk.lzma file, and then add the lines to the
lines to the grub.conf....
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Why do I need to do it? Because I like to know what I am building/installing
on my machine! In the "development" archive 98% of the files are binaries! So,
how do I know what's in there?
Besides, by building it from source I have a greater deal of flexibility to
tailor the build to my own configuration/architecture/cpu etc.
Reading your answer above, I take it I have to check all of the (s)bin
directories to see what packages are used, is that what you are suggesting? Do
you not know what packages are you using in your own project?
What about the scripts, object files and executables in the root directory
(xg4l, zexg4l, ifcheck2.sh, getkey.sh, bicode.o3, cleandrive*, grldr,
blank6.exe - to name just a few files scattered in the root directory)? What
are they part of?
There is also a directory in the development package called
resources/g4lscripts, but that contains just g4l* scripts - I don't believe
that is everything as far as the "g4l" package is concerned (no xg4l, zexg4l
etc either). What else is needed to form just the g4l package?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
What I presented is to see if you want to have all of the things that are
included on the g4l build.
The development kit is for building a custom CD image not to develop an RPM.
You could also just take the g4l main script, and go thru it, and add only
those programs that the options you use.
I don't think you mentioned NTFSCLONE, which is for backing up ntfs
partitions.
The little scripts in the root directory, are mostly linked in the bin
directory, but are just front end loaders to the main g4l script. ifcheck
script is to setup the IP, but if you are booting from a running system, it
has already setup the ip. The getkey.sh is to get the key files from an usb
for use with the AESPIPE. The bicode.o3 is used to decrypt front end scripts
that are downloaded via ftp. cleandrive scripts are to clear unused sectors
for raw backups. blank6.exe is a dos/windows program to clear unused space.
grldr is the grub4dos boot loader that can be used to run g4l from a ntfs
partition.
The G4L main script is probable 99% of what the program is, and with the
jetcat-mod it uses a number of other commands.
ncftp, ntfsclone, dialog, dd, sed, etc.
I once did a 64 bit rebuild of the g4l and the difference in speed was less
than 1%.
Don't know if there is something that could scan the g4l script, and pull out
everything that is a program to list it. It would be a big job to go thru
every line, and write down all the programs it uses?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'm pretty sure I've seen discussion of how to integrate g4l into an existing
Linux installation, but I'm not finding it now.
I have a 640Gb portable hard drive that I'd like to put Fedora on and
integrate several toolsets, including g4l, into it. I know I can put g4l on
its own partition as though it's a usb drive and hook it into grub.conf as a
boot option, but since I have several toolsets I want to integrate, it seems
cleaner to hack it into the main Fedora install. Or maybe not. Any pointers to
existing discussion of this?
The method that I use to run g4l with various versions is to make it an option on the grub boot menu. Simple option is using the 40_custom option, and then use grub2-mkconfig -o /boot/grub2/grub.cfg
Contents of the 40_custom file.
!/bin/sh
exec tail -n +3 $0
This file provides an easy way to add custom menu entries. Simply type the
menu entries you want to add after this comment. Be careful not to change
the 'exec tail' line above.
menuentry G4L {
linux /bz3x17.3 root=/dev/ram0 telnetd=yes
initrd /ramdisk.lzma
}
One then puts the kernel file (in this case the bz17.3 kernel, and then the g4l file system ramdisk.lzma. Then the grub menu will provide option to load the full g4l running from the ram that allows for backing up all partitions.
ftp://amd64gcc.dyndns.org/g4l0.48alpha/
Contains the latest versions that I am working on
ftp://amd64gcc.dyndns.org/g4l0.47alpha/
Is the files for the release version
ftp://amd64gcc.dyndns.org/g4l_kernels
Has the kernel files.
ftp://amd64gcc.dyndns.org/g4l_kernels/bz3x17.3
Is the current latest version
Recently, I've been also making flashes that can boot the g4l using grub4dos.
Have a USB 3.0 128GB flash on which I created a 100M Fat32 or NTFS partition, and then installed the files from one of the g4l-lite-fc20 files.
ftp://amd64gcc.dyndns.org/g4l0.48alpha/g4l-lite-fc20.7z
ftp://amd64gcc.dyndns.org/g4l0.48alpha/g4l-lite-fc20.exe
ftp://amd64gcc.dyndns.org/g4l0.48alpha/g4l-lite-fc20.zip
All the same files, just different compression.
The files are extracted to the 100M partition, and then use bootlace or the windows exe file to make the flash bootable.
Then created an ext4 partition with the rest of the flash for storing image files.
Can then boot from the flash, and do images from or to it.
In testing, I can restore a 160G Windows 7 image that uses about 40+G of space in about 8 minutes using USB 2 port, but takes just 4 1/2 minutes using the USB 3 port.
g4l can be added directly to the grub using the boot partition.
copy the kernel that you would use from the cd to the /boot and the ramdisk
file.
In the past this would be the ramdisk.gz file, but I have recently changed to
using ramdisk.lzma since it is about 25% smaller.
My grub.conf file has this lines, added, and it is just a boot option and
loads the kernel and ramdisk from the boot.
title G4l
kernel /bz36.2 ramdisk_size=65536 root=/dev/ram0
initrd /ramdisk.lzma
Thanks, Mike. That's definitely cleaner than dedicating a disk partition. It
didn't occur to me that everything would be inside the initrd.
My original thought was to integrate g4l directly into the Fedora installation
so I could, for example, attach to a machine, boot off my drive, and run
ophcrack on the XP hashes, mount the volume(s) and zero out the free space,
and pull an image, all from one boot environment. Thinking about it further,
that may be too ambitious, anyway. I'd not only have to install anything
necessary for your script(s) to run, I'd have to devise a way to get necessary
drivers loaded for unpredictable hardware (as you have done so well).
I've said it before, but thanks again for g4l and your ongoing
support/maintenance of it. I use it in an enterprise environment at work, on
my farm of disparate machines at home, and for pro bono support work.
The g4l scripts can be run from fedora (or other linux) directly. I run the
cleandrive scripts from the fedora to clear things out. In the latest alphas,
I have a cleandrive8.dialog script that can mount and clear space.
As long as the support package are installed and the jetcat-mod program is
there. It should work, except for not being able to backup mounted partitions.
In my classroom, I put an ntfsclone image of the XP on a linux partition, and
have a script that can restore the XP in about 10 minutes. Even added an
option on the grub menu to do this, so a user can restore the XP by just
picking a grub option to restore and then reboot in about 10 minutes.
Note: with ntfsclone images, there is no real reason to clear free space.
If you can hold my hand just a bit more, what support packages are needed?
Other than those that will be there in a basic Fedora 14 installation? IIRC,
you static compile everything, so I can just grab jetcat-mod off the iso, but
if you know the other packages offhand, you can save me crawling through a
bunch of "yum whatprovides". Thanks for the help.
The g4l uses a number of packages that may or may not already be installed as
default on Fedora or other distros.
The ones that are my own would be jetcat-mod. Note: There is a regular jetcat-
mod that works on 32 bit installs, but also a 64 bit version since the
libraries may not be there for the 32 bit one on 64 bit systems. I just copy
the 64 bit version to the 32 bit name on those machines.
In the past, I've found that lzop, dialog, ntfsclone, ncftp, udpcast may need
to be added. There is also aespipe, but only used if encrypting image.
I have a script on the cd called chkldd that I use to make sure I have all the
support libraries needed for the programs in each directory. It uses the ldd
program to check if anything for the programs is missing.
In the past ncftp needed to be added, but I found that it is not included at
least with my install of fedora 14.
Don't hesitate to ask for any help.
Here's what ended up working.
1) Basic workstation install of Fedora 14 onto USB hard drive (had to use
advanced storage options to find the USB drive). I used a single 50Gb /
physical partition, and made a separate 500Gb /home for data.
2) Unpacked the latest g4l iso as /opt/g4l
3) Yum installed lzop and vsftpd
4) ln -s /opt/g4l/g4l /usr/local/bin/g4l;ln -s /opt/g4l/usr/bin/jetcat-mod
/usr/local/bin/jetcat-mod
With this as root I can launch g4l from a command line and use network mode
with localhost as the ftp server and do basic lzop images and restores. There
were too many quirks and presumptions in local mode to work with that, e.g. it
seems to want the images in the root level of an unmounted filesystem on a
physical partition. Have not tested nfts stuff, etc.
There are some quirks to getting the external USB drive booting consistently
on different platforms. Booting rescue mode off the F14 install CD 1 ON A
MACHINE WITH NO OTHER LINUX INSTALLATIONS, e.g. a Windows box, chrooting
/mnt/sysimage, and doing a "grub-install /dev/sda" seems to have made it
consistent (make sure /dev/sda is what you're chrooted on by looking at
"mount") . Don't know what Anaconda may have done differently that made its
boot work on some machines and not others.
If you have problems with some hardware booting from USB, you can use a PLOP
Boot Manager CD or diskette in conjunction with the USB drive. Google "plop
boot manager", download the zip, and use the iso or img from the cli folder.
Hi
What's g4l file in new versions?
1) Basic workstation install of Fedora 14 onto USB hard drive (had to use
advanced storage options to find the USB drive). I used a single 50Gb /
physical partition, and made a separate 500Gb /home for data.
2) Unpacked the latest g4l iso as /opt/g4l
3) Yum installed lzop and vsftpd
4) ln -s /opt/g4l/g4l /usr/local/bin/g4l;ln -s /opt/g4l/usr/bin/jetcat-mod
/usr/local/bin/jetcat-mod
Two things.
The 0.36 version is now based on Fedora 14, so it should only require the
actual g4l script (g4l30o9) and the jetcat-mod in one of the path directories.
The other files would be those not included in the install.
The second thing would be you would need the 64bit jetcat-mod if your using a
64bit version of Fedora or have the 32 bit libraries installed.
Thanks for all the help, Mike, on this and previous issues.
I don't understand "The other files would be those not included in the
install.", though. In any case, I have something that works for walking up to
a machine and imaging it with the plus that I also have gparted, etc,
available in the same session. I'm looking now at configuring this to also
serve as a pxeboot server so I could walk into a lab or datacenter, boot one
machine from my drive, then pxe boot others and image them. Looks pretty
simple.
The g4l script uses other support programs, and many are included with a
regular install of Fedora.
Others that may not have been included like the lzop, ntfsclone, jetcat-mod,
udpcast, and others.
As an example. I have in the past created little archives that allow g4l to
work on other live cds.
Did one for finnix from last november ftp://amd64gcc.dyndns.org/finnix3.gz
It includes most of the extra programs that were not included with the finnix
lived .
aespipe
bicode.o3
g4l
ifcheck2.sh
jetcat-mod
lzop
ncftp
ncftpget
ncftpls
ncftpput
sshfs
udp-receiver
udp-sender
That doesn't have the latest version of the script, but would work.
I also have been working on g4l-win.exe that has the latest kernel, and
ramdisk that can be added to a windows partition by using grub4dos to load it.
With my Fedora machines, I have just added the kernel and ramdisk to the boot
directory and grub menu.
I have not yet done a PXE setup of g4l, so that is something I have yet to do.
Have done a flash boot, but have found that one has to play with bios settings
on many machines to get it to work.
Thanks for the messages.
I use FC15 and would like to prepare a kickstart file which would install two
separate versions of g4l - 32 and 64bit - on a separate usb stick. After
downloading the development package I found the whole thing very confusing!
Could you enlighten me as to what packages do I need to add in my kickstart
file please? I do not wish to use any of the binaries present in that
"development" archive as these are tied up/compiled for a specific arch/kernel
versions and I don't want that really.
The kickstart file also gives me the possibility of configuring and executing
script actions and that is where I plan to add the g4l scripts at the end of
the installation process - I understand that this is the only essential part
of g4l - the rest are package dependencies, which can be compiled/downloaded
with most common distros currently available.
The end product would be a single usb stick capable of using g4l - both 32 and
64 bit (via grub-selected boot options) - depending on the target machine on
which g4l is to be executed. I can easily update that image by executing the
kickstart file when needed to do refresh the packages already installed.
I have been using version 0.26 up until now where I used to copy files3.tar.gz
on top of my installation image, but that was only for 32 bit and only for a
specific version(s) of the kernel. I see that much has changed since then!
Help would be appreciated! Thanks in advance!
Running g4l from grub or grub4dos or syslinux is generally just a matter of
adding the kernel and ramdisk.lzma files to the boot directory, and then
adding the lines to the grub.conf or menu.lst or syslinux.cfg.
On my Fedora 14 machines (both 32 and 64 bit) I just add the following to the
grub.conf.
title G4L
kernel /bz3x0.4 ramdisk_size=65536 root=/dev/ram0 telnetd=yes
initrd /ramdisk.lzma
The kernel is a 32 bit, but it runs fine on 64 bit machines, and the file
system has all the all the support files and libraries.
The files option was normally to add the g4l as a run option for other live
cds in which the g4l cd would not work, but I've generally found that the new
kernel builds work. They don't have the xwindows, but g4l doesn't use it.
As for 32 versus 64 bit, I don't think there would be much advantage in a 64
bit version running, since it is mostly io based. Might be some advantage in
the compression, but it is usually the io part that holds things back.
As far as the development kit, I copy the libraries that the programs require.
I have a script called chkfiles on my built machine that compares the files
currently in the g4l directories with those in the matching system
directories, and can then copy the newer versions when updates change them. To
test on the cd image that I didn't miss anything, I have another script called
chkldd, and it runs ldd on all files in the current director, and I check for
not found messages.
Using the kernel and ramdisk.lzma file works best as I see it, since it is all
running from ram, and can copy all images.
Also, one doesn't have to bother with the development kit.
Note: one can also add the run="options..." to the menu line to get it to
automate the process.
Example: In my lab, I have dual boot with Fedora 14 and XP, and XP can get
messed up in many ways, so wanted a quick way to restore it. Have an ntfsclone
image of the xp partition on /dev/sda6. The following lines allow for an user
to just select it, and it will restore the XP in about 12 minutes.
title G4L Restore XP
kernel /bz3x0.4 ramdisk_size=65536 root=/dev/ram0 telnetd=yes run="mount
/dev/sda6 /mnt/local && cd /mnt/local && ./reimagexp "
initrd /ramdisk.lzma
Loads the kernel and filesystem in ram, and then mounts /dev/sda6 and then
runs the reimagexp script that does and ntfsclone restore to partition
/dev/sda1 and then reboots.
msetzerii, you've completely missed my point!
I wish to prepare and build the entire g4l - both 32 and 64 bit versions -
from sources/rpms using kickstart. For that, I need to know what part forms
the g4l "package" (scripts etc) and what are its dependencies.
I am also building the kernel separately - in Fedora environment. So far from
reading the posts above, I figured that the following packages are required:
lzop: lzop
ncftp: ncftp ncftpget ncftpls ncftpput
fuse-sshfs: sshfs
udpcast: udp-receiver udp-sender
aespipe: aespipe
What is also missing, at least from your post dated 30 Jan 2011, is what forms
the "g4l" package itself as well as ifcheck2.sh and jetcat-mod. I figured the
jetcat-mod bit as it is in the contrib directory and I can build a separate
.spec file from this and create the rpm necessary for kickstart installation.
I also presume there must also be a dependency (not specified in any of the
posts above) on ntfs-3g package as well as g4l deals with ntfs partitions.
What I need to know is all the other stuff which forms "g4l" - stuff I can
package into a separate rpm (called g4l) and include it as part of the
kickstart installation process. In the "development" archive there are a lot
of scripts, particularly in the rootfs directory, which I am not certain
whether they form part of g4l or whether they are part of some obscure program
I haven't heard of.
I hope I've made my points a bit clearer this time.
I still don't see why you are trying to it that way???
I use a single kernel for both 32 and 64 bit Fedora 14 and 15 machines, but if
that is what you need.
You would have to basically check for all the programs in the following
directories to see if they exists on the machine.
/bin
/sbin
/usr/bin
/usr/sbin
/usr/local/bin
/usr/local/sbin
On my fedora 14 build machine, I have all programs available and can just run
the g4l script and it works.
Note: the /mnt/local directory does need to exist for many operations.
Some programs are not Fedora RPMs, but newer vresions of programs complied
from source.
I had created the filesx.gz stuff long ago to add to knoppix, since it booted
machines that the g4l kernels didn't.
Back then I had programs statically compiled, so they were large, but didn't
require the libraries.
I recall created a script about a year ago that used which to check for all
the files that were in any of the binary directories.
Ran this on a finnix livecd, since it had libraries that match for the most
part, knoppix had much older libraries. Most of the files were already there,
but a few were not, and those I included in the finnix3.gz file on the
amd64gcc.dyndns.org server.
I didn't include all the missing programs, but the important ones that were
not included.
Note: Running the g4l script from a mounted system doesn't allow for imaging
that disk or mounted partitions. Running it from the grub is all in ram, so it
is free to backup everything, and it does see LVM partitions as well.
I just made this script with the g4l0.39 that I am working on to do the check
I mentioned.
Used this to run script.
./g4lwhich >g4lwhich.out 2>g4lwhich.miss
The g4lwhich.miss shows the files that are not on the system.
I should also note that jetcatmod program needs to be included, which is my
own program for piping the data thru for the progress bar. The 32 and 64 bit
binaries are on the cd, since the clean scripts use them.
The g4lwhich script to check if the files are available.
which bash
which bicode.o3
which bkey
which busybox
which cat
which chgrp
which chkldd
which chmod
which chown
which cleandrive8
which cleandrive8.dialog
which comm
which cp
which date
which dd
which df
which diff
which dmesg
which dumpkmap
which echo
which efg4l
which egrep
which eject
which exg4l
which false
which fdflush
which fg4l
which fgrep
which ftpd
which fusermount
which g41
which g4l
which g4lmenu
which getopt
which grep
which gunzip
which gzip
which hostname
which iostat
which ipaddr
which ipcalc
which kill
which lblank7
which less
which ln
which login
which ls
which lzop
which lzopcat
which mdev
which mkdir
which mknod
which more
which mount
which mpstat
which mv
which nbd-client
which netstat
which ntfs-3g
which ntfs-3g.probe
which ntfs-3g.secaudit
which ntfs-3g.usermap
which ntfscat
which ntfsck
which ntfsclone.txt
which ntfscluster
which ntfscmp
which ntfsdecrypt
which ntfsdump_logfile
which ntfsfix
which ntfsinfo
which ntfsls
which ntfsmftalloc
which ntfsmount
which ntfsmove
which ntfsreloc
which ntfstruncate
which ntfswipe
which ntpclient
which openvt
which pidof
which ping
which powertop
which printenv
which ps
which pwd
which rm
which rmdir
which sed
which sh
which sleep
which startftpd
which strings
which stty
which sync
which tar
which tcpsvd
which telnetd
which touch
which true
which udhcpc
which umount
which uname
which unlzop
which which
which xg4l
which xprogress
which zexg4l
which zxg4l
which agetty
which blkid
which blockdev
which dosfsck
which e2fsck
which ethtool
which fdisk
which freeramdisk
which fsck
which fsck.cramfs
which fsck.ext2
which fsck.ext3
which fsck.ext4
which fsck.ext4dev
which fsck.jfs
which fsck.minix
which fsck.msdos
which fsck.vfat
which fsck.xfs
which halt
which hdparm
which hwclock
which ifconfig
which init
which insmod
which jfs_fsck
which jfs_fscklog
which klogd
which loadkmap
which losetup
which lsmod
which lvchange
which lvconvert
which lvcreate
which lvdisplay
which lvextend
which lvm
which lvmchange
which lvmconf
which lvmdiskscan
which lvmdump
which lvmsadc
which lvmsar
which lvreduce
which lvremove
which lvrename
which lvresize
which lvs
which lvscan
which makedevs
which mkdosfs
which mke2fs
which mkfs
which mkfs.cramfs
which mkfs.ext2
which mkfs.ext3
which mkfs.ext4
which mkfs.ext4dev
which mkfs.jfs
which mkfs.minix
which mkfs.msdos
which mkfs.vfat
which mkfs.xfs
which mkswap
which modprobe
which mount.cifs
which mount.fuse
which mount.nfs
which mount.nfs4
which mount.ntfs
which mount.ntfs-3g
which mount.ntfs-fuse
which nologin
which ntfsclone
which ntfscp
which ntfslabel
which ntfsresize
which ntfsundelete
which parted
which pivot_root
which poweroff
which pvchange
which pvck
which pvcreate
which pvdisplay
which pvmove
which pvremove
which pvresize
which pvs
which pvscan
which reboot
which resize2fs
which rmmod
which route
which rpcbind
which sfdisk
which swapoff
which swapon
which syslogd
which tune2fs
which umount.cifs
which umount.nfs
which umount.nfs4
which vgcfgbackup
which vgcfgrestore
which vgchange
which vgck
which vgconvert
which vgcreate
which vgdisplay
which vgexport
which vgextend
which vgimport
which vgimportclone
which vgmerge
which vgmknodes
which vgreduce
which vgremove
which vgrename
which vgs
which vgscan
which vgsplit
which watchdog
which arping
which awk
which base64
which basename
which bc
which beep
which bootchartd
which bunzip2
which bzcat
which bzip2
which chvt
which clear
which cmp
which cut
which ddrescue
which dd_rescue
which dd_rhelp
which deallocvt
which dialog
which dirname
which dnsd
which dnsdomainname
which dos2unix
which du
which env
which expr
which fdformat
which fgconsole
which find
which flock
which fold
which free
which gpg
which head
which hexdump
which hexedit
which hostid
which id
which inetd
which install
which ip
which ipcalc
which iplink
which iproute
which iptunnel
which jetcat-mod
which killall
which ldd
which lddlibc4
which length
which loadfont
which logger
which logname
which md5sum
which mesg
which mkfifo
which modinfo
which nano
which nc
which nslookup
which pmap
which printf
which raidautorun
which rdate
which renice
which reset
which rev
which scp
which setkeycodes
which setterm
which smemcap
which sort
which split
which ssh
which stat
which strace
which tail
which tee
which telnet
which test
which time
which top
which tr
which traceroute
which tty
which tune2fs.busybox
which uniq
which unix2dos
which unxz
which unzip
which uptime
which uudecode
which uuencode
which vi
which vim
which wc
which who
which whoami
which xargs
which xz
which xzcat
which yes
which chroot
which fbset
which gpm
which rpcinfo
which aespipe
which ftp
which ncftp
which ncftpget
which ncftpls
which ncftpput
which sshfs
which testdisk
which testdisk-6.13WIP
which fsarchiver
which fsarchiver7
which partimage
which partimaged
which udp-receiver
which udp-sender
A number of those files are little scripts that are not needs by g4l main
script.
On my system where the main g4l script works, I find that 56 files don't show
in the which, some are mount support files that are there, but which does see
as others.
which: no bicode.o3
which: no bkey
which: no chkldd
which: no cleandrive8
which: no cleandrive8.dialog
which: no dumpkmap
which: no efg4l
which: no exg4l
which: no fdflush
which: no fg4l
which: no ftpd
which: no g41
which: no g4l
which: no g4lmenu
which: no iostat
which: no ipaddr
which: no lblank7
which: no lzopcat
which: no mdev
which: no mpstat
which: no nbd-client
which: no ntfsclone.txt
which: no ntfsreloc
which: no ntpclient
which: no powertop
which: no startftpd
which: no tcpsvd
which: no telnetd
which: no udhcpc
which: no unlzop
which: no xg4l
which: no xprogress
which: no zexg4l
which: no zxg4l
which: no freeramdisk
which: no fsck.minix
which: no klogd
which: no loadkmap
which: no makedevs
which: no mkfs.minix
which: no mount.ntfs
which: no mount.ntfs-3g
which: no mount.ntfs-fuse
which: no syslogd in
which: no umount.cifs
which: no watchdog
which: no bootchartd
which: no dnsd
which: no inetd
which: no iplink
which: no iproute
which: no length
which: no loadfont
which: no raidautorun
which: no smemcap
which: no tune2fs.busybox
I was thinking of making a setup option similar to memtest, which it would
just put the kernel file, ramdisk.lzma file, and then add the lines to the
lines to the grub.conf....
Why do I need to do it? Because I like to know what I am building/installing
on my machine! In the "development" archive 98% of the files are binaries! So,
how do I know what's in there?
Besides, by building it from source I have a greater deal of flexibility to
tailor the build to my own configuration/architecture/cpu etc.
Reading your answer above, I take it I have to check all of the (s)bin
directories to see what packages are used, is that what you are suggesting? Do
you not know what packages are you using in your own project?
What about the scripts, object files and executables in the root directory
(xg4l, zexg4l, ifcheck2.sh, getkey.sh, bicode.o3, cleandrive*, grldr,
blank6.exe - to name just a few files scattered in the root directory)? What
are they part of?
There is also a directory in the development package called
resources/g4lscripts, but that contains just g4l* scripts - I don't believe
that is everything as far as the "g4l" package is concerned (no xg4l, zexg4l
etc either). What else is needed to form just the g4l package?
What I presented is to see if you want to have all of the things that are
included on the g4l build.
The development kit is for building a custom CD image not to develop an RPM.
You could also just take the g4l main script, and go thru it, and add only
those programs that the options you use.
I don't think you mentioned NTFSCLONE, which is for backing up ntfs
partitions.
The little scripts in the root directory, are mostly linked in the bin
directory, but are just front end loaders to the main g4l script. ifcheck
script is to setup the IP, but if you are booting from a running system, it
has already setup the ip. The getkey.sh is to get the key files from an usb
for use with the AESPIPE. The bicode.o3 is used to decrypt front end scripts
that are downloaded via ftp. cleandrive scripts are to clear unused sectors
for raw backups. blank6.exe is a dos/windows program to clear unused space.
grldr is the grub4dos boot loader that can be used to run g4l from a ntfs
partition.
The G4L main script is probable 99% of what the program is, and with the
jetcat-mod it uses a number of other commands.
ncftp, ntfsclone, dialog, dd, sed, etc.
I once did a 64 bit rebuild of the g4l and the difference in speed was less
than 1%.
Don't know if there is something that could scan the g4l script, and pull out
everything that is a program to list it. It would be a big job to go thru
every line, and write down all the programs it uses?