From: David H. L. Jr. <dh...@dl...> - 2006-11-25 23:31:13
|
I am making heavy use of colinux as a tool for embedded systems development. But I am dealing with clients/co-workers that feel a compelling need to live on the Windows/NTFS side of things. I have been asked to look at trying to fix cygwin so that it can compile Linux kernels - rather than build them under colinux, specifically because that eliminates the need to work on a different file system inaccessible to windows. I noticed recently that the linux-ntfs project has finally released a beta version of a new NTFS driver that has support for everything needed to host Linux on NTFS. Initially I thought that might be relevant to hosting colinux on NTFS - gaining the ability to run Linux and windows concurrently on an NTFS partition. but the ntfs-3g driver is a block driver and concurrent block access to the same disk would probably scramble the disk. But after thinking further is there a particular reason that cofs can not support a linux root file system ? Has there been any thought to this ? has anyone tried it ? I am not familiar with the cofs innards but NTFS does support all the critical features needed by Linux. -- Dave Lynch DLA Systems Software Development: Embedded Linux 717.627.3770 dh...@dl... http://www.dlasys.net fax: 1.253.369.9244 Cell: 1.717.587.7774 Over 25 years' experience in platforms, languages, and technologies too numerous to list. "Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction." Albert Einstein |
From: peter g. <plugwash@P10Link.net> - 2006-11-26 03:42:54
|
> I am making heavy use of colinux as a tool for embedded=20 > systems development. ok =20 > But I am dealing with clients/co-workers that feel a=20 > compelling need to live on the Windows/NTFS side of things. > I have been asked to look at trying to fix cygwin so that it=20 > can compile Linux kernels - rather than build them under colinux, > specifically because that eliminates the need to work on a=20 > different file system inaccessible to windows. you should be able to do most of your work on a cofs or smbfs area without too much trouble. > I am not familiar with the cofs innards but NTFS does support=20 > all the critical features needed by Linux. permissions would be the most significant thing, you'd have to decide how to map linux partitions to NTFS ones and then implement that also i'm not sure if cofs is stable/bug free enough to be used as a root filesystem yet. |
From: David H. L. Jr. <dh...@dl...> - 2006-11-26 08:11:51
|
peter green wrote: > >> But I am dealing with clients/co-workers that feel a >> compelling need to live on the Windows/NTFS side of things. >> I have been asked to look at trying to fix cygwin so that it >> can compile Linux kernels - rather than build them under colinux, >> specifically because that eliminates the need to work on a >> different file system inaccessible to windows. >> > you should be able to do most of your work on a cofs or smbfs area > without too much trouble. > > I do not care much (under coLinux) about smbfs - as it requires a working network. >> I am not familiar with the cofs innards but NTFS does support >> all the critical features needed by Linux. >> > permissions would be the most significant thing, you'd have to decide > how to map linux partitions to NTFS ones and then implement that > > also i'm not sure if cofs is stable/bug free enough to be used as a > root filesystem yet. > I also monitor the linux-ntfs mailinglist and now the ntfs-3g mailinglist - aparently there is somekind of spat going on. ntfs-3g claims to be strong enough to support booting linux over ntfs - though I have not seen actual evidence beyond the claim. That said there was some discussion regarding limitations at the w32 API level, vs. the NT internals, vs. NTFS itself, regarding hard case issues like file names greater than approx. 255 characters - I think the windows API breaks, but obviously they work internally, you can subst or net use against a reference point (recursively if need be) and go as deep as you want. Symlinks. I though that W2K actually implimented them for HSM, and sysinternals "junction" does them atleast for directories - though lots of windows apps barf, but aparently SFU impliments them differently. There is also name case matching issues, and file name character set issues. Is it reasonable to presume that cofs is constrained by whatever the limits of the windows API ? I beleive name case matching under a cofs mount follows windows rules as an example. -- Dave Lynch DLA Systems Software Development: Embedded Linux 717.627.3770 dh...@dl... http://www.dlasys.net fax: 1.253.369.9244 Cell: 1.717.587.7774 Over 25 years' experience in platforms, languages, and technologies too numerous to list. "Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction." Albert Einstein |
From: <ds...@uc...> - 2006-11-27 03:00:07
|
> peter green wrote: >> >>> But I am dealing with clients/co-workers that feel a >>> compelling need to live on the Windows/NTFS side of things. >>> I have been asked to look at trying to fix cygwin so that it >>> can compile Linux kernels - rather than build them under colinux, >>> specifically because that eliminates the need to work on a >>> different file system inaccessible to windows. >>> >> you should be able to do most of your work on a cofs or smbfs area >> without too much trouble. >> >> > I do not care much (under coLinux) about smbfs - as it requires a > working network. > This is not exactly correct. smbfs can be run without a working network (at least in the physical sense). Just setup the coLinux side for samba, install the virtual network interface on the windows side, and then map the coLinux from Windows. You could also enable filesharing on the windows side and access your coLinux build which resides on the NTFS by mounting the windows shared folder in the coLinux build directory. There are also tools that you can download that allow a windows host to access an ext2/3 filesystem using a client that functions like WSFTP. I can't remember the name of the one that I have used at the moment though. -Dave |
From: Juergen H. <jue...@gm...> - 2006-11-27 13:34:40
|
-------- Original-Nachricht -------- Datum: Sun, 26 Nov 2006 19:59:48 -0700 (MST) Von: ds...@uc... An: "David H. Lynch Jr." <dh...@dl...> Betreff: Re: [coLinux-devel] Hosting coLinux directly on NTFS > > peter green wrote: > >> > >>> But I am dealing with clients/co-workers that feel a > >>> compelling need to live on the Windows/NTFS side of things. > >>> I have been asked to look at trying to fix cygwin so that it > >>> can compile Linux kernels - rather than build them under colinux, > >>> specifically because that eliminates the need to work on a > >>> different file system inaccessible to windows. > >>> I did that a few times, but most of the work went always into the kernel and uClinux-dist build system. Also (maybe this has changed) there are a few places in the kernel (and the build system, but this could be fixed now) that relay on case sensitive file systems (netfilter?). Also compiling on cygwin is much slower than on coLinux. Additionally you can't use two cygwin installations at the same time and when you want to install new software (with setup.exe), you may end with an installation that is incompatible with your embedded toolchain. > >> you should be able to do most of your work on a cofs or smbfs area > >> without too much trouble. > >> > >> > > I do not care much (under coLinux) about smbfs - as it requires a > > working network. > > > > This is not exactly correct. smbfs can be run without a working network > (at least in the physical sense). Just setup the coLinux side for samba, > install the virtual network interface on the windows side, and then map > the coLinux from Windows. You could also enable filesharing on the > windows side and access your coLinux build which resides on the NTFS by > mounting the windows shared folder in the coLinux build directory. > Exporting the build tree via SMBFS to Windows over tun/tap works quite good. This way the case problems could also be fixed. > There are also tools that you can download that allow a windows host to > access an ext2/3 filesystem using a client that functions like WSFTP. I > can't remember the name of the one that I have used at the moment though. > AFAIK these tools have all the same problem, as they don't/can't support case sensitive file access under Windows. They have therefore the same problems as cygwin. You can download an (rather old) example installer for such a setup (exports the directory containing the uClinux-dist with SAMBA) from http://blackfin.uclinux.org/projects/bfin-colinux. Also since the coLinux system can connection with a Windows X-Server can function like a normal Linux box, it could support quite a number of additional applications which could be very interesting for a developer in the embedded area. Juergen > -Dave > > |
From: David H. L. Jr. <dh...@dl...> - 2006-11-27 22:46:43
|
Juergen Hennerich wrote: > > I did that a few times, but most of the work went always into the kernel and uClinux-dist build system. Also (maybe this has changed) there are a few places in the kernel (and the build system, but this could be fixed now) that relay on case sensitive file systems (netfilter?). > > Also compiling on cygwin is much slower than on coLinux. Additionally you can't use two cygwin installations at the same time and when you want to install new software (with setup.exe), you may end with an installation that is incompatible with your embedded toolchain. > But after persuasion and argument has failed, what is left is what the client wants. Unfortunately this client is brilliant - and understands nothing is impossible, stuborn - they want Linux on their embedded systems, but they want to work from windows, and a close friend so blowing him off is not an option. But atleast I get to tell him he is an idiot, before trying to give him what he wants. I personally dislike cygwin - with respect to the people who put enormous effort into it. It is a horrible bag on the side of windows attempting to fix the unfixable. colinux is a much more elegant solution to the same problem - as well as many others. >> This is not exactly correct. smbfs can be run without a working network >> (at least in the physical sense). Just setup the coLinux side for samba, >> install the virtual network interface on the windows side, and then map >> the coLinux from Windows. You could also enable filesharing on the >> windows side and access your coLinux build which resides on the NTFS by >> mounting the windows shared folder in the coLinux build directory. >> >> > Exporting the build tree via SMBFS to Windows over tun/tap works quite good. This way the case problems could also be fixed. > I started playing with colinux about 2 years ago. I never got TUN/TAP working. I have not used slirp, but I have had good luck with bridging - but I do nto have the time to fix the install system to get all the bridge parameters right automatically. We decided to drop networking colinux - it is really nice to have, but not critical to operations. Also I do not think CIFS/SMB resolves the filename case problems. I am fairly certain that in non-colinux environments I have had really evil things happen accross Samba connections based on the difference between the client and host OS's case rules. I think this even effects cofs with colinux. I think a cofs mount uses windows filename case rules even though it is on the Linux side. > AFAIK these tools have all the same problem, as they don't/can't support case sensitive file access under Windows. They have therefore the same problems as cygwin. > We (my client and I looked thoroughly at the filename case issues - and they are solvable. It is possible to force casesensitive operations under Win32 - I am not sure why cofs and cygwin don't but it can be done. We were preparing to try to patch cygwin when we hit a few other problems: Most w32 api's are limited to approximately 240 character path's. There may be work arrounds - but they are going to be ugly. legal linux filename characters are superset of legal windows filename chacters. > You can download an (rather old) example installer for such a setup (exports the directory containing the uClinux-dist with SAMBA) from http://blackfin.uclinux.org/projects/bfin-colinux. > Also since the coLinux system can connection with a Windows X-Server can function like a normal Linux box, it could support quite a number of additional applications which could be very interesting for a developer in the embedded area. > My problem is my client is trying to get the colinux install (root file system particularly) as small as possible. I am down to 225Mb with a minimal compressed Debian system with a complete patched linux source and build environment and crosstools. I tried building a smaller image starting with Arch - but by the time I had everything I needed to compile Linux I was at probably 80% of the size of a minimal Debian image anyway, and I have probably a decade of Debian experience. I am not trying to create my own - or even the clients development environment. I have a fairly sophisticated coLinux install, multiple network connections, ssh, X, .... I am trying to create something my client can put on their web site for their clients to download. My client - http://www.picocomputing.com. sells very small and expensive embedded systems. They happen to run GreenHills and Linux, but primarily they are used for immage procession, or cryptography, or other things were their clients develop special hardware APU's in the FPGA. Anyway their clients are mostly firmware engineers doing everything under windows. Now I would personally like to see cofs evolve to the point of reducing my personal need for a large disk image, and since there is some significant synergy between some aspects of colinux and the embedded systems I am working on I might get time to do some work on cofs - basically I want to piggyback on colinux for the Pico Hosted development environment. I want to tunnel the cofs, conet, .. colinux<->windows link between the windows X86 host and my ppc405 linux target. -- Dave Lynch DLA Systems Software Development: Embedded Linux 717.627.3770 dh...@dl... http://www.dlasys.net fax: 1.253.369.9244 Cell: 1.717.587.7774 Over 25 years' experience in platforms, languages, and technologies too numerous to list. "Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction." Albert Einstein |
From: David H. L. Jr. <dh...@dl...> - 2006-12-14 02:30:27
|
Henry; I took a quick look at the colinux "installer creates images" on your site that looks like it builds a root file system, and I think that is what I need. Can I get a copy of the .nsi file for that installer ? I can not seem to find it. -- Dave Lynch DLA Systems Software Development: Embedded Linux 717.627.3770 dh...@dl... http://www.dlasys.net fax: 1.253.369.9244 Cell: 1.717.587.7774 Over 25 years' experience in platforms, languages, and technologies too numerous to list. "Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction." Albert Einstein |
From: Henry N. <Henry.Ne@Arcor.de> - 2006-12-14 09:00:48
|
David H. Lynch Jr. wrote: > Henry; > I took a quick look at the colinux "installer creates images" on > your site that looks like it builds a root file system, and I think > that is what I need. > Can I get a copy of the .nsi file for that installer ? I can not > seem to find it. You need a full installed coLinux, all exe/sys files and the special initrd from such installation. The nsi file, shell scripts and some tools are in the source arcive ftp://terminal-systems.de/software/colinux-trm/sources/colinux-installer-0.7.1-ssv2.src.zip It's based on Blackfin installer from Juergen. But, it creates image files, not runs directly on ntfs (your subject line suggest it). -- Henry Nestler |
From: Brian D. <br...@de...> - 2006-11-27 14:06:33
|
Juergen Hennerich wrote: > AFAIK these tools have all the same problem, as they don't/can't support case sensitive file access under Windows. They have therefore the same problems as cygwin. If that is the issue then use Cygwin's managed mounts. AFAIK people have been cross-compiling linux kernels in Cygwin using this feature for years, so I don't know what OP is talking about. Brian |
From: David H. L. Jr. <dh...@dl...> - 2006-11-27 22:51:35
|
Brian Dessent wrote: > Juergen Hennerich wrote: > > >> AFAIK these tools have all the same problem, as they don't/can't support case sensitive file access under Windows. They have therefore the same problems as cygwin. >> > > If that is the issue then use Cygwin's managed mounts. AFAIK people > have been cross-compiling linux kernels in Cygwin using this feature for > years, so I don't know what OP is talking about. > > Brian > > I have been looking for information regarding cross-compiling linux kernels from cygwin. Thus far everything I have found says: 1). Don't - it is not worth the agrevation. 2). No one who has tried has succeeded. 3). The demands of the Linux Kernel Build system exceed the capabilities of cygwin - file name case issues just being one of many problems. I am NOT trying to claim to know this. I personally love working under coLinux and have never been happy working on cygwin, But if I could cross-compile a Linux kernel under cygwin I have a client I could make very happy. If you have any references, links etc. regarding anyone who has successfully cross compiled Linux kernels on windows - using anything except colinux - or maybe QEMU, or something like that please, please provide them. I can't even succcessfully build cross tools under cywin (obviously others have succeeded at that). > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > coLinux-devel mailing list > coL...@li... > https://lists.sourceforge.net/lists/listinfo/colinux-devel > -- Dave Lynch DLA Systems Software Development: Embedded Linux 717.627.3770 dh...@dl... http://www.dlasys.net fax: 1.253.369.9244 Cell: 1.717.587.7774 Over 25 years' experience in platforms, languages, and technologies too numerous to list. "Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction." Albert Einstein |
From: Brian D. <br...@de...> - 2006-11-28 03:34:22
|
> "David H. Lynch Jr." wrote: > I have been looking for information regarding cross-compiling > linux kernels from cygwin. Thus far everything I have found says: > > 1). Don't - it is not worth the agrevation. It won't be fast, that is for sure. > 2). No one who has tried has succeeded. Provably incorrect. There are plenty of people on the Cygwin list that do this regularly, including Chris Faylor, co-lead of the project. You need a functioning cross compiler of course, but crosstool also works out of the box on Cygwin. > 3). The demands of the Linux Kernel Build system exceed > the capabilities of cygwin - file name case issues just being one of > many problems. Managed mounts solve the case issue and the problem with filenames that are illegal in windows like "CON" or "AUX". The only other possible problem would be total path length and you can solve that easily by mounting your source and build directories such that they have a short prefix. > But if I could cross-compile a Linux kernel under cygwin I have a > client I could make very happy. This question belongs on the Cygwin list. It's OT here. Brian |
From: Henry N. <Henry.Ne@Arcor.de> - 2006-11-28 10:49:13
|
Hello David, David H. Lynch Jr. wrote: > Now I would personally like to see cofs evolve to the point of > reducing my personal need for a large disk image, and since there is > some significant synergy between some aspects of colinux and the > embedded systems I am working on I might get time to do some work on > cofs - basically I want to piggyback on colinux for the Pico Hosted > development environment. I want to tunnel the cofs, conet, .. > colinux<->windows link between the windows X86 host and my ppc405 linux > target. cofs is currently not usable as compile platform. There are some limits, some are the same as you have under cygwin: - You can not create links. Kernel build need this for include/asm - Cofs works async, that means the file was updated some delays after a compiler has cloesed it. There exist a raise condition for a typical: creat foo - close foo - open foo. You can check it with a single command line on a cofs mount: rm -f foo; touch foo; ls -l foo; sleep 1; ls -l foo and see the result for differ mode and user rights. - cofs has an error on some compilers. The compiler reads behind the end of file. Some gcc use a kernel function "mmap2", instead a normal "read". This is perhaps also a async problem inside kernel. (in same coLinux session failed with gcc 3.3, but runs under gcc 3.4.x) On cofs you can loop mount image files to make your working dir grater. You can create the image for loop device directly under cofs (dd from /dev/zero). I use mkFile.exe under windows with sparse option and than I create a filesystem and mount it as loop. This allocate only the space I need, not all the empty space. You can also convert (mkSparse.exe) your typical 2GB root filesystem into a sparse file. Than it only use the allocated blocks. Sparse file tools: http://www.henrynestler.com/colinux/tools/file-utils-mingw-bin-stripped.zip -- Henry Nestler |
From: David H. L. Jr. <dh...@dl...> - 2006-11-30 09:27:08
|
Henry Nestler wrote: > Hello David, > > David H. Lynch Jr. wrote: >> Now I would personally like to see cofs evolve to the point of >> reducing my personal need for a large disk image, and since there is >> some significant synergy between some aspects of colinux and the >> embedded systems I am working on I might get time to do some work on >> cofs - basically I want to piggyback on colinux for the Pico Hosted >> development environment. I want to tunnel the cofs, conet, .. >> colinux<->windows link between the windows X86 host and my ppc405 >> linux target. > > cofs is currently not usable as compile platform. There are some > limits, some are the same as you have under cygwin: > - You can not create links. Kernel build need this for include/asm I do not recall the details, but both M$ SFU and ntfs-3g purportedly handle most of the incompatiblities - such as symbolic Links. SFU is particularly interesting as anything it does coLinux should be able to do. I am potentially interested in working to improve cofs - though as I beleive I mentioned my first task - after alot of other first tasks, is to look at adapting coLinux as a whole to become a host support for a ppc 405 Linux target, in a CardBus slot in a Windows Host. But once I have some idea what I am doing I might take a look at cofs. > - Cofs works async, that means the file was updated some delays after > a compiler has cloesed it. There exist a raise condition for a > typical: creat foo - close foo - open foo. You can check it with a > single command line on a cofs mount: > rm -f foo; touch foo; ls -l foo; sleep 1; ls -l foo > and see the result for differ mode and user rights. > - cofs has an error on some compilers. The compiler reads behind the > end of file. Some gcc use a kernel function "mmap2", instead a normal > "read". This is perhaps also a async problem inside kernel. (in same > coLinux session failed with gcc 3.3, but runs under gcc 3.4.x) > > On cofs you can loop mount image files to make your working dir > grater. You can create the image for loop device directly under cofs > (dd from /dev/zero). I am not sure I understand this ? I thought cofs was basically a filesystem driver that mapped Linux read/write/open/... to windows read/write/open. My understanding of loop drivers is they basically map a block device to a file. > > I use mkFile.exe under windows with sparse option and than I create a > filesystem and mount it as loop. This allocate only the space I need, > not all the empty space. > > You can also convert (mkSparse.exe) your typical 2GB root filesystem > into a sparse file. Than it only use the allocated blocks. > Sparse file tools: > http://www.henrynestler.com/colinux/tools/file-utils-mingw-bin-stripped.zip > This is interesting but I am not sure how it applies to my specific problem - when I am creating a root file system image for distribution, I create a blank of the size I want, zero it, then copy a working image to it. Then I use bzip2 to compress it. All the unused space should compress to near nothing whether it is a windows sparse file or filled with zero's. - I can see how sparse files would be useful for a working system. I did look at your other tardump stuff - and I have to look at it further, it might be interesting Fortunately - for the moment my client has backed down. They have decided that the 225Mb stripped compressed rootfs I created will do - I really, really did not want to have to solve this with cygwin. coLinux is much more elegant, and I would rather put effort into coLinux than cygwin. -- Dave Lynch DLA Systems Software Development: Embedded Linux 717.627.3770 dh...@dl... http://www.dlasys.net fax: 1.253.369.9244 Cell: 1.717.587.7774 Over 25 years' experience in platforms, languages, and technologies too numerous to list. "Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction." Albert Einstein |