user-union-discuss Mailing List for user-union
Status: Beta
Brought to you by:
dwheeler
Menu
▾
▴
user-union-discuss — Discuss the user-union tool
You can subscribe to this list here.
| 2014 |
Jan
|
Feb
(3) |
Mar
(5) |
Apr
(20) |
May
|
Jun
(22) |
Jul
|
Aug
|
Sep
(43) |
Oct
(11) |
Nov
(6) |
Dec
(4) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2015 |
Jan
|
Feb
(7) |
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Stas S. <st...@li...> - 2015-02-08 00:27:48
|
08.02.2015 00:47, David A. Wheeler пишет: > On Thu, 05 Feb 2015 20:27:54 +0300, Stas Sergeev <st...@li...> wrote: >> Hi, attached are the patches I had to do when >> porting my project from uclibc to glibc. > Excellent. All but patch #3 included and posted (per your later request). And this one? http://sourceforge.net/p/user-union/mailman/message/33345291/ |
|
From: David A. W. <dwh...@dw...> - 2015-02-07 21:47:35
|
On Thu, 05 Feb 2015 20:27:54 +0300, Stas Sergeev <st...@li...> wrote: > Hi, attached are the patches I had to do when > porting my project from uclibc to glibc. Excellent. All but patch #3 included and posted (per your later request). --- David A. Wheeler |
|
From: David A. W. <dwh...@dw...> - 2015-02-05 22:53:51
|
On February 5, 2015 1:09:54 PM EST, Stas Sergeev <st...@li...> wrote: >05.02.2015 20:27, Stas Sergeev пишет: >> Hi, attached are the patches I had to do when >> porting my project from uclibc to glibc. >Oh, please ignore the patch that disables C99. >It appeared -D_FORTIFY_SOURCE=2 together with -std=c99 >makes the compilation to fail. I removed -D_FORTIFY_SOURCE=2 >from my CFLAGS instead. That's odd. Any idea why D_FORTIFY_SOURCE=2 would cause a failure? --- David A.Wheeler |
|
From: Stas S. <st...@li...> - 2015-02-05 22:38:48
|
06.02.2015 01:32, David A. Wheeler пишет: > On February 5, 2015 1:09:54 PM EST, Stas Sergeev <st...@li...> wrote: > > 05.02.2015 20:27, Stas Sergeev пишет: > > Hi, attached are the patches I had to do when porting my > project from uclibc to glibc. > > Oh, please ignore the patch that disables C99. > It appeared -D_FORTIFY_SOURCE=2 together with -std=c99 > makes the compilation to fail. I removed -D_FORTIFY_SOURCE=2 > from my CFLAGS instead. > > That's odd. Any idea why D_FORTIFY_SOURCE=2 would cause a failure? Not really. Please use the attached test-case to find out. Compile it with cc -std=c99 -D_FORTIFY_SOURCE=2 -Os -Wall getcw.c |
|
From: Stas S. <st...@li...> - 2015-02-05 18:24:54
|
05.02.2015 20:27, Stas Sergeev пишет: > Hi, attached are the patches I had to do when > porting my project from uclibc to glibc. Oh, please ignore the patch that disables C99. It appeared -D_FORTIFY_SOURCE=2 together with -std=c99 makes the compilation to fail. I removed -D_FORTIFY_SOURCE=2 from my CFLAGS instead. |
|
From: Stas S. <st...@li...> - 2015-02-05 17:28:15
|
Hi, attached are the patches I had to do when porting my project from uclibc to glibc. |
|
From: Stas S. <st...@li...> - 2015-02-04 15:21:28
|
Hi David. This is another attempt to solve the atomicity problems that arise when copying files to overlay. |
|
From: David A. W. <dwh...@dw...> - 2014-12-14 01:11:08
|
Excellent. Applied. On December 11, 2014 4:59:07 AM EST, Stas Sergeev <st...@li...> wrote: >08.12.2014 03:41, David A. Wheeler пишет: >> On Fri, 05 Dec 2014 23:49:57 +0300, Stas Sergeev <st...@li...> >wrote: >>> Hi David, here's the drop of my latest development. >> Awesome! They all look great, and pass all tests. Pulled and posted. >Here's another small patch. --- David A.Wheeler |
|
From: Stas S. <st...@li...> - 2014-12-11 10:57:48
|
08.12.2014 03:41, David A. Wheeler пишет: > On Fri, 05 Dec 2014 23:49:57 +0300, Stas Sergeev <st...@li...> wrote: >> Hi David, here's the drop of my latest development. > Awesome! They all look great, and pass all tests. Pulled and posted. Here's another small patch. |
|
From: David A. W. <dwh...@dw...> - 2014-12-07 23:41:23
|
On Fri, 05 Dec 2014 23:49:57 +0300, Stas Sergeev <st...@li...> wrote: > Hi David, here's the drop of my latest development. Awesome! They all look great, and pass all tests. Pulled and posted. --- David A. Wheeler |
|
From: Stas S. <st...@li...> - 2014-12-05 20:50:31
|
Hi David, here's the drop of my latest development. |
|
From: Stas S. <st...@li...> - 2014-11-14 21:35:08
|
14.11.2014 19:36, David A. Wheeler пишет: > On Fri, 14 Nov 2014 18:18:59 +0400, Stas Sergeev <st...@li...> wrote: >> Hi, please apply. > Awesome. Pulled and posted. Adding remove() is a no-brainer. And suddenly I recalled that EXIST should now be used for it. :) I'll properly test the patch next week only though. |
|
From: David A. W. <dwh...@dw...> - 2014-11-14 17:02:44
|
I said: > > Eventually it'd probably be good to identify a total list of filename-using functions. On Fri, 14 Nov 2014 19:55:01 +0400, Stas Sergeev <st...@li...> wrote: > I think we can extract it from fakechroot. Useful, though I know it doesn't wrap some key functions. > Otherwise I don't know how. grepping the glibc headers might be boring. Looking at the Linux kernel system calls, and the stdio.h interface, would probably be a really good start. In the end it just has to be "good enough". Lots of calls are rarely used in practice. --- David A. Wheeler |
|
From: Stas S. <st...@li...> - 2014-11-14 16:55:25
|
14.11.2014 20:36, David A. Wheeler пишет: > On Fri, 14 Nov 2014 18:18:59 +0400, Stas Sergeev <st...@li...> wrote: >> Hi, please apply. > Awesome. Pulled and posted. Adding remove() is a no-brainer. > > Eventually it'd probably be good to identify a total list of filename-using functions. I think we can extract it from fakechroot. Otherwise I don't know how. grepping the glibc headers might be boring. |
|
From: David A. W. <dwh...@dw...> - 2014-11-14 16:36:16
|
On Fri, 14 Nov 2014 18:18:59 +0400, Stas Sergeev <st...@li...> wrote: > Hi, please apply. Awesome. Pulled and posted. Adding remove() is a no-brainer. Eventually it'd probably be good to identify a total list of filename-using functions. I suspect it includes many functions not actually used in practice, though. --- David A. Wheeler |
|
From: Stas S. <st...@li...> - 2014-11-14 15:19:22
|
Hi, please apply. |
|
From: David A. W. <dwh...@dw...> - 2014-11-14 03:40:36
|
On Thu, 13 Nov 2014 15:46:58 +0400, Stas Sergeev <st...@li...> wrote: > Hi, the attached patches allow to skip or fail the > redirection. They close the sf bug #2 and also fix > the horrible data corruption problem with unlink(). Excellent. I pulled them all, and closed bug #2. --- David A. Wheeler |
|
From: Stas S. <st...@li...> - 2014-10-31 21:04:19
|
30.10.2014 06:56, David A. Wheeler пишет: > Okay, that seems reasonable for now. And the patch is... not pushed? |
|
From: David A. W. <dwh...@dw...> - 2014-10-30 03:57:38
|
Okay, that seems reasonable for now. On October 28, 2014 4:16:54 PM EDT, Stas Sergeev <st...@li...> wrote: >27.10.2014 01:58, David A. Wheeler пишет: >> On Mon, 27 Oct 2014 02:28:08 +0400, Stas Sergeev <st...@li...> >wrote: >>> OK. >>> Maybe instead of repeating creation, one needs to add the unlink() >call >>> before open()? Also, to make it thread-safe, perhaps the suffix can >>> include tid. >> Neither would help. You need to create random filenames and put them >in a loop. >But the unlink patch is attached nevertheless. >I am really afraid that your O_EXCL patch can break the >whole thing if the stalled tmp file exists. >This has nothing to do with the security - a plain bug fix. --- David A.Wheeler |
|
From: Stas S. <st...@li...> - 2014-10-28 21:18:45
|
27.10.2014 01:58, David A. Wheeler пишет: > On Mon, 27 Oct 2014 02:28:08 +0400, Stas Sergeev <st...@li...> wrote: >> OK. >> Maybe instead of repeating creation, one needs to add the unlink() call >> before open()? Also, to make it thread-safe, perhaps the suffix can >> include tid. > Neither would help. You need to create random filenames and put them in a loop. But the unlink patch is attached nevertheless. I am really afraid that your O_EXCL patch can break the whole thing if the stalled tmp file exists. This has nothing to do with the security - a plain bug fix. |
|
From: Stas S. <st...@li...> - 2014-10-27 00:20:54
|
27.10.2014 02:58, David A. Wheeler пишет: > On Mon, 27 Oct 2014 02:28:08 +0400, Stas Sergeev <st...@li...> wrote: >> 27.10.2014 01:29, David A. Wheeler пишет: >>> All 5 patches pulled! >> Thanks. >> Note that not all are very well tested. >> For example the alias thingy looks useless on uclibc, >> so it may be a good idea if you double-check that the >> trick works as expected. > I always run the test suite on Linux+GNU lib; the code seems to work in my setups. > If it's not tested, that's something that needs to be added to the test suite. I wonder if the test suite covers the need for _# and __# aliases. I think it is something esoteric, not covered by your test suite. >> OK. >> Maybe instead of repeating creation, one needs to add the unlink() call >> before open()? Also, to make it thread-safe, perhaps the suffix can >> include tid. > Neither would help. You need to create random filenames and put them in a loop. > The tid, in particular, doesn't help, since an attacker can guess tids. I didn't mean security-wise. For robustness you may need to first call unlink() to minimize the chances of O_EXCL failure. Then you may need to add tid to avoid threads stepping on each other's feet. These are not the security concerns, I'll leave the security concerns to you. :) But the above suggestions should still stay IMHO. They will also minimize the chance of your loop to actually ever loop. >> I think GNU extensions are the most portable extensions. >> Others are likely pretty much not, and are most likely useless for >> our purposes anyway. > GNU is great, but not everything is GNU, and I doubt that all > the GNU extensions are implemented elsewhere. We need to have this > working on GNU systems, but I want to expand as far as we can reasonably do so. Then I would suggest a case-by-case approach: when something breaks, enable the needed extension, ifdef the replacement code, etc. I don't trust the "handle-all" approach in this case. E.g. having the cygwin hack out of ifdefs will really-really unlikely to help in some non-cygwin case. Enabling system extensions will very-very unlikely help on systems where the GNU extensions are not supported (without coding the proper replacement of course) etc. Your example with shl_load() just confirms this: no matter how many extensions you enable on hp-ux, you will need to write the replacement code for dlopen() under the proper ifdefs. And then perhaps no new extension will need to be enabled at all. :) |
|
From: David A. W. <dwh...@dw...> - 2014-10-26 22:58:40
|
On Mon, 27 Oct 2014 02:28:08 +0400, Stas Sergeev <st...@li...> wrote: > 27.10.2014 01:29, David A. Wheeler пишет: > > All 5 patches pulled! > Thanks. > Note that not all are very well tested. > For example the alias thingy looks useless on uclibc, > so it may be a good idea if you double-check that the > trick works as expected. I always run the test suite on Linux+GNU lib; the code seems to work in my setups. If it's not tested, that's something that needs to be added to the test suite. > > However, 0001-make-COW-atomic-by-the-use-of-tmp-file.patch > > uses a fixed filename when it creates a tempfile.... > OK. > Maybe instead of repeating creation, one needs to add the unlink() call > before open()? Also, to make it thread-safe, perhaps the suffix can > include tid. Neither would help. You need to create random filenames and put them in a loop. The tid, in particular, doesn't help, since an attacker can guess tids. More info here: http://www.dwheeler.com/secure-class/Secure-Programs-HOWTO/avoid-race.html > Anyway, I am leaving these security researches up to you. :) I'm hoping to get to it, but for now, documenting the problem is the first step. It'd need to get fixed before giving it a version 1.0 number. > Surely but I think they provide the sufficient level of GNU compatibility. Not necessarily. They certainly didn't when I used them. > Those extensions that are specific to solaris and are not part of > _GNU_SOURCE (which includes everything, not only GNU extensions) > are likely too obscure and unportable to even consider, not speaking > about enabling them by default. Or do you have the evidence that > _GNU_SOURCE in solaris doesn't enable something it does on linux? I no longer have direct access to them, but my recollection is that they do not. I know that HP-UX doesn't have dlopen, but instead has shl_load: https://www.gnu.org/software/libtool/manual/libtool.html#Using-libltdl > I think GNU extensions are the most portable extensions. > Others are likely pretty much not, and are most likely useless for > our purposes anyway. GNU is great, but not everything is GNU, and I doubt that all the GNU extensions are implemented elsewhere. We need to have this working on GNU systems, but I want to expand as far as we can reasonably do so. --- David A. Wheeler |
|
From: Stas S. <st...@li...> - 2014-10-26 22:30:02
|
27.10.2014 01:29, David A. Wheeler пишет: > All 5 patches pulled! Thanks. Note that not all are very well tested. For example the alias thingy looks useless on uclibc, so it may be a good idea if you double-check that the trick works as expected. > However, 0001-make-COW-atomic-by-the-use-of-tmp-file.patch > uses a fixed filename when it creates a tempfile. In many environments that's fine, > but if there's an attacker who can manipulate that temp file, that'd be a problem. > I've added O_EXCL, so that at least the attacker can't own the temp file. > In the longer term there should be a loop that repeatedly tries to create the file. > I did that as a separate patch... for now, O_EXCL removes the worst problems, and > there's a comment in the code and log file about it. OK. Maybe instead of repeating creation, one needs to add the unlink() call before open()? Also, to make it thread-safe, perhaps the suffix can include tid. Anyway, I am leaving these security researches up to you. :) There seem to still be the undebugged atomicity problems that I need to solve. >>> Again, I'm happy to use GNU extensions if they help, but I'd like to >>> minimize the effort to port it to other systems. >> I've yet to see any such a system and a port. >> I think most systems these days provide some gcc/glibc compatibility. >> So I am really not sure what are the concerns. > Market share of Unix systems like Solaris are very slowly dwindling in the server world, but there's still Surely but I think they provide the sufficient level of GNU compatibility. Those extensions that are specific to solaris and are not part of _GNU_SOURCE (which includes everything, not only GNU extensions) are likely too obscure and unportable to even consider, not speaking about enabling them by default. Or do you have the evidence that _GNU_SOURCE in solaris doesn't enable something it does on linux? > Anyway, I just want to make the majority of the code to be as portable as reasonably possible. I think GNU extensions are the most portable extensions. Others are likely pretty much not, and are most likely useless for our purposes anyway. |
|
From: David A. W. <dwh...@dw...> - 2014-10-26 21:29:55
|
All 5 patches pulled! However, 0001-make-COW-atomic-by-the-use-of-tmp-file.patch uses a fixed filename when it creates a tempfile. In many environments that's fine, but if there's an attacker who can manipulate that temp file, that'd be a problem. I've added O_EXCL, so that at least the attacker can't own the temp file. In the longer term there should be a loop that repeatedly tries to create the file. I did that as a separate patch... for now, O_EXCL removes the worst problems, and there's a comment in the code and log file about it. > > Again, I'm happy to use GNU extensions if they help, but I'd like to > > minimize the effort to port it to other systems. > I've yet to see any such a system and a port. > I think most systems these days provide some gcc/glibc compatibility. > So I am really not sure what are the concerns. Market share of Unix systems like Solaris are very slowly dwindling in the server world, but there's still a lot of them. Fourth quarter numbers (I believe 2013) say: Unix server share (NOT Linux) was 13.6%, Linux 28.5%, Windows 45.7%. http://www.zdnet.com/oracle-launches-solaris-11-2-exadata-openstack-fueled-relevance-7000028897/ I.e., there are about half as many Unix systems as Linux systems in the server market. Now in the *embedded* world Unix systems are completely non-existent, but I think it's clear that user-union would be useful on some servers too. Anyway, I just want to make the majority of the code to be as portable as reasonably possible. --- David A. Wheeler |
|
From: David A. W. <dwh...@dw...> - 2014-10-13 22:57:31
|
Sorry for the delay, I've been doing Shellshock-related stuff for a little while. I've pulled all 3 of your patches, and posted them to the site. Thanks!!! Per the previous emails: > 28.09.2014 07:56, David A. Wheeler: > > I want to take advantage of any extensions where they exist, On Sun, 28 Sep 2014 14:32:02 +0400, Stas Sergeev <st...@li...> wrote: > Could you please list an advantages you get from an extensions > that are not used in the code? The potential disadvantages are > an overhead. And the advantages are ... ? :) Well, the intent is that the other extensions will be *eventually* used in the code. But there's no problem in not enabling other extensions until they're actually used. So I'm fine with just using AC_GNU_SOURCE for now; eventually that may need to be replaced with AC_USE_SYSTEM_EXTENSIONS. > How come it is commented to affect only cygwin, but is instead > made as global? Oh, I'm fine with limiting it to Cygwin, as you did in your patch. Thanks, pulled. > On systems that do _not_ support GNU extensions, user-union > will never work. Do you disagree? The *current* code requires some GNU extensions, but not many, and alternative systems generally have similar functionality. IIRC, the ability to call down to the "next" library, or at least the underlying C library, is very common, even though there's no officially-portable way to do it. Again, I'm happy to use GNU extensions if they help, but I'd like to minimize the effort to port it to other systems. --- David A. Wheeler |
1 message has been excluded from this view by a project administrator.