From: Jeff H. <jha...@va...> - 2001-07-16 22:50:36
|
Just thought I would give everyone a heads up. . . I have added XFree86 version numbers to the kernel module names. This allows multiple Xserver installations to co-exist on the same machine. All the kernel modules are now of the form : <driver name>-<XFree86 Major>_<XFree86 Minor>_<XFree86 Patch>.o Please everyone which depends on the old naming (installation scripts, package maintainers), update your usage of the module name to the new convention. -Jeff |
From: Jeff H. <jha...@va...> - 2001-07-18 15:10:34
|
Jeff Hartmann wrote: > Just thought I would give everyone a heads up. . . > > I have added XFree86 version numbers to the kernel module names. This > allows multiple Xserver installations to co-exist on the same > machine. All the kernel modules are now of the form : > > <driver name>-<XFree86 Major>_<XFree86 Minor>_<XFree86 Patch>.o > > Please everyone which depends on the old naming (installation scripts, > package maintainers), update your usage of the module name to the new > convention. > > -Jeff > > > _______________________________________________ > Dri-devel mailing list > Dri...@li... > http://lists.sourceforge.net/lists/listinfo/dri-devel > I've had a private discussion with Daryll, and he seems to think that driver version numbers should be used instead of the XFree86 version number. While this might perhaps be an good alternative solution, it does have a few additional maintainance headaches. It also might have some technical issues. Heres some of the issues if we use driver version numbers: 1. We can not use any number except the major one in the version number, since older Xservers will attempt to load a different module name then required. This could be solved by the brute force method, and loading all combos of minor number until one loads. For simplicity I think this means we are restricted to the following naming scheme: <driver name>-<driver major>.o 2. Our versioning scheme will prevent a module that does not contain the correct minor revision from being used. Since minor number bumps must be backwards compatible, then perhaps leaving out the minor version isn't a big issue. We will have to be more religious about bumping the major number though, which is something we haven't done perfectly in the past. 3. We will further complicate an already complicate kernel Makefile. Each module could have multiple version numbers, and the way the defines are setup for backwards compatibility we will still have to keep the XFree86 version around. We will also have yet another place to keep driver version numbers up to date. 4. This will remove the requirement of having new module names each time XFree86 is revised. While we are currently revising the driver specific API's quite often, we probably will not do so forever. With the currently implemented scheme, we have to create new modules each time XFree86 is revised from here to eternity. 5. The DRI common layer does not know what the version number of the kernel module has to be. To implement this scheme we will have to add a mechanism for the DRI common layer to query the version needed in some manner. Using the XFree86 version number requires no such changes. What do people think, should we use driver version numbers, or stay with the planned use of the XFree86 version number? I think both have pros and cons, but I'm not 100% sure which one makes the most sense. What do other people think? -Jeff |
From: Jeff H. <jha...@va...> - 2001-07-18 18:43:54
|
Jeff Hartmann wrote: > Jeff Hartmann wrote: > >> Just thought I would give everyone a heads up. . . >> >> I have added XFree86 version numbers to the kernel module names. This >> allows multiple Xserver installations to co-exist on the same >> machine. All the kernel modules are now of the form : >> >> <driver name>-<XFree86 Major>_<XFree86 Minor>_<XFree86 Patch>.o >> >> Please everyone which depends on the old naming (installation >> scripts, package maintainers), update your usage of the module name >> to the new convention. >> >> -Jeff >> >> >> _______________________________________________ >> Dri-devel mailing list >> Dri...@li... >> http://lists.sourceforge.net/lists/listinfo/dri-devel >> Okay well I've done some more thinking about this one and I have an idea which addresses the biggest concern that Daryll had with using XFree86 version. Here goes: We stay along with the new module naming scheme using XFree86 version number, however the Xserver will attempt to load all possible modules in the following way: Current release version first All previous release versions in backwards order if the first one fails. This allows something like tdfx.o (which hasn't really ever changed since version 1.0) to still be used. All version clashes will be caught by the versioning system we have in place now. This allows a Makefile to only build the old version of a module, and not to require that it builds the newer ones. The Xserver will have to do a little more work to find the module, but this scheme should work reasonably well, and not require much more development. Can anyone poke holes in this? If I don't hear anything against this by the end of the day, this will be the scheme that is implemented. -Jeff |
From: Mike A. H. <mh...@re...> - 2001-07-18 19:52:00
|
On Wed, 18 Jul 2001, Jeff Hartmann wrote: >>> allows multiple Xserver installations to co-exist on the same >>> machine. All the kernel modules are now of the form : >>> >>> <driver name>-<XFree86 Major>_<XFree86 Minor>_<XFree86 Patch>.o >>> >>> Please everyone which depends on the old naming (installation >>> scripts, package maintainers), update your usage of the module name >>> to the new convention. >>> >>> -Jeff >>> >>> >>> _______________________________________________ >>> Dri-devel mailing list >>> Dri...@li... >>> http://lists.sourceforge.net/lists/listinfo/dri-devel >>> > >Okay well I've done some more thinking about this one and I have an idea >which addresses the biggest concern that Daryll had with using XFree86 >version. Here goes: > >We stay along with the new module naming scheme using XFree86 version >number, however the Xserver will attempt to load all possible modules in >the following way: > >Current release version first >All previous release versions in backwards order if the first one fails. > >This allows something like tdfx.o (which hasn't really ever changed >since version 1.0) to still be used. All version clashes will be caught >by the versioning system we have in place now. This allows a Makefile to >only build the old version of a module, and not to require that it >builds the newer ones. The Xserver will have to do a little more work to >find the module, but this scheme should work reasonably well, and not >require much more development. > >Can anyone poke holes in this? If I don't hear anything against this by >the end of the day, this will be the scheme that is implemented. Jeff, please hold off on locking this in stone just yet. I like the idea so far, but I would like to bounce it off our kernel team and see what they think. The more eyes that see this before the decision is final, the better IMHO. If XFree86 version numbers are used: As a distributor, we could potentially have to ship kernel DRM modules for 4.0.3 + 4.1.0(versioned as ie: radeon-4_1_0), and in the future theoretically radeon-4_1_1, radeon-4_1_2, radeon-4_1_3, radeon-4_2_0, etc. I don't think our kernel guys would go for that myself, although I haven't asked them yet. If DRM version is used, we would have to ship DRM modules for specific interface changes only. This means I believe that if the code doesn't change in an incompatible way from XFree86 4.1.0 to XFree86 4.3.5 in a given module, we would not have to change that module at all. Whatever produces the least number of modules is IMHO the best solution. I've Cc'd Gareth, Linus, Alan, Arjan also for comment. I'm going to forward your ideas to our devel team also and see if they can offer any suggestions as well. Thanks for working on this. ---------------------------------------------------------------------- Mike A. Harris Shipping/mailing address: OS Systems Engineer 190 Pittsburgh Ave., Sault Ste. Marie, XFree86 maintainer Ontario, Canada, P6C 5B3 Red Hat Inc. Phone: (705)949-2136 http://www.redhat.com ftp://people.redhat.com/mharris ---------------------------------------------------------------------- |
From: Jeff H. <jha...@va...> - 2001-07-18 20:07:59
|
Mike A. Harris wrote: > On Wed, 18 Jul 2001, Jeff Hartmann wrote: > >>>> allows multiple Xserver installations to co-exist on the same >>>> machine. All the kernel modules are now of the form : >>>> >>>> <driver name>-<XFree86 Major>_<XFree86 Minor>_<XFree86 Patch>.o >>>> >>>> Please everyone which depends on the old naming (installation >>>> scripts, package maintainers), update your usage of the module name >>>> to the new convention. >>>> >>>> -Jeff >>>> >>>> >>>> _______________________________________________ >>>> Dri-devel mailing list >>>> Dri...@li... >>>> http://lists.sourceforge.net/lists/listinfo/dri-devel >>>> >> Okay well I've done some more thinking about this one and I have an idea >> which addresses the biggest concern that Daryll had with using XFree86 >> version. Here goes: >> >> We stay along with the new module naming scheme using XFree86 version >> number, however the Xserver will attempt to load all possible modules in >> the following way: >> >> Current release version first >> All previous release versions in backwards order if the first one fails. >> >> This allows something like tdfx.o (which hasn't really ever changed >> since version 1.0) to still be used. All version clashes will be caught >> by the versioning system we have in place now. This allows a Makefile to >> only build the old version of a module, and not to require that it >> builds the newer ones. The Xserver will have to do a little more work to >> find the module, but this scheme should work reasonably well, and not >> require much more development. >> >> Can anyone poke holes in this? If I don't hear anything against this by >> the end of the day, this will be the scheme that is implemented. > > > Jeff, please hold off on locking this in stone just yet. I like > the idea so far, but I would like to bounce it off our kernel > team and see what they think. The more eyes that see this > before the decision is final, the better IMHO. > > If XFree86 version numbers are used: > > As a distributor, we could potentially have to ship kernel DRM > modules for 4.0.3 + 4.1.0(versioned as ie: radeon-4_1_0), and in > the future theoretically radeon-4_1_1, radeon-4_1_2, > radeon-4_1_3, radeon-4_2_0, etc. I don't think our kernel guys > would go for that myself, although I haven't asked them yet. > > If DRM version is used, we would have to ship DRM modules for > specific interface changes only. This means I believe that if > the code doesn't change in an incompatible way from XFree86 4.1.0 > to XFree86 4.3.5 in a given module, we would not have to change > that module at all. Whatever produces the least number of > modules is IMHO the best solution. > Since the Xserver will search through the available modules if the module doesn't change you don't have to rev it. When I finally get this all sorted out and patches off to Linus the Makefiles will not build a tdfx-4_1_1 module for example, only the tdfx.o will be built since there have been no interface changes from 4.0.3 to 4.1.1. "tdfx.o" will be loaded by the Xserver since now the Xserver will search through old modules. With this change the number of modules will be equal to if we used driver version. Besides David Dawes told me that there will be NO binary API differences between 4.1.0 and 4.1.1 (which will actually contain the versioning code.) He also says that we need to be able to use an old drm driver with a new Xserver, and that a newer drm module must function on a 4.1.0 Xserver (it doesn't have to autoload in this case though.) This basically means we can only version by XFree86 number, since the versioning by driver version is most likely going to require a binary API change in libdri.a. The DRI common layer doesn't know anything about what kernel version we need, and it would have to have that information to version things. I think that versioning by XFree86 version works just fine, and its already implemented (I have a cleanup patch to commit which makes sure old kernel modules are tried as well, it will be in CVS shortly. > > I've Cc'd Gareth, Linus, Alan, Arjan also for comment. > > I'm going to forward your ideas to our devel team also and see if > they can offer any suggestions as well. > > Thanks for working on this. No problem. -Jeff |
From: Linus T. <tor...@tr...> - 2001-07-18 21:54:05
|
On Wed, 18 Jul 2001, Jeff Hartmann wrote: > > Since the Xserver will search through the available modules if the > module doesn't change you don't have to rev it. If you don't allow compiled-in modules, and if you don't allow two different versions of the X server running at the same time on the same machine, then there is _zero_ reason for having the DRM tree included with the kernel. If it's going to be a question of "in which order do we search for the right module to load", you might as well remove me from the Cc list and just tell me to remove the DRM directory from the kernel. Because I really really do not want to have multiple tdfx drivers in my tree. Two is my limit - that's been the case for a lot of hardware over time, but even then it's usually considered a _bad_ thing and a source of confusion rather than a versioning design. We need more stricter versioning than just "which version do we load". Some people do not like kernel modules at ALL. I don't use them, for example. And even if you have modules, do you seriously want to play with switching between two different X servers, and having to load and unload modules in between, or having to worry about name clashes from having two different modules (for the same card) loaded at the same time. How are you going to handle the case (for debugging - I've done it myself, and I'm not even a X developer) where you run two different versions of X at the same time? Linus |
From: Linus T. <tor...@tr...> - 2001-07-18 21:47:22
|
On Wed, 18 Jul 2001, Mike A. Harris wrote: > > Jeff, please hold off on locking this in stone just yet. I like > the idea so far, but I would like to bounce it off our kernel > team and see what they think. The more eyes that see this > before the decision is final, the better IMHO. Actually, as far as the kernel is concerned, this is REALLY easy. Either the DRM interfaces maintain backwards compatibility, or they get dropped from the kernel source tree. End of story. There's a single exception on the grounds of "anybody can mess up once by not just thinking about it", but that's it. So right now, I'm perfectly willing to accept two DRM trees or something else, on the theory that hey, it's really hard to get right the first time, and nobody was even THINKING about backwards compability because everybody was so focused on just getting it working in the first place. But that is not something that people can continue doing forever. The kernel tree cannot have 15 different versions of DRM. This is not something we discuss - this is a FACT. Which means that when people add new interfaces, they had damn better make sure that the old interfaces still work, and are 100% backwards compatible. That way the kernel DRM version can be incremented, without breaking old X servers. What does this mean? - When you add an ioctl, you MAKE A NEW NUMBER. You NEVER EVER re-use an old one, like DRM did between 4.0.x and the CVS tree. - When you change a user-exported structure argument in incompatible ways, THAT IS A NEW IOCTL. Same way that when the "stat" structure changes in the kernel, we have to make a new sytem call for it. - When the X server starts up, it queries the version number of the particular driver, and will refuse to run with a version "too old". But there is no need for fancy version number schemes, because they are all backwards compatible. Thus, if the X server wants version 7, it should be happy with version 9. It's not about DRM version or XFree86 version, it's _only_ about the version of the particular video driver interface. So in the same tree, the Radeon driver may be at version 4, while the i810 driver is at version 9. Don't get stuck with global versions - why should you tie things up more than they deserve? Now, we already have one case where this broke, which is why we probably should have a major version number too, which indicates that things start from a clean slate. So the old 4.0.x DRM should be called version 0.0, and the new 4.1 DRM should be called 1.0. And everybody should strive for never ever even having a 2.x release. Add new functionality, but don't break old stuff. And realize that when a completely new card comes out, you just start from 1.0 for that card. You don't make a new DRM version for everybody else. They DO NOT CARE. Note that this really is not negotiable as far as the kernel is concerned. If the DRM team cannot bother with backwards compatibility, then the DRM tree from the kernel just gets dropped. There's no point to having it at all, and that means that when the kernel changes, there will be version skew, and people will be unhappy. But that's better than having confusion about fiver different versions and having to compile in the right one. And if the DRM stuff only works as modules, and only works when combined with the right version of XFree86, then what's the point of having it in the kernel tree AT ALL? I want to have one Radeon driver, compiled into the kernel, and I want it to work with both the new X server and the old one. But it's ok if that means that the kernel driver has to be the _newest_ one. It's _not_ ok if it means that I have to have two different kernels. I hope that this can happen. I hope that the people working on DRM will realize that the ioctl's they use aren't just temporary random numbers, they are driver-specific system calls. And the same way the kernel still supports the old-style 32-bit "lseek()" system call while at the same time _also_ supporting the new 64-bit llseek(), you have to do the same thing. Does this mean that you need to actively _think_ about forwards compatibility before adding an ioctl? Hell yes. Does it mean that some things have to be designed up-front a bit more sometimes? Oh, yes. Does it mean somewhat more work? Sure it does. But does everybody agree that it is worth it? I personally sure hope we do. Because I'm not in the least interested in seeing many different DRM versions in the kernel tree. I can take two. If I have to see three, I'll just say "f*ck those incompetent bastards", and let you keep track of your own damn mess. Linus |
From: Jeffrey W. B. <jw...@ac...> - 2001-07-18 22:03:01
|
On Wed, 18 Jul 2001, Linus Torvalds wrote: > And if the DRM stuff only works as modules, and only works when combined > with the right version of XFree86, then what's the point of having it in > the kernel tree AT ALL? I have often wondered this myself from the perspective of the user. The DRI stuff in the kernel releases has NEVER been useful. I have always had to use either the DRI CVS tree or, now, an XFree86 release. Even the new major release of Slackware used the kernel modules from XFree86, not from the kernel. So, why keep it in the mainline kernel sources? -jwb |
From: Arjan v. de V. <ar...@re...> - 2001-07-18 22:07:49
|
On Wed, Jul 18, 2001 at 03:00:12PM -0700, Jeffrey W. Baker wrote: > I have often wondered this myself from the perspective of the user. The > DRI stuff in the kernel releases has NEVER been useful. I have always had > to use either the DRI CVS tree or, now, an XFree86 release. Even the new > major release of Slackware used the kernel modules from XFree86, not from > the kernel. > > So, why keep it in the mainline kernel sources? So that at least r00tholes get found more easily, and that you make life on the people building their own kernels simpler. Also, not being in the main kernel doesn't mean Linus is wrong with his arguments. Those hold, imho, even if DRM is outside the kernel. The net effect will be that only distro's will actually use DRI, if they decide to bother with all the different versions, but that endusers who happen to read the howto on how to configure a kernel, will loose 3D support. This also means the DRI/Xfree people lose a lot of testers. Maybe it would work if 2.5 would be kept current with CVS, in BOTH ways. Greetings, Arjan van de Ven |
From: Jeffrey W. B. <jw...@ac...> - 2001-07-18 22:19:36
|
On Wed, 18 Jul 2001, Arjan van de Ven wrote: > On Wed, Jul 18, 2001 at 03:00:12PM -0700, Jeffrey W. Baker wrote: > > > I have often wondered this myself from the perspective of the user. The > > DRI stuff in the kernel releases has NEVER been useful. I have always had > > to use either the DRI CVS tree or, now, an XFree86 release. Even the new > > major release of Slackware used the kernel modules from XFree86, not from > > the kernel. > > > > So, why keep it in the mainline kernel sources? > > So that at least r00tholes get found more easily, and that you make life on > the people building their own kernels simpler. Also, not being in the main > kernel doesn't mean Linus is wrong with his arguments. Those hold, imho, > even if DRM is outside the kernel. > > The net effect will be that only distro's will actually use DRI, if they > decide to bother with all the different versions, but that endusers who > happen to read the howto on how to configure a kernel, will loose 3D > support. This also means the DRI/Xfree people lose a lot of testers. But that's exactly my point. The kernel's version of DRM is not useful for testing. It is always out of date relative to the DRI CVS, and if you post on dri-devel that you had such-and-such problem with the code from kernel x.y.z, you'll be told that the kernel version is out of date and either that problem is already fixed or could you please test the CVS version. Also I'm not seeing why regular users would be excluded. Lots of people use and test ALSA, and it is entirely outside the mainline kernel. Most people who currently test the DRI are building XFree86 from scratch anyway. -jwb |
From: Linus T. <tor...@tr...> - 2001-07-18 22:34:41
|
On Wed, 18 Jul 2001, Jeffrey W. Baker wrote: > > But that's exactly my point. The kernel's version of DRM is not useful > for testing. It is always out of date relative to the DRI CVS, and if you > post on dri-devel that you had such-and-such problem with the code from > kernel x.y.z, you'll be told that the kernel version is out of date and > either that problem is already fixed or could you please test the CVS > version. Note that this is at least partly due to the fact that there has been no point to even try to import things from the DRI tree (or even the standard XFree86 CVS tree, which I _do_ have uptodate on all machines I personally use, for example). Because the kernel historically has not been able to import the DRI tree due to binary incompatibilities. > Also I'm not seeing why regular users would be excluded. Lots of people > use and test ALSA, and it is entirely outside the mainline kernel. Most > people who currently test the DRI are building XFree86 from scratch > anyway. And do you ever want DRI to be a feature that people take for granted? That's the question, really. Do you want DRM to remain a "for X hackers only" thing? Or do you want to make it available to people who do want to recompile their kernel for some reason? If DRM wants to be taken for granted, then that implies that DRM has to take these kinds of versioning problems seriously. It should just work. Which implies that it should have a BIG push to make sure that things are binary compatible, and that people realize that bianry incompatibilities are really really bad because it means that you cannot mix and match things. This is not DRM-specific, or even kernel-specific. Remember why people get excited about LSF and things like that? You want your programs to work across multiple vendors, and across different versions of libraries and kernels. The library people are spending a lot of effort on being binary compatible across releases. So are the compiler people. And so are we kernel people. I don't require that people upgrade their libc when they upgrade their kernels. Even if libc obviously is the main user of most of the interfaces the kernel exports. Libc people don't require that people upgrade all their applications when the library is exported. Yes, you have major upheavals every once in a while. They're considered bad, and are done only in extreme cases to rectify major problems and design mistakes. Do you think DRM would be any different if it expects to be ubiqutous? Linus |
From: Linus T. <tor...@tr...> - 2001-07-18 22:15:57
|
On Wed, 18 Jul 2001, Jeffrey W. Baker wrote: > > On Wed, 18 Jul 2001, Linus Torvalds wrote: > > > And if the DRM stuff only works as modules, and only works when combined > > with the right version of XFree86, then what's the point of having it in > > the kernel tree AT ALL? > > I have often wondered this myself from the perspective of the user. The > DRI stuff in the kernel releases has NEVER been useful. I have always had > to use either the DRI CVS tree or, now, an XFree86 release. Even the new > major release of Slackware used the kernel modules from XFree86, not from > the kernel. > > So, why keep it in the mainline kernel sources? Indeed. I actually had this same discussion with some people during the 4.1.x CVS time, and thought it had been resolved and that the binary incompatibilities would be resolved by the time the CVS tree became a 4.1 release. Obviously, not all ends were tied and not everybody communicated well. No point in trying to rewrite history or argue about it any more. And I'm definitely not married to having DRM in the kernel tree. In particular, I'm also not married to have all versions of DRM in the kernel tree. Which means that we have multiple options: a) drop it from the kernel tree altogether. This is the ONLY option if the DRM people aren't interested in keeping backwards binary compatibility. b) replace the current works-for-4.0 DRM tree with the new one, assuming people are willing to try to maintain binary compatibility and really think of it like a system call interface c) Same as (b), except we have both the old and the new version. Now, note that there is not a "option d", to keep just the old DRM tree in the kernel. It's not useful, and if the DRM tree isn't getting maintained anyway and people have to rely on external modules, it's pointless. So "status quo" is not an option, and at some point turns into (a) - drop it. I still think it would be a pity - quite frankly we have many times seen that quality control of modules outside the kernel tree is pretty much nil and non-existant. Security bugs, updates to infrastructure changes, things like that tend to be completely ignored, because people work with the kernel tree and just ignore third-party modules because it's such a pain to hunt them down. We see that right now - there are security-fixes in the mainline kernel trees (mainly -ac, which is certainly "mainline" too ;) that apparently have had very little impact on the DRM team. Linus |
From: Gareth H. <gar...@ac...> - 2001-07-19 01:35:39
|
"Jeffrey W. Baker" wrote: > > On Wed, 18 Jul 2001, Linus Torvalds wrote: > > > And if the DRM stuff only works as modules, and only works when combined > > with the right version of XFree86, then what's the point of having it in > > the kernel tree AT ALL? > > I have often wondered this myself from the perspective of the user. The > DRI stuff in the kernel releases has NEVER been useful. I have always had > to use either the DRI CVS tree or, now, an XFree86 release. Even the new > major release of Slackware used the kernel modules from XFree86, not from > the kernel. > > So, why keep it in the mainline kernel sources? Because *it should be useful*. We fucked up on that. -- Gareth |
From: Alan C. <al...@re...> - 2001-07-19 23:34:52
|
> I have often wondered this myself from the perspective of the user. The > DRI stuff in the kernel releases has NEVER been useful. I have always had Oh its useful 99% of the time, because users generally don't care what XFree86 they get unless they hit problems. They update it a lot less than kernels and only for minor revisions. > to use either the DRI CVS tree or, now, an XFree86 release. Even the new > major release of Slackware used the kernel modules from XFree86, not from > the kernel. Yes I know. I added Slackware to my 'things I dont support'. I can't track DRI code so any bug in X11 or after running X11 on Slackware I assume to be DRI > So, why keep it in the mainline kernel sources? Maintainability. It means for example that the stupid security holes DRM has been plagued with get found |
From: Allen A. <ak...@po...> - 2001-07-20 03:00:40
|
On Thu, Jul 19, 2001 at 07:34:47PM -0400, Alan Cox wrote: | | ... It means for example that the stupid security holes DRM has | been plagued with get found For the record, security has been a major design consideration for the DRI from day one -- security is one of the main reasons there *is* a DRI project distinct from Utah GLX in the first place. Not all apparent security bugs are real -- for example, many of the problems found in the DRI code by the Stanford checker are false positives. And it's probably not obvious to folks outside the DRI project just how many security problems (usually precipitated by flawed hardware) are already solved by the DRI design and driver implementations, at significant cost in performance when necessary. Undoubtedly there are more DRI security problems to be solved, but in the meantime it's important for us to maintain mutual respect and be aware that the issue is absolutely not being ignored. Nor, in my opinion, is it being handled carelessly. Allen (Not a DRI implementor, but a close observer.) |
From: Alan C. <al...@re...> - 2001-07-20 08:54:42
|
> Undoubtedly there are more DRI security problems to be solved, but in > the meantime it's important for us to maintain mutual respect and be > aware that the issue is absolutely not being ignored. Nor, in my > opinion, is it being handled carelessly. The point stands, and its not about people being careless, its about the fact that because it was in the kernel both security holes and programming errors were found. |
From: Allen A. <ak...@po...> - 2001-07-20 16:28:03
|
On Fri, Jul 20, 2001 at 04:54:36AM -0400, Alan Cox wrote: | > Undoubtedly there are more DRI security problems to be solved, but in | > the meantime it's important for us to maintain mutual respect and be | > aware that the issue is absolutely not being ignored. Nor, in my | > opinion, is it being handled carelessly. | | The point stands, and its not about people being careless, its about the fact | that because it was in the kernel both security holes and programming errors | were found. My point is that we need to know the history and be civil during the current discussion. I got the impression that some of the folks on the kernel side don't believe the DRI people are acting in good faith, and/or don't know how much work has already gone into solving security issues (from the very beginning of the project) and therefore how much the DRI people are aware of the problems. If those matters are understood, I'm happy to move the conversation back to solving the remaining real technical issues. Allen |
From: Alan C. <al...@re...> - 2001-07-23 20:25:57
|
> current discussion. I got the impression that some of the folks on > the kernel side don't believe the DRI people are acting in good faith, Oh I believe they are acting in good faith. I have differences with them over technical matters like version management |
From: Jeff H. <jha...@va...> - 2001-07-18 22:48:46
|
Linus Torvalds wrote: > On Wed, 18 Jul 2001, Mike A. Harris wrote: > >> Jeff, please hold off on locking this in stone just yet. I like >> the idea so far, but I would like to bounce it off our kernel >> team and see what they think. The more eyes that see this >> before the decision is final, the better IMHO. > > > Actually, as far as the kernel is concerned, this is REALLY easy. > > Either the DRM interfaces maintain backwards compatibility, or they get > dropped from the kernel source tree. > > End of story. There's a single exception on the grounds of "anybody can > mess up once by not just thinking about it", but that's it. > > So right now, I'm perfectly willing to accept two DRM trees or something > else, on the theory that hey, it's really hard to get right the first > time, and nobody was even THINKING about backwards compability because > everybody was so focused on just getting it working in the first place. > > But that is not something that people can continue doing forever. > > The kernel tree cannot have 15 different versions of DRM. This is not > something we discuss - this is a FACT. > > Which means that when people add new interfaces, they had damn better make > sure that the old interfaces still work, and are 100% backwards > compatible. That way the kernel DRM version can be incremented, without > breaking old X servers. > > What does this mean? > > - When you add an ioctl, you MAKE A NEW NUMBER. You NEVER EVER re-use an > old one, like DRM did between 4.0.x and the CVS tree. > > - When you change a user-exported structure argument in incompatible > ways, THAT IS A NEW IOCTL. Same way that when the "stat" structure > changes in the kernel, we have to make a new sytem call for it. > > - When the X server starts up, it queries the version number of the > particular driver, and will refuse to run with a version "too old". But > there is no need for fancy version number schemes, because they are all > backwards compatible. Thus, if the X server wants version 7, it should > be happy with version 9. It's not about DRM version or XFree86 version, > it's _only_ about the version of the particular video driver interface. > > So in the same tree, the Radeon driver may be at version 4, while the > i810 driver is at version 9. Don't get stuck with global versions - why > should you tie things up more than they deserve? > > Now, we already have one case where this broke, which is why we probably > should have a major version number too, which indicates that things start > from a clean slate. So the old 4.0.x DRM should be called version 0.0, and > the new 4.1 DRM should be called 1.0. > > And everybody should strive for never ever even having a 2.x release. Add > new functionality, but don't break old stuff. And realize that when a > completely new card comes out, you just start from 1.0 for that card. You > don't make a new DRM version for everybody else. They DO NOT CARE. This makes sense, and is where I guess we have to go. I don't want the DRM to be removed from the kernel on my watch. I do have a concern right now though: Currently the 4.1 drm has no notion of which version the XServer is expecting. This is a very significant problem because the ioctl interface is actually the least of my concern. We should have been handling that part correctly from the beginning. The kernel modules interface with the 3D driver very tightly using a shared memory area. The layout of the shared memory area is nearly impossible to keep constant. What we need is some sort of mechanism to communicate the version the XServer is expecting, so we can insure that the sarea is laid out correctly. This will require that each revision of the sarea in a driver will have to have to duplicate code (at least the final binary will, most of this can be handled by templating) for each function which accesses the sarea. This can potentially become a huge amount of duplicate code, and I want to bring that to your attention. I would imagine that we are going to have to overload the entire driver dependant on the version number at runtime. This is probably not the design you had in mind but it is the only one I can think of right now. We already have changes in the works which drastically change the layout of the sarea for all the dma drivers. This vastly improves performance but it breaks compatibility currently. The only way we can make these changes is if we duplicate large portions of code. Since we have to go this way, I suppose the DRM kernel module will have to assume if it doesn't get this special ioctl telling it what version it should use, that it should assume that it is running on a 4.1 XServer. Other revisions of the XServer must send this ioctl down to the kernel module for it to function correctly. > > Note that this really is not negotiable as far as the kernel is concerned. > If the DRM team cannot bother with backwards compatibility, then the DRM > tree from the kernel just gets dropped. There's no point to having it at > all, and that means that when the kernel changes, there will be version > skew, and people will be unhappy. But that's better than having confusion > about fiver different versions and having to compile in the right one. > > And if the DRM stuff only works as modules, and only works when combined > with the right version of XFree86, then what's the point of having it in > the kernel tree AT ALL? > > I want to have one Radeon driver, compiled into the kernel, and I want it > to work with both the new X server and the old one. But it's ok if that > means that the kernel driver has to be the _newest_ one. It's _not_ ok if > it means that I have to have two different kernels. > > I hope that this can happen. I hope that the people working on DRM will > realize that the ioctl's they use aren't just temporary random numbers, > they are driver-specific system calls. This shouldn't have happened, however look at my comments on the sarea. The sarea is really the changing interface between the kernel and the device driver that I worry about. The ioctl's should have never been a problem, but go reused because we didn't have strict guidelines in place. > And the same way the kernel still > supports the old-style 32-bit "lseek()" system call while at the same time > _also_ supporting the new 64-bit llseek(), you have to do the same thing. > > Does this mean that you need to actively _think_ about forwards > compatibility before adding an ioctl? Hell yes. Does it mean that some > things have to be designed up-front a bit more sometimes? Oh, yes. Does it > mean somewhat more work? Sure it does. > > But does everybody agree that it is worth it? I personally sure hope we > do. Because I'm not in the least interested in seeing many different DRM > versions in the kernel tree. I can take two. If I have to see three, I'll > just say "f*ck those incompetent bastards", and let you keep track of your > own damn mess. > Agreed. -Jeff |
From: Keith W. <ke...@va...> - 2001-07-18 23:12:41
|
Jeff Hartmann wrote: > > Linus Torvalds wrote: > > > On Wed, 18 Jul 2001, Mike A. Harris wrote: > > > >> Jeff, please hold off on locking this in stone just yet. I like > >> the idea so far, but I would like to bounce it off our kernel > >> team and see what they think. The more eyes that see this > >> before the decision is final, the better IMHO. > > > > > > Actually, as far as the kernel is concerned, this is REALLY easy. > > > > Either the DRM interfaces maintain backwards compatibility, or they get > > dropped from the kernel source tree. > > > > End of story. There's a single exception on the grounds of "anybody can > > mess up once by not just thinking about it", but that's it. > > > > So right now, I'm perfectly willing to accept two DRM trees or something > > else, on the theory that hey, it's really hard to get right the first > > time, and nobody was even THINKING about backwards compability because > > everybody was so focused on just getting it working in the first place. > > > > But that is not something that people can continue doing forever. > > > > The kernel tree cannot have 15 different versions of DRM. This is not > > something we discuss - this is a FACT. > > > > Which means that when people add new interfaces, they had damn better make > > sure that the old interfaces still work, and are 100% backwards > > compatible. That way the kernel DRM version can be incremented, without > > breaking old X servers. > > > > What does this mean? > > > > - When you add an ioctl, you MAKE A NEW NUMBER. You NEVER EVER re-use an > > old one, like DRM did between 4.0.x and the CVS tree. > > > > - When you change a user-exported structure argument in incompatible > > ways, THAT IS A NEW IOCTL. Same way that when the "stat" structure > > changes in the kernel, we have to make a new sytem call for it. > > > > - When the X server starts up, it queries the version number of the > > particular driver, and will refuse to run with a version "too old". But > > there is no need for fancy version number schemes, because they are all > > backwards compatible. Thus, if the X server wants version 7, it should > > be happy with version 9. It's not about DRM version or XFree86 version, > > it's _only_ about the version of the particular video driver interface. > > > > So in the same tree, the Radeon driver may be at version 4, while the > > i810 driver is at version 9. Don't get stuck with global versions - why > > should you tie things up more than they deserve? > > > > Now, we already have one case where this broke, which is why we probably > > should have a major version number too, which indicates that things start > > from a clean slate. So the old 4.0.x DRM should be called version 0.0, and > > the new 4.1 DRM should be called 1.0. > > > > And everybody should strive for never ever even having a 2.x release. Add > > new functionality, but don't break old stuff. And realize that when a > > completely new card comes out, you just start from 1.0 for that card. You > > don't make a new DRM version for everybody else. They DO NOT CARE. > > This makes sense, and is where I guess we have to go. I don't want the > DRM to be removed from the kernel on my watch. I do have a concern > right now though: > > Currently the 4.1 drm has no notion of which version the XServer is > expecting. This is a very significant problem because the ioctl > interface is actually the least of my concern. We should have been > handling that part correctly from the beginning. The kernel modules > interface with the 3D driver very tightly using a shared memory area. > The layout of the shared memory area is nearly impossible to keep > constant. What we need is some sort of mechanism to communicate the > version the XServer is expecting, so we can insure that the sarea is > laid out correctly. This will require that each revision of the sarea > in a driver will have to have to duplicate code (at least the final > binary will, most of this can be handled by templating) for each > function which accesses the sarea. This can potentially become a huge > amount of duplicate code, and I want to bring that to your attention. I > would imagine that we are going to have to overload the entire driver > dependant on the version number at runtime. This is probably not the > design you had in mind but it is the only one I can think of right now. > > We already have changes in the works which drastically change the layout > of the sarea for all the dma drivers. This vastly improves performance > but it breaks compatibility currently. The only way we can make these > changes is if we duplicate large portions of code. I'm actually pretty dubious about the viability of the sarea for communicating with the kernel module under the constraints of backwards compatibility. As I don't see how it can be made to work, I think we've got to look at dropping it for that purpose altogether, retaining the sarea to just hold the lock and doing all other communications through the ioctls. I doubt this will have a major impact on performance, though I obviously once believed it would... Keith |
From: Gareth H. <gar...@ac...> - 2001-07-19 01:41:58
|
Keith Whitwell wrote: > > I'm actually pretty dubious about the viability of the sarea for communicating > with the kernel module under the constraints of backwards compatibility. As I > don't see how it can be made to work, I think we've got to look at dropping it > for that purpose altogether, retaining the sarea to just hold the lock and > doing all other communications through the ioctls. I doubt this will have a > major impact on performance, though I obviously once believed it would... Agreed on both counts. -- Gareth |
From: Jens O. <je...@je...> - 2001-07-19 17:58:06
|
Linus Torvalds wrote: > Either the DRM interfaces maintain backwards compatibility, or they get > dropped from the kernel source tree. > Which means that when people add new interfaces, they had damn better make > sure that the old interfaces still work, and are 100% backwards > compatible. That way the kernel DRM version can be incremented, without > breaking old X servers. Linus, I will take the heat for letting the versioning on our interfaces roll too quickly. Early on, we were unstable and needed the latitude--but now, you're absolutely right, we need to support stable interfaces. This will place a much larger burden on a small number of graphics subsystem developers that are already under an enormous work loads. However, our inability to address this issue effectively over the last year has hurt the broader acceptance of open source graphics solutions in the industry. This is an area where we as the DRI development team can change our priorities and make a substantial difference. Keith Whitwell wrote: > I'm actually pretty dubious about the viability of the sarea for communicating > with the kernel module under the constraints of backwards compatibility. As I > don't see how it can be made to work, I think we've got to look at dropping it > for that purpose altogether, retaining the sarea to just hold the lock and > doing all other communications through the ioctls. I doubt this will have a > major impact on performance, though I obviously once believed it would... Keith, By viability, do you mean the amount of work supporting both versions of the interface? Or is there a technical reason these two versions can't co-exist? Can you help me understand how changing the SAREA is any different than changing the layout of a data structure passed via an IOCTL? Please be gentle, I haven't touched this code in almost three years :-/ Regards, Jens -- /\ Jens Owen / \/\ _ je...@je... / \ \ \ Steamboat Springs, Colorado |
From: Keith W. <ke...@va...> - 2001-07-19 18:17:40
|
Jens Owen wrote: > > Keith Whitwell wrote: > > > I'm actually pretty dubious about the viability of the sarea for communicating > > with the kernel module under the constraints of backwards compatibility. As I > > don't see how it can be made to work, I think we've got to look at dropping it > > for that purpose altogether, retaining the sarea to just hold the lock and > > doing all other communications through the ioctls. I doubt this will have a > > major impact on performance, though I obviously once believed it would... > > Keith, > > By viability, do you mean the amount of work supporting both versions of > the interface? Or is there a technical reason these two versions can't > co-exist? Can you help me understand how changing the SAREA is any > different than changing the layout of a data structure passed via an > IOCTL? Please be gentle, I haven't touched this code in almost three > years :-/ I'm refering mainly to the amount of work and the potential for fuckups. The problem with changing the sarea is that changing any aspect of it will usually change all aspects of it as everything's now at a different offset. The analogy is modifying one ioctl causes all ioctls to change. A partial solution is obvious: Never change the contents of the sarea, but only append fields to the end of it. This is analogous to 'never change an ioctl, just add new ones (as infrequently as possible)'. Perhaps now that things have stabilized signficantly this is possible, but in the days of rapid versioning, we would have had a rapidly growing sarea -- the trouble is that the sarea is a real physical resource (locked down pages), and growing it has a cost, unlike adding ioctl numbers. Keith |
From: Linus T. <tor...@tr...> - 2001-07-19 18:34:02
|
On Thu, 19 Jul 2001, Keith Whitwell wrote: > > The problem with changing the sarea is that changing any aspect of it will > usually change all aspects of it as everything's now at a different offset. > The analogy is modifying one ioctl causes all ioctls to change. Shared memory is always painful - the whole point of shared memory tends to be to share internal data structures without having to copy them around and marshall them into a nice portable implementation. > A partial solution is obvious: Never change the contents of the sarea, but > only append fields to the end of it. Also, please be careful when adding things to the shared area, and ask yourself a few things. Is it _really_ performance critical? Does it have to be shared? Also, shared memory implementations tend to have _really_ subtle security issues, and let's say that you use a shared memory area for passing in data that ends up being used by something asynchronous like an interrupt routine or directly by a DMA hardware engine or something. That's the perfect example of something where you as a software engineer might think "Oh, let's just let the user put the data in the right format in the first place, and we can just validate it and let the engine rip". Bzzt! Wrong answer. You cannot validate shared memory. Another thread may be changing it maliciously at the same time, so you HAVE to copy it away to a private area anyway while you're validating it. At which point the whole point of using shared memory is completely lost, and your shared memory optimization actually turns out to be a pessimization on many architectures (shared memory has to have stricter cache consistency guarantees on some architectures, so it can actually be slower to access from a pure CPU hardware standpoint). So be very very careful about shared memory even if you ignore the versioning problems. To a first-order approximation you should have the rule that shared memory is used for - locks - mapping in "safe" memory (ie things like a frame buffer) directly into the application. Putting things like drawing engine command lists in shared memory (I don't know if DRI does) would count as a REALLY bad mistake from a security standpoint. Can you say "Microsoft Windows" five times really quickly? And security is the _one_ place where performance simply doesn't matter. If it's not secure, it doesn't matter _how_ fast it is, because it is not working right. Linus |
From: Keith W. <ke...@va...> - 2001-07-19 18:54:12
|
Linus Torvalds wrote: > > On Thu, 19 Jul 2001, Keith Whitwell wrote: > > > > The problem with changing the sarea is that changing any aspect of it will > > usually change all aspects of it as everything's now at a different offset. > > The analogy is modifying one ioctl causes all ioctls to change. > > Shared memory is always painful - the whole point of shared memory tends > to be to share internal data structures without having to copy them around > and marshall them into a nice portable implementation. > > > A partial solution is obvious: Never change the contents of the sarea, but > > only append fields to the end of it. > > Also, please be careful when adding things to the shared area, and ask > yourself a few things. Is it _really_ performance critical? Does it have > to be shared? > > Also, shared memory implementations tend to have _really_ subtle security > issues, and let's say that you use a shared memory area for passing in > data that ends up being used by something asynchronous like an interrupt > routine or directly by a DMA hardware engine or something. > > That's the perfect example of something where you as a software engineer > might think "Oh, let's just let the user put the data in the right format > in the first place, and we can just validate it and let the engine rip". > > Bzzt! Wrong answer. You cannot validate shared memory. Another thread may > be changing it maliciously at the same time, so you HAVE to copy it away > to a private area anyway while you're validating it. At which point the > whole point of using shared memory is completely lost, and your shared > memory optimization actually turns out to be a pessimization on many > architectures (shared memory has to have stricter cache consistency > guarantees on some architectures, so it can actually be slower to access > from a pure CPU hardware standpoint). > > So be very very careful about shared memory even if you ignore the > versioning problems. To a first-order approximation you should have the > rule that shared memory is used for > - locks > - mapping in "safe" memory (ie things like a frame buffer) directly into > the application. > > Putting things like drawing engine command lists in shared memory (I don't > know if DRI does) would count as a REALLY bad mistake from a security > standpoint. Can you say "Microsoft Windows" five times really quickly? > > And security is the _one_ place where performance simply doesn't matter. > If it's not secure, it doesn't matter _how_ fast it is, because it is not > working right. Linus, I appreciate the comments. We've put a fair bit of thought into the security aspects of what we do with the various drivers, and looking at the possibilities for malicious use of shared memory (both the sarea and agp/framebuffer regions) has been high on the list. I'm not saying that we've caught everything, of course, but we've identified some pretty thorny issues both in our modules and in the hardware (of various vendors) itself... Keith |