Now that Vista is also restricting who and what can make changes in the general software installation area, it seems now is a good time to revisit the Plugins folder location issue.
To address a couple of issues I would suggest something along the following lines:
1. Have a user-settable preference which lists the locationS of the Plugins folderS.
2. Have a structure which handles versions correctly.
In this way, a large variety of structures and combinations can be accomodated:
A) Global plugins. These are kept in the <AOI>/Plugins folder for any given installation. This area will increasingly be unmodifiable by non-admin users.
B) per-user plugins. These can be stored anywhere, but <USER-HOME>/AOI/Plugins or similar would be typical
C) shared user-updatable Plugins folders. Multiple users add a particular path to their Plugin list, ensure averyone has the required permissions, and away you go!
This approach would also provide a more deterministic solution to the "where-oh-where is the Plugins folder?" problem. Instead of reverse-engineering the location from the ArtOfIllusion.jar location, the paths are explicitly listed in an easily preferences located file.
I'd go for something *really* simple. The problem is always the same : if you do not store information within AoI distro and somewhere else, how do you cope with different versions ? I mean let's suppose a user stores plugin in HOME/Plugins. How can he distinguish between his 2.4 plugins and 2.5 plugins?
So my best guess is first of all check AoI/Plugins for write access. If this works then silently store all plugins there. If access is denied, then store into .aoi/2.x/Plugins. SPManager should tell the user where is stored which plugin.
Of course .aoi/2.x/Plugins location is valid for unices only. For Vista I suppose there is a convention for user specific files. Let's hope there is one, it can't be such a bad OS (or can it?)
I'd stop right there and I think the user shouldn't be bothered about where his plugins are stored unless he wants to put them on a pendrive, in which case the SPManager tells him where to look.
I would like to keep it so that i can have a number of AoI installed and using there own plugin dirs. In other words keeping the ablitiy to locally install under the users home dir like i do in linux.
I would be very problematic if you install a new versions, it freaks out because the plugins are out of date, you update them, and then a older versions stops working.
I too was thinking of something simple, and definitely with version awareness.
> 2. Have a structure which handles versions correctly.
My concept was something like:
prefs file contains the line:
And the /home/nik directory contains the following:
The sub-directory names "install-2007-Nov-11-2.5" and "install-2007-Nov-14-2.5.1beta" would be generated by AOI on first startup.
Obviously this can be adjusted/refined as others point out other needs or requirements.
So, we have already established fairly clearly that we should *not* by relying on, nor even encouraging giving users write permissions to /usr/... under linux, which is where shared software is typically installed. Similarly now Vista is restricting user write permissions to shared software areas (C:\Program Files). So increasingly, any test for write permissions to the AOI installation Plugins directory is going to return false. So I am suggesting we plan for this and come up with a clear and robust design for that.
And please don't forget that there are still situations in which AOI cannot determine the location of the AOI installation directory, and so starts up with no plugins available. If we're going to make changes to how we find plugins, let's at least consider a design which eliminates this problem.
How about this design:
- Built in plugins still go in ArtOfIllusion/Plugins. This means that installing the application still consists of putting a single folder in a single location. It doesn't need to worry about putting different files in different places.
- User installed plugins go in .aoi/Plugins/2.5 (or whatever the appropriate version number is).
- The version number in the above path should be the major version. A minor version should not break compatibility with plugins, and users shouldn't have to reinstall their plugins when a minor version comes out.
- If there are two versions of the same plugin, one in each location, the one in the application directory is ignored. This allows users to install upgrades of the preinstalled plugins.
- SPManager could potentially take care of migrating your plugins automatically when a new version of AoI comes out. It would look at each plugin you had installed for the old version, check whether it was compatible with the new version, and if not, see whether there was a new version that was compatible.
100% yes, except maybe for Windows Vista which may have a different user file location. Can anyone with Vista or access to Vista tell us?
Your suggestion is simple, and addresses many of the issues I think are important. I would like to clarify or extend a couple of your points:
* I think having preinstalled plugins within the install tree is excellent - these effectively become Global plugins;
* your suggestion of having the user-installed directory override the preinstalled directory solves the write-to-global problem;
* I would still like to have a definition in the user preferences which specifies the path to each plugins directory, including the Global plugins directory (which will typically be <path-to-AOI>/Plugins). Having this definition for the Global directory would provide a number of benefits:
(i) it allows users to move this path if they wish, or for us to fix future problems by relocating the Global Plugins path;
(ii) it allows AOI (specifically ModellingApp) and other plugins (particularly standalone capable plugins such as SPManager and HelpPlugin) to find the Plugins path without resorting to analysing the ArtOfIllusion.jar path, with the problems that that entails;
(iii) it would simplify the code for handling Plugins (presuming that user plugin paths have such a definition), since then all Plugin path lookups would refer to a definition in the preferences, rather than the Global path being the exception.
* having a definition in preferences for each user-defined plugin directory simplifies having multiple different installations of AOI-version-X.Y, each with different plugin (both global and user) directories. I normally have 3+ installations per AOI release:
(iv) and one or more temporary installations to test download and installation procedures for new plugin releases, and for changes to SPManager.
So I really need to be able to have user-level plugin directories as separate as the global directory.
One other solution for this is to consolidate the user Plugins path into a single directory (possibly per release), and then allow each AOI installation to specify which files in that folder to load. However, this solution is more complex, and still has issues with trying to test an update of a plugin whilst keeping the released version available for live use.
Without some support for this, I would be forced to test plugin updates using a different user login on my machine, which I definitely do *not* want. The apparently simple solution of performing tests in isolated installations and relying on the isolation of the global plugin directory falls at the first hurdle, because by definition this cannot test the code path which includes using user-level plugins directories.
I'd kind of prefer not to store the plugin location in the preferences. For one thing, the whole reason for having a preferences file (rather than using the Java Preferences API) is so that people can copy their preferences file from one computer to another. If that file contains the path for loading plugins, that immediately makes it non-transportable.
It also seems unnecessarily complicated for the end user. Most people shouldn't have any reason to care what folder their plugins are being stored in, and putting it into the preferences window will just confuse them - especially since the default location is inside an invisible folder!
For the case where you have multiple versions installed at once, that can be dealt with just as easily by putting the plugins into the application directory, exactly as you do now, which should continue to work. (One possibility is that when you install plugins, SPManager could give you the option of installing them in the application directory or your home directory. JEdit does something like this. But those are the only options; you can't specify an arbitrary directory.)
I recognize that this does make things more complicated for you when you want to test SPManager changes, since as you say, you need to test the code path which puts them in the user directory. But you may literally be the only person in the world for whom this is an issue! I know, being a developer is a lot less convenient than just using a program someone else has written. :)
You make a good point regarding preferences file transportability.
Since plugin directory paths are presumed installation-dependent, then they would probably best be stored using the Java Preferences API. AOI already uses this for other information.
If you don't want the paths displayed in the main preferences window, then perhaps in an "Advanced" panel?
I suggested storing plugin directory paths way back in release 1.something, to address the "where-oh-where-is-the-plugins-folder?" problem, which is still being encountered from time to time. It just seemed that with recent events requiring changes to the location of Plugins directories, we could eliminate that problem as well.
> putting the plugins into the application directory, exactly as you do now, which should continue to work.
But that's the whole point of this thread. It has not been working on a range of installations already (/usr on linux), and is now not working on another environment ("C:\Program Files" on Vista) for which the proportion of users capable of adjusting the permissions of a system folder is arguable smaller. The empirical evidence is that it isn't continuing to work, and in fact will continue to work less.
More specifically, your solution as described won't work ever. The problem is that if the per-user plugins folder overrides the global plugins folders (those in the application structure), then as soon as a plugin is written to the per-user location for any installation, it will override all of the (possibly different versions) of that plugin in every global Plugin directory of the same AOI release.
... Unless, of course, there is a control which allows the user to specify per-installation, which plugin directories are used.
The basic issue I am trying to solve is that hard-coded directory locations are per-machine, whereas the granularity that is needed is per-installation.
And finally, I don't believe I'm the only user who needs (or wants) completely independent installations of the same AOI release, without the restriction that the user must avoid *any* use of the per-user directory (for that AOI release).
Certainly plugin developers in general will find it useful to have multiple installations: one for regular modelling; one for development; and (usually) one for testing. While I don't think we should overly complicate AOI just for the sake of plugin developers, I think we should also try to avoid making things difficult for them.
And I can certainly see the benefit for some non-developer users to have multiple installations of the same AOI release configured with different plugins. I can see situations in which regular users would like to install an experimental plugin into their experimental installation without having the plugin "magically" appearing in all installations of the same AOI release.
I've recently took a very simple orientation towards deciding if features were actually beneficial to the user. My opinion is that computers are here to let people do things, not to make them waste time. System administration entirely falls in the 'wasting time' category, whereas some people mistakenly take it it for 'doing things'.
I mean who would bother with where the plugins are as long as it works? Having base plugins installed along with distribution at installation time (when obviously system wide locations are granted write access) is very sensible IMO. I understand that Peter's proposition would raise problems when one wants to run similar distributions with a different set of plugins, but I guess this is a usage limited to developers and competent users. We can install AoI in a local dir and keep all plugins in the distribution folder with user write access. Honestly this seems the best design to me. The rest is just too complex whithout any benefit for the average user.
> Since plugin directory paths are presumed installation-dependent, then they would
> probably best be stored using the Java Preferences API.
I don't think that will work. The Preferences API stores all your settings in a global registry that is shared by all applications. So if you installed two copies of AoI for testing different plugins, they would still use the same plugin directory.
> It has not been working on a range of installations already (/usr on linux), and is
> now not working on another environment ("C:\Program Files" on Vista)
It will work fine as long as you install in a location you have write access to, such as inside your home directory. Remember, this is only for the small set of users who want to install multiple copies of the same version of AoI with different plugins. Most people will simply install a single copy in the default location, and it will work fine. But for power users who have more complicated needs, there's a straightforward solution that works well.
(Peter said)> The Preferences API stores all your settings in a global registry that is shared by all applications.
I don't think that limitation is different to that of storing one or more preferences file(s) in a single .artofillusion directory in the user's home directory. In either case, release-specific information will require release-specific keys.
(Peter said)> It will work fine as long as you install in a location you have write access to, such as inside your home directory.
This is very much a key to the problem :o)
One part of the puzzle should probably be to ensure that the installer code handles this well, thereby making it easy for users (as opposed to administrator-users :o) to install AOI correctly. Given the changes we are seeing in Vista, we probably need to avoid installing AOI in "C:\Program Files"; or at worst, provide "C:\Program Files" as a non-default option. Similarly, linux installation logic probably needs to default to the user's home directory; and I don't know what issues there may be with installing software in different areas on MacOS.
A separate part of the puzzle is documenting the requirements to users who for whatever reason, don't use the installer.
This is currently an issue on Windows, since many Windows users would install a program in "C:\Program Files", because that is where programs usually go on Windows. It just doesn't work the same way anymore on Vista.
And as I pointed out in my previous post, the suggested design of a single per-user plugins folder for each AOI release will break multiple AOI installations of a given release, regardless of the location of those multiple installations.
(Francois said)> My opinion is that computers are here to let people do things [...]
I would word my "programming credo" differently to Francois, and probably Peter. My view is that software should do whatever tasks the user needs to perform repetively, as well as those that otherwise need to be explained to the users. So instead of writing a Wiki on how to calculate RGB colour from HSV, I would try to write logic into the program that could perform the conversion for the user.
I see handling plugin folder problems in the same light: instead of documenting to users the folders where they should and should not install AOI (and their relative merits), I am looking at how to write code that implements that knowledge such that users don't need to sort it out for themselves.
(Framcois said)> I mean who would bother with where the plugins are as long as it works?
That's exactly the point, although not the way you intended. No one wants to know so long as it works. It's when it doesn't work that they suddenly get all interested.
I'm trying to implement changes so that yet another "it doesn't work" situation is changed into a "it just works".
@Francois & Peter. I obviously haven't made part of my suggestion particularly clear.
I am not proposing storing the paths to plugin folders into a perferences file so that users can tinker with them all day, nor even be required to tinker with them 99% of the time.
The primary purpose of storing the paths into a file is so that they are no longer hard-coded into the AOI logic.
This solves a number of problems:
1. AOI can find the plugin directory location without having to cast arcane spells over the corpse of an ArtOfIllusion.jar file.
For those of you who haven't looked, the code that a Java program needs to go through to find where on the file-system the running Jar file is, and hence deduce where related directories might be (eg the Plugin directory), is not pretty. It works most of the time, but it is essentially implementation-dependent, and hence is not fully deterministic.
2. Multiple installations of AOI can easily have non-conflicting plugins directory paths by having different values set for each installation. This can (and hopefully is) organised by the AOI software itself rather than the user.
3. In the infrequent situations in which we discover that a change in some external system (eg Java, Windows, Linux, OpenGL, etc) has changed what was once a suitable location into one which is no longer suitable (as Vista seems to have done with "C:\Program Files"), then the problem can be resolved for existing users by changing these paths in the preferences file, rather than requiring users to upgrade to a new version of the AOI software release, and having to wait for that new version to be made available.
Point 3. is not a high-use feature I agree, but instead of our releasing a number of trial versions of AOI, each with different hard-coded plugins folder locations, and getting various users to test the new version and report the results, we could get some users to modify their path preferences and report which paths worked, and which ones didn't. The second process would be significantly quicker, since it doesn't involvew building and releasing a new version of AOI for each experiment.
Those users who needed to get going immediately with the current version of AOI could just change their path preferences and get on with modelling. Meanwhile we would change the control files and/or logic that is used to install these directories, and so the next version of AOI would work with the changed external system out of the box, and we could all forget about the fact that paths are stored in a preferences file until the next external system change came along.
Hopefully I've cleared a few things up :o)
"Given the changes we are seeing in Vista, we probably need to avoid installing AOI in "C:\Program Files"; or at worst, provide "C:\Program Files" as a non-default option."
This is how ALL programs are going to be developed, so you'd better get used to the arrangement. Linux has been doing this type of program/user separation for a long time. I really don't see why AOI has to be different from any other piece of software in this respect.
The problem isn't where AOI is installed; it's where the plugins are installed. If AOI always looks in the same place for the plugins, then there is no problem.
"And as I pointed out in my previous post, the suggested design of a single per-user plugins folder for each AOI release will break multiple AOI installations of a given release, regardless of the location of those multiple installations."
So we do some type of versioning, as Peter suggested.
And frankly, having several installations of AOI is the exception, not the rule. Most users only want a modeler and renderer - they will only be using one copy of the latest stable version of AOI.
"instead of documenting to users the folders where they should and should not install AOI (and their relative merits), I am looking at how to write code that implements that knowledge such that users don't need to sort it out for themselves."
Yup. If AOI is installed in Program Files and the Plugins in a user's folder, they will never know or care where AOI is or where their plugins are.
"AOI can find the plugin directory location without having to cast arcane spells over the corpse of an ArtOfIllusion.jar file."
I recommend that the installer does all the "arcane spells" and moves the Plugins from wherever they were at and into where they should be from now on. In addition, if AOI asks the OS where to place stuff instead of it being hard coded, then it won't matter where AOI is installed, and it won't matter if the OS changes stuff, because the OS knows what changed.
I agree, it should not be hard coded; AOI should ask the OS where stuff goes.
"Multiple installations of AOI . . ."
Again, this is generally only the case with us developers, not with your average user. If we want to do something to allow us to do this while developing, fine. We can implement something that can help us in this respect while developing - something that won't prevent us from placing AOI in Program Files.
The final release that the user sees should follow the normal conventions that most other software follows. I see no reason why AOI should be any different from any other software.
You've raised some good points:
Firstly, I agree that multiple installations of AOI is the exception, but I highly doubt that it is only developers who have multiple installations of AOI. I'll ask in the other forums, and see if we can get some responses from others. For example, I know it is quite common for Scribus users to have multiple installations of AOI for a single non-developer user.
"I really don't see why AOI has to be different from any other piece of software in this respect."
I don't think anyone is suggesting that AOI persists with installing AOI in "C:\Program Files". What we are discussing is what shape the new arrangements should take.
"The problem isn't where AOI is installed; it's where the plugins are installed."
Well, it's not even quite that. The problem is where new (optional) plugins get saved to.
"So we do some type of versioning, as Peter suggested."
My comments were regarding the type of versioning which Peter had suggested - which will break in the scenario I had described.
"I recommend that the installer does all the 'arcane spells'"
Yes I agree. And a natural result of having the installer doing this is that the installer must document the result it got from the 'arcane spells' by writing it in a file somewhere. If it doesn't, then AOI will have to perform some 'arcane spell' of its own to find out where the installer put things.
"In addition, if AOI asks the OS where to place stuff..."
This is a really interesting suggestion. I have no idea what facilities the OSes provide to answer such questions, but it's certainly worth looking into. In code I wrote for a plugin recently, I have relied on the "user.home" property, to the almost universal approval of Windows users...
"For example, I know it is quite common for Scribus users to have multiple installations of AOI for a single non-developer user."
Interesting. Under what scenarios do they require multiple installations?
"My comments were regarding the type of versioning which Peter had suggested - which will break in the scenario I had described. "
I'm sorry - which scenario?
"And a natural result of having the installer doing this is that the installer must document the result it got from the 'arcane spells' by writing it in a file somewhere. If it doesn't, then AOI will have to perform some 'arcane spell' of its own to find out where the installer put things. "
Nope. What I'm trying to say is that the installer moves the existing plugins AWAY from the 'arcane' places and places them where the OS says the user files are. AOI then asks the OS where the user files are instead of having to do any 'arcane spell's.
"I have no idea what facilities the OSes provide to answer such questions"
Usually environment variables.
HOMEPATH and APPDATA are two possible candidates in a Windows environment.
> I don't think anyone is suggesting that AOI persists with installing AOI in
> "C:\Program Files".
Actually, I think that's precisely what Jeremiah was suggesting. :) C:\Program Files is the standard location for applications to be installed on Windows. That's where users expect it to be installed, and while it's fine to give an option of installing it somewhere else, people will get confused if that isn't the default location.
> AOI can find the plugin directory location without having to cast arcane spells over
> the corpse of an ArtOfIllusion.jar file.
I wouldn't worry about the arcane spells. They may look arcane, but they're actually robust and work very reliably. I think there were some Java 1.1 JVMs where it didn't work, but falling back to user.dir hasn't been necessary for a long time.
And what do you mean, "corpse"??? This is a LIVE jar we're talking about! :)
> Multiple installations of AOI can easily have non-conflicting plugins directory paths
> by having different values set for each installation.
I'm afraid I still don't understand your design. You want to have several copies of the same AoI version installed on your computer, and have each one use a different plugin directory. How will the location of that plugin directory be determined? Not by the user, you say; not by the installer, because all the installations will be identical; not by the version number, because they're all the same version. Where will the path to that location be stored? It can't be in the preferences file, because that's global; it can't be stored with the Preferences API, because that's also global. So where?
So, he wants to use several copies of the SAME version with each copy having a DIFFERENT set of plugins?
IMHO, that really sounds like an improbable scenario - the only time when you'd really NEED a different set of plugins is if you want multiple versions of the same plugin. Which pretty much sounds like a development environment.
Perhaps each plugin can have its own folder based on the plugin version, or some other way of separating plugin versions?
Quite frankly, I'm drawing blanks on where this scenario would happen outside a development environment.
> Actually, I think that's precisely what Jeremiah was suggesting.
Ok, on re-reading Jeremiah's posts I can see that is also an interpretation. I'll let Jeremiah clarify, if he feels the need.
> And what do you mean, "corpse"??? This is a LIVE jar...
Well yes, it is quite live in AOI terms, but the arcane spells most closely resemble a post-mortem, in my opinion anyway. And if the JAR is so LIVE, how come it can't answer a simple question like "Excuse me, but where *are* you exactly?" - huh? :o)
> I'm afraid I still don't understand your design.
Ok, the basic principle is that the location of the plugins directory(ies) is documented in a file.
The file in which the information is stored can be global or discrete. What is important is that each installation can easily determine a unique key or "installation id" that it is associated with and which can be used to identify the data pertaining to that installation. Whether this key is used to identify a discrete file, or specific entries in a global file is immaterial.
(As an aside, the pathname of the installation directory is currently used as such a unique installation id.)
The concept of such an id is not that uncommon. Thunderbird generates unique directory names in the .thunderbird directory for each installation, for example.
There is more than one way of implementing this, but creating a properties file in the installation directory and including that in the classpath of the jar is certainly one of the easier ways. With this, each running copy of AOI will find a unique "installation id" by looking up the same property key.
In fact, thinking on it, this could be used to replace the "arcane spell" also.
A variation on this is to store a sequence number instead of a unique id, and generate the unique id by appending the sequence number to the AOI version number.
How this properties file is written, and when, can be adjusted to give the preferred behaviour.
1. One simple approach is that the id is generated and written by the installer process, since this obviously has the required write permissions on the installation directory.
The installer could create a new id for each installation, or only do so if a checkbox is selected, which could make the default behaviour to have only one per-user plugins directory for each AOI version.
2. The almost opposite approach is to supply a standard properties file with the installation, containing the first sequence number (eg, zero). If a user, such as myself, wants parallel installations then he makes the change - either manually, or by entering the new path in the preferences advanced panel which causes the properties file to be modified. Obviously, this action will require the correct permissions.
There are certainly other implementation options as well.
>> For example, I know it is quite common for Scribus users to have multiple installations of AOI for a single non-developer user.
> Interesting. Under what scenarios do they require multiple installations?
Firstly, I've only just noticed a bizarre typo in my original sentence. I wrote:
"...for Scribus users to have multiple installations of AOI..."
when I actually meant:
"...for Scribus users to have multiple installations of Scribus..."
Secondly, the predominant reason Scribus users to have multiple copies of Scribus installed is so that different versions of the software are available at the same time.
From some of the responses I got on FriendlySkies, there are AOI users other than myself who do something similar:
However, I note that the number of people doing this is relatively small.
> Quite frankly, I'm drawing blanks on where this scenario would happen outside a development environment.
1. A new version of a plugin is released. Some users want to use it, but don't wish to change their production AOI environment because they are mid-way through a critical project.
2. A particular combination of plugins is causing a problem. Some users who want/need each plugin might then create parallel installations, each with only *one* of the problem plugins installed.
3. A particular plugin has beneficial features in some situations which are an inconvenience in others. Insead of continually installing/deinstalling that plugin, users may elect to have parallel installations, one with the plugin installed, and one without. Currently, the LogPlugin falls into this category for some people.
4. All ModellingTool plugins create an entry in the Tools menu. It is not inconceivable that in the future, the Tools menu starts getting unwieldy in length. Users may find it more efficient to have multiple installations of AOI, each with a subset of plugins tailored to more specific types of project, eg animation, physics modelling, architectural-style modelling, etc.
5. With Open Source, many more users are ivolved in the testing of beta or experimental quality code. It is not uncommon for users to wish to test in a separate environment to their production environment.
6. One of the main driving forces of the pace of AOI's advancement is through plugins developed by ... developers. Why then reject a feature out of hand, simply on the basis that it benefits only developers? How does inconveniencing developers help with the advancement of AOI? Is that really the way you wish to thank plugin developers?
I'm certainly not saying that AOI should evolve only to please developers; but by the same token, I do not see it as reasonable to dismiss all features that might make developers' tasks more convenient, simply because only developers might benefit from them (AFYT).
Y'know my solution to the whole admin problem is to simply install aoi in a folder in my documents called "My Programs". This allows me to do several things including accessing everything there without being logged in as an admin and also being able to easily have many different installs of different versions at the same time. The only problem of course is that there is only one preferences folder so changes made to one version's preferences also affect the other versions preferences.
These are just my thoughts :)
Yes, the "private" (non-shared) installation of AOI has been a viable option for quite some time now.
I seriously think we'll probably have to support both shared and non-shared installations so users can choose what works best for them.
Thanks for your input.