Interesting presentation about changing Blenders UI

Developers
Harald G
2013-10-17
2013-12-16
  • Harald G
    Harald G
    2013-10-17

    http://www.youtube.com/watch?v=UWacQrEcMHk

    and also interesting debates.
    :)
    But Andrew is making some points in his series about GUI proposals that wouldn´t hurt Art of Illusion either.
    In some areas (layermanagement) AoI has already what Blenderusers keep theirs thumbs pressed for.

    Worth the time watching.
    Harald

     
  • John_S
    John_S
    2013-10-19

    Seen that. And the AoI specific UI discussions on Friendlyskies. Lots of detail there.

    Usability Issue: Most used button? Render. Where is it? In a menu -- not in the toolbar. Correction: Big honking Render button, when pressed renders with current settings. Right out on the workspace.

    Every other usability issue is where that button is now.

    I've used a bunch. Commercial. Free. Blender UI is not a great starting point for this kind of discussion.

    Having such issues as installing the ViewLayout plugin taking out whatever is in position five of the tools menu -- over several versions, without correction -- makes this discussion a mute point. You can't use it if it doesn't function.

    Right now (1600xx900) AoI ver 3.1 Polymesh Editor, I don't see the controls at the bottom. I have to resize -- and only do that because I know what's cut off. A new user would not know the OK and Cancel controls were hidden off screen, thus giving the impression there are no controls for getting back to main screen in AoI.

    You don't want to be having this discussion in version three. ...Of a program with the ease-of-use rep AoI purports. Blender can get away with it because it makes no such claim.

    Just about every Blender description or tut starts by explaining the visual complexity. AoI puts itself forth as different. Hence user expectations are different.

    Which makes the inevitable comparison to Blender by coders a natural escape clause for why UI improvements don't happen in AoI. It's assumed AoI is, by default, usable because it is not Blender.

    Additional escapes like "plugin authors did it" or "XML headers did it" makes UI discussion fruitless. And comparison to UIs of commercial apps? You don't even want to go there.

    Likewise taskflow and workflow discussions. AoI should be here at version three. This isn't event a UI concept in version three. The UI is for hobbyists doing test renders, no more.

    Open Source, by and large, uses its Get Out of UI Design FREE card way too much.

     
    Last edit: John_S 2013-10-19
  • Luke S
    Luke S
    2013-10-20

    @Harald: Yes that was a rather interesting video. I actually found the two preceding it to also be interesting - they discuss some basics of UI design in the context of blender. (They were intended as a setup for the one you linked.) And yes, AOI could probably learn something from them as well.

     
  • Luke S
    Luke S
    2013-10-20

    @John_S: I've tried, and I can't reproduce that issue you are having with Polymesh. Could you post what OS and Java version you are using? Also, are you using the OpenGL option?

    Regarding some of your other points, I'd agree that AOI's interface could be improved, probably a lot. (And yes, that... oddity... in the ViewLayout Plugin needs to be fixed.) But:

    • You object that "Plugin authors did it" isn't a valid excuse. Maybe not. (And definitely not when the issue is in a core function.) As far as I know, there is no way to prevent plugin authors from creating the craziest UI anyone ever thought of. Any thoughts in this direction?
    • Please remember that the only really active developers of AOI at this time are Peter Eastman and Nick T-Jones. And They both do this as a hobby. There is bound to be a little triage.
    • You seem to be focusing a little much on version numbers. Version numbers on one piece of software don't compare with version numbers on another. (Which I believe is where you were going.) As examples:
      • Blender - 2.65ish.
      • Inkscape - 0.48.4 (Not even version 1! and many professional graphic designers use it every day.)
      • Google Chrome just passed 30.something.
      • Java platform is going to release 8 next, (Except it's really 1.8! That's marketing for you.)

    My point is, "Version 3.0" does not imply anything about the maturity of an application other than in comparison to other versions of the same application.

    I do believe, though, that the devs would welcome suggestions on how AOI could be improved, so don't go away. :)

     
    Last edit: Luke S 2013-10-20
  • Peter Eastman
    Peter Eastman
    2013-10-20

    I totally know what you mean about plugins, and I don't like the situation either. I just don't know what to do about it.

    Someone writes a plugin and posts it on the repository. It does something useful and people start using it. Maybe they stick around a while and add some features in response to user requests. But eventually they move on to other things and drop out of touch. When bugs get reported in the plugin, they don't get fixed, because there's no one around maintaining it. Maybe a new AoI version, or an update to another plugin, creates an incompatibility, so a plugin that used to work doesn't anymore - or sort of works, but not perfectly.

    Ideally someone would test every plugin before each release, and someone would be maintaining every one of them. But I certainly don't have the time (or interest) to do that, and no one else is doing it. Like Luke said, Nik and I are the only active developers right now, and even I've been feeling guilty about how little time I've spent on AoI lately. For better or worse, this is something I do as a hobby in my spare time, and it's only one of many hobbies at that.

    So I don't really know how to improve the situation, but I'd welcome suggestions. I think this is an important discussion to have.

    Peter

     
  • John_S
    John_S
    2013-10-20

    There are a lot of issues raised. I'll try to tease out a few and list them in no particular order.

    Polymesh Workspace and OS. Win XP Pro SP3, just upgraded to Java 7 update 45 -- upon rechecking; nothing changed. UI/usability issue: No scrollbar or indicator the window isn't maximized. This despite plenty of room to adjust the window size to show OK/Cancel controls, or anything else at the bottom, like the tip text or the context sensitive controls like Smoothness slider.

    Again, minor inconvenience for a regular AoI user. Confusion for the new and casual user.

    You seem to be focusing a little much on version numbers.

    This is an elementary problem. One I have addressed previously. Simply announce what the numbers mean. So, dot zero: Plugins have not been fully checked. dot one: Plugins have been checked and issues solved. Just like an interface this provides, let's call it a system state or release state -- where a "version numbers mean just what I choose them to mean, no less, no more" lack-of-focus seems a bit capricious.

    As far as I know, there is no way to prevent plugin authors from creating the craziest UI anyone ever thought of.

    If only that were true we might have better UIs. They are all the same: Function dictates form. There is no following going on with Java controls, more like the edict of the Java toolkit control. Otherwise we might have a Kai Krause. That ain't happening.

    To restate, this isn't a Blender problem -- it is an AoI problem because ease-of-use is AoI's turf.

    Solution? One is to elevate UI bugs to the status of code bugs. Another can be to take a version and unify around some guidelines. It could be as frequent as dot one being functional bugs and dot two being UI. If the version doesn't get to dot two, then perhaps the next can.

    Thing to remember is "Plugin authors did it" isn't usable. You wanna talk UI, let's pull up a seat for user perspective.

     
    Last edit: John_S 2013-10-20
  • Peter Eastman
    Peter Eastman
    2013-10-20

    So, dot zero: Plugins have not been fully checked. dot one: Plugins have been checked and issues solved.

    Are you volunteering to be the person who checks the plugins and fixes the problems? If so, that could work well.

    Peter

     
  • Harald G
    Harald G
    2013-10-21

    I don´t believe in a "ready to use solution with full compatibility check", but I do believe in a simple check during the alpha and beta stages.
    Why don´t we use a simply Excel- or similar sheet online where all available plugins are listed. In this sheet we add the AoI Versions horizontally and make a remark if the plugin works just fine or behaves irregular somehow.
    If so, Nik could add a remark to that plugin in the Script and plugin manager.

    We could move this task into the forum communities (Friendlyskies, AoI Board,...)

    Regarding the view laypout plugin:
    This - along with enhanced functionality should become part of the core, so that plugin devs use just that.

    I´m amazed by the way how much discussion came up by posting that link.

    Harald

     
  • John_S
    John_S
    2013-10-23

    Are you volunteering to be the person who checks the plugins and fixes the problems? If so, that could work well.

    Are you offering me retail rights and consequently development control? Or can I, euphemistically speaking, as coders like to say "go fork myself."

     
    Last edit: John_S 2013-10-23
  • Peter Eastman
    Peter Eastman
    2013-10-23

    I think you misunderstood me. This is an open source project. There are no retail rights - or rather, you and everyone else in the world has the right to distribute it however you want. It's developed entirely by volunteers, working for free in their spare time. Neither I nor anyone else has ever been paid a cent for our work on this project. The only way anything gets done is if someone volunteers to do it. If you want to volunteer some of your time to help (as we volunteer ours), that's great. If you don't want to, I understand. But if no one volunteers to do something, that means it doesn't get done.

    Peter

     
  • John_S
    John_S
    2013-10-25

    But if no one volunteers to do something, that means it doesn't get done.

    Okay. Consequently, do it yourself and don't suggest anything you won't build yourself.

    What was the point of having a thread on UI then? Talking to oneself doesn't really require computer mediation.

    Which is also why the Blender thread won't go anywhere. The guy in the video can fork the project. And the programmer's evergreen response is they should. Usability or UI bugs aren't seen as bugs -- they are considered issues of taste; opinion.

    Volunteers don't want to clean up anybody's messy room. They want to make their own mess. Which, somehow, in a thread about UI design, is my problem. And, apparently, mine alone.

    And this, hilarious, after the lament of my initial refusal to beta test and subsequent draft into the corps anyway. This really does raise a smile. Maybe it's better put as if nothing happens, there won't be volunteers.

    Just shut down UI threads. Simpler. And more to the point.

     
    Last edit: John_S 2013-10-25
  • Harald G
    Harald G
    2013-10-25

    This is the discussion thread and I allow myself from time to time to "copy and paste" ideas or thoughts from other sources or sometimes even put my own here.
    I see this more as a "collector".
    If the idea is later going "into production", because someone (a volunteer) grab the idea, or it was somehow congruent with his/her own thoughts about improvement, then the discussion about these issues are moved into the development section.

    I have to admit that I´m not a coder.
    AoI is often (but not always)slower in progress than "the big projects" and even at these big projects there is often no chance for changing the UI (see Tons perspective of view*) immidiatley just because someone has good ideas about it.

    *what you´ve (John) done for sure, if I read your post correct. And here you´re correct about your opinion.

    Btw. there was already an AoI clone (or fork?) with some nice approaches in the UI section,
    but it´s not alive anymore.

    Conclusion:
    If I want to have a big render button I would put it into the bugs & feature request section. If I like to see overall enhancements in the UI of AoI, I put it here.
    Sometimes nothing will happen, sometimes we have a discussion, sometimes (maybe years later) we see things moving...

    Harald

     
  • Luke S
    Luke S
    2013-10-26

    @John S: You said:

    Polymesh Workspace and OS. Win XP Pro SP3, just upgraded to Java 7 update 45 -- upon rechecking; nothing changed. UI/usability issue: No scrollbar or indicator the window isn't maximized. This despite plenty of room to adjust the window size to show OK/Cancel controls, or anything else at the bottom, like the tip text or the context sensitive controls like Smoothness slider.

    I managed to reproduce this - on Win Vista. It does not happen on my own system. (Win 7) I've patched to improve - there is still a bit to go for a perfect start up. There is a thread started over on

    https://sourceforge.net/p/aoisp/discussion

    The patched file is attached there. Feel free to test and give feedback.

     
  • Luke S
    Luke S
    2013-10-26

    RE: Improving plugin compatibility.

    Peter said:

    Someone writes a plugin and posts it on the repository.

    Maybe a new AoI version, or an update to another plugin, creates an incompatibility,

    I'd suggest that one solution to catching the second early starts at the first. This may be a little simplistic, but here goes:

    For a new plugin to make it into the repository, it needs to be production ready -

    1. Must have the complete source and build dependencies to be checked in to the repo.
    2. Must have a standardized build script. (The ant targets that are convention in AOI might make a good starting point.) The purpose of this is made clear lower down.
    3. Must compile with no errors, and preferably with no warnings, against the current AOI jar. (Even errors that appear to leave a functional plugin would be a no-no)

    I don't mean to imply that the community shouldn't support plugin developers, and test new plugins that are in development, just don't put it in the repository unless it compiles cleanly. (Some larger projects would call this 'Don't break the build.')

    There should be a master script of some sort that goes through each of the plugins, and runs the distribution target on it's build script.

    Whenever a new AOI version is released, (take it through beta first)

    1. Someone can run the master plugin-compiler script.
    2. If a plugin compiles cleanly, the dist artifact is upgraded to the repository, suffixed with the current AOI version number.
    3. If the plugin doesn't compile, spit the compiler errors to an appropriately named text file.
      3a. Open a bug ticket on AOISP and attach the error message file. (It would be lovely if the script could do this itself, but I'm not sure this is allowed)

    The three main goals I have here:

    1. Automated basic compatibility. Once such a script were built, and all plugins set up to use it, checking plugins should be a one-command process.
    2. Source compatible. If a plugin won't compile, there is definitely a bug hiding in the back of existing .jars, even if they seem to work on a cursory inspection.
    3. Binary compatible. Even if the source compiles, there may be subtle differences in the final jar. (I ran into this just last week. Hologram plugin. See aoisp.)

    The other thing about this solution is that it does not require any extra tooling. If you are set up to compile AOI from source, you have everything you need to use this.

    Whoa!!! Turned out to be a longer post than I expected.

     
  • Peter Eastman
    Peter Eastman
    2013-10-27

    That all sounds like a really good plan. And really we probably should have been requiring full source to be in the aoisp repository from the very beginning. Then we wouldn't have these problems of trying to track down source for plugins if the original author has disappeared. Currently it looks like there are only a handful there, all of which were written by either me or Francois.

    So as a starting point: any plugin authors out there, can you post a pointer to the source code for your latest version, so we can add it to the repository?

    Peter

     
  • Luke S
    Luke S
    2013-10-27

    The shout-out is a good idea, but there is source for more of the plugins than you might think. It's just not under source control. If you take a look under the 'files' tab, you will find 27 entries that are all source tarballs or zip archives for plugins that are in the repository. Some of them are not up-to-date, but they would be better than nothing.

     
  • A few points on the excellent suggestions here:

    a) a good CI (Continuous Integration) tool will automate much of the compile-and-check-correctness of the plugins. I haven't yet determined if we can run Hudson or similar on the SF servers, but I am pretty sure we could.
    - So, there would be no need for any master scripts, per se.

    b) I have already put some thought into automating the submission of plugin code. I actually wrote a similar system for a customer - based on my experience with the AOI plugin system (kind of circular-improvement path, there :)

    c) I have also put some work into some automated functional test generation for plugins.

    Cheers!
    Nik

     
  • Luke S
    Luke S
    2013-11-02

    @Nik: Wow, that's a little more ambitious than I expected!

    a)I'd basically tried to come up with the absolutely simplest thing that could work. Hence a script that one person would run on their local machine... I generally associate CI setups with an environment where multiple people are making contributions to a codebase, overlapping in time and files modified... Which kind of brings us to:

    b)How are you planning on handling access control? I can understand wanting to minimize developer friction for a new plugin, but I don't think you want to give everyone direct write access. I also wonder what you have in mind for third-party maintenance. (I've recently posted what are intended to be bugfixes for both PME and Hologram. I don't have, nor should I have the ability to just overwrite what is already in the repository, but posting a .jar and some text files in a forum area that few read means that not many people will benefit, either.)

    c) I'd imagine such a test generation system would work better on a pluign that has a decoupled UI, or at least a well defined API that can access everything the UI does?

    As always, just putting the thoughts out there to see what happens...

     
  • John_S
    John_S
    2013-11-02

    don't put it in the repository unless it compiles cleanly. (Some larger projects would call this 'Don't break the build.')

    This is why you're not ready for a UI design thread.

     
  • Luke S
    Luke S
    2013-11-02

    This is why you're not ready for a UI design thread.

    Why can't we have both? You might notice that discussions here tend to spread out and cover a rather wide ground. That's what happens when an entire project is supported by just a couple of people. Those people tend to address concerns about many aspects of the project at once.

    BTW: Did you get a chance to try out my tweaked PMesh Build? If so, a short comment about whether or not my changes improved the UI issue you had noted would be appreciated.

     
  • Luke S
    Luke S
    2013-11-02

    @Harald:

    Regarding:

    Why don´t we use a simply Excel- or similar sheet online where all available plugins are listed. In this sheet we add the AoI Versions horizontally and make a remark if the plugin works just fine or behaves irregular somehow. If so, Nik could add a remark to that plugin in the Script and plugin manager.

    We could move this task into the forum communities (Friendlyskies, AoI Board,...)

    I'm sorry, I didn't mean to ignore your opinion. I certainly can see that human testing is going to be important for many purposes. (Such As UI testing!) The forum communities would be a great resource for this, and a more organized way of collecting their feedback would be great.

    Automated builds and testing is not a be-all, end-all solution, but once put together, It should eliminate the most common plugin problem scenario that I have seen:

    1. User says: "My AOI install won't start" or "Crashes all the time."
    2. Others ask about scene files, error messages, etc. Nothing seems to make sense.
    3. User is prompted to uninstall plugins one at a time...
    4. Turns out that this user is the only person who has tried running plugin(x) with new AOI version(y)
    5. Developer finally looks at the code of plugin(x), and finds that the whole thing is due to a two line change, and takes 5 minutes to fix.

    This sort of thing makes up a large part of the support zone on friendlyskies, and causes lots of frustration. I'd rather that users spend time building things, and finding more subtle bugs ;)

    There is a programmer's rule of thumb (don't know where it started) something along the lines of "If you need to do it more than three times, write a script to do it for you."

     
  • John_S
    John_S
    2013-11-09

    BTW: Did you get a chance to try out my tweaked PMesh Build? If so, a short comment about whether or not my changes improved the UI issue you had noted would be appreciated.

    I am not entirely sure; but it seems just a little bit better -- you can now see where some of the controls are cut off. Thus indicating there is something outside the viewable window area, which might then get some people to try to resize it.

    Now (?) the new Ploymesh window cuts the Loose Select ...etc etc etc controls in half.

    Why can't we have both?

    Well, some trivial details are I find myself moving over PolymeshTemplates -- primitives. These basic building blocks are trivial in terms of development. Almost non existent in terms of basic dev. Yet nontrivial for use and saving the user time.

    From edge case city, the view is foggy. Determining the correct set of primitives is impossible. And yet I find a rounded cube or torus is not really edge case. Just a study in use.

    This requires no code. This requires no library. Not hundreds of objects and a million variables. Yes, the tools can provide the necessary shapes. It's still part of the usability puzzle and valid for such discussion.

    What this requires is a different world view outside edge case city. Not everything anybody might ever do. What real users of this and of similar products actually do on a day-to-day basis.

    This is why we might as well be talking about terraforming.

     
    Last edit: John_S 2013-11-09
  • Luke S
    Luke S
    2013-11-11

    @John: you tried my experimental build if you pulled it from this thread:

    https://sourceforge.net/p/aoisp/discussion/457423/thread/7f29436a/?limit=25#19ed

    Your results were what I expected. I do have one other change that I will make shortly to further fix this issue - I just wanted to make sure I was on the right track for the problem you are seeing on your system.

    PolymeshTemplates -- primitives. These basic building blocks are trivial in terms of development. Almost non existent

    More Polymesh primitives could be a very valid idea, though I'd be careful about adding to many. Though, Polymesh has a way for you to keep your most commonly used starting points. See below.

    This requires no code. This requires no library.

    Polymesh actually uses both. The basic primitives that you can use out of the box are hard-coded into the plugin. If you find yourself using a certain shape over and over again as a starting point, (such as a torus, or maybe a rough humanoid mesh) You can save the mesh as a template. This sticks it in a library, and then you can choose it from the dropdown box when you set what shape the polymesh creator makes. It would be nice to ship some extra templates with polymesh, but as you said, deciding which ones would be common enough to be useful would be a tricky proposition. Any suggestions?

    edge case city

    You used this expression a couple of times, and I'm not sure what you mean by it. I got the gist that it is a state of mind that you believe AOI developers currently inhabit, but that's about it. Could you clarify?

     
    Last edit: Luke S 2013-11-11
  • Luke S
    Luke S
    2013-11-12

    @John: I've posted another tweak to Polymesh that should eliminate the "hiding buttons" issue. Could you try it out and confirm? If you have any comments, please put them in that thread.

     
  • John_S
    John_S
    2013-11-12

    I've posted another tweak to Polymesh that should eliminate the "hiding buttons" issue.

    All I see using SPM is v1.63b18 2011/11/24 , Size: 1354kb.

    Where is this posted? And, in getting back to the topic of the thread, why is this dated 2011? Is anybody thinking about people who are supposedly trying to help you having to makes sense of this?

    You used this expression a couple of times, and I'm not sure what you mean by it.

    Developing a program about anything anybody might do, rather than what a specified, studied user would actually do. This makes the software lowest common denominator rather than usable.

    An article on Edge Cases and how they go wrong.

     
    Last edit: John_S 2013-11-12
  • Luke S
    Luke S
    2013-11-13

    @John: Since my Polymesh build is in no way official, It is not in the repository. it will probably not go there until it has been tested - specifically, unless it fixes the problem you have been seeing. (And does not cause any others)

    The Polymesh that you see in the repository has a date in 2011 because that is the last time the binary has been updated.

    Go to this thread:

    https://sourceforge.net/p/aoisp/discussion/457423/thread/7f29436a/?limit=25#19ed

    It is part of the discussion forums dedicated to supporting AOI plugins
    (It is still on sourceforge. Your ID will work. In fact, if you are signed in when you follow the link, you won't have to sign in over there.)

    Once you get there:
    - download the file. (There are actually two, the more recent is the one that I believe fixes your problem.)
    - copy the file into your AOI plugins directory. (It's DriveLetter\Where\AOI\Lives\OnYourComputer\plugins)
    - You might need to manually delete the existing PMesh plugin file. Or move it to another place for safekeeping.
    - Start AOI and use the polymesh tool.
    - To go back to the official version, if you prefer, just uninstall the plugin through SPManager, and re-install.

    Is anybody thinking about people who are supposedly trying to help you having to makes sense of this?

    That is one of the things that I hope would be addressed by some of the stuff Nick and I have bounced back and forth on this thread. Currently, there is no way for just any developer to make a bugfix, and push it to somewhere that the adventurous (Or those who were bothered by the bug in the first place) can say "Hey! I want the beta test version to see if it works better for me!" through the SPManager.

     
  • Luke S
    Luke S
    2013-11-13

    An article on Edge Cases and how they go wrong.

    Read the article. Tend to agree with it. In what specific ways do you feel that AOI devs are falling into the trap outlined?

     
  • Currently, there is no way for just any developer to make a bugfix, and push it to somewhere that the adventurous (Or those who were bothered by the bug in the first place) can say "Hey! I want the beta test version to see if it works better for me!" through the SPManager.

    That's not entirely true.

    There are 2 features in SPManager that support this, but both turned out to be confusing enough for regular users that they are not in common usage.

    1) Firstly, SPManager supports switching repository, so different versions of code can be installed and tested from different repositories.
    - The problem was that users would forget to switch back to the regular repository.

    2) A plugin in any repository, including the regular one, can be flagged as "alpha", "experimental", or have an included warning and "confirm to continue" message.
    And users can set in preferences how SPManager should filter out and hide plugins that match particular flags.
    - However, there is no support for multiple instances of the same plugin in the same repository matching the same version of AOI.

    Thinking about this, I can probably add support for that fairly quickly.
    Such a feature would allow a later version of a plugin to be uploaded with some additional flag (eg "experimental", "beta", or "alpha"), and users could either manually select the version they wanted, or have settings automatically avoid or select versions of plugins with certain flags.

    Would any users like to discuss the UI and operation of such a feature before I start implementing it?

    Cheers!
    Nik

     
  • John_S
    John_S
    2013-11-13

    unless it fixes the problem you have been seeing. (And does not cause any others)

    Well, it does come in at the bottom of Ok/Cancel, so pretty much gets the user a complete UI.

    In what specific ways do you feel that AOI devs are falling into the trap outlined?

    Well for one thing AoI doesn't save much. Render settings. Anything.

    The default settings for render are 400 width, 300 height? This is so anybody, anywhere, on any thing -- Including WebTV (remember WebTV??) will get a result. Because, you know, somebody just might want to render a 3D scene on their watch browser.

    The edge case is served. The user is not.

     
  • Peter Eastman
    Peter Eastman
    2013-11-15

    Well for one thing AoI doesn't save much. Render settings. Anything.

    In the latest version it does save render settings.

    Peter

     
  • Luke S
    Luke S
    2013-11-15

    Would any users like to discuss the UI and operation of such a feature before I start implementing it?

    I'm going to approach this from two directions, so please bear with the multiple posts, and the digressions a little beyond the subject at hand.

    First, from the end-user perspective:

    1. One thing that kind of annoys me about the SPManager's interface right now is the line I've circled in SPM_FunkyIcons. To see what any of this is about, I have to choose 'flags' from the dropdown on the left. The ability to reset the filter settings is also more complicated than it needs to be. What I would propose in it's place:
      • Use that space to display the flags that are currently set on the selected plugin. Perhaps have an icon for each of them. (I don't have any handy, but might be willing to search for some.)
      • Remove the filtering options that I have crossed out in SPM_Options. Keep only enable, confirm and hide. In fact, I think you could go further. Don't have these options in the first place. Anything with no flags (or, having -stable flag... We'll see how some of the rest turns out) is treated as 'enabled.' Anything with a flag triggers a confirmation. ("Are you sure? This plugin is marked as xyz")
    2. I'd appreciate some way for SPM to disable an installed plugin. This would mostly be useful when tracking down which plugin in a user's system is causing runtime issues. (See my common troubleshooting sequence above. Asking users to manually adjust their plugins folder is not a UI at all, plus, you run the risk of seriously messing something up.) For implementation:
      • On my own system, I often simulate disabling plugins that I don't want to delete by moving them to plugins/hidden, where AOI core can't find it. I don't see why SPManager couldn't do something like this behind the scenes.
      • It would be nice if SPManager could automatically "Bounce" AOI after re-configuring plugins. One approach would be to use JMX to get the runtime parameters that the JVM is running with, start a new instance, and call Runtime.exit() on the existing one.

    Running out of time tonight. I'll come back and finish this post tomorrow... when I can think clearly.

     
    Last edit: Luke S 2013-11-15
  • Luke S
    Luke S
    2013-11-17

    Okay, on with SPManager UI:

    1. For handling bugfix/enhancemant candidates:
      • All bugfix builds are delineated by suffix. eg, [plugin_name]_bugfix[bugID]
      • Enhancements are handled the same way, [plugin_name]_Enhance_TweakerTool if this enhancement adds something called 'Tweaker Tool'
      • Either of the above will almost certainly have a '-beta' tag.
      • SPManager should allow you to have any/all versions downloaded to your local machine, but enforce that only one is in the active plugins folder at once, perhaps through the use of a '-conflicts' tag? Re-enabling the standard version of a plugin should place a bugfix version in the 'disabled' folder, and vice-versa.
      • Perhaps, downloading a plugin should download all bugfix and enhancement versions? Which is active could then be set from the 'manage' tab - perhaps with a visual marker in the tree view that there are multiple configurations for that plugin?

    This is the part that I feel needs more thought and input. My point here is for most people not to have to think about how the plugin system works, just give them a quick way to try out the bugfixes.

    That's it for the end-user UI. Next time, some thoughts about how this should look from a plugin developer point of view.

     
    Last edit: Luke S 2013-11-17
  • I'd like to make 2 comments on the side-topic of edge-case programming.

    1. In my direct experience, of offering code to Peter Eastman for inclusion to AOI, Peter's code is very much the opposite of edge-case coding.

    In the cited example, John is correct that the 400x300 default setting for rendering in AOI is no longer relevant, but incorrect in inferring it was coded to satisfy an edge case.
    In the case of the render settings, when the defaults were first decided, 400x300 was a very reasonable value. What has happened since is that hardware has evolved significantly, and those defaults have not been revisited - so the cause is not edge-case programming, but simply historical.

    I, on the other hand, may be guilty of supporting edge-cases at times in plugins, although I have tried to put the options that support those edge cases into secondary or "Advanced" UI panels (if I recognised it was an edge case).

    1. This is not to say that users should not start discussions on any and all of these issues, whether caused by edge-case programming, advances in hardware, or other historical issues.

    And in all cases, progress will be quickest if people could please report exactly what is not working, or could be improved; rather than reporting their inferred cause or solution.

    For example, pointing out that 400x300 is no longer a reasonable rendering default is far more informative than stating there are too many edge-cases in AOI.

    @John (and everyone else) - I am sure there are other examples where you can see that the default setting in AOI is no longer (or may never have been) optimal to the 80% use-case. Please post your thoughts so we can discuss, and decide on changes.

    Cheers!
    Nik

     
  • @Luke:

    Awesome feedback - thank you.
    I have been postponing various UI changes to SPManager in the past, as we have planned a more general replacement for some time now - for which I apologise.

    However, I think I should definitely sort some of these issues out in the current SPManager.

    Here are the issues that I see, based on how SPManager currently thinks (WARNING - long post):

    1. Overview. There are 2 concepts that need to be separated:

    a) Flags: (Alert, Mark, Disable, Confirm, Required) these represent some additional information that SPManager has inferred about the plugin - usually associated with some corresponding action.
    SPManager uses icons representing each flag to alert the user of the implied action.

    b) Filters: (earlyAccess, Beta, alpha, largeSize, Experimental, etc)
    these are metadata added by a plugin developer to categorise a plugin and help users filter (in or out) plugins by category.
    Each filter has an associated level of visibility within SPManager (Enable, Mark, Confirm, Disable, Hide).

    1. Important: the list of filtering tags (earlyAccess, alpha, experimental, largeSize, etc) is not hard-coded. Any plugin developer can create a new filter flag at any time, simply by defining it in the metadata of a plugin.
      However, while the list of visibility tags (enable, mark, confirm, disable, hide) is not entirely hard-coded, SPManager only recognises those listed.

    2. SPManager currently makes no assumptions about a plugin from its filename. All metadata about a script or plugin is contained within the plugin or script file (currently in XML).

    3. The current filtering logic (including how the Beta flag is handled) works from this XML metadata.

    4. It is not possible to eliminate the XML metadata - there is more information there than could be squeezed into the filename.

    5. I would currently prefer to have all the metadata in just one place - rather than have some in the XML and some in the filename.

    4a. however, to have multiple versions of a plugin in the repository, there will need to be different files with different names, and in that situation, it would be most efficient to have the reason for the different files in only one place - the filename.

    4z. So my current suggestion is that I modify SPManager to infer a filtering flag from part of the filename - unless that inference is overridden in the XML metadata.

    **Discuss :)

    1. I am not sure we actually have that many bugfix release plugins, but I guess that if a developer adds that information to the filename, I can handle it usefully.
      I am even less convinced about putting enhancement info into filenames - many releases will include multiple new features - however, I am again happy to infer information from the filename if it is there.

    2. I like your suggestions on the filtering. I think we will still need some way to display to users what filters have been set on any plugin; but I am happy to improve the automation of the handling of these flags.

    6b. I think we will still need to provide some way for users to specify that they will or will not accept beta/alpha/experimental code.
    And I'm not yet convinced that asking the user for each affected plugin is a viable alternative.

    6z. My current suggestion is that SPManager continues to add an icon to the display of any plugin that has a filter set.
    - Then if there are multiple flagged plugins selected when the user starts a download, a single window opens showing which flags are involved and offering: "Include All", "Exclude All", "Let me decide Now", "Prompt for Each".
    The "Let me decide Now" option would open a dialog showing only those plugins with an associated filter, the description of why it has that filter, and a checkbox to include/exclude it from the download.

    1. I agree that asking users to manually adjust their plugins folder is not a UI, but I would point out there is a "Delete plugin"/"Delete Script" button in SPManager that allows any script or plugin to be uninstalled from within the UI.

    7a. Notwithstanding, I agree a "Disable" checkbox would be useful too... :)

    1. Instead of the pull-down that controls the content of the details panel, I could provide a row of tabs. That way users would be more aware of the other information available.

    2. I can provide a second line in the area between the plugin summary window (above) and the plugin details window (below) to show any filters. It would usually be possible to show all filters set for a plugin.

    3. If we are going to support multiple versions of a single plugin, then we should always display that information to the user - both in the repository, and in the installed plugins (under the Manage tab).
      I guess I should try to create open-close leaf logic (I hate coding tree displays in AWT/Swing) to allow users to navigate the different versions of the plugin and install either a single version - or all of them (install/update) or select which is active (manage).

    For example, f a plugin has multiple available versions, then it could be displayed as a folder - and if the folder is opened, the available versions displayed.

    1. thanks for your suggestions on restarting AOI.
      There has been code in SPManager from the beginning to do just that, but we could never get it to work cleanly. I'll see if I can fix that with the help of your suggestions.

    Thanks for all your input - and thank for readin :)

    Cheers!
    Nik

     
  • Luke S
    Luke S
    2013-11-20

    @Nik Thanks for taking a look at my suggestions. I'll wait and see what you have in mind for the filtering/flagging UI before I comment on it.

    First a couple of responses:

    suggestions on restarting AOI.
    There has been code in SPManager from the beginning to do just that, but we could never get it to work cleanly.

    My suggestions on this point were entirely lifted from some Questions & Answers on

    http://www.programmers.stackexchange.com

    (Very handy site if you don't know it. Try the internal search, or run a google search that ends: site:programmers.stackexchange.com) My impression is that no matter which way you slice it, this is going to be slightly messy.

    I would currently prefer to have all the metadata in just one place - rather than have some in the XML and some in the filename.

    I agree in principle. The question is 'can metadata do everything that we need?' See the next point, and follow-on post.

    I am not sure we actually have that many bugfix release plugins,

    I am even less convinced about putting enhancement info into filenames - many releases will include multiple new features

    This was the part of my suggestion I was least certain about, though my comments (and the problems I was trying to solve) about Bugfix/Enhancement versions would really apply to Release Candidates, and they made some assumptions about how I might like to see an automated build engine/submission system for aoisp work from a Plugin Dev standpoint. Which will be the topic for my next post...

     
    Last edit: Luke S 2013-11-20
  • Luke S
    Luke S
    2013-11-20

    Part of this post is growing out of the discussion of UI for SPManager, and part of it is growing from the thoughts on Automated builds/Automated submission, so bear with me.

    I'm going to start by listing a few of the constraints that I believe help define the problem:

    1. For the long term health of AOI's plugin repository, It is desirable to reduce friction in getting new plugins, by new developers, added to the repository.
    2. It is further desirable to make it easy for any developer to contribute bugfixes & enhancements to existing plugins.
    3. Bugfixes should not over-write the existing stable version until they have been user-tested.
    4. Bugfixes will not be user-tested with any reasonable dispatch unless they are available through SPManager.
    5. It is not desirable to give any and all developers direct commit access to the aoisp source repository. (Vandalism, or a person with good intentions but limited skills)
    6. It is not desirable for a maintainer (Nik?) to have to intervene in this process for every bugfix/new feature.

    As a developer adding a plugin, I would like my workflow to be:

    1. Create my lovely code.
    2. Notify the aoisp build system of where it is. Done.

    As a developer enhancing/fixing a plugin:

    1. Get a copy of the plugin code.
    2. Make my changes.
    3. Notify the aoisp build system of where they are. Done.

    What the build system should be able to do:

    1. Retrieve the source, and put it in an appropriate place.
    2. Do a basic compile. Notify the contributing Dev of any errors. (email?) Generate and run any automated functional tests (If Nik has added that capability)
    3. Place a .jar in the binary repository. Clearly mark as -new or -bugfix[bug description] (I'm deliberately not specifying how this metadata is stored/displayed) (I would call this stage a release candidate, at least on the context of this project)
    4. When certain requirements are met, (See below) Cause this to become the primary -stable version of this plugin.

    As a user:

    1. See that a new plugin/bugfix is available. If a bugfix, allow me to seamlessly slide between the original version and the bugfix. (I know we need to restart AOI, but I shouldn't need to uninstall one to use the other.)
    2. Give feedback. Could be as simple as a "thumbs up" to say the plugin works for me. I'd love to be able to do this inside the SPManager. (The more the user has to look around to make this feedback, the less likely he is to do so. More complex feedback, such as bug reports and UI suggestions, should go to the aoisp forums, just as they do now.

    'certain requirements'

    A plugin should be moved from beta status to stable when:

    • It receives enough positive feedback from users ('Enough' still needs to be defined!)
    • One of the project coordinators (Currently Nik or Peter, probably) can short-circuit a plugin to stable status. (The way AOI seems to go, I wouldn't be surprised to see this happen more often or not)

    One Possible way to implement the Dev. side.

    Looking at the workflow I've marked out here, I'm reminded very much of what some people would call "Fork and Pull" development.

    I think that a lot of the workflow functionality is supported, or could be built on top of, one of the DVCS platforms. From here on, I'll speak specifically of git, though Hg is also supported on sourceforge.

    • Each plugin's source could be kept in an 'orphan branch' of the repository.(No common history)
    • sourceforge has built in ability to create a fork for a dev to work in.
    • git has the ability to create 'pull request' data that has a specific format. Such a request could be sent to an email address that is watched by a 'bot that can parse the pull request.
    • Pull requests of a source tree that does not have any history from the main repo (such as a new plugin would be) Would be pulled onto a new 'orphan' branch
    • Pull requests that are a modification of one of the existing plugins are parsed as follows: (The automatic system should be able to do all of these things by issuing a few git commands)
      1. If the dev created a feature branch off of the plugin master branch, (as he probably should) keep the dev's branch name, etc.
      2. If the dev committed to the plugin master branch, create a feature branch for this pull request.
      3. If the plugin master has been changed since the dev's branch, (as would happen if another dev has had a change promoted to stable - probably a rare situation for aoisp) Insist that the dev merges the plugin master into his branch before adding the branch to the repository. If this has not been done, the 'bot should reject the pull request, with an explanation. This prevents painful merge conflicts later if two or more are working on the same plugin. (Rare)
    • Build system builds a .jar for the most recent commit in every branch. (unless a branch is marked -dead somehow?)
    • When one of the [plugin]branches gets promoted to stable, the branch is merged back to [plugin]master. (If point 3 above has been enforced, this should be a simple fast-forward merge, done by the automated system)

    This solution does assume:

    • That plugin devs are able to understand, or learn, a few basic git commands. (All of the advanced stuff is actually done by the build system)
    • Or that someone else would be willing to input their files into the system to start.
    • That any home-brew solution will also have a learning curve, not necessarily simpler, that no other experience will prepare a developer for, and that is not applicable anywhere else.
    • That such a system, whether implemented as I've suggested, or home-brew, would be better than posting files on the forums and waiting for Nik to have time to take a look at it.

    If you've stuck with me this far, thanks for the patience. Lets take a break.

     
    Last edit: Luke S 2013-11-20
  • Hi Luke,

    So far, your approach works for me - and the approach I've been considering for a while now.

    I built an automated plugin BOT some time ago, based on my experience with AOI, and what I wanted to do for the AOI repository rewrite.

    From that experience, I developed a simple plan for AOI, which is very much along the same technology lines you haev described.

    • I need to re-read and consider some of your finer points.

    • I plan to use GIT, allowing the repos to be hosted on SF, or any other GIT-based hosting system.

    • GIT will be used for both the source management and the binary management.

    • There is an issue you haven't covered yet: that we need to support multiple versions of AOI - and now each of those versions could have multiple corresponding versions of a plugin.

    • submission of a new plugin is automated using a back-end BOT, with submitter's identity proven and checked using GPG certificates.

    • There will certainly be a manual (human) override for accepting/rejecting and approving/blocking a plugin.

    • A CI tool (assume Hudson/Jenkins) will be used to build and test all uploaded code.

    • I'm not quite sure how to handle updates submitted by someone who is not the code owner.
      My current thinking is to follow the linux kernel approach of using GIT to prepare an email that is forwarded by the BOT to the code owner for checking and inclusion into the plugin.

    • We can use the jGIT library to automate the client side
      7.1 certainly for binary downloads ie installing/updating plugins and scripts;
      7.2 possibly for devs - although I am hoping that I can provide some Ivy files that will allow regular Ivy build rules to download dependencies, and publish changes.

    • Either way, I don't want to force devs to learn GIT.

    I'll re-read your finer points, and think on them some more.

    All in all, I think your suggestions are awesome, and add a lot to my original thoughts.

    Thanks! :)

    Cheers!
    Nik

     
  • Luke S
    Luke S
    2013-12-16

    @Nik: Sorry for the delay in replying. It sounds like you are a lot more conversant than I am in the nuts and bolts of using a version control system for something like this, but I do have a couple of further thoughts/comments.

    Either way, I don't want to force devs to learn GIT.

    Fair enough. Two things to keep in mind, though:

    • Whatever you do build needs to have a simple, clear UI.
    • If at all possible, allow those who do know how to use GIT the option to use it directly. This includes structuring the code repo(s) in a way that a GIT trained dev will find idiomatic.

    I'm not quite sure how to handle updates submitted by someone who is not the code owner.
    My current thinking is to follow the linux kernel approach of using GIT to prepare an email that is forwarded by the BOT to the code owner for checking and inclusion into the plugin.

    My concern with this approach is what happens in the case where the plugin developer has moved on. Case in point: Polymesh. (Yes, I will probably keep coming back to that) IIRC, Francois G. posted something here a couple of months back, but otherwise seems to have dropped out of the AOI community entirely. Who should 'own' that plugin? This is why I was proposing allowing a bugfix/enhancement branch to be created by anyone, (gated with ID checks of course.) and then allowing users to try it out side-by-side with the original.

    There is an issue you haven't covered yet: that we need to support multiple versions of AOI - and now each of those versions could have multiple corresponding versions of a plugin.

    First Question: How many AOI versions do you want to support? I know that for a while, some users were resisting upgrading from 2.8.1 because they didn't like the progressive renderer. (Windows installer issues added to this feeling) but you can't support everything.

    Proposal:

    • One folder in the repository for each AOI version supported.
    • SPManager only looks in the folder that matches the version that is running on the user's machine.
    • Contents of each folder are automatically compiled against the supported AOI version.
      -If the newest main-branch version of a plugin will not compile cleanly against the AOI binary, go back up the version history until you find a version that does. Mark the plugin as -outOfDate, perhaps give a warning when installing: "There is a newer version of this plugin available, but it is not supported by your version of AOI. You may want to think about updating AOI"
    • If there is no version of the plugin that compiles against a certain AOI binary, it is not visible in SPManager.
    • If a plugin fails to compile against the latest supported version of AOI, There should be an alert sent - probably to the original contributor, possibly to any who have asked to receive such alerts, - unless the plugin has been marked -deprecated, in which case it probably shouldn't even try to compile. (Think the meta-data plugin. No-one should try to use it today.)
    • There should probably be a folder of plugins compiled against the latest HEAD of the AOI repository. This gives people a chance to fix plugin incompatibilities before they affect any non-dev users.
    • Any multiple versions of plugins should be short lived. The whole point of putting them in the repository in the first place is to get them user-tested quickly so they can be folded into the 'mainline' version.

    GIT will be used for both the source management and the binary management.

    This one has me scratching my head a little. What are you hoping to gain by putting the binaries in version control? I understand that some projects do this to make sure that a certain (usually tagged) version of their code is only ever compiled against a certain build of a third-party library, but I don't see that here. The way I'm thinking of this:

    • An old binary should not be run against a new version of AOI. Make a new compile. This will catch a lot of the version-based incompatibilities.
    • Once a bugfix or feature is accepted into the main version of a plugin, the older version should not be available from the repository.
    • If a change to a plugin was a mistake, its simple to revert the source history and rebuild.

    You don't want old binaries hanging around. Over-write them. Or am I missing something?