UCOSPLogMatthewRodusek

Each UCOSP participant is asked to log all the steps they take as they make progress through the semester. This includes difficulties they faced (getting set up, understanding how to model in Umple, using UmpleOnline, code that is difficult to figure out, and development tools that are hard to figure out, or don't seem to work properly). The more log entries added the more we can better get a grip on the usability issues with Umple, and the more we can help future new contributors come up to speed.

Please record entries in reverse chronological order

Log Entries

Date

April 4, 2016:

Note to self: Never run grep piped into sed in the root directory of a git project unless you use either the `--exclude flag to exclude specific files, or --include to restrict them. Last night I completely wiped my Umple repo by trying to do a bulk-replace for a job that could have been done in a matter of minutes. Not my finest decision. Spent a good while cloning everything and setting up the remotes, then redoing my work on the branch (since my work was nuked). Thankfully most of my work has already been pushed, so not much was lost.

After working on extracting everything into a separate directory for parsing I came across a small problem (not necessarily an bug or issue). It seems extending from classes marked external don't generate automatic constructors or call the super constructor. It's possible this is intended behavior since external classes and interface are coming from Java, so they wouldn't have contextual information from Umple. It's also likely this is already thought-of and I just misunderstand how to handle externals.

At any rate, this lead me to re-evaluate where the Parser class, and a few related components were located. I originally moved them to be part of cruise.umple.parser namespace expecting they had a more important purpose to the overall RuleBasedParser, but it turns out that only the Token,Position`, and error handling classes were important. So I extracted those out, which not only helped my original problem, but also makes it more usable.

After I managed to get it working I issued a pull request and had it accepted sometime this morning.

Mar 28, 2016:

Update 11:50PM

Good news, finally. After working through the test-cases one-by-one to find the source of the issue, it turns out that part of the issue was coming from the handlers being assigned as non-static. Given the way that the RuleBasedParser is constructed and contains static variables that are kept across instantiations, it is also important to keep the handlers static as well -- otherwise not assigning it somewhere may change the analyzer's data -- which is not good.

Another source of the issue was the way that I was passing linked files. My method expected the source file, along with the files to be linked with the main file. Now, this shouldn't itself be a problem -- except for the fact that in init(File file) I call init(file, null) -- effectively passing an empty list of linked files. This was giving -- depending on the case -- nullpointer exceptions along with index-out-of-bounds errors. This, obviously, is not desireable -- and thus I have since fixed it.

I need to clean up and squash some of my commits together to make them a little more coherent, but after that I will be pushing to origin and will wait for the travis build. If all is good, I should be issuing a PR soon and then all that's left is extracting it to a separate project/directory/repository (depending on how it's handled) and then adding it to the build step.

It is such a relief that all is working well now. It was really intimidating to see a huge collection of errors like that.

Update 5:00PM

I'm still working on the failing tests. For some reason, once I switched everything over from UmpleFile to File, every test began to fail for different reasons. Some were "NullPointer" exceptions, others were "Index out of bounds", or just simply "expected <some random thing>". I'm not sure what exactly is the problem here, but it's quite annoying that this is occurring.

I'm beginning to suspect this has something to do with the static variables in RuleBasedParser. The parser is created in many tests (and originally it had to be changed in tests in order to make it compile properly, since it originally expected an UmpleFile instead of a File), and it often is only constructed without ever reading the rules. My understanding is that it is doing this since the rules and everything are handled statically, meaning that by constructing a new one it still points to the one with generated rules from somewhere outside of the test-cases.
I feel like this may be part of the problem though. Perhaps the tests aren't being run in the right order, or perhaps the construction isn't quite working right; I'm not sure exactly what the cause is, but it's the only reason that makes sense to me.

Currently, Umple compiles anything properly -- including itself. It's strictly the test-cases that are dying a horrible, horrible death and making this even harder. I'm not sure why the original design was to create multiple RuleBasedParsers instead of just simply doing model.getRuleBasedParser() or something. Right now, the static variables appear to be the cause of virtually all the problems. I've reverted back to a previous building version that still contained UmpleFiles, and will work from here for now.

3:00AM

I'm very close to releasing a pull request that should remove all ties of umple from the parser, but I encountered a very difficult issue that has held me up for almost an hour.

In attempting to update the parser to take File instead of UmpleFile, I had to modify many different tests in order to make it work. However, one of the tests constructed a large quantity of java/php/ruby files that then caused a series of huge compilation issues at command line! I'm not sure which file in particular is the cause of this, or why the files weren't deleted, but this caused even further problems. This issue couldn't be simply fixed by using a previous Umple jar or even doing a first-build. Because it failed midway through the tests, the java files were attempting to be compiled -- but failing -- even in the first build. I had to reset back to a previous revision and manually delete the generated files (wrote a script to do it quickly). That was a pain that took away a decent amount of work.

I'll be working more on this throughout the day to hopefully get the PR in. Then it will be just figuring out where the parser will be moved to, and how it will be imported into the project.

I made sure to test that the changes so far work independently of Umple. I created a separate umple project that contains symlinks to all the umple files used in the Parser (e.g. RuleBasedParser.ump, etc), and a Master.ump file that brings them all together. Then I wrote a small script that simple compiles it with Umple, followed by with java into a separate directory.

As long as I can get all the tests to pass successfully, then this work should be mostly done.

Mar 27, 2016:

I didn't do any work on Umple yesterday because it was my birthday and I had guests over, but I'm back to work today.
I finally came up with a way to get around the UmpleFile and to pass and handle linked files. To do this, I created a new interface LinkedFileHandler that requires users to implement String onFileLink( String input, File[] linkedFiles ). This method takes in the input of the current file, along with the File objects for the linked files and then handles the linkage in this delegate handler. In Umple, this will be producing a bunch of use <filename>.ump strings and appending it to input before returning the value, but it can easily be changed for other people's uses.

I also managed to find a way to pass in the dependence on GrammarAnalyzer to the ParserAction by adding the analyzer to ParserDataPackage. This way the required information is still accessible.

With all that done, I'm down to a few smaller, but manageable, problems:

1) I now need to find a way to pass in the linked files from the main. This requires me to pass the dependency all the way from UmpleModel -> UmpleInternalParser -> RuleBasedParser. I'm thinking that I might be able to hackishly fix this by just doing a split on the linked string by doing getLinkedFiles().split("\nuse ") or something to produce the string names from the UmpleFiles. At least for now. If I made changes to the constructor for UmpleModel, it would require a massive change overall since many tests require the current constructor.

2) I need to make sure that all the pieces will work together at once. Currently, by design, it's been a pain to test things due to it building successfully but failing the next build (which Tim provided some helpful suggestions on going around). Still nonetheless, it has made testing to be quite difficult to ensure that it still provides the same output as the original work.

3) I may have to modify almost every test that includes the RuleBasedParser due to the fact that I have removed the coupling from inside the class to be done externally through hooks now. This means after construction I have to do things like addParserAction(...) on the RuleBasedParser.

4) My final issue is just getting this all into a separate project/repository (however this may be handled) and having it import in correctly. After thoroughly reviewing the code I am nearly positive that there exists no more coupling between other classes after I finish these fixes -- but some things have snuck up on me in the past, meaning I can't be sure.

All of this is manageable. That said, I suspect I won't have this done by Monday necessarily, but shortly thereafter at least.

Mar 25, 2016: 1:15PM

Every time I compile I risk the possibility that something has broken but compiles correctly, resulting in the next compile breaking everything. This is getting absolutely tedious, as it has resulted in many first-builds having to be done since subsequent builds won't work.

Every time I think I've found a solution to this problem, it causes further problems instead. It will compile successfully, pass the tests, and then fail on the next compile because something must have changed just enough to not be caught by the tests, but to die a horrible death on the next build. This is entirely caused by trying to remove the heavy and unnecessary integration of UmpleFile all over the parser.

I was trying to make this class work without requiring any polymorphism, but the more i work with this the more I think about how much headache I could be saving if I did just decide to make GrammarAnalyzer into a superclass, and have an UmpleGrammarAnalyzer that implements it that will be used in Umple directly. My only dislike with this comes from the fact that it would require users of this library to now have to create their own analyzer class and have knowledge of how it works.

I might just have to do this though, since it will simplify many of the problems due to Umple's heavy integration. Part of the issue with doing it using methods and hooks like I was trying to do, is that the GrammarAnalyzer currently needs to set up a ParserAction in the constructor. And due too the cyclic dependency and construction of the parser, having RuleBasedParser -> GrammarAnalyzer -> RuleBasedParser, it needs to have the GrammarAnalyzer have the same ParserAction constructed. This means it may not work nicely attempting to assign them with a method, since there could be various points where an assignment may be done.

Mar 25, 2016: 2:20AM

I've spent a lot of time today working on removing the UmpleFile dependency, and it has not been an easy task.

One change I am looking to make is to move the parser action for the useStatement to be done from outside the class, but the current design forces the use of the UmpleFile currently inside of it. I was hoping to use the ParserDataPackages 'getFileName()` method, but even this isn't helpful due to the fact that the file name is just the name and not the path. I could change it to remove that, but apparently it's something that is hoped to be refactored out.

I was also considering changing the data package to contain a File object instead of a string filename, but this hasn't been particularly useful as it has resulted in a lot of errors from across many sections of Umple.

I'm also still working on getting the linked files into the RuleBasedParser by other means instead of having to use getLinkedFiles from UmpleFile. It looks like one of the main classes actually keeps a list of filenames to be parsed, and so this may prove to be useful. This will require modifying the Model though to change what can be passed to it.

Still not 100% sure what to do for the linkage after that though. The current method just appends a series of use <filename>; statements to the current file input, effectively shivving the recursive statements in. It's a hackish solution, but without better knowledge of the way in which the couples and everything are produced, I can't see how to change it from its current state. I might just make an overridable hook that will allow the user to specify how to add new files, but I'm not sure just yet.

Mar 22, 2016: 3:30AM

I've spent quite a few hours going over the GrammarAnalyzer code, and I think I finally understand what it's doing, and how to deal with it.

I've been concerned about the Analyzer classes being constructed using reflection in the makeAnalyzer(...) method, particularly because it tight quite tightly to cruise.umple.analysis. After going through what the actual Analyzer class is doing, however, it begins to make a bit more sense.

Analyzer is just a superclass of all the different rules that need to be analyzed through specific means. Analyzers can implement up to two methods: prepare(Token token) and analyze(Token token). prepare(...) is called before analyze(...), acting almost like a constructor/initializer by giving time to initialize different data elements that will then be used in analyze(...). Alternatively, this could be used to sanitize/strip token input for consumption after. analyze(...), as the name suggests, is the actual analysis of the token. These analyzers are often called from outside for consumption, such as in the UmpleInternalParser.

Analyzer is simply used to construct other analyzers that inherit from it, constructing the appropriate ones by the rule names so that the rules get analyzed correctly.

So that's how the Analyzer class alone works; now how does this all tie into GrammarAnalyzer? Currently, the GrammarAnalyzer constructs all analyzers by using reflection, attempting to instantiate it given the fully qualified classpath and the string rule-name to use. If it succeeds, then it's added to a running list of analyzers. If it fails because it doesn't exist, then it quitely ignores it and continues. Basically, at the moment, it brute-forces instantiating every type of analyzer for all rules in the grammar, even though only a few actually will exist.

Right now, this method is only working with cruise.umple.analysis.<some analyzer>Analyzer -- but I plan to change that shortly. For the time being I will make the classpath assignable (setClasspath(String) or something), but I would like to add to that in the future. I dislike using this reflection method because, though it's automated, it's also quite limited in that it requires the full path, and would need to be changed if ever the package name changes (instead of having a lot of it automated or successfully inferred).

It'd be much nicer if instead the user could manually assign which Analyzers exist for each string either individually, or in bulk. Still using java's reflection, this could be done with Class objects and string identifiers, or using generics in a function and string identifiers. Something to the effect of:

void makeAnalyzer<T extends Analyzer>(String identifier){
  // do something with identifier and T.getClass()
}

That's just my thoughts on the matter. Might be a little more cumbersome to write for many classes (such as Umple's ~64 analyzers), but even that could be automated through other means -- for example by loading the data from a file.

Anyway, that was off-topic. Since this Analyzer class has no real ties with cruise.umple.compiler, I figured it was quite suiting to have it moved to cruise.umple.parser.analysis instead. Now, any and all Analyzers must extend cruise.umple.parser.analysis.Analyzer any time they wish to add new analyzers to their parser.

The other thing I figured out while working was how I can get around the use <umplefile> statement. The GrammarAnalyzer actually came pre-designed with the idea of ActionTokens; tokens that, when parsed, execute useful actions (ParserAction). Currently, only one is being assigned in the constructor of RuleBasedParser that handles "useStatement"s by reading the source file and parsing/analyzing it as well. I plan to take this functionality and make it work by use of a simple function (addAction or something), and this will be done in the construction of the RuleBasedParser somewhere in the model or compiler instead.

After that, all that should be left is removing UmpleModel. And now that I understand how the parser is automatically-parsing new files, I think I can solve this issue; I just need to figure out how. Hopefully after this I will be all set to bring this to a separate repo to compile and run as an independent project.

In My Opinion: Having had to pour through a lot of the RuleBasedParser code, I really do have to say it is a wreck and could use an overhaul. Right now it has many static variables that get initialized only once, but the class is never handled like a singleton (which would at least indicate that it's a single-instantiated class). Instead other classes construct a RuleBasedParser that refers to the same parser's grammar, but not to the same things like root tokens. Not only does this kill readability, but it's also created it's own form of spaghetti code.

GrammarAnalyzer and RuleBasedParser both pingpong off each other to produce results. RuleBasedParser constructs a GrammarAnalyzer which constructs a RuleBasedParser (this one has the same grammar as the first, but different tokens) and handles dispatch of some of the RuleBasedParserThreads. When a parser parses, it gets the parse result from the GrammarAnalyzer and assigns the root token to its parser's root-token.

Most of the statics could have been eliminated by creating a copy constructor or reference-passing important variables like the grammer. Or, alternatively, the GrammarAnalyzer could have worked as an internal class that would have direct access to the important data that was required. Sadly, due to the current design and how often it's used, changing it would be a rather large feat.

This has made it a nightmare for readability and even more-so for maintenance. It may work and/or be efficient, but it comes at a painful cost. Not to mention this could minimize usefulness of the parser, since it can only be useful for a single grammar.

Moving forward I really strongly believe that this should, at some point, be overhauled in favor of a cleanly written one.

Mar 20, 2016

Apparently I missed a few emails from the forums where it was discussed that the jet generation was to be shut-down for good. After the last conference call I was under the impression that some of the Jet-related code was going to remain open and available for work while others were shut-down, but I guess this is not the case. Not entirely sure what my next course of action is with regards to the one fix -- but it looks like it will have to wait. I guess it's my fault for not keeping up with the emails from the forums (Google auto-sorts them into a 'forums' section and doesn't give me notifications when they arrive, so I often miss them). I've reached out to Vahdat about this, but I suspect it will just have to wait

Otherwise I've returned to the parser work. I've finally narrowed down the coupling to cruise.umple.compiler.UmpleFile, which I'm working on removing, and cruise.umple.compiler.exception.UmpleCompilerException which is used in the ErrorTypeSingleton. Other than that I need to find a way to remove cruise.umple.analysis from the RuleBasedParser since it is heavily tied with it. Some things may be easy to extract, but others will likely be difficult.

Mar 18, 2016

Today I've been working quite heavily on fixing a few line comments that have been broken since the previous refactor with getRelativePath(...). The original refactor seems to have caused a lot more issues than intended, and this is in-part due to me attempting to achieve a syntax of getRelativePath(<lang>), as I was informed to do.

Given the results of this, it appears that having the syntax that simple may not actually be feasible after all, since the relative path is calculated multiple times for various elements -- for example between method declarations, class declarations, etc. Thus the only way that I can think for that syntax to be feasible is to do an obfuscated form of parameter-passing with a method. I originally used this since it worked properly for evaluating the @umplesource tags, thus it didn't appear to be broken at all. However, in retrospect it caused many issues for individual line comments -- and not always in easy-to-find areas either.

While I've been doing this I've also taken the time to implement @umplesource tags and line comments in the Java Interface generator. For some reason they were never implemented in interfaces before this. This may be due to the fact that UmpleInterface never contained any associations with Comments, and because it was unable to acquire a relative path from a Position, back with the way it was written before.

That's what I'm working on so far. After this I am going to be back to removing all traces of the compiler from the parser, which includes UmpleFile. I think I'm finally getting close to completing this part of the project; it's required an awful lot more refactoring than I initially realized, and a lot of understanding of different components around the system.

Mar 15, 2016

1:00AM

I finally found the problem with the tests! The ErrorTypeSingleton was causing problems between the two different test-cases. I'm not 100% sure on the specifics of how or why, but it appears that a call to ErrorTypeSingleton.getInstance().reset() solved this. I found this out by looking through other JUnit files that made use of the singleton, and noticed that this was often the first call prior to constructing an UmpleModel. I am guessing that, without resetting, it won't always properly throw exceptions.

The reason I think this is because the Collector's analyze(..) method makes use of a try-catch block, where it only returns false on a thrown exception (both failing tests were expecting false but receiving true instead). This was made harder to discover since it only catches a generic Exception instead of specifying which Exception to look out for. It's complicated even further by the fact that the throwing methods don't have throws declarations to easily find methods that throw Exceptions. Gah.

I'm going to guess the tests were able to successfully pass in the past by luck due to the order in which tests were executed. By moving the tests to a new directory, it must have also changed the order the tests were executed which resulted in this issue.

I'm just gonna tack this whole event on as another reason as to why I dislike Singletons. It took quite a few hours to find out, only to be fixed by a 1-liner.

Mar 14, 2016

Before I went to bed last night, I took one last stab at Umple and corrected most of my failing JUnit tests. It was actually a lot easier to fix than I had realized, as it turns out that the directory structure needed to match the package name. Some of the tests were renamed from cruise.umple.compiler to cruise.umple.parser accordingly, which meant the tests originally in cruise/umple/compiler needed to be moved to a different directory of cruise/umple/parser. I'm not sure why that made a difference, but it seemed to fix it.

All that's left now is some strange failing MetricsCollector test. I'm not sure what's causing this one, as I didn't edit anything relating to it (short of just source imports), and the tests that are failing are fileNotFound and analyze_unknownFile -- both of which are expecting a result of false from the method collector.analyze(...). Somehow, it's always returning true now even when the input is invalid -- so I need to find out why.

After this, I'll issue a pull request and implement the next few necessary changes, which are removing cruise.umple.compiler from all parser-related files and moving them to a /parser director. After that I can figure out what the cruise.umple.analysis is for, and whether it's crucial to the parser. Hopefully this won't take as long though.

EDIT: 5:50PM

I can not for the life of me figure out this test failing issue. After moving some tests from cruise/umple/compiler/ to cruise/umple/parser it causes the MetricsCollector test to fail. But if the tests were kept in the old location, the parser tests fail -- but MetricsCollector passes. I don't understand how the tests could be connected at all like this.

The other curious thing is that once (and only once), the testJava multipleDoActivitiesInNestedStateMachine failed on me with the reason:

Index: 1, Size: 1

java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
at java.util.ArrayList.rangeCheck(ArrayList.java:653)
at java.util.ArrayList.get(ArrayList.java:429)
at cruise.statemachine.test.CourseWMultiDo.getLog(CourseWMultiDo.java:59)
at cruise.statemachine.test.DoActivityTest.multipleDoActivitiesInNestedStateMachine(DoActivityTest.java:128)

I've done multiple compiles and tests since, and this one hasn't reoccurred. It's just strange since I never touched this file directly, or any of the files relating to it.

EDIT: 8:10PM

Tim emailed me a little while ago with information regarding the multipleDoActivitiesInNestedStateMachine failure. Apparently it's a problem that occurs due to timing about 1% of the time it is run. I'm glad I'm not responsible for it!

I'm still trying to single out the problem regarding the other failing umple test.

I decided to start from basics and run the tests without any of the parser tests present, and the MetricsCollectionsTest didn't fail. Then I tried running it with only the CoupleTest, and it also didn't fail. Then, running it with CoupleTest and ParserTest caused a failure.

It looks like one of the parser tests being present in the parser/ directory is somehow causing the MetricsCollectionsTest to fail for some reason. I have no idea why, but I guess the next little bit will be singling out the problematic file (or files) and isolating it from the rest. I'm hoping this is due to a test itself, and not due to some obscure error like the number of tests (or combination of tests) that cause this failure.

EDIT: 8:50PM

I've isolated the test that seems to be sparking the MetricsCollectionsTest failure (although I still fail to understand why it's the problem). Any time ParserTest is present in cruise/umple/parser instead of cruise/umple/compiler it causes 2 of the 10 metrics tests to fail.

I wonder if this has something to do with the order in which JUnit tests are executed?

The really weird thing that I've just discovered is that the sheer existence of a test named ParserTest seems to cause it to fail, even if the entire file is commented out (except for the class declaration). This is insanely bizarre.

EDIT: 11:30PM

Apparently I was wrong about it failing as long as the class exists in the last edit. I wasn't aware that the tumple script doesn't actually compile the tests, it just runs them. After many. many. many... tests I have discovered that the issue is somehow related to the ErrorTypeSingleton tests in ParserTest. It looks like any of the tests that invoke the singleton somehow cause the MetricsCollectorTest to fail. Still no idea why it is, but at least I have a solid lead now.

It's just strange that the failing tests are relating to files not existing, not from any directly visible reference to the ErrorTypeSingleton. I need to look into this even further.

Mar 13, 2016

Yesterday it took me under an hour to find the appropriate place to fix issue [#768], and yet -- due to a reinstall -- it took me many hours to get it to generate code with JET.

Background on my issue

To understand my problem, first it's best to know my current set-up. I have my Windows computer(s) that I do work on, but I use my linux server to compile the code. With the linux server I virtualize a filesystem that I mapped to my windows computers to make accessing/editing Umple code easier (the alternative is using another separate git repo that pushes to my linux server that then pushes to GitHub, but I didn't want to go that route). The main reason for this is I want to be able to compile all my code straight through Linux instead of through Windows, as the compile times jump from ~4.5 minutes to almost ~15 minutes.

I wanted to try something new with my setup, and that was to generate JET code from my Windows computer using a network mapped project (since I primarily avoid GUI on my linux server). So I started with Eclipse Mars on my Windows computer.

First I tried installing all the appropriate plugins needed to produce the JET code on Eclipse Mars. This straight-up refused to work, not installing a few necessary portions (even after following the answer on stackoverflow). This was to be expected, as the last time I used JET it didn't work properly on Mars for me -- and so I attempted to use Luna instead.

Luna proved to work a little bit better, with only some minor conflicts for installation. One somewhat major issue is that it never produced the expected "JET Settings" menu, meaning I could not properly set up the JET project. The other problem is that finding and dealing with projects over the network slowed down Eclipse to a grinding halt. This would take upwards of 10 minutes to produce meaningful data just so I could hit "next" to just continue the process.

And so I figured this wouldn't work. Instead, I decided to RDP over to my Linux server and just install Eclipse on that. I went through with the setup, and this time I even attempted using Eclipse Juno (the last one recommended to use with JET). I tried installing the appropriate plugins, and everything went through without complaining -- so I thought "yay, it's working". However, then I realized that -- according to Eclipse -- it had absolutely no plugins installed. Despite having all the appropriate *.jar files and having the update sites, the information claimed there were no plugins installed short of the defaults.

I looked into various methods to attempt to get around this, but I could not find what the issue was, and so I resolved to try Luna again, only on linux this time. This gave no different results for reasons that I could not fathom.

Eventually after all of the struggling, I went back into my network-mapped Eclipse set-up -- which is the first one that was closest to correct and tried running the clean command to see if it would generate the jet files. It struggled and chugged along for over 10 minutes, but eventually it did produced the expected code. This came as a surprise to me, given that it doesn't give any editable options for JET settings in the menu.

This task took a lot longer than expected all because JET wasn't behaving as intended -- and for that I am excited to be moving to the Umple Template system.

EDIT 1

Man am I ever thankful for the javadocs, and the automatically generated @umplesource tags, because I'd be so lost on this refactor without it. I've already had to go through changing 14 different files to change references from cruise.umple.compiler to cruise.umple.parser namespaces. Each time I clear up all the javac compiler errors, it throws up 20+ new ones at me to fix -- and the javadocs have aided in locating each file.

Unlike other languages that stick to a "1-file-per-class" ideology, Umple allows multiple classes in multiple files -- which makes navigation and traversal quite difficult without auxiliary documentation. I understand the idea behind it, as it allows for a mixin-esque feature where a class is defined in more than one location -- but as far as maintainability goes, it's a feature I would rather not have since it complicates locating where classes exist.

As I've been changing the dependencies on the namespace, I'm also taking the liberty to decrease the scope of the dependency to just the required classes. Prior, most classes just had a generic depend cruise.umple.compiler.* bringing in the entire compiler namespace. Often I'm finding that most classes only had that as a crude way to import just ParserResult, Token, or Position -- and so I've narrowed the scope to just the requirements on most classes that I've had to refactor so far.

EDIT 2

I've been working at this for 5 hours straight, and have modified 41 files so far just changing the dependencies. So far there is no end in sight to the compiler errors. They keep coming, and each time I fix them it delivers new waves of them to me. I wish that initially the imports were handled explicitly instead of importing the entire namespace.

If all dependencies were done as depend cruise.umple.compiler.<specific class> then this could have been sped along a lot faster by just doing a bulk rename from cruise.umple.compiler.<specific class> to cruise.umple.parser.<specific class>. Sadly this is not the case, and so I have to manually find each and every point of error, of which there are many.

This work has actually been kind of fun, albeit painful, as it has led me to almost all corners of Umple -- exposing me to parts that I hadn't seen before.

While on this magical journey I have noticed a few things that I wish to/need to get to as well.

• ErrorMessage and ErrorTypeSingleton were originally part of cruise.umple.compiler and are now part of cruise.umple.parser. Neither location is particularly fitting, given that they are used in both namespaces. This probably belongs in a more utility-style namespace since it could be used just about anywhere. However, this change would require me duplicate the classes in cruise.umple.parser to avoid the parser having dependency back to Umple (which is the main thing I'm working on).
• UmpleFile really doesn't seem to have much practical use, it's a small wrapper around simple path methods. The one main difference that an UmpleFile has that a Java File does not is the getLinkedFiles() which seems to only return a string of use <umpfile>.ump; \n [ use <umpfile>.ump; ... ]. This method only really gets used in one location so far as I can tell, short of the tests. Strangely, the method it is used in is titled AddExtraFiles in title-case notation. This is only ever called once to set up the main *.ump file to translate, and it gets called in init.
• Interfaces in the JavaDocs for Umple never generate @umplesource information. This doesn't seem to be related to my original refactoring work, as it never appeared for interfaces before I did that work. I think this may have something to do with the fact that UmpleInterface, prior to the refactor, never kept any information relating to its source package -- meaning a relative path could not be established. At any rate, given that I've had to make extensive use of the JavaDocs, I'd like to implement this to make it easier to navigate the sea of *.ump files.
• I need to remove any imports in the cruise.umple.parser package that reference other Umple packages if I wish to properly separate this project.
• The GrammarAnalyzer uses elements from cruise.umple.analysis. I'm still trying to figure out whether or not these classes are part of the Analysis stage or whether it would better belong in cruise.umple.parser.analysis. There are quite a few poorly-chosen names that I've encountered that make deciphering the functionality to be quite difficult (for example, UmpleInternalParser isn't a parser.)

There is so much work to be done here! When I started isolating the Parser project, I had no idea how much work it'd entail since my initial overview made it appear rather separated. It wasn't until I began looking into the functionality of objects, and the namespacing that this became more evident.

EDIT 3

This will probably be my final edit of the day. I spent around 8 hours today working on the refactoring. I changed a total of 50 files (according to git), most of which were changing dependency statements. This task should have really been a lot faster, but took long for 3 reasons.

1. Most depend statements tend to take in the entire namespace (e.g. depend cruise.umple.compiler.*) instead of only importing specifics (cruise.umple.compiler.ParseResult). If it were done via specifics, I could have had this done hours ago since I would have been able to run sed or some other replacement program through all the files that import them.
2. When a java file fails to compile, it spits out the java file name that failed -- not the umple file. This means that every failure to compile gave me just the name of the class or interface that I had to find. This becomes complicated, since Umple can have multiple classes per file, or multiple files per class. This resulted in an awful-lot of me having to search for files individually -- using the JavaDocs @umplesource attribute to find where to begin looking. This issue became even more complicated due to my next point.
3. When javac spat out errors to me, it didn't do it with any reasonable order. It would throw ~20 errors at a time, and often it would throw new errors from a class I already fixed a few iterations prior. I'm not sure why the errors occurred in this order, but it resulted in me often backtracking after I thought an issue was properly resolved. Given what I mentioned in point 2, it resulted in a lot of jumping around.

I've updated my pull request that fixes the issue from yesterday. When I committed my changes yesterday it attempted to claim the entire JET-generated files were modified by me -- which isn't a good thing.
So today I found out what the issue was and pushed the changes back to origin to update my Pull Request. Turns out this issue is the result of my git server not properly stripping CRLF to LF. This was a frustrating endeavor that took a while to properly fix (resulting in a lot of soft-resets and re-commits). Now I'm just going to hope that it's accepted and I can continue on with other work.

Today has been a long day as a result of this. I have a few test cases that I need to fix (currently failing), but Umple at least compiles successfully -- which is a step forward.

My next few steps are to work on the following:
- correcting the currently-failing tests
- re-arranging some of the tests from umple/compiler to umple/parser (for tests strictly dealing with parser-related files).
- remove dependency to UmpleFile from the parser. Ideally UmpleFile should be removed in general, but this may be a huge change depending (lots of tests require this)
- Figure out why cruise.umple.analysis is used in the GrammarParser and see if it can be refactored out or removed.

Mar 12, 2016

For this past week I have been working on navigating files from the cruise.umple.compiler namespace to the cruise.umple.parser namespace, since a good portion of those symbols are crucial to the parser more-so than the compiler. This has resulted in the unintended side-effect of a lot of compiler errors resulting from "Ambiguous" references to the different classes. Apparently during the compilation stage javac seems to think that certain classes exist in both namespaces simultaneously.

I originally thought that perhaps I missed a class that was defined in separate files (since Umple allows you to do that), and worried that it was defining the same symbol in 2 different namespaces -- however after a lot of grepping through the source, I feel I can rule that one out. I'm not entirely sure what's up with that one, though I'm still looking into this further. I'm wondering if perhaps there is some cache of files not being cleared, which is why it thinks there are two of them -- or perhaps that Umple is having some difficulty compiling itself due to the change in namespace during the self build. There are a number of possibilities here, so I have to try to narrow them down.

Aside from that, today my work was deviated for a small amount of time to fix bug [#768]. Apparently when I was moving getRelativePath over to the UmpleClassifier and changed the syntax of it, I forgot (in a couple places) to assign the sourceFilename to the classifier. This results in it coming up with the wrong line comments, pulling in the wrong filename.

Thankfully this doesn't seem to be too hard of a fix, as I've already narrowed down the issue (it seems to have come from a jet-file). That said, the fact that this slipped past tests worries me given that that means there aren't any tests that have verified that the // line comments are accurate at all. It's possible that fixing this isolated case won't fix all issues relating to it that could spring up in different instances. I certainly hope this isn't the case, and that this is the only issue.

Mar 3, 2016

I've had a fever since Feb 28 that was getting progressively worse until just yesterday. I went to the nearby clinic out of concern for my health, and was diagnosed with pneumonia and put on antibiotics.

Until today, I've barely left my room or even been on my computer due to lack-of-focus from my fever -- meaning I haven't had much time to put toward the Umple project this week (or any other project for that matter).

I'm hoping to blitz some work in the following couple of days, but I'm not sure I will have a PR ready in time for the next conference call.

Feb 28, 2016

After my last post on the 22nd, I was contacted by Tim both by email and through the issue relating to the refactoring. Apparently the code is there for a cache system to help speed up repeated compilations, but for unknown reasons was set to never happen a few years ago.

At the request of Tim I ran some performance evaluations and found that, for some reason, the cache system actually performs approximately 3 to 4% slower than the non-cached version. Closer analysis suggests this may be due to the amount of string operations being done with the cached versions that is not done in the non-cached. There are a lot of split operations being done, which only get used once per function. This may be invoking the garbage collector due to the number of repeat calls, which would ultimately slow it down.

I hope, at some point, to get the chance to enhance it. With proper work, this could likely be more performant (though i don't know how much more comparative to the non-cached one). If it made use of a binary file instead of a text file, sorted the lines such that they can enhance branch prediction, and cut down on string operations it could increase performance a large amount while decreasing the file size.

Other than that, I've been looking into other parser coupling and came across something that I need to bring up. It appears that a lot of the classes used in the Parser belong to various different namespaces. For example, the ParseResult class belongs to cruise.umple.compiler and not cruise.umple.parser. A lot of these things wind up being used in both the parser and the compiler which makes the decision to migrate it from one namespace to the other a little more difficult.

The other more difficult part of it is that many of the needed classes in cruise.umple.compiler are there for the Parser interface, of which the RuleBasedParser mysteriously is not one of.

From this I can only think of two possible actions.

1. Move necessary classes from cruise.umple.compiler to cruise.umple.parser. This may bring some unnecessary classes that also depend on it though, which could bloat the project.
2. Duplicate some important classes from cruise.umple.compiler to cruise.umple.parser. This is not ideal, but would serve the purpose

Neither of these solutions seem all that great, and so I need to look into other, hopefully better alternatives. All I know for now is that separating the parser from the Umple project is looking to be more work than initially anticipated.

Feb 22, 2016 (Late addition)

I managed to complete my first task a lot faster than anticipated by simply using linux commands. It was mostly a lot of changes in constructing a GrammarAnalyzer/RuleBasedParser/RuleBasedParserThread by changing it from taking an UmpleModel to not. This resulted in easy changes able to be done with grep -rl '<thing to change>' ./ | xargs sed -i 's/<thing to change>/<what to change to>/g'. I was able to change almost everything before the conference call was even over, so that's a plus.

Everything compiled properly, passed Travis and appveyor, so I issued a pull request and it's since been merged.

So after that I started looking into a mysterious hardcoded file of "rules.grammar" that's floating around in the RuleBasedParser. It's really strange, since as far as I can tell it's not even used -- and the only path of execution automatically deletes the file. Here's a small snippet, starting at line 74 of GrammarParsing_Code.ump.

    boolean mustParse = true; <-- This never changes
    try {
      File rulesfile = new File(new File("cruise.umple").getAbsolutePath()+File.separator+"bin"+File.separator+"rules.grammar");

      if(rulesfile.exists()&&rulesfile.canRead())
      {
        reader = new BufferedReader(new FileReader(rulesfile));
      }
      else
      {
        resourceStream = getClass().getResourceAsStream(File.separator+"rules.grammar");
        if(resourceStream == null)
        {
          mustParse = true; <-- This is always true
          reader = null;
        }
        else
        {
          reader = new BufferedReader(new InputStreamReader(resourceStream));  
        }          
      }
      if(mustParse || forceParse) <-- mustParse is always true, this is the only flow of execution
      {
        readGrammarFiles();
        if(rulesfile.exists()&&rulesfile.canWrite()){
          rulesfile.delete(); <-- Delete the file ?
        }
        // ...
      }
      else if(reader!=null) <-- Never reached
      {
        // ...
      }

mustParse exists in only 3 places in this entire method. It exists in the declaration, beginning as true. It changes to true if the resource is not found, and then the if statement checks if it's true.

This means that the entire flow of execution will always hit that first if statement, and never reach anywhere else. Even more curiously, this file gets deleted in any instance that it exists, and everything continues on without ever making use of it. Why does it exist in the first place?

I suspect this is relic from previous code. A git blame shows that these lines were written in late 2013, so it's possible that this entire execution series is entirely deprecated. Removing them entirely seems to cause no changes to any of the tests or functionality of Umple.

Will have to look further into this, and possibly consult with someone to make sure that it's alright to be removed -- but so far as I can tell, this code has no use.

Feb 22, 2016

Over the week I was reworking my previous pull-request to satisfy all participants in the discussion. It took a little work due to some NullPointerExceptions that occurred regarding state machines (mentioned on Feb 17), but that was able to be taken care of by a fix suggested by Vahdat in his fix_3 branch.

After successfully changing the syntax from <position>.getRelativePath( <class/trait>, lang ) to <classifier>.getRelativePath( lang ), I was ready for a pull request. Or more accurately, I would have been -- except I had to rebase onto the new master. This had strange consequences in that I began failing tests that my code never altered (part of the 'ECore' failing tests). Thankfully this was a known-issue, and when it was fixed the next day I was able to issue my pull request without any failing builds on Travis.

Right now I am working on 2 things.

1. Remove UmpleModel from RuleBasedParser

As far as I can tell, the UmpleModel is only used for a call in getModel().setAnalyzer( getAnalyzer().getAnalyzerMap() ) -- which could be done from outside of the RuleBasedParser. Stranger yet, this model gets passed to multiple other classes seemingly only to be passed back to this. The GrammarAnalyzer requires the UmpleModel so it can pass it to an RuleBasedParserThread that uses the model to construct a RuleBasedParser.

At least, this is how it appears at a first glance. I'm hoping I'm right about this, since it would be as simple as removing references to the model and extracting the getModel().setAnalyzer(..) line to somewhere else.

2. Documenting umple_code.ump

(This one is less of a priority, and I primarily do it wherever I find the time.)

There are a lot of functions in umple_code.ump, very few of which actually are documented.

In between my other work this week, primarily while waiting for responses on my Pull Request, I have been writing up javadoc information for each function in that file. It's not a particularly easy task, given that some of the method names don't describe exactly what they are doing -- for example, getAllTranslators() doesn't just simply do a get, it actually calls newGenerator for any presently unmapped language generators. Behavior that doesn't match the method name should be better documented to make it easier to understand for future maintainers.

A lot of this work has been trying to figure out what the methods are actually doing vs what the name of the method is. Most of them are adequately named, but just have certain behavioral things that should be noted -- such as a method returning silently instead of throwing an exception.

I will be issuing a pull request with all the documentation once I have documented all methods in the umple_code.ump file -- but this may take a while, since it's not my top priority.

Feb 17, 2016

I keep forgetting to update this the past couple of days. This week I have been trying to work on my refactoring project of Umple, however I am not getting very far. My initial pull request was rejected for using static methods, and after I made some modifications and inquired further about a proper change, I didn't hear back for 3 days due to the long weekend.

My task was simple: remove the tight coupling between the Position class and UmpleClass/UmpleTrait. The getRelativePath method in Position takes one of the latter two classes and returns a string representing the relative path between the position of the class/trait and the current position token. This is logically an easy fix, as a Position token really has no reason to have knowledge about an UmpleClass or UmpleTrait, and so the method shouldn't exist in that class.

My original suggestion was to extract that method into a ParserUtil class using static methods. This would change the call from positon.getRelativePath( umpleclass, "Java") to ParserUtil.getRelativePath( umpleClass, position, "Java" );. I chose this notation as it acts more similarly to a C# extension method, wherein a method can be defined from outside the class statically, but is treated as a method that is part of the class. This doesn't violate OO principles since encapsulation is still kept, and methods are still treated as existing as part of the class (and thus are not functionally free functions, even though they are under-the-hood). I guess this is not considered the case in Java.

This was rejected due to it being a code-smell (use of public static functions), and also since I could modify the method to accept all descendents of UmpleClassifier instead of UmpleClass/UmpleTrait. I was also suggested to use a mixin class instead of using a static class.

Since then I have fixed the former and have been aiming toward accomplishing the latter, but am now met with the problem that state machines will often produce null UmpleClassifiers. This, of course, leads to <classifier>.getRelativePosition(...) causing a NullPointerException anywhere that the state machine is tested, resulting in java tests failing, and the compileJava build step to fail as well.

The original implementation of getRelativePath(...) was designed to accept null UmpleClassifier values, and in the case that it was null it would instead return the full path to the position token's file.
As a result, it's not trivial to fix this issue in the state machines. It would result in a series of nested if statements that look something like the following:

String filename = p.getFilename();
String path = null;
if( sm.getUmpleClass() == null ){
  if( filename == null ){
    path = "";
  }else{
    path = Paths.get(filename).getFileName().toString();
  }
}else{
  path = sm.getUmpleClass().getRelativePath( ... );
}

This leaves an awful lot of work on the caller rather than the callee, and so it doesn't feel like a clean solution to me.

One of my alternative suggestions was to wrap the dependencies in another class, say UmplePath that accepts as parameters the required arguments for the functions. This would result in an OO-proper calling notation like this new UmplePath( uClass, position, "Java" ).getRelativePath();. It would be easier to keep the null-checks in the method, and the class could even be used for alternative purposes, such as finding the relative difference between two UmpleFiles. This suggestion wasn't even acknowledged, so I'm not entirely sure where this stands.

I regret that we can't come to a more agreed upon solution to the problem, as it's debilitating my progress and jeopardizing my ability to finish my project. I hope to see some resolution in the future, as I'd prefer a lot more progress to show for my efforts.

Feb 7, 2016

I've been looking into the RuleBasedParser for a little bit now to try to understand how it works so that I can easily extract it later on, and I've noticed a few other things that also need to be done.

• UmpleFile is often passed to RuleBasedParser or related classes. This class seems to be an adapter around what a java File already does, with the one exceptional case that it contains a list of strings that amount to a bunch of Umple's use <file>.ump statements. This can probably be refactored out entirely, or alternatively be made to extend Java's File class so that it can be interoperable with regular files. If it extends File, then most areas that use UmpleFile in the parser can be changed to just File to reduce the coupling. It would also be a good transition to completely removing it from the system
• The RuleBasedParser doesn't just return a parse tree, it performs a grammar analysis in the same stage. I'm not sure if I'm confusing this with the same analyzer that I will be working with after this project or not, but if it is the same one then that will complicate extracting the parser. This Analyzer also hard-codes a few of the grammar files in Umple, which I need to change to use injected strings so that it can be more generic.
• this line. Enough said.
• A bunch of the parse-tree classes like Position often have calls to UmpleClass or UmpleTrait. This one may be tricky to get around. Some of these are utility methods like getRelativePath -- and so they can likely be extracted into a helper/utility class instead.

The coupling with traits/classes shouldn't be too hard to fix from what I've encountered so far, I'm more concerned with the deep-tie between parser and GrammarAnalyzer. I am hoping this won't be in need of a complex refactoring, as I don't want to hurt any performance.

Feb 6, 2016

I finally know what I'll be doing for the rest of the semester (or rather, I finalized it on Wednesday). To help prepare for the eventual shift to a plugin architecture, I will be doing generic refactoring around the Umple codebase to help clean up unnecessary coupling, among other things.

As part of this, I will be extracting the RuleBasedParser, and all the related files, into a self-contained module either in the root of the project or perhaps in a new Github project. In the case of the latter, it will require modifying the Ant build to bring in the source at build-time.

I will also be working on extracting the Analyzer class into more self-contained classes -- a project that has already been started by another student in the past, though not completed. This may take a little time to try to find out what he was doing initially.

For the time being I've been pouring over the codebase and looking for ways to reduce coupling for the RuleBasedParser. It's quite interesting since I have almost no idea why it is so tightly tied to the the UmpleModel. It looks like it accepts an UmpleModel strictly to used model.setAnalyzer( getAnalyzer() ) later on -- which could be done outside the class entirely. Other than that, there are a few strange locations that pass on the UmpleModel to other classes in the parser, such as the RuleBasedParserThread, which only seems to use it to construct yet another class, etc, etc.

As far as I can tell, the model serves next to no purpose. I hope to have this all removed by mid-week if possible.
After that I will also need to look into extracting the error messaging system. All error messages are currently going through a central class in the core, and so I need to find a way to attempt to reduce this coupling as well.

I'm a little more concerned about the actual parse tree produced however. It looks like the tree contains a lot more than just tokens; but rather a lot of Position elements and Comments, which then reference a lot of classes like UmpleClass, UmpleTrait, etc. This is making me think this may be a larger undertaking than I initially suspected.

Jan 31, 2016

I've had quite a few challenges posed this week. What started as a suggestion for a plugin system has resulted in a debate among previous contributors, with suggestions that lead in various directions.

I was originally expecting to be researching into topics brought up in issue [#706] (originally discussed in email), but now have multiple different directions to look into. Andrew recommends splitting code into different repos that then get pulled together at build time instead of a plugin architecture. Tim suggests starting by working on refactoring the Analyzer into different classes and files in order to prepare for a shift to a plugin-environment. Vahdat recommends a similar action of refactoring the core component of Umple into different layers for better preparation, and to start with a system more feasibly ported to a plugin like the Analyzer (which is more consistent across versions).

All are great suggestions, but are somewhat deviating in different directions, which is a little overwhelming.

Before this conversation took place I was working on a Java project that simply implemented a plugin architecture and loaded them at runtime; just something I could use as a starting-guide for integrating into Umple (though this will have to be put on the side for now).

I didn't get it fully working, but it was in the form of:

Plugin

Where Plugin is the interface that all Plugins would extend from. It gives the functions for loading/unloading plugins such as onEnable(), onDisable(), onLoad(), and onUnload(). It also requires that features for permissions, configuration, and description data be fulfilled in the form of getPermissions(), getPluginConfig(), and getPluginDescription().

Umple Plugin

UmplePlugin is an abstract class that fulfills this interface by providing a concrete definition. The class is abstract so that it cannot be directly instantiated. Any plugin designed would extend from this plugin.

Plugin Description

PluginDescription is a small class used to contain descriptive information about a Plugin. It contains details like the name, version number, a list of dependencies, a list of requested permissions, and -- most importantly -- the plugin entry-point. The latter of which is necessary to actually instantiate the jar, because otherwise it would be impossible to know which class extends UmplePlugin.

Plugin Configuration

PluginConfig a generic method for handling configuration for a plugin. I haven't fully fleshed this out, but a standardized pattern for local configuration files would be helpful -- and this would be the main method for providing access to it.

Plugin Permissions

PluginPermissions is a small class that contains a collection of requested Java permissions that will be passed to the Umple core. These permissions are in the form of Java's security package, and may be modified from a generate configuration file.

Plugin Loader

PluginLoader is the generic interface that loads the plugin file itself. It requires that the following methods be defined:

• Plugin loadPlugin( File file )
• void enablePlugin( Plugin p )
• void disablePlugin( Plugin p )

Umple Plugin Loader

UmplePluginLoader extends the PluginLoader. The method loadPlugin( File file ) will, given a File, open up the resource of the jar and look for a plugin manifest (perhaps something like plugin.xml). It will first load this file to produce a PluginDescription. If successful, it would use this description to instantiate an instance of Plugin from the jar.

Not seen in this example

UmplePlugin would, when instantiated, somehow gain an instance of the current UmpleModel. How I can manage this without requiring it explicitly passed during its construction is making this hard though. I'm considering UmplePlugin to have a setModel(...) method that is called after successful instantiation of a plugin, which would also mean that UmplePlugin would contain an instance of UmpleModel. This would give any UmplePlugin support for retrieving the current model.

Listeners could also solve the above issue of passing the model, as it would be possible for listeners that require the model to pass it as a parameter. For example, onGenerate( UmpleModel model, ... ) would pass the instance of model to a method onGenerate that would be written by a Listener class. This would likely be the preferred method, and it also guarantees that the UmpleModel would be fully instantiated before being passed to the Plugin

Listener would be a generic interface for listeners. The UmpleModel would contain a list of Listeners that it stores for various purposes. Listeners can be added through a method registerListener( Listener ), where the signature takes a Listener for a specific purpose.

All descendants of the Listener will be either abstract or interface classes, as they will give a concrete requirement for implementation. This could be simple like a CommandListener which operates at runtime, requiring onCommand( String command, ... ) to be satisfied -- and would register a new command-line option to the Umple Main.

The Listener classes would be the primary hook, and could be added just about anywhere throughout Umple for future integration.

That's the main part of what I was working on this week. I've also looked into other things mentioned by Tim (outlined in the issues page), but I haven't found solutions to all problems.

• Dependency still stands as a pretty strong issue, of which my primary suggestion is to keep the majority of Umple within the core. Instead, hooks will be made available that override features with plugins -- instead of making the core component, itself, a plugin. This would also solve the distribution problem, allowing Umple to be distributed in a single jar
• Plugin dependencies can be managed with Java using apache ivy. I wasn't aware that this existed until I was told by Kevin Brightwell (major thanks to him for that one)
• If plugins are still made internal to the Umple core, testing wouldn't be changed.
• I haven't been able to look too far into performance yet, since I still don't have a working test model
• I've been thinking about what I want as a doable proof-of-concept implementing this framework. My current thought is to produce a simple logging system that is capable of logging detailed information at different steps of the compilation process, all done using Listener hooks. This could be configured with a configuration file for the plugin, allowing different levels of print verbosity. This would be a good way of demonstrating the capabilities of the Plugin framework, and is reasonable enough to produce a working prototype within the given time constraints. My only concern is the concept sounds almost underwhelming.

All that said: I'm not 100% sure where to go from here. I've received a few different views that could have me starting in completely different directions, making it hard to know where to begin.

Jan 24, 2016

Not a huge update this week. After the code sprint I have had to focus on a couple school assignments, a job-fair, and preparing for an upcoming interview. Overall, this has been a hectic-week as a result.

In between all that work, I have mostly been looking into a replacement issue I can work on while the Jet Java generation issue is resolved. The issues that keep interesting me most are either already assigned, java-generation (which requires jet), or seem particularly easy. I might just take some of the smaller very-easy tasks like the wiki cleanup/update/fixes just to power-through a bunch until I can return to my originally assigned one.

Another Umple-project I've been looking into is the feasibility of a plugin-framework (which I still need request permission to work on). One problem that I am worried about occurring is the same one that occurred when I was trying to solve the issue relating to the case-sensitive names. When I was attempting to read class files from within the jar, this would only work when run from command-line, but junit testing failed -- a problem I suspect is due to a Java security issue.

I need to run some tests to see whether or not a plugin system would fail once it reaches Junit, because if it does then this whole idea may have to be approached differently. It is possible that Junit may disallow the jar that it is testing from itself executing another external jar not found in the classpath, and this would likely be a permission-related issue if it were the case.

Jan 17, 2016: Code Sprint Day 3

Started the day with the pull-request being accepted. Began looking into Issue [#295] relating to Java generation. I spent most of the time setting up the environment, and installing the Jet plugin for Eclipse. The first install on Eclipse Mars didn't work properly -- resulting in it showing as installed, but not giving me context options in the project preferences. As a result I had to download an older version of Eclipse (Luna), which I managed to get working.

It took a little bit for me to get used to working with the generation stuff, so mostly it was reading over the wiki and getting some help from Morgan who had already begun to use Jet. After a little while it became apparent that there was a strange Java generation issue that could not be avoided. The generated jet code on GitHub did not match the code actually generated using Jet, likely resulting from a previous user either making changes to the generated code directly, or not testing the Jet code. Since the full build only copies the jet-generated code to the appropriate directories, it wasn't discovered until now.

I spent a little time looking into this issue, though Morgan was doing a lot more on it than I was. As a result of this though, my current assigned issue will have to be put on hold. Until then, I will likely just assign myself another issue.

Overall

I really enjoyed this whole Code Sprint. It was nice to actually meet my teammates in person, and speak with them 1-on-1 without any feedback, network issues, etc.. Though I didn't write as much code as I originally had hoped to do over this weekend, I did get the chance to become acquainted with a lot of the umple core model, and a little bit of the parser and generators. I have a far better understanding now than I did before of its overall structure, which should certainly aid in future work. I also learned important things like issuing pull-requests through github, and updating the remotes in a git repository through command-line (Thanks to Victoria for that one!).

Moving Forward

Vahdat mentioned that we would be allowed to develop a new feature/add onto existing features in an area we are comfortable with. This has got me excited and thinking about possible tasks.

One thing that I have considered is that Umple's current structure lends itself quite nicely to a pluggable interface, allowing plugins to provide additional functionality outside of the original scope. My thought is that it could be used as a new way to provide generators to the core compiler without requiring as much interdependency (For example, Umple_Code.ump has methods that pertain to translating to comments of all possible languages; a functionality that may be better encapsulated in a module). If this transition were done properly, it would require additional build steps to construct individual jars for each plugin at build time -- but may also decrease time for quickbuild options for people attempting to construct their own generators (since only the jar would have to be constructed).

My current thought for a general interface is the following:

With a structure like this, the compiler can poll for all plugins at runtime -- loading each jar file. It would search the root of the jar for a manifest file that specifies the entrypoint for the jar (e.g. the class that extends UmplePlugin) and attempt to construct it by passing it the UmpleModel (necessary for parsing .ump files). If successful, the UmplePlugin would automatically load a configuration file (also in the jar) specifying key details (the name of the plugin, version, the type of plugin functionality it provides, etc) to set it up.
All that's left is to attempt to cast it to the expected plugin type specified in the configuration. If the cast is successful, then it can be stored in a master list that will have pre-set plugin hooks called at specific areas throughout the system.

Policies are also able to be manually assigned (using Java 7+), so that executing external jars are sandboxed within a set of rules.

By extending it this way, it would allow for other organizations that may wish to use Umple to add their own features, perhaps with support for UmpleOnline, without having to clone the repo and work with intricate system knowledge; instead they can work straight from a Java jar API (generated with ant) to make development faster.

This idea would not strictly be limited to generators; my thought is to develop a generic API that only requires UmplePlugin to be extended in order to develop new functionality. Perhaps adding new hooks at various stages of the build process, using an event system (or possibly Java traits?) to pass messages to the various hooks.

Though this idea is a very large-scale idea for such a short time, my thought is to start small; build a (working) plugin API, and attempt to migrate one of the less-used generators to the new structure as a proof-of-concept.

Of course, this is just an idea. I haven't posed this to Tim yet, so I need to get around to that during the next hangouts call.

Jan 16, 2016: Code Sprint Day 2

JUnit tests continued to fail using the method I wrote yesterday, in which I traverse Umple.jar's structure, searching for all class files.
As a result I explored a different approach and wrote a small resource parser in 10 minutes that was capable of extracting the generate token. This was capable of satisfying Issue [#602] in the JUnit tests, and so I made a pull request before noon.

I later assigned myself Issue [#592], and began working on a solution. The fix is simple enough; change the comments from =begin =end block-style syntax to # inline (common ruby) syntax. I constructed a small StringBuilder in Umple_Code.ump and append # at the start of every new line (indented for the internal comments).

A small annoyance kept occurring where the comment text was 2 characters after the comment symbol, which took a little while to trace. I had to find the Generator code, which I then followed to Jet to try to change before realizing that the code made a call to Comment.format(...) in UmpleCode which forced a 2 space delimiter.

All there is to work on is the test cases, which immediately failed since they test against the previous commenting-format.
One thing I noticed is the previous author of the code forced test-cases that could not compile correctly if ran. All test cases compared against the =begin and =end delimiters as indented by 2 spaces, which cannot be run by ruby (it must be at the start of the line for block-syntax to work).

I have a lot of test-cases to correct with the new comments, and then I should be good to issue a pull request. My target is to have the pull-request issued by tonight.

Update Issued a pull request after a successful build. It took a little while to properly correct the test-cases to get the formatting accurate.
Once this pull request is accepted, I'd like to make a small change to the main -h/--help instruction and auto-generate languages based on my revision to Issue [#602].

Jan 15, 2016: Code Sprint Day 1

Today I spent hours working on solving Issue [#602]. The solution would have been a lot simpler had I started looking in a better area first.

The majority of the time was spent attempting to extract tokens from the RuleBasedParser. I first began by trying to grab the parser and read a list of generates that were read from umple_core.grammar; however this proved to not be useful, as the parser is constructed after the generator is selected with newGenerator().

I then attempted to construct a second parser that was used strictly to acquire the appropriate generation languages, and this led to other problems. It appears as though Umple can only have one RuleBasedParser at a time, otherwise it will cause other build steps to fail.

As a result of this, I had to look into other approaches. I spoke to Vahdat for help regarding this, and he suggested 2 possibilities: build a custom parser to read the .grammar file, or attempt to read all .class files and compare against that. Given the way newGenerator() works by substituting the language string into {0}.Generator.class, I found that to be the most effective way of solving the problem.

A little while later I had a working solution, which walked the files existing in the .jar file, extracting only ones that were in cruise/umple/compiler/ and ending in Generator.class. Finally, a solution that works. Mostly.

Somewhere down the line, this change caused a few JUnit tests to crash (not just fail). This problem took a lot of searching to discover where it was coming from. It turns out, if you use System.exit() early on a JUnit test expecting a result, it treats it as an unhandled crash.

Last on the list is writing a few test cases to ensure that this is working properly. I intend to have this done tomorrow morning, and from there I can issue my first pull request.
The current test cases I have written seem to be failing, as the runtime-generated list of valid strings doesn't appear to populate if run through JUnit (go figure). I need to figure out how to successfully make a call while still making the test meaningful.

Ultimately, this problem has taken a lot longer than I was originally anticipating. I spent far too long looking in the wrong direction for a solution, and I am thankful for Vahdat's suggestion.

Jan 14, 2016

I checked in at the Chelsea Hotel around 11pm in preparation for the Code Sprint this weekend (sadly I couldn't make the Meet & Greet at U of T).

While waiting around in my hotel room, I decided to quickly look into the JUnit issues that are happening with Umple when compiling on Windows (as discussed with Vahdat during our last video conference).
The issue of cruise.umple.UmpleConsoleMainTest failing was simple, a few JUnit test cases relied on the Unix-style line endings \n instead of the system-specific System.getProperty("line.separator"). Changing that solved the issue quickly.

The second issue of cruise.umple.compiler.UmpleImportTest seems a little more complicated, as the data is being read from two different files. Glancing over the test files doesn't show any glaring issues, and so I will need to look more in-depth into this in the future.

Side note: Compiling Umple on Windows is killer. It takes around 12 minutes, as opposed to ~3 minutes on Linux.

On the topic of Issue [#602]: I need to look further into Umple to provide a decent fix for the issue. It's simple in nature, and has many naive solutions, but I want to provide one that is generic and scalable if ever future languages are added. My current attention is on the languages supplied in umple_core.grammar, which is made accessible by iterating through model.getGenerates() -- only model doesn't seem to be initialized until a while after main.

Jan 12, 2016

I spent a couple hours searching through the Umple source code to get a feel for it, and to understand how compiler parameters are parsed from command line. I did quite a lot of greping through the different files to follow the flow of execution relating to parameter handling and code generation, which I need for Issue [#602].

I discovered that all (command line) compiler parameters use an Optional object from a library that simulates POSIX's getopts() functionality in an object-oriented fashion. I also found that parameters to the argument -g or --generate all are sent to an object, <[inline_block>3](https://github.com/umple/umple/blob/master/cruise.umple/src/Umple.ump#999, that is used as a means to hold the target language. This object is used in the method newGenerator(...) by providing the language string to it, which it then uses to to construct the appropriate compiler at runtime using the fully-qualified path.
This appears to be why all the parameters to the generate argument require a specific case, because otherwise it will not refer to the appropriate object due to case-sensitivity.

Since GenerateTarget seems to be a common denominator for selecting the language, I have suggested the possibility of making the modification in this class instead of in the main. This could (if I understand it properly) correct the case-problem in a generic manner that will support any manner of entry point, not just the console compiler (since UmplePlayground also makes a call to GenerateTarget). Given my still limited knowledge of Umple's code-base, this could easily not be the case and so I am waiting for a response back before pursuing it officially.

A small modification made to Umple.ump adding before constructor to GenerateTarget did appear to solve the error in testing, however it caused an unspecified failure in a JUnit test (Something I need to look into).

Jan 10, 2016

I have been struggling since late January 8th to properly get Umple compiling on my Linux server running OpenSuse 13.2. Every time I would try, I would encounter one problem after another. Initially it was ant configuration issues, such as improperly set JAVA_HOME, ANT_HOME, and even ANT_OPTS variables (the latter of which resulted in various out-of-memory exceptions from Java since I was only allocating 256M). At one point, I needed to created a symbolic link pointing from /etc/lib/java-1.8.0 to /etc/lib/java (since the original /etc/lib/java-1.8.0 was, strangely, created as an empty directory).
Some of the required libraries weren't brought in with Ant as dependencies, which required me to either find a working update site for zypper, or find a working rpm.

After various package installations, updates, and configuration I finally got compilation working... almost. For some reason, Umple would compile properly (and even be runnable, generating correct code), yet somehow it would always fail all junit tests immediately. Strangely, Umple worked better on my Windows computer than on OpenSuse. Searching online yielded various known complications with some versions of Java and xerces-j2 which result in warnings and exceptions being thrown in junit tests, which ultimately result in all tests failing. The primary fixes were to reconfigure the project itself, which I am not (yet) equipped to do, or to remove xerces from the classpath, which I couldn't do without generating other errors.

For this reason, I decided to follow advice from previous UCOSP participants and migrate everything to Ubuntu. I'm not particularly partial on Ubuntu given the way they manage files (which differs from more-common Unix directory structures), but I figured having a better-maintained distro with a larger support community would be beneficial in the long-run.

It took a little getting used to, especially following symbol-link chains that Ubuntu loves to do (for example, /usr/bin/java -> /etc/alternatives/java -> /usr/lib/jvm/java-8-oracle/jre/bin/java instead of simply /usr/lib/java -> /usr/bin/java-1.8 like other distros), but I quickly managed to set it up.

Running the ant build commands resulted in a quick and easy build without any errors or failures, so that's already a major step ahead from using OpenSuse. It's a shame the fix was installing a whole different OS, but I figured it was about time I upgraded anyway.

Jan 8, 2016

• Successfully compiled Umple from command line in Windows without any hassle. Used a MSYS2 console to compile, which may have simplified the process since it includes gnu utilities.
• Successfully tested the compiler from command-line with example Umple programs. Commands seem pretty basic and simple to use.
• Ran into an issue running the umpleonline server, but not sure why. The command php -S localhost:8001 runs a server successfully, but accessing the page just throws an error. I think it isn't able to find an index.php file and as a result isn't running properly. Need to look into this further.
• Read through some of the UCOSP logs for insight/pitfalls to learn from.
• Took a suggestion from previous UCOSP participant Tyler McConnel and began reading Test-Driven Development: By Example by Kent Beck. Still not sure how helpful it will be, but it can't hurt to read.

Jan 6, 2016

• Read over a few Umple tutorials
• Played around with Umple online for a bit to get a better feel for the language
• Cloned Umple using git on my Windows computer.
• Will attempt to do a build tomorrow
• Ideally I would like to work on this on my Linux server, but I am waiting for a part I ordered before I can use it.
• Read up on (and played around with) Travis CI. I've never personally used any Continuous Integration software, as I haven't had much need for it -- but it's something I have been meaning to learn.
• Might consider adopting this for some of my larger personal projects given that it is free to integrate and supports compiling with multiple c++ compilers (g++ and clang).
• Contacted UCOSP mentors about rationale behind some C++ code generation tactics
• Learned that Umple generates a macro constant labeled _{namespace(s)}_BEGIN and _{namespace(s)}_END for scoping namespaces. On a single-deep namespace, this is senseless, but on multi-layers it expands to multiple namespaces (which ultimately saves on code bloat while still remaining readable)
• I brought up possible symbol-collisions from the macro constants since it conflicts with C++ standard (symbols beginning with an underscore followed by a capital letter are reserved for use by the compiler). This suggestion will apparently be resolved in the next delivery

Overall, I'm quite excited to work on this project.