*** Mistakenly posted on help forum, should be here ***
Dear JPivot Community Colleagues:
About this same time last year, I made my first post on JPivot and Mondrian forums, to gage interest in developing an XMLA interface for JPivot so it can report off MSAS. Since then, I've gotten to know a lot of the active participants in both communities including the project founders (all nice people :-), and I'm also very happy to say that our company completely replaced its ASP.NET based OLAP reporting application with a JPivot-based application proving more efficient in every aspect -- response time, UI aesthetics, manageability, scalability, etc. etc.
Our most pleasant surprise was how well the open source development model has worked for our R&D efforts. Developing software via community input and support has been extremely beneficial. To continue that trend of community-based development, we will soon release this application in the open source space under the name of OpenI (I will make a separate forum post with details on that). However, this brings up a key challenge that begs the attention of the JPivot -- How do we make it easier for contributors to contribute code on the JPivot part of the application?
In our experience, the current process of submitting code by email while having no visibility to the CVS repository makes it hard to work with. This IMHO discourages the community from actively participating in the JPivot project. I also understand that the sponsoring company behind JPivot (Tonbeller) has a commercial BI product of which JPivot is one of the components, and there isn't a clear separation of the CVS repository and the build scripts between the commercial product and the JPivot piece of it. This has resulted in the current practice of publishing periodic sourcecode snapshots on JPivot site, and requiring all code submissions come in the form of modified snapshots via emails -- which doesn't address the root problem of community participation.
To address this, I propose that there should be an independent CVS repository for JPivot. Ideally it would be under this same site, but if this is difficult for Tonbeller to do so, perhaps the community should put up the latest JPivot snapshot in a different sourceforge project CVS respository -- maybe something like a Jpivot-plus or Jpivot-X, so that it is easier for the community to post their contributions. That way, Tonbeller can continue publishing JPivot source snapshots, and the committers community will merge Tonbeller changes with the independent CVS repository (JPivot-X). And each member of the community can decide whether they want the latest Tonbeller snapshot of JPivot, or the one that has outside community contributions as well (Jpivot-X).
Again, the goal here is foster a greater level of community participation, and not to "fork" the codebase. Overall, the community (including Tonbeller) should benefit from the improvements on JPivot, and these imporvements should really come from the overall community including those people outside of Tonbeller.
Of course, it is hard to say how big of a committers community JPivot will have if we pursue the path of JPivot-X. And I don't know if we can count on the support of the original JPivot authors for this path because their management could completely disagree with this approach because of potential conflicts with their commercial product. However, a good number of you who I have talked to in the past few months have expressed great interest in this concept, and that's why I'm posting this here to get some more feedback. I sincerely believe that an open source project thrives when there is participation from a community wider than the original authors -- not just in terms of code, documentation, testing, etc., but also in terms of shaping its roadmap and raising the overall product quality.
By opening up the CVS, I strongly believe that JPivot will see a much increased participation from the community. Our company for one will want to be a major contributor and also a major factor in spreading the word around on JPivot. We really believe it has great potential to become a really important project for open source business intelligence, and we want to collaborate with the rest to make that happen.
I hope you don't read me wrong here. I have a lot of respect for what Tonbeller and the original authors have done with JPivot. It is a groundbreaking piece of work, and my company wouldn't have its current product in production had it not been for JPivot. For that, I am very thankful to the JPivot team, and have a lot of respect for you all. I just think it will be a shame if JPivot doesn't get the community participation it deserves just because we as a community couldn't figure out how to open up the CVS repository.
I'd love to hear your thoughts and comments on this approach.
Loyalty Matrix, Inc.
I've been following JPivot for some time, and I agree that an open CVS repository is an important step in moving JPivot forward. JPivot is a very useful project, but development does seem a bit sporatic right now. I whole heartedly support the idea of establishing an open repository for JPivot.
I'm one of the contributers on the JasperReports project, and think that it might be interesting to look into integrating the two projects. I can easily imagine a use case where JPivot is used for analysis, and when you drill down to the raw relational values or need highly formatted reports you jump into JasperReports. Let me think about what it would take on the JR side of things.
We have been talking with Sandeep about this, and are willing to contribute a JBoss Portal implementation of JPivot, continued integration with Mondrian and ongoing support of the community.
I have talked with Sandeep and Sherman about this issue. We are using JPivot in the Pentaho BI Platform that we are building (http://www.pentaho.org). We also have enhancements for JPivot that we would like to share.
Will are offering to provide:
* Hosting of JPivot snaphot(s)
* CVS access
* Issue and feature tracking in JIRA
* Discussion forums
* Project managment and product management
* Hosting of web conferences for design and direction
* Transparent roadmaps
* Administration and development with no commercial conflict of interest
* No commercial software versions or commercial upgrade paths
good news: I have management approvement to open up the CVS at sourceforge.
This is a step in the direction of a more open JPivot project. We know that the lack of an open CVS was discouraging contributors. We hesitated to open it up because our build environment is quite complex and if JPivot moves elsewhere, we have to change it. Ok, on public demand we will do so now ;-)
It may take a few days until the CVS is up because I want to decouple the jpivot and the wcf packages, include the wcf tests etc. This is difficult when everything is checked in.
If you want write access to CVS, please tell me. But I'd like to review contributions before checkin to make sure that they fit into the JPivot architecture (at least the first few contributions).
This is great news. One of the best news I've heard in a long time ;-) I'm sure all the rest of us who have been using JPivot in our product and services are equally happy to hear this as well. So -- thanks and congratulations!
We will keenly look forward to the CVS repository opening up on this sourceforge site, and Paul Lucas from our team will definitely be one who'd like to have write access for his XMLA submissions and other fixes/ehnacements. I'm sure Sherman, Bill Parker's team, et al would want write access as well for their contributions.
Once we're there, we should start some community discussion re roadmap and outline tasks. With the development proecss being more open, I'm sure that there will be a good deal of community participation. And that should also help your management rationalize the decision to go open source -- once the community contributions to JPivot start helping improve your commercial product line as well.
Anyway -- I'm happy. You guys made my day. I'll definitely buy you and Kurt a beer if we get to meet some day.
This is excellent news. I'm delighted that we managed to work this out.
One exciting prospect is the re-use JPivot's model component. JPivot contains classes for building MDX queries, formatting the results into a pivot table, et cetera. If another project were to build an alternative user interface -- say 'SPivot', a hypothetical Swing-based pivot table -- then SPivot could and should make use of JPivot's model component.
SPivot would inherit all of the good stuff from JPivot's model classes, including the ability to run on top of a generic XMLA backend. Now that JPivot's repository is public, this is a much more realistic prospect.
A more immediate matter is that Mondrian's 1.2 release is coming out shortly. If changes are required to JPivot to support the new Mondrian release (we tried not to make incompatible changes to Mondrian, so it's not likely), and JPivot has not been migrated to the public CVS at that point, then we'll need to find another way to propagate these changes into JPivot. I'm sure I can work something out with Andreas.
Once again, thanks to everyone for working out a solution to this problem. It will help to make Mondrian, JPivot and open source BI stronger.
I am definitely interested in doing an SPivot (Swing and/or SWT). I was really hoping that the person who had done something like this already would have contacted me so I could get their code.
I'd prefer the SWT version. I think its simpler to develop because you can reuse all of the eclipse infrastructure. Also it could be used in different scenarios, one of course being an RCP application for end-users.
I currently develop Mondrian schemas using the XML editor from the eclipse webtools project which has built-in schema support. So it "knows" the structure of Mondrian schema files and gives quick fixes, content assist, syntax highlight etc. To view the database I use a plugin like quantum. If I could immediately try out my schema and run queries in the eclipse IDE this would be a very productive environment to develop JPivot (swt based and web based) applications (queries, schemas etc).
Andreas & Sandeep,
This is probably the best of all possible results. While an open CVS is essential if Jpivot is to get more active participation and get even better the reality of keeping two CVS up to date was a danger of drifting into two projects. Jpivot is an excellent product now because of the rigors of production use in the Tonbeller product environment, to lose this would have been a big negative, but now we have the best of all worlds. The only missing piece is perhaps a bit of co-ordination / roadmap management so we arent working on different versions of the same great enhancement while other bits and neglected.
As some of you know we (Meridian Informatics) made some enhancements to the Jpivot interface back in late 2003 / early 2004 but did not get around to getting them ready for submission (some of you have seen our live demo and movies). With the release of Jpivot 1.3 with XMLA support we got excited again and have almost finished reapplying the enhancements to the latest version and have submitted some of these and will be submitting the rest in the next week or so.
Our enhancements include
1. An enhanced drill-thru report which will do multi-column sorting and grouping an makes a more focused report by eliminating (hiding) redundant columns (ie all the row / column hierarchies and filters for which all rows of the report will be the same)
2. Access to some navigation functions and addition of some new functions (eg expand / collapse all members of a level, simultaneous drill down on both row & column, ..) via mouse right-click in dimension / member headers and cells.
3. Drag & drop functionality in the pivot chart for adding / removing dimensions to rows / columns
4. Some other bits, a bookmark interface, enhancements to bookmark functionality,
We are testing these now and adding cross browser support (at least IE & Firefox)
When we are finished we will put a public live demo on our site for people to play with (and hopefully comment on)
Andreas it is not unreasonable for you as project co-ordinator to publish some submission guidelines, it is in no ones interest to have the CVS filled with contributions that are inconsistent etc, perhaps you should discuss guidelines with some of the larger current contributors covering desirable levels of test cases to accompany submissions and suitable standard project test cases and some basic level of documentation to explain (technical & business) the contribution.
Good news all round
Meridian Informatics (www.meridianinfo.com)
All this sounds very promising. I'm happy that we could turn it this way, looks like JPivot is on its way to a truly open source project.
Currently at Tonbeller we are exploring the best way to synchronize the sourceforge CVS repository with our own. When we have set up a mechanism for this, I will check in the code.
I'm looking forward to see new features in JPivot, for example what Bill announced sounds great. Significant progress after opening the CVS also will prove that this is the right way to go.
Except for new features I see the following areas where contributions would be very desired:
Documentation + Examples: JPivot could be much simpler to use if there was some documentation, examples, tips + tricks, tutorial, howto, wiki, etc. Also a nicer looking demo thats easier to install would help. JPivot deserves better than index.jsp and testpage.jsp ;-)
More Tests: When more developers are working with the code, changes that break existing behaviour become more likely. We use JPivot in several production environments and we must make sure that the code is correct, performant and the API stays compatible with previous versions. I will upload our tests for the WCF packages (GUI classes, independent of JPivot). But there are only few tests for JPivot itself currently. I maybe can provide a few example tests using httpunit, but these will not cover a significant amount of the code.
Everything else: I'd like to focus on developer support and code reviews the next time to make sure everything fits together and production quality is maintained. Since I wrote most of JPivot, I think I can do that best. So help with everything else would be greatly appreciated. Someone volunteers for release management?
I can help you with some release eng. Also, with regards to a tutorial, I can write up some step by step jpivot+xmla setup, tips, tricks.
I'd like to see some tests for the model component of JPivot. In principle, the same tests could be run against the two implementations: the Mondrian implementation talking to Mondrian (obviously) and the XMLA implementation talking to MSAS via XMLA.
I want to promote the model component as a separate module which can be used to build other front-ends.
If this is successful, JPivot will have sister projects SPivot (built on Swing), SwtPivot (built on SWT), maybe MacromediaPivot, CursesPivot ... :)
By the way, I'd also like to run the XMLA model against Mondrian's XMLA implementation, to see whether Mondrian's XMLA implementation is up to snuff.
So the JPivot CVS is now live at SourceForge by the looks.
What are the next steps, contributors?
I'm currently working on an ANT script that helps to mirror the sourceforge CVS with our internal CVS. When this works, I will add you and the others as developers and do a posting here.
When ready let us know what point in the snapshots the starting CVS will reflect. Denis ( stflourd ), Paul ( jacksonpd ) and I ( parkerb ) will then update as required with the drill-thru extensions , drag & drop etc.
I'm trying to login to CVS using the command provided on this site. The following is the error.
cvs [login aborted]: connect to jpivot.cvs.sourceforge.net:2401 failed: No conne
ction could be made because the target machine actively refused it.
I have populated CVS with the current versions of WCF and JPivot.
WCF is a separate CVS module now, it contains the GUI library that JPivot is built upon. We use WCF in several applications, not just JPivot - so its some kind of general purpose GUI library and should not have dependencies to JPivot. It was designed to interoperate with other GUI libraries like Struts or JSF. Also I have added all the WCF tests which may also serve as examples for how to use WCF components. To build JPivot you will either have to build WCF first or use the precompiled version.
To build JPivot things have not changed much, except it expects the wcf directory as a sibling of jpivot directory where it takes the wcf.jar and other files from.
Also I have added most of you guys to the list of developers - if I missed somebody let me know. I left the default permissions which include full CVS access and added full access to the trackers. I would be glad if someone wants to take over other tasks too. Unfortunately it looks like I can not spend too much time for JPivot in the next weeks as it has been the case in the past. This maybe bad news but the upside is that JPivot is getting more independent of Tonbeller and you will get the Sourceforge permissions to do whats needed to make JPivot successful.
I have created a new release 1.4.0 which reflects the initial CVS code. It may also be helpful to get the CVS code to compile and tests suites to pass.
Regarding a roadmap: Do you think its possible to use Sourceforge trackers and/or Tasks for this? I'm not familiar with these, so I'm not sure if this would work. The advantage could be that the items could be edited by all of us, tasks could be assigned to developers, can have priorities and state (e.g. 'completed'). What do you think?
Regarding the initial version: because of the splitting into separate WCF and JPivot modules, it does not correspond to any CVS snapshot published so far. There should be not too many changes between what went into CVS and the latest snapshot but there are definitely differences. If this causes difficulties I can provide a diff that lists the changes between latest snapshot and current CVS. Also if you send me a modified version of a snapshot, I could do a "cvs update" that merges all changes into your working copy.
So this is sort of an official lauch of the open JPivot CVS - happy hackig,
This is defnitely a monumental moment for JPivot. Just reading this discussion thread makes me a believer in open source development model (again!). I understand the challenges you and your team have gone through to make JPivot independent of Tonbeller, so congratulations on this "official launch".
To quote from Spiderman "with great power comes great responsibility" -- so your including of new people in the developers list means we (the community) now share the responsibility of making JPivot a success. Paul Lucas and myself from the OpenI project will start working on the build engineering aspects so the issues of cvs versions and release snapshots are clear. We will ping you separately for any issues we run into.
As for roadmap -- I'd have opted for a wiki (or wiki-like) environment, and I'm not sure what the sourceforge equivalent is. Maybe Trackers and/or Tasks are the possible place to start, or we could independently host a wiki instance somewhere outside of Sourceforge.
Anyway, this is great, we are very excited, and you guys should feel very proud of having pulled this one off. Congratulations once again!
I'll second that. Many thanks for all you've done so far Andreas -- and here's hoping you'll find enough time to continue leading the project, even if in a time-reduced capacity.
As regards roadmap, I think that the SF RFEs are a good starting point. Then someone in a "product management" role should put these into a document which spells out the cost/benefit of each feature, how they interrelate, and which features are targeted for which release. (I have attempted to follow this process in producing Mondrian's roadmap, http://mondrian.sourceforge.net/head/roadmap.html.\)
My vote for the roadmap would be to formalize JPivot's "model" (as in model-view-controller) classes in such a way that they can be used by other user-interface components. Mondrian and JPivot have had a healthy relationship because they aren't too interdependent: JPivot can talk to other XML/A providers, and Mondrian can work with other user-interfaces. A productized model API would make it more economical to build other OLAP user-interfaces, but it would also benefit JPivot because the model classes would be more rigorously tested.
Log in to post a comment.