First I'd like to congratulate everybody here on creating a truly outstanding project, especially you John.
Although I maintain that the Gedcom specification is what is holding back the genealogy folks from unleashing its full potential (understandable that companies want to tie users into their software by clinging to the industry "standard" Gecdom which 'unfortunately' does not allow exporting of all these nifty features that their own software understands), that is far outside the point of this post.
Some background:
I have a website running WordPress and Gallery2. I wanted a way to share the existing information about my relatives (relationships, addresses etc.) with them and have them able to edit it.
I looked around and quickly decided that PGV is the most feature-rich (judging from its activity).
Then I thought it might be nice to be able to link any FACT (i.e. any event, INDI or whatever) to a pic or an album in G2 and then have PGV display an imageblock next to it or just pulling out a random pic.
You could even link a type of fact to a specific album and find the respective album or pic by matching the FACT and the pic/album-name (or note, comments etc).
After looking at the Gal2 module I realized that it was just an embedding, so I decided to write my own and started reading PGV's code.
For a couple days of sifting I still maintained that I could pull it of, and tried to locate a place to begin.
But finally, comments like "Does anybody know what this is for?" dissuaded me from pursuing that goal any further and instead gave me the impression that PGV is in need of a major rewrite/cleanup.
I am aware that a rewrite is going to meet some resistance for obvious reasons but in the end it is going to benefit everybody, as it makes finding/localizing bugs as well as extending, modding and theming a lot easier.
The longer it is procrastinated, though, the harder it is going to get, so better do it soon.
Please do not get me wrong, I wouldn't say that the code of projects like G2 or WP is easier to understand (actually, I think they both employ overkill in some respect), but I think that by being more organized they give people the ability to develop plugins and themes.
Let's face it, if this is not done at some point in time, PGV is going to die a slow, painful death with more and more developers leaving for better/more organized projects.
These are the points that need addressing, as far as I can tell:
- Consolidating the HTML output (Theming): This has been brought up various times in the forums, mostly as the suggestion to move to a template engine like Smarty. I think Smarty is a very good alternative, and the arguments brought forth against it are not convincing. Though I do not think it needs to be smarty, while you are already consolidating the HTML you might as well move to a template engine to make theming/skinning easier. If you'd rather write your own, please do so. Along the same lines, the Javascript code needs to be consolidated and taken out of the various output methods to be put in '.js' files, to remove clutter from the HTML source (as well as grouping it at the beginning of the page where it belongs - in the header).
Picking up on the GUI thread (https://sourceforge.net/forum/message.php?msg_id=4080347) this will help alleviate the whole problem of programming-people trying to build a good UI (because as we all know, this doesn't work) by allowing UI-people to design the theme/view of the site. This is going to separate the HTML (presentation-logic) form the PHP source (application-logic) thus getting one step closer to pure MVC.
- Getting MVC right: The *_ctrl.php in the 'controllers' folder do most of the printing/displaying of content which is not their function according to "pure" MVC. Let's recapitulate the MVC model quickly:
-- M, the Model: defines an API through which the data of the software can be accessed. This defines everything, from the relations to types, ranges and meta-data. How this data is then stored is the sole concern of the Model, as long as it is uniformly accessible through the API (it can be stored in a DB, a text-file, XML - you name it). Parsers and serializers thus belong in the Model.
-- V, the View: describes a "View" of the Model, that is, a particular way of looking at it, as for example, the source-code-view or the UML-view (in the case of code). The model does not concern itself with anything else, e.g. security, just with how to represent the data. Views are exchangeable (i.e. pluggable), and get their data from the Model, so as soon as you change something in one view, the other views of the same data change accordingly. Themes and a little bit more for that matter, belong in the View.
-- C, the Controller: Glues the two other parts together and provides all the programming logic behind the app, like security, user management, file management and the like.
Note on the Model: I am not sure about having two storages for the model as is currently the case in PGV (DB and the Gedcom file) because this introduces a whole lot of sync trouble (which one is to be trusted if they are not in sync?). Since the DB is of course a whole lot faster than parsing the Gedcom every time, maybe they should only be synced when the gedcom is im-/exported.
- Rewriting the Gedcom parser: As far as I can tell, there are two locations in the code where a gedcom file is parsed (--GEDCOM functions-- in function.php and in functions_import.php). Also, it seems to me that this process is very complicated and thus error-prone partly because it uses RegExes for the parsing. 40 years of research into efficient automated parsing of text-based files, and you still write your own parser? This should be done by a using lexer, which have helped compiled programs outperform hand-coded Assembly programs more than 20 years ago. PEAR has classes for parser- and lexer-building which is all that's needed. I am aware of international character issues (UTF-8), but I'm sure lexers can handle this.
- Consolidating the SQL statements: A collection of lots of SQL statements (like those for creating the tables) belong in their own .SQL files to separate languages (like HTML from PHP). It is also advisable to group all - or most - other SQL in a single file that provides functions to call from the outside - for easier debugging. This should be part of the Model anyway.
- Moving from "Modules" to Plugins: The current approach does not define enough hooks to make PGV really customizable. The very existence of remedy code in the general code-base for the most common modules like googlemap or lightbox is witness to this. The ultimate goal is for plugin writers to not have to touch or modify any code of the 'core' - preventing them from breaking anything and allowing the core developers to change the underlying functionality without breaking any of the modders code. That is, the mod does not have to be re-inserted into every new release. For that hooks need to be defined for every little function call in PGV, every section of the output, in effect - for everything. Look at WP or G2 for examples. It does require some effort on the part of the developers, though, for defining a sensible API that is reasonably stable over long periods of releases.
- General organization: Files ending in .php should be files that need to be executed directly in the browser, like index.php or individual.php. These - and only these - should reside in '/'. Every other file that is included by the main files should reside in a logically named subfolder ('includes' for example) and have an ending of .inc, class files by convention have the ending of .class; thus preventing them from being executed and accessed directly. Files that logically belong together should be placed in the same subfolder; all files comprising the back-end (admin functions) should be in a folder named something like 'admin' or 'pgv-admin'. This makes it clearly visible to anyone what function can be found where and also which part of the HTML output comes from where.
With all this said, I would like to offer my help and start a branch off the official release (maybe name it 'Code Cleanup and Consolidation Project' - CCCP *eg*) which will be merged slowly into the mainline release. Anybody who is willing to contribute is welcomed to. This will not be quick and may break some functionality here and there (well, hopefully not), but it will be well worth the effort.
Other projects have successfully done this, and so can we!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The code for phpgedview makes me think of some sort of spaghetti code. I can't help but think that, despite seemingly being one of the most popular online genealogy programs around, the state of the code hinders phpgedview and results in a much smaller number of developers working on it than it has the potential for. The documentation I read says phpgedview follows MVC and then I go read about MVC and can't agree with the documentation like Vitus here is trying to explain.
Take for example this... I have a problem with phpgedview where I can't upload GEDCOM files because the file gets uploaded but for some reason the name gets doubled on the server and then the software can't find the file and doesn't process it. That's being all done it looks like by editconfig_gedcom.php. Now why is the upload stuff in there to begin with? The upload functionality gets duplicated in media.php, where I can upload media files without this doubling up on the filename. It is things like this that make this software difficult to maintain and probably drives away potential developers.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I apologise, if I overlooked something of importance and have not included it...it's a wiki so feel free to contribute to the article. I understand that the needs of the developers differs to that of the users, but I feel that with some effort put in now to address some concerns of current (and of course future) developers, it will make it far easier for new devs to come on board and start contributing.
This is a "call to arms" for developers - I hope I'm not stepping on any toes, but as it's opensource I'm trying to take the interest expressed in this thread by several people and trying to get some actions clarified etc and hopefully a roadmap will emerge from the other side.
I'm hoping that the above PGVWiki article can be a place to help organise thoughts and actions with regards to some of the issues brought up in this thread, so if you are at all interested in helping out, please take a look, get involved and add your name to the list at the bottom of the article if you are willing/able to make contributions and make things happen.
Cheers
Nath
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
-
2008-01-27
If what you are proposing is a more structured approach to development (albeit with a small backwards step to provide a firmer basis for progress) then I wholeheartedly agree with you. I have said for some time now that what PGV lacks most of all is a sense that it is a 'project', clearly managed and directed. Believe it or not, even an open-source project (if it is going to last) needs to be structured and managed much like a commercial venture.
I could point out a few errors in your summary (e.g. "Since the DB is of course a whole lot faster than parsing the Gedcom every time, maybe they should only be synced when the gedcom is im-/exported." which already happens if you choose to uncheck the option "Synchronise edits into GEDCOM file") but your overall point seems sound, at least to a 'layman'.
I noted especially your comment "Let's face it, if this is not done at some point in time, PGV is going to die a slow, painful death with more and more developers leaving for better/more organized projects." I have said that offline to others earlier. My own prediction is that without something such as you are suggesting this could turn out to be a very bad year for PGV.
I can't help with programming, but would be more than happy to do some testing.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
<snip>
I noted especially your comment "Let's face it, if this is not done at some point in time, PGV is going to die a slow, painful death with more and more developers leaving for better/more organized projects." I have said that offline to others earlier. My own prediction is that without something such as you are suggesting this could turn out to be a very bad year for PGV.
</snip>
If PGV were going to die because of poor leadership or poor code it would have died a long time ago.
I believe in the OSS software process and I believe that this project has flourished because we have kept to OSS principles of community development and involvement. If you want "structured" (I also add oppressive) leadership then go to GenMod or start your own project.
--John
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
That's a fairly accurate description of PGV, and you make sensible/standard suggestions. You don't mention the data model at all - but that's another story...
PGV's biggest problem is its success. I've been working on it for a couple of years, and I still find features that I didn't know existed. It supports pretty much every server configuration out there. Hence any change you make is likely to break/remove some feature that someone has been using for years and is very attached to - and you will make yourself very unpopular.
A simple example would be to move all layout/presentation from the html and put it into css. This would just leave the content in the html file. It would be faster (less to download and shift processing from the server to the client), it would be far more flexible. It would be a classic "good thing" (tm). But it would break all the custom themes that people have created, and force theme developers to learn CSS. So I can't see it happening.
I have tried to refactor the code many times, to implement many of the suggestions you've made. Each time I come across some stumbling block that would involve some loss of backward compatibility, and that puts a spanner in the works.
IMHO, the best thing to do would be to start from scratch. Pick out the reusable code, promise no backward compatibility, ditch support for old versions of PHP/MySQL/etc.
I'm sure I'm not the only developer who does more work on their "side project" than on PGV.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
First let me say that more than anyone I am well aware of these problems. I agree with most of the points that have been made.
Most of the code examples mentioned here as bad examples (such as editconfig_gedcom) are also the code examples that bug me the most.
You don't know how many times I've wanted to start from scratch. But I also believe that starting from scratch is anti-open-source it also throws away thousands of hours of volunteer work.
Now let me give you some history.
PGV started in 2002 as an experiment to see if I could make a genealogy web application which could dynamically parse a GEDCOM file on the fly and produce displays. The experiment turned out to be a success and it grew up from there gaining momentum.
Since 2005 we started moving in the direction you are suggesting. We have been rewriting things, we have been adding OOP code, we have been trying to make things more modular. But I have purposely made the transition a slow one. Maybe that was a bad decision on my part, but when you have tens of thousands of users, you don't just abandon them.
The other problem is that code from PGV has come from many different developers with many different backgrounds. Not all of them necessarily the best coders with professional training. But their code works, and I don't personally have time to rewrite it all.
But honestly is that really a bad thing? From a professional coders perspective it is a nightmare... but from an open-source KISS perspective it is wonderful how many people have been able to contribute to PGV.
Anyway, what I really want to say is:
I whole-heartedly agree... we welcome all the help that we can get... but it is going to be a long hard journey.
--John
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
May I suggest that we open up a PGVWiki article to help co-ordinate efforts in particular areas of PGV development. I wouldn't intend the resulting article(s) to replace forum threads such as this, but a place to gain a better consensus of the community by collating thoughts in one place. A TODO list could link to dedicated articles dealing with things such as improving MVC which in turn lists areas/files involved in a particular aspect of PGV. Hopefully, this would lead to the community helping and organising itself without the heavy burden laying on a single person - after all, people are doing this work out of the goodness of their hearts.
I'm happy to set up some stub articles to help the organisation and can try to summarise/organise threads such as this if others are interested in contributing?
On a similar note, what are the things that need to be completed before a 4.2 release is made? Are these documented anywhere? I know there is a "new features under development" tracker, but I assume items can be added to this list and the list me never really be completed - just wondering where the line is drawn between the 4.1.x release series as the 4.2.x release series.
Cheers
Nath
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Something else I wanted to add is that the 4.2 branch of SVN is where the work of this nature is happening. There you will find much more OOP code. Again progress is slow but it is moving forward.
--John
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have been going over the code these past few weeks and agree that it is in need of a cleanup. I do not know PHP though I am learning by going through things. I am willing to get involved in a major code clean-up. Currently I am working on getting the PostgreSQL database calls working without error (in the 4.2 branch, thought I should probably submit patches for 4.1.x branch as well).
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have mixed feelings. I do not think it will “die a slow painful death.” The worst that could happen would be that some people bail out for a "start-from-scratch" project, leaving others to continue with PGV as is.
On the lighter side, CCCP looks too much like СССР which stands for Союз Советских Социалистических Республик
Anyway, though I agree that working with PhpGedView code is difficult, I'm not willing to give up what it can do while changing it. If it’s as tangled up as you say (and some parts are), you can’t really fix it gradually.
Database: Lifelines was fairly successful at letting the GEDCOM, with some supporting indexes, be the database.
There are some stylistic things that could be done which wouldn’t buy much but would assist future development, such as
1. Any construct of any complexity should have a comment at its end to help match the end to the beginning. Examples:
} // if edit allowed
</div><!-- class="blockhc" -->
2. DOM elements should begin and end in the same function or even the same set of braces.
Hard to change a set of table rows/cells to a set of divs when the the parts and the tags
between them are printed from a utility function.
3. Lots more line breaks. SVN can't merge two unrelated changes when they are both in different parts of the same mile long line of code.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
As a user but not a developer may I offer a word of encouragement to those contributing to this thread? PhpGedView is brilliant! Sure it has its problems but what complex project does not? Since I switched to PGV from my own home made site late last year it has been a huge success. My members and visitors love it and I can see my way to opening up to a much wider authorship. You have a great deal of goodwill from developers, users and other project groups such as Drupal (super disciplined techos that they are) who would love to improve integration with PGV. I am sure that a review of your formal processes such as functional and technical road maps, architecture, APIs, documentation, integration, validation, version control and security would go a long way to restoring your confidence. No need to be discouraged or too impatient. I believe that PGV is an outstanding product with outstanding support and a very promising future. Congratulations to a fabulous concept and a fabulous team.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have heard several people offer to help in this area and I have said that we would love the help but I never said how you can get involved.
If you want to get involved in helping with code cleanup or with other areas of PGV, first send me an email (yalnifj at users dot sourceforge dot net) and include a specific area that you would like to clean up. Include a brief outline for how you plan to make your changes.
I'll get you access to the SVN and hopefully be able to get you in touch with other stakeholders who will have an interest in what you are working on.
As an example, if you would like to improve on the MVC in the program, pick an area of PGV where you think conversion to MVC would best help the program and other developers to build on PGV. A great example that badly needs it is the whole import process (I would love to see the ability to develop a command-line import tool).
Thanks,
--John
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I hesitated to post to this thread as I have only been interested in PGV for a very short time. Perhaps my perspective will be of interest however. Feel free to reject my input as the ravings of a newcomer who does not know what he is talking about.
Firstly to reafirm that which has been said by others that PGV is a great package and a credit to those that have worked on it. The question being asked is where should it go now?
Greg Roach briefly mentioned the data model. I have just performed a global search for the string INDI and counted those files where the string appeared to be used in the context of the GEDCOM file format. Ignoring language files I came to a rough estimate of 67 files where a detailed knowledge of the GEDCOM file format is embedded. One could argue that data abstraction is even more important than MVC. For a newcomer to the code this is also an issue that makes the code difficult to understand. A typical example (from person_class) is:
$famlink = get_sub_record(1, "1 FAMC @".$family->getXref()."@", $this->gedrec);
$ft = preg_match("/2 PEDI (.*)/", $famlink, $fmatch);
Presumably at some stage in the future, whether in 1, 2, 5 or 10 years time an improved GEDCOM format will appear. Is this area of data abstraction for the GEDCOM format one that should be treated as a priority in the Code Cleanup? Conveniently of course what you end up with is a good chunk of what is known as the Model in the latest Three Letter Acronym. Good Lord I have just realised that I have been writing MVC code for 30 years and didn't realise. (Well the M part anyway, admittedly the V and C have had a tendency to get a bit entangled).
Although this would be a very significant amount of work, the first part (defining and coding the interface to the GEDCOM data) is new code and could be written more or less in isolation. Once it is functional then it's use could be gradually phased in one file at a time. Well that is the theory anyway, it is never quite that simple of course.
Colin
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I agree with many of the conventions mentioned, but ask that the files retain the .php extension. If the webserver does not recognize the file type, it will do one of two things:
1) serve it as plain text. This would allow hackers who know the file layout to read our settings
2) serve it as a binary file. This prompts visitors to download the file. Once on their local machine, it is readable by any text editor.
Instead, I suggest two levels of extensions, based on the function of the file. As an example, the filename gedparser.class.php clearly shows that it contains a class used to parse GEDCOM.
I agree 110% that plugins / modules should not require editing any core file.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Well, that is certainly true: Anyone can read our files.
But since PGV is an open-source project, anyone can go to phpgedview.net, download the source and then know our file layout, and _even_: the contents of those files.
And because of that you put all the relevant settings in a file appropriately named config, which is the only file that needs to be protected - which is why it is mostly a .php file that displays nothing.
The thing that you do not want, is that people are able to execute all those files directly, which is a greater security risk than them reading them. And by making all files have a .php ending you just allowed them to.
Because then all you need is one file that does not rigorously check if it was executed or included and you have a much bigger security risk than you would have, had the user just seen the source code (which he can get anyway). Now the attacker can pass variables, try to set options etc. And if you are dangerous-loving enough and have "register globals" still on - well that's just too bad, you basically handed him your server on a silver platter.
-V
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
This is a most interesting thread. I read it with growing hopes, but, as it has developed, reality has taken over.
Using web-based data to generate home-based web pages is not very well known to many people - nor do they have the knowledge or aspiration to try to get it to work for them. Some of us have seen the possibilities (largely, but not only, people with some computing background) and we have enjoyed the experience. A talented group of developers have fed us, and we proudly showed our work to our families and other interested folk.
PGV, like all projects, started well, attracting developers..and in turn users, so creating impetus. And as PGV was not developed to a plan, it got a bit tatty at the edges - something that happens with all like projects, I should add. As the picture became clearer about users and their likes/dislikes, so the original PGV looked to be in need of a serious overhaul, even a restart because, unplanned, that's how things go. But gravity had taken over, and there was too much investment in the current model to contemplate anything major.
And, as the number of interested developers has tapered off, so to has the possibility of such an event taking place. Realistically, PGV is PGV. Apart from some titivation, there is no practical way to redesign the model, rewrite the code, redesign the UI or do other than minor edits and small code tidy-ups. That's not to discourage anyone from trying, just be aware of the limited scope that is available to you.
Paul
Australia
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Remember that this thread is a developer level discussion. In the end users don't care what the code looks like under the hood. They just want something that is stable and works. And if you take a look at all of the Feature Requests, the vast majority of users are not demanding a code rewrite, nor would they want the development resources reassigned to one.
It is potential developers who want to customize or build on PGV that want more modular, more abstracted, code.
The fact of the matter is that this is a volunteer developed open source project and if you want to see these changes made, YOU are going to have to work up enough motivation to start the work. Honestly, I think that is the problem... nobody has enough motivation to do it.
I personally try to work some of these changes in with each release and with each major revision.
Based on the many priorities in the project, I am personally doing as much as I can to make PGV a better platform for people to build on.
--John
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I think the wiki page is the way to go to keep track of what needs doing. Last night I was reading "UML Distilled" by Martin Fowler and came across words that were relevant to this subject.
"... when you add a new function to a program, you build on top of the existing program, often in a way that the existing program was not intended to support ..."
"Although in theory it is better to redesign your program, this usually results in extra work because any rewriting of your existing program will introduce new bugs and problems... However, if you don't redesign your program, the additions will be more complex than they should be... Gradually this extra complexity will exact a stiff penalty. Therefore, there is a trade-off: Redesigning causes short-term pain for longer-term gain."
He goes on to talk about refactoring techniques that make this process easier. A quick Google search comes up with more information on it.
Doing this kind of work I think is not at all contrary to OSS principles. Like John pointed out, people are volunteering their time to help on the project. If the project code is a mess, a potential contributor has to spend more time than should be necessary to understand the extra complexities unless they were extremely dedicated. Of course not many people are going to be willing to spend their time doing this so you end up losing these people.
One key to refactoring requires testing to make sure functionality isn't broken. Is there some sort of testing suite available for phpgedview? Perhaps making a series of tests should be the first priority to put up on the wiki page.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
-
2008-02-02
No, there are no formal testing routines, although I believe nathan has been attempting to develop some.
The current process for testing anything is to rely on PGVs large user base. In particular there are a number of users (including myself) who have been using PGV for some time (years generaly), have a range of differing installations (hosted web, own server, MAC, PC, etc etc); operate in languages from English to Hebrew and beyond; and have GEDCOMs containng from 100s of INDIs to over 40,000.
I'm certainly not suggesting a series of standard tests wouldn't be useful, just that there are additional options as well.
One of the defining components (at least according to Gregorio Robles and the 'Bazaar' model) is:
the following patterns:
"Users should be treated as co-developers
The users are treated like co-developers and so they should have access to the source code of the software. Furthermore users are encouraged to submit additions to the software, code fixes for the software, bug reports, documentation etc. Having more co-developers increases the rate at which the software evolves. Linus's law states that, "Given enough eyeballs all bugs are shallow." This means that if many users view the source code they will eventually find all bugs and suggest how to fix them. Note that some users have advanced programming skills, and furthermore, each user's machine provides an additional testing environment. This new testing environment offers that ability to find and fix a new bug."
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
This should help you understand why I manage the project the way I do and help you better understand the philosophies governing OSS development.
--John
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
-
2008-02-04
An excellent suggestion John. Unfortunately like any great work (try the Koran for example) it is open to widely differing interpretations. Take this paragraph as an example:
<<Rather, I want to suggest what may be a wider lesson about software, (and probably about every kind of creative or professional work). Human beings generally take pleasure in a task when it falls in a sort of optimal-challenge zone; not so easy as to be boring, not too hard to achieve. A happy programmer is one who is neither underutilized nor weighed down with ill-formulated goals and stressful process friction. Enjoyment predicts efficiency.>> [for chapter "On Management and the Maginot Line"]
I respectfully suggest this explains the issues many have raised - especially the less competent programmers who have tinkered in the development arena - "ill-formulated goals and stressful process friction" and "...too hard to achive".
As the creator of PGV (and one who is extremely highly respected for that) - can you comment on the likely success and acceptability of the process occurring on the WIKI pages regarding this discussion?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I think if you read that article and compare it to PGV you will find lots of interesting connections including Linus' law and ego-less programming.
<<< As the creator of PGV (and one who is extremely highly respected for that) - can you comment on the likely success and acceptability of the process occurring on the WIKI pages regarding this discussion? >>>
I can comment based on my experiences in the open source world (which are not limited to just PGV by the way).
Unfortunately this is all talk, and the one thing that an open source project needs to be successful is code that people can play with. I've never seen a successful open source project which started out like this. Instead someone needs to take the initiative and do something and then make that code available. If it has "potential" then others will jump on it.
I've seen this process over and over again in the genealogy world as well. Take GEDCOM or XML as an example. Everyone wants to talk about it but that talk is never converted into anything that someone can use. All successful open source projects that I am aware of start out as an idea that someone implements, then others jump on and contribute. But the key is that there has to be an initial implementation.
The average person who gets involved in PGV just wants to "scratch a personal itch" and so they aren't going to be interested or involved in these high-level design discussions. And these discussions have come about because people who wanted to scratch a personal itch haven't been able to figure out how to do that and looking at the code and not understanding its history and why things are the way they are, suggest that the code is flawed or mismanaged.
Now I am not dooming this initiative nor do I think this discussion is a waste of time. I think that this discussion is healthy and helpful to PGV. For the most part these are good ideas. This discussion builds on other concepts from the article such as "parallel design" and the importance of having users.
There are many places in PGV where the code is flawed and needs to be rewritten to be more scalable or use a better OOP design. But in OSS the average volunteer developer is not going to be the one to do that.
What then does this mean for PGV and its future? It means that these ideas will be implemented slowly over time by people who have enough initiative and motivation to do it. There are less than a handful of PGV developers--maybe only 2 or 3--who I think would even start to take projects of this magnitude on. I am one of those, but my time is extremely stretched. Just this response has taken me at least 30 minutes to write, not to mention all of the other things I deal with everyday for this project. Fortunately I have students who can help with these kinds of projects.
Speaking of which, it is time for me to go to my next class. But let me end with my vision for the future of PGV and where I think PGV needs to go. My vision for the future of PGV is in "genealogy as a platform" and "PGV as a platform" that many people can build off of either through direct access to the PHP APIs, or through web services, or through mash-ups and widgets. I think implementing this vision would meet the needs which spurred this discussion as well as meet the UI needs and meet the future of the genealogy industry as a whole.
The future of the genealogy industry is an exciting one and PGV will continue to be a major force in realizing that future.
--John
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
John, I did not want to criticise your way of managing the project.
I fully understand that keeping this many developers pointed in the same/right direction over long periods of time is very hard - and even more to get them to right good code. Because PGV depends on people contributing you cannot turn down additions just because they dont hold up to an artificial (your) measurement of quality - that surely varies even from person to person. The problem is that people contribute code all over the place because one function of PGV is not centralized in one place.
That is why the issue of plugins and templates is critical.
Once you manage to create a clean and well programmed core that does not need to be changed for users to successfully be able to scratch their 'itch' as you put it, you can have them play with their own code in templates and plugins as much as they want. In this way you do not allow 'dirty' code into the core and still give everybody the ability to costumize their PGV to their liking.
And if you look at my two favorite role-projects (WP and G2) you will see that this is exactly the way they work.
Over time plugins will naturally converge on better code anyway, as their developers get more experienced.
I admit that this necessitates a slightly stricter reign over the core developers, as well as keeping their number as small as feasible - but is well worth the effort.
If you do want to take advantage of the "bazaar-style" approach as Linus did with Linux you have to keep in mind that developers contributing to the linux kernel had already reached a rather high coding skill in order to understand what was going on (even proved computer-savvy for getting Linux installed at all, back in the day) - ensuring that their contributions weren't badly written. And also that Linus and very few others still look over all the changes before they get accepted ensuring high quality.
I think the situation with PGV is different - as most of the users are not experienced programmers but still have the wish to scratch an itch - and therefore needs a slightly different handling.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
PGV does have a "core" as you call it, it's just hard to see it for all of the clutter. It is composed largely the files in the includes directory. Everything else is a plug-in that uses the "core" APIs. The code is in need of refactoring not throwing away.
The thing that is not modular enough is the UI. If I add a new file, the UI is not going to automatically pick up on it. Unfortunately that is the thing that people most want to scratch, and rightly so. I believe you should be able to present and honor your ancestors in the way you want.
I am trying to make the UI more modular. I have a team of students right now working on making the tabs on the indi page modular. I have tried many times to make the menus modular... but I am afraid that is one area I will have to completely scrap and start over on. The themes are already modular, and you can do a lot with the themes, again the menus being the biggest problem with the themes.
There is also a lot of legacy code, most of the older charts, etc, were written using the old APIs. They just need to be rewritten or thrown out. This old code is serving as a poor model for new developers.
There was also a time when a previous developer and I disagreed. I was pushing for more MVC, more separation, and he was going around combining files and taking out separation and making things less modular.
This discussion of a "core" has come up before as well as the notion that we have to protect the core and we shouldn't let anyone modify it (ironically with the same developer I was talking about above). If we did "protect the core", then Greg wouldn't be able to make the DB changes he is proposing in the other thread. I also don't want to setup some hierarchy that says one developer is better or more valuable than another.
My philosophy is that if code really is "Core", then it naturally won't be changed because it won't need to be changed. I don't consider anything in the UI to be "core".
>> If you do want to take advantage of the "bazaar-style" approach
I do want to use the open source or "bazaar-style" for 2 reasons:
1. I believe it produces better products faster
2. It allows me the flexibility of not having to be a hands-on manager which I don't have the time to be
I believe that once the majority of the legacy code has been fixed and updated, and good documentation produced, the new code will serve as a model for new developers and will encourage them towards producing good code themselves no matter what there level or experience.
--John
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
First I'd like to congratulate everybody here on creating a truly outstanding project, especially you John.
Although I maintain that the Gedcom specification is what is holding back the genealogy folks from unleashing its full potential (understandable that companies want to tie users into their software by clinging to the industry "standard" Gecdom which 'unfortunately' does not allow exporting of all these nifty features that their own software understands), that is far outside the point of this post.
Some background:
I have a website running WordPress and Gallery2. I wanted a way to share the existing information about my relatives (relationships, addresses etc.) with them and have them able to edit it.
I looked around and quickly decided that PGV is the most feature-rich (judging from its activity).
Then I thought it might be nice to be able to link any FACT (i.e. any event, INDI or whatever) to a pic or an album in G2 and then have PGV display an imageblock next to it or just pulling out a random pic.
You could even link a type of fact to a specific album and find the respective album or pic by matching the FACT and the pic/album-name (or note, comments etc).
After looking at the Gal2 module I realized that it was just an embedding, so I decided to write my own and started reading PGV's code.
For a couple days of sifting I still maintained that I could pull it of, and tried to locate a place to begin.
But finally, comments like "Does anybody know what this is for?" dissuaded me from pursuing that goal any further and instead gave me the impression that PGV is in need of a major rewrite/cleanup.
I am aware that a rewrite is going to meet some resistance for obvious reasons but in the end it is going to benefit everybody, as it makes finding/localizing bugs as well as extending, modding and theming a lot easier.
The longer it is procrastinated, though, the harder it is going to get, so better do it soon.
Please do not get me wrong, I wouldn't say that the code of projects like G2 or WP is easier to understand (actually, I think they both employ overkill in some respect), but I think that by being more organized they give people the ability to develop plugins and themes.
Let's face it, if this is not done at some point in time, PGV is going to die a slow, painful death with more and more developers leaving for better/more organized projects.
These are the points that need addressing, as far as I can tell:
- Consolidating the HTML output (Theming): This has been brought up various times in the forums, mostly as the suggestion to move to a template engine like Smarty. I think Smarty is a very good alternative, and the arguments brought forth against it are not convincing. Though I do not think it needs to be smarty, while you are already consolidating the HTML you might as well move to a template engine to make theming/skinning easier. If you'd rather write your own, please do so. Along the same lines, the Javascript code needs to be consolidated and taken out of the various output methods to be put in '.js' files, to remove clutter from the HTML source (as well as grouping it at the beginning of the page where it belongs - in the header).
Picking up on the GUI thread (https://sourceforge.net/forum/message.php?msg_id=4080347) this will help alleviate the whole problem of programming-people trying to build a good UI (because as we all know, this doesn't work) by allowing UI-people to design the theme/view of the site. This is going to separate the HTML (presentation-logic) form the PHP source (application-logic) thus getting one step closer to pure MVC.
- Getting MVC right: The *_ctrl.php in the 'controllers' folder do most of the printing/displaying of content which is not their function according to "pure" MVC. Let's recapitulate the MVC model quickly:
-- M, the Model: defines an API through which the data of the software can be accessed. This defines everything, from the relations to types, ranges and meta-data. How this data is then stored is the sole concern of the Model, as long as it is uniformly accessible through the API (it can be stored in a DB, a text-file, XML - you name it). Parsers and serializers thus belong in the Model.
-- V, the View: describes a "View" of the Model, that is, a particular way of looking at it, as for example, the source-code-view or the UML-view (in the case of code). The model does not concern itself with anything else, e.g. security, just with how to represent the data. Views are exchangeable (i.e. pluggable), and get their data from the Model, so as soon as you change something in one view, the other views of the same data change accordingly. Themes and a little bit more for that matter, belong in the View.
-- C, the Controller: Glues the two other parts together and provides all the programming logic behind the app, like security, user management, file management and the like.
Note on the Model: I am not sure about having two storages for the model as is currently the case in PGV (DB and the Gedcom file) because this introduces a whole lot of sync trouble (which one is to be trusted if they are not in sync?). Since the DB is of course a whole lot faster than parsing the Gedcom every time, maybe they should only be synced when the gedcom is im-/exported.
- Rewriting the Gedcom parser: As far as I can tell, there are two locations in the code where a gedcom file is parsed (--GEDCOM functions-- in function.php and in functions_import.php). Also, it seems to me that this process is very complicated and thus error-prone partly because it uses RegExes for the parsing. 40 years of research into efficient automated parsing of text-based files, and you still write your own parser? This should be done by a using lexer, which have helped compiled programs outperform hand-coded Assembly programs more than 20 years ago. PEAR has classes for parser- and lexer-building which is all that's needed. I am aware of international character issues (UTF-8), but I'm sure lexers can handle this.
- Consolidating the SQL statements: A collection of lots of SQL statements (like those for creating the tables) belong in their own .SQL files to separate languages (like HTML from PHP). It is also advisable to group all - or most - other SQL in a single file that provides functions to call from the outside - for easier debugging. This should be part of the Model anyway.
- Moving from "Modules" to Plugins: The current approach does not define enough hooks to make PGV really customizable. The very existence of remedy code in the general code-base for the most common modules like googlemap or lightbox is witness to this. The ultimate goal is for plugin writers to not have to touch or modify any code of the 'core' - preventing them from breaking anything and allowing the core developers to change the underlying functionality without breaking any of the modders code. That is, the mod does not have to be re-inserted into every new release. For that hooks need to be defined for every little function call in PGV, every section of the output, in effect - for everything. Look at WP or G2 for examples. It does require some effort on the part of the developers, though, for defining a sensible API that is reasonably stable over long periods of releases.
- General organization: Files ending in .php should be files that need to be executed directly in the browser, like index.php or individual.php. These - and only these - should reside in '/'. Every other file that is included by the main files should reside in a logically named subfolder ('includes' for example) and have an ending of .inc, class files by convention have the ending of .class; thus preventing them from being executed and accessed directly. Files that logically belong together should be placed in the same subfolder; all files comprising the back-end (admin functions) should be in a folder named something like 'admin' or 'pgv-admin'. This makes it clearly visible to anyone what function can be found where and also which part of the HTML output comes from where.
With all this said, I would like to offer my help and start a branch off the official release (maybe name it 'Code Cleanup and Consolidation Project' - CCCP *eg*) which will be merged slowly into the mainline release. Anybody who is willing to contribute is welcomed to. This will not be quick and may break some functionality here and there (well, hopefully not), but it will be well worth the effort.
Other projects have successfully done this, and so can we!
Hear, hear!
The code for phpgedview makes me think of some sort of spaghetti code. I can't help but think that, despite seemingly being one of the most popular online genealogy programs around, the state of the code hinders phpgedview and results in a much smaller number of developers working on it than it has the potential for. The documentation I read says phpgedview follows MVC and then I go read about MVC and can't agree with the documentation like Vitus here is trying to explain.
Take for example this... I have a problem with phpgedview where I can't upload GEDCOM files because the file gets uploaded but for some reason the name gets doubled on the server and then the software can't find the file and doesn't process it. That's being all done it looks like by editconfig_gedcom.php. Now why is the upload stuff in there to begin with? The upload functionality gets duplicated in media.php, where I can upload media files without this doubling up on the filename. It is things like this that make this software difficult to maintain and probably drives away potential developers.
I've tried to summarise what has been said/suggested on this thread over on the PGVWiki site:
http://wiki.phpgedview.net/en/index.php/Code_cleanup
I apologise, if I overlooked something of importance and have not included it...it's a wiki so feel free to contribute to the article. I understand that the needs of the developers differs to that of the users, but I feel that with some effort put in now to address some concerns of current (and of course future) developers, it will make it far easier for new devs to come on board and start contributing.
This is a "call to arms" for developers - I hope I'm not stepping on any toes, but as it's opensource I'm trying to take the interest expressed in this thread by several people and trying to get some actions clarified etc and hopefully a roadmap will emerge from the other side.
I'm hoping that the above PGVWiki article can be a place to help organise thoughts and actions with regards to some of the issues brought up in this thread, so if you are at all interested in helping out, please take a look, get involved and add your name to the list at the bottom of the article if you are willing/able to make contributions and make things happen.
Cheers
Nath
If what you are proposing is a more structured approach to development (albeit with a small backwards step to provide a firmer basis for progress) then I wholeheartedly agree with you. I have said for some time now that what PGV lacks most of all is a sense that it is a 'project', clearly managed and directed. Believe it or not, even an open-source project (if it is going to last) needs to be structured and managed much like a commercial venture.
I could point out a few errors in your summary (e.g. "Since the DB is of course a whole lot faster than parsing the Gedcom every time, maybe they should only be synced when the gedcom is im-/exported." which already happens if you choose to uncheck the option "Synchronise edits into GEDCOM file") but your overall point seems sound, at least to a 'layman'.
I noted especially your comment "Let's face it, if this is not done at some point in time, PGV is going to die a slow, painful death with more and more developers leaving for better/more organized projects." I have said that offline to others earlier. My own prediction is that without something such as you are suggesting this could turn out to be a very bad year for PGV.
I can't help with programming, but would be more than happy to do some testing.
<snip>
I noted especially your comment "Let's face it, if this is not done at some point in time, PGV is going to die a slow, painful death with more and more developers leaving for better/more organized projects." I have said that offline to others earlier. My own prediction is that without something such as you are suggesting this could turn out to be a very bad year for PGV.
</snip>
If PGV were going to die because of poor leadership or poor code it would have died a long time ago.
I believe in the OSS software process and I believe that this project has flourished because we have kept to OSS principles of community development and involvement. If you want "structured" (I also add oppressive) leadership then go to GenMod or start your own project.
--John
That's a fairly accurate description of PGV, and you make sensible/standard suggestions. You don't mention the data model at all - but that's another story...
PGV's biggest problem is its success. I've been working on it for a couple of years, and I still find features that I didn't know existed. It supports pretty much every server configuration out there. Hence any change you make is likely to break/remove some feature that someone has been using for years and is very attached to - and you will make yourself very unpopular.
A simple example would be to move all layout/presentation from the html and put it into css. This would just leave the content in the html file. It would be faster (less to download and shift processing from the server to the client), it would be far more flexible. It would be a classic "good thing" (tm). But it would break all the custom themes that people have created, and force theme developers to learn CSS. So I can't see it happening.
I have tried to refactor the code many times, to implement many of the suggestions you've made. Each time I come across some stumbling block that would involve some loss of backward compatibility, and that puts a spanner in the works.
IMHO, the best thing to do would be to start from scratch. Pick out the reusable code, promise no backward compatibility, ditch support for old versions of PHP/MySQL/etc.
I'm sure I'm not the only developer who does more work on their "side project" than on PGV.
First let me say that more than anyone I am well aware of these problems. I agree with most of the points that have been made.
Most of the code examples mentioned here as bad examples (such as editconfig_gedcom) are also the code examples that bug me the most.
You don't know how many times I've wanted to start from scratch. But I also believe that starting from scratch is anti-open-source it also throws away thousands of hours of volunteer work.
Now let me give you some history.
PGV started in 2002 as an experiment to see if I could make a genealogy web application which could dynamically parse a GEDCOM file on the fly and produce displays. The experiment turned out to be a success and it grew up from there gaining momentum.
Since 2005 we started moving in the direction you are suggesting. We have been rewriting things, we have been adding OOP code, we have been trying to make things more modular. But I have purposely made the transition a slow one. Maybe that was a bad decision on my part, but when you have tens of thousands of users, you don't just abandon them.
The other problem is that code from PGV has come from many different developers with many different backgrounds. Not all of them necessarily the best coders with professional training. But their code works, and I don't personally have time to rewrite it all.
But honestly is that really a bad thing? From a professional coders perspective it is a nightmare... but from an open-source KISS perspective it is wonderful how many people have been able to contribute to PGV.
Anyway, what I really want to say is:
I whole-heartedly agree... we welcome all the help that we can get... but it is going to be a long hard journey.
--John
May I suggest that we open up a PGVWiki article to help co-ordinate efforts in particular areas of PGV development. I wouldn't intend the resulting article(s) to replace forum threads such as this, but a place to gain a better consensus of the community by collating thoughts in one place. A TODO list could link to dedicated articles dealing with things such as improving MVC which in turn lists areas/files involved in a particular aspect of PGV. Hopefully, this would lead to the community helping and organising itself without the heavy burden laying on a single person - after all, people are doing this work out of the goodness of their hearts.
I'm happy to set up some stub articles to help the organisation and can try to summarise/organise threads such as this if others are interested in contributing?
On a similar note, what are the things that need to be completed before a 4.2 release is made? Are these documented anywhere? I know there is a "new features under development" tracker, but I assume items can be added to this list and the list me never really be completed - just wondering where the line is drawn between the 4.1.x release series as the 4.2.x release series.
Cheers
Nath
Something else I wanted to add is that the 4.2 branch of SVN is where the work of this nature is happening. There you will find much more OOP code. Again progress is slow but it is moving forward.
--John
I have been going over the code these past few weeks and agree that it is in need of a cleanup. I do not know PHP though I am learning by going through things. I am willing to get involved in a major code clean-up. Currently I am working on getting the PostgreSQL database calls working without error (in the 4.2 branch, thought I should probably submit patches for 4.1.x branch as well).
I have mixed feelings. I do not think it will “die a slow painful death.” The worst that could happen would be that some people bail out for a "start-from-scratch" project, leaving others to continue with PGV as is.
On the lighter side, CCCP looks too much like СССР which stands for Союз Советских Социалистических Республик
Anyway, though I agree that working with PhpGedView code is difficult, I'm not willing to give up what it can do while changing it. If it’s as tangled up as you say (and some parts are), you can’t really fix it gradually.
Database: Lifelines was fairly successful at letting the GEDCOM, with some supporting indexes, be the database.
There are some stylistic things that could be done which wouldn’t buy much but would assist future development, such as
1. Any construct of any complexity should have a comment at its end to help match the end to the beginning. Examples:
} // if edit allowed
</div><!-- class="blockhc" -->
2. DOM elements should begin and end in the same function or even the same set of braces.
Hard to change a set of table rows/cells to a set of divs when the the parts and the tags
between them are printed from a utility function.
3. Lots more line breaks. SVN can't merge two unrelated changes when they are both in different parts of the same mile long line of code.
As a user but not a developer may I offer a word of encouragement to those contributing to this thread? PhpGedView is brilliant! Sure it has its problems but what complex project does not? Since I switched to PGV from my own home made site late last year it has been a huge success. My members and visitors love it and I can see my way to opening up to a much wider authorship. You have a great deal of goodwill from developers, users and other project groups such as Drupal (super disciplined techos that they are) who would love to improve integration with PGV. I am sure that a review of your formal processes such as functional and technical road maps, architecture, APIs, documentation, integration, validation, version control and security would go a long way to restoring your confidence. No need to be discouraged or too impatient. I believe that PGV is an outstanding product with outstanding support and a very promising future. Congratulations to a fabulous concept and a fabulous team.
I have heard several people offer to help in this area and I have said that we would love the help but I never said how you can get involved.
If you want to get involved in helping with code cleanup or with other areas of PGV, first send me an email (yalnifj at users dot sourceforge dot net) and include a specific area that you would like to clean up. Include a brief outline for how you plan to make your changes.
I'll get you access to the SVN and hopefully be able to get you in touch with other stakeholders who will have an interest in what you are working on.
As an example, if you would like to improve on the MVC in the program, pick an area of PGV where you think conversion to MVC would best help the program and other developers to build on PGV. A great example that badly needs it is the whole import process (I would love to see the ability to develop a command-line import tool).
Thanks,
--John
I hesitated to post to this thread as I have only been interested in PGV for a very short time. Perhaps my perspective will be of interest however. Feel free to reject my input as the ravings of a newcomer who does not know what he is talking about.
Firstly to reafirm that which has been said by others that PGV is a great package and a credit to those that have worked on it. The question being asked is where should it go now?
Greg Roach briefly mentioned the data model. I have just performed a global search for the string INDI and counted those files where the string appeared to be used in the context of the GEDCOM file format. Ignoring language files I came to a rough estimate of 67 files where a detailed knowledge of the GEDCOM file format is embedded. One could argue that data abstraction is even more important than MVC. For a newcomer to the code this is also an issue that makes the code difficult to understand. A typical example (from person_class) is:
$famlink = get_sub_record(1, "1 FAMC @".$family->getXref()."@", $this->gedrec);
$ft = preg_match("/2 PEDI (.*)/", $famlink, $fmatch);
Presumably at some stage in the future, whether in 1, 2, 5 or 10 years time an improved GEDCOM format will appear. Is this area of data abstraction for the GEDCOM format one that should be treated as a priority in the Code Cleanup? Conveniently of course what you end up with is a good chunk of what is known as the Model in the latest Three Letter Acronym. Good Lord I have just realised that I have been writing MVC code for 30 years and didn't realise. (Well the M part anyway, admittedly the V and C have had a tendency to get a bit entangled).
Although this would be a very significant amount of work, the first part (defining and coding the interface to the GEDCOM data) is new code and could be written more or less in isolation. Once it is functional then it's use could be gradually phased in one file at a time. Well that is the theory anyway, it is never quite that simple of course.
Colin
I agree with many of the conventions mentioned, but ask that the files retain the .php extension. If the webserver does not recognize the file type, it will do one of two things:
1) serve it as plain text. This would allow hackers who know the file layout to read our settings
2) serve it as a binary file. This prompts visitors to download the file. Once on their local machine, it is readable by any text editor.
Instead, I suggest two levels of extensions, based on the function of the file. As an example, the filename gedparser.class.php clearly shows that it contains a class used to parse GEDCOM.
I agree 110% that plugins / modules should not require editing any core file.
Well, that is certainly true: Anyone can read our files.
But since PGV is an open-source project, anyone can go to phpgedview.net, download the source and then know our file layout, and _even_: the contents of those files.
And because of that you put all the relevant settings in a file appropriately named config, which is the only file that needs to be protected - which is why it is mostly a .php file that displays nothing.
The thing that you do not want, is that people are able to execute all those files directly, which is a greater security risk than them reading them. And by making all files have a .php ending you just allowed them to.
Because then all you need is one file that does not rigorously check if it was executed or included and you have a much bigger security risk than you would have, had the user just seen the source code (which he can get anyway). Now the attacker can pass variables, try to set options etc. And if you are dangerous-loving enough and have "register globals" still on - well that's just too bad, you basically handed him your server on a silver platter.
-V
This is a most interesting thread. I read it with growing hopes, but, as it has developed, reality has taken over.
Using web-based data to generate home-based web pages is not very well known to many people - nor do they have the knowledge or aspiration to try to get it to work for them. Some of us have seen the possibilities (largely, but not only, people with some computing background) and we have enjoyed the experience. A talented group of developers have fed us, and we proudly showed our work to our families and other interested folk.
PGV, like all projects, started well, attracting developers..and in turn users, so creating impetus. And as PGV was not developed to a plan, it got a bit tatty at the edges - something that happens with all like projects, I should add. As the picture became clearer about users and their likes/dislikes, so the original PGV looked to be in need of a serious overhaul, even a restart because, unplanned, that's how things go. But gravity had taken over, and there was too much investment in the current model to contemplate anything major.
And, as the number of interested developers has tapered off, so to has the possibility of such an event taking place. Realistically, PGV is PGV. Apart from some titivation, there is no practical way to redesign the model, rewrite the code, redesign the UI or do other than minor edits and small code tidy-ups. That's not to discourage anyone from trying, just be aware of the limited scope that is available to you.
Paul
Australia
Remember that this thread is a developer level discussion. In the end users don't care what the code looks like under the hood. They just want something that is stable and works. And if you take a look at all of the Feature Requests, the vast majority of users are not demanding a code rewrite, nor would they want the development resources reassigned to one.
It is potential developers who want to customize or build on PGV that want more modular, more abstracted, code.
The fact of the matter is that this is a volunteer developed open source project and if you want to see these changes made, YOU are going to have to work up enough motivation to start the work. Honestly, I think that is the problem... nobody has enough motivation to do it.
I personally try to work some of these changes in with each release and with each major revision.
Based on the many priorities in the project, I am personally doing as much as I can to make PGV a better platform for people to build on.
--John
I think the wiki page is the way to go to keep track of what needs doing. Last night I was reading "UML Distilled" by Martin Fowler and came across words that were relevant to this subject.
"... when you add a new function to a program, you build on top of the existing program, often in a way that the existing program was not intended to support ..."
"Although in theory it is better to redesign your program, this usually results in extra work because any rewriting of your existing program will introduce new bugs and problems... However, if you don't redesign your program, the additions will be more complex than they should be... Gradually this extra complexity will exact a stiff penalty. Therefore, there is a trade-off: Redesigning causes short-term pain for longer-term gain."
He goes on to talk about refactoring techniques that make this process easier. A quick Google search comes up with more information on it.
Doing this kind of work I think is not at all contrary to OSS principles. Like John pointed out, people are volunteering their time to help on the project. If the project code is a mess, a potential contributor has to spend more time than should be necessary to understand the extra complexities unless they were extremely dedicated. Of course not many people are going to be willing to spend their time doing this so you end up losing these people.
One key to refactoring requires testing to make sure functionality isn't broken. Is there some sort of testing suite available for phpgedview? Perhaps making a series of tests should be the first priority to put up on the wiki page.
No, there are no formal testing routines, although I believe nathan has been attempting to develop some.
The current process for testing anything is to rely on PGVs large user base. In particular there are a number of users (including myself) who have been using PGV for some time (years generaly), have a range of differing installations (hosted web, own server, MAC, PC, etc etc); operate in languages from English to Hebrew and beyond; and have GEDCOMs containng from 100s of INDIs to over 40,000.
I'm certainly not suggesting a series of standard tests wouldn't be useful, just that there are additional options as well.
One of the defining components (at least according to Gregorio Robles and the 'Bazaar' model) is:
the following patterns:
"Users should be treated as co-developers
The users are treated like co-developers and so they should have access to the source code of the software. Furthermore users are encouraged to submit additions to the software, code fixes for the software, bug reports, documentation etc. Having more co-developers increases the rate at which the software evolves. Linus's law states that, "Given enough eyeballs all bugs are shallow." This means that if many users view the source code they will eventually find all bugs and suggest how to fix them. Note that some users have advanced programming skills, and furthermore, each user's machine provides an additional testing environment. This new testing environment offers that ability to find and fix a new bug."
One of the first things everyone interested in this topic should do is read some literature on OSS and how it should be managed.
You should start with the original classic by Eric Raymond entitled "The Cathedral and the Bazaar"
http://www.catb.org/~esr/writings/cathedral-bazaar/cathedral-bazaar/
This should help you understand why I manage the project the way I do and help you better understand the philosophies governing OSS development.
--John
An excellent suggestion John. Unfortunately like any great work (try the Koran for example) it is open to widely differing interpretations. Take this paragraph as an example:
<<Rather, I want to suggest what may be a wider lesson about software, (and probably about every kind of creative or professional work). Human beings generally take pleasure in a task when it falls in a sort of optimal-challenge zone; not so easy as to be boring, not too hard to achieve. A happy programmer is one who is neither underutilized nor weighed down with ill-formulated goals and stressful process friction. Enjoyment predicts efficiency.>> [for chapter "On Management and the Maginot Line"]
I respectfully suggest this explains the issues many have raised - especially the less competent programmers who have tinkered in the development arena - "ill-formulated goals and stressful process friction" and "...too hard to achive".
As the creator of PGV (and one who is extremely highly respected for that) - can you comment on the likely success and acceptability of the process occurring on the WIKI pages regarding this discussion?
I think if you read that article and compare it to PGV you will find lots of interesting connections including Linus' law and ego-less programming.
<<< As the creator of PGV (and one who is extremely highly respected for that) - can you comment on the likely success and acceptability of the process occurring on the WIKI pages regarding this discussion? >>>
I can comment based on my experiences in the open source world (which are not limited to just PGV by the way).
Unfortunately this is all talk, and the one thing that an open source project needs to be successful is code that people can play with. I've never seen a successful open source project which started out like this. Instead someone needs to take the initiative and do something and then make that code available. If it has "potential" then others will jump on it.
I've seen this process over and over again in the genealogy world as well. Take GEDCOM or XML as an example. Everyone wants to talk about it but that talk is never converted into anything that someone can use. All successful open source projects that I am aware of start out as an idea that someone implements, then others jump on and contribute. But the key is that there has to be an initial implementation.
The average person who gets involved in PGV just wants to "scratch a personal itch" and so they aren't going to be interested or involved in these high-level design discussions. And these discussions have come about because people who wanted to scratch a personal itch haven't been able to figure out how to do that and looking at the code and not understanding its history and why things are the way they are, suggest that the code is flawed or mismanaged.
Now I am not dooming this initiative nor do I think this discussion is a waste of time. I think that this discussion is healthy and helpful to PGV. For the most part these are good ideas. This discussion builds on other concepts from the article such as "parallel design" and the importance of having users.
There are many places in PGV where the code is flawed and needs to be rewritten to be more scalable or use a better OOP design. But in OSS the average volunteer developer is not going to be the one to do that.
What then does this mean for PGV and its future? It means that these ideas will be implemented slowly over time by people who have enough initiative and motivation to do it. There are less than a handful of PGV developers--maybe only 2 or 3--who I think would even start to take projects of this magnitude on. I am one of those, but my time is extremely stretched. Just this response has taken me at least 30 minutes to write, not to mention all of the other things I deal with everyday for this project. Fortunately I have students who can help with these kinds of projects.
Speaking of which, it is time for me to go to my next class. But let me end with my vision for the future of PGV and where I think PGV needs to go. My vision for the future of PGV is in "genealogy as a platform" and "PGV as a platform" that many people can build off of either through direct access to the PHP APIs, or through web services, or through mash-ups and widgets. I think implementing this vision would meet the needs which spurred this discussion as well as meet the UI needs and meet the future of the genealogy industry as a whole.
The future of the genealogy industry is an exciting one and PGV will continue to be a major force in realizing that future.
--John
John, I did not want to criticise your way of managing the project.
I fully understand that keeping this many developers pointed in the same/right direction over long periods of time is very hard - and even more to get them to right good code. Because PGV depends on people contributing you cannot turn down additions just because they dont hold up to an artificial (your) measurement of quality - that surely varies even from person to person. The problem is that people contribute code all over the place because one function of PGV is not centralized in one place.
That is why the issue of plugins and templates is critical.
Once you manage to create a clean and well programmed core that does not need to be changed for users to successfully be able to scratch their 'itch' as you put it, you can have them play with their own code in templates and plugins as much as they want. In this way you do not allow 'dirty' code into the core and still give everybody the ability to costumize their PGV to their liking.
And if you look at my two favorite role-projects (WP and G2) you will see that this is exactly the way they work.
Over time plugins will naturally converge on better code anyway, as their developers get more experienced.
I admit that this necessitates a slightly stricter reign over the core developers, as well as keeping their number as small as feasible - but is well worth the effort.
If you do want to take advantage of the "bazaar-style" approach as Linus did with Linux you have to keep in mind that developers contributing to the linux kernel had already reached a rather high coding skill in order to understand what was going on (even proved computer-savvy for getting Linux installed at all, back in the day) - ensuring that their contributions weren't badly written. And also that Linus and very few others still look over all the changes before they get accepted ensuring high quality.
I think the situation with PGV is different - as most of the users are not experienced programmers but still have the wish to scratch an itch - and therefore needs a slightly different handling.
PGV does have a "core" as you call it, it's just hard to see it for all of the clutter. It is composed largely the files in the includes directory. Everything else is a plug-in that uses the "core" APIs. The code is in need of refactoring not throwing away.
The thing that is not modular enough is the UI. If I add a new file, the UI is not going to automatically pick up on it. Unfortunately that is the thing that people most want to scratch, and rightly so. I believe you should be able to present and honor your ancestors in the way you want.
I am trying to make the UI more modular. I have a team of students right now working on making the tabs on the indi page modular. I have tried many times to make the menus modular... but I am afraid that is one area I will have to completely scrap and start over on. The themes are already modular, and you can do a lot with the themes, again the menus being the biggest problem with the themes.
There is also a lot of legacy code, most of the older charts, etc, were written using the old APIs. They just need to be rewritten or thrown out. This old code is serving as a poor model for new developers.
There was also a time when a previous developer and I disagreed. I was pushing for more MVC, more separation, and he was going around combining files and taking out separation and making things less modular.
This discussion of a "core" has come up before as well as the notion that we have to protect the core and we shouldn't let anyone modify it (ironically with the same developer I was talking about above). If we did "protect the core", then Greg wouldn't be able to make the DB changes he is proposing in the other thread. I also don't want to setup some hierarchy that says one developer is better or more valuable than another.
My philosophy is that if code really is "Core", then it naturally won't be changed because it won't need to be changed. I don't consider anything in the UI to be "core".
>> If you do want to take advantage of the "bazaar-style" approach
I do want to use the open source or "bazaar-style" for 2 reasons:
1. I believe it produces better products faster
2. It allows me the flexibility of not having to be a hands-on manager which I don't have the time to be
I believe that once the majority of the legacy code has been fixed and updated, and good documentation produced, the new code will serve as a model for new developers and will encourage them towards producing good code themselves no matter what there level or experience.
--John