ADempiere ERP Business Suite

https://sourceforge.net/forum/message.php?msg_id=5784201

Hi - I just wanted to cross post this to make sure eveyone was happy with this. I've proposed to re-structure the migration directory to be - [database type]/[version]/[patch] instead of [version]/postgres/[patch] and [version]/[patch]

The idea is that you can do a svn checkout just for your database type and we could add new databases types within the proposed structure.

Mike

I don't like the proposal.

Anyways I understand your "problem" (if it can be considered a problem)

My counter-proposal would be to move oracle scripts to a subdirectory called oracle.

Example:
Currently we have:
352a-trunk/349_MarkBetaFunctionalities.sql
352a-trunk/postgresql/349_MarkBetaFunctionalities.sql

Then my proposal is to move the first line this way:
352a-trunk/oracle/349_MarkBetaFunctionalities.sql

Note: if you plan to do this move then you need to take care of not breaking the nigthly-build strategy in build.xml files there


__________
Carlos Ruiz

The reason I proposed the database type first is it is easier to check out from the svn if it is recursively all in one place rather than in separate subdirectories.

One of the issues for us has been migrating existing customers (which may have extensive mods for previous versions). We have been working to extract these and package them around the core so we can update clients. We do migrate people through two versions at a time and this would need two or more svn checkouts with your nomenclature.

What would be the benefits of using version/database/patch over database/version/patch?

Thanks for the tip on the build - I intended to look at that.

Mike

Hi,

> The reason I proposed the database type first is it is easier
> to check out from the svn if it is recursively all in one place
> rather than in separate subdirectories.

Yes, it's just the point of view of the downloader - in my case I find easier to download all the scripts from a specific version instead of traversing several directories to get them.


> One of the issues for us has been migrating existing customers (which may have
> extensive mods for previous versions). We have been working to extract these
> and package them around the core so we can update clients. We do migrate people
> through two versions at a time and this would need two or more svn checkouts
> with your nomenclature.

My idea is to release one single script to migrate between stable releases - like 320-340/320-340_or.sql and 320-340/postgresql/320-340_pg.sql

Just between stables.


> What would be the benefits of using version/database/patch over
> database/version/patch?

As I said version view instead of db view.

Comparing scripts (to found potential bugs) is also easier if they're close.


> Thanks for the tip on the build - I intended to look at that.

Another tip - there are two scripts in migration directory called migrate.sh and migrate_postgresql.sh - you need to take care also to avoid breaking such functionality.


Cordially,

Carlos Ruiz

Hi!

Now that migration scripts are getting some attention I would like to ask about the numbering strategy.

What I mean is that for example if you are trying to help testing the trunk frequently there some scripts:

"Placeholder"

that's mean that those scripts will change in the future?

Another issue that I have faced is when something went wrong with an script. The same script gets modified rather than creating a new one.

I would like to hear some comments about those issues.

Regards!

I believe that:
> "Placeholder"
means a number has been reserved by a developer for a script which is part of a commit that will be made in the future.

> Another issue that I have faced is when something went wrong with an script. The same script gets modified rather than creating a new one

The rule is that you should not fix the old scripts once committed but create a new script to fix your old one. I believe this is a 'rule' which developers with commit rights should be aware of.

Mike

> Yes, it's just the point of view of the downloader - in my case I find easier to download all the scripts from a specific version instead of traversing several directories to get them.

At the moment I do a:
svn co https://adempiere.svn.sourceforge.net/svnroot/adempiere/trunk/migration Adempiere/migration
(so I get all scripts)

If we organise version/database the developer will always have to download each specific version like this:

svn co https://adempiere.svn.sourceforge.net/svnroot/adempiere/trunk/migration/316-318/postgresql/ Adempiere/migration/316-318

or all databases like this:

svn co https://adempiere.svn.sourceforge.net/svnroot/adempiere/trunk/migration// Adempiere/migration/

(Which is the 'problem' I have now)


If we organise database/version then the developer can choose to get

svn co https://adempiere.svn.sourceforge.net/svnroot/adempiere/trunk/migration/postgresql/ Adempiere/migration
(all postgres migrations)

or

svn co https://adempiere.svn.sourceforge.net/svnroot/adempiere/trunk/migration/postgresql/316-318/ Adempiere/migration/316-318
(just one version)

>My idea is to release one single script to migrate between stable releases - like 320-340/320-340_or.sql and 320-340/postgresql/320-340_pg.sql
>Just between stables.

That's a good idea - perhaps these could be in a different tree - so people who need nightly's can get scripts as they are committed to a tree with a svn update but others who work off stable could get the summarised patches more like traditional software release.

> Comparing scripts (to found potential bugs) is also easier if they're close.

In the db/version/script method, the scripts are still close together unless you are talking about both oracle and postgres - in which case they are an extra directory away.

> Another tip - there are two scripts in migration directory called migrate.sh and migrate_postgresql.sh - you need to take care also to avoid breaking such functionality.

Thanks - I use migrate_postgresql.sh - it's a good helper script.

Could people please review this and advise if they are happy with this change or not.

One further issue is what do we do with the directories "dist", "src", "tools_migrate_with_insert" - should they be moved down a level or just left where they are?

I've done the work incorporating Carlos' hints. Ready to commit if people vote in the positive.

Regards,

Mike

Hi everyone,

I like well organized things.
[database type]/[version]/[patch] is fine for me.

This also makes thing more clear for new developers as they can see which are oracle and which are postgre sql scripts.

[+1]

Trifon

This thread seems to be going the right way, taking ideas from others and thinking carefully about impact.

+1 vote.

About src/dist placement here, i wonder if it is redundant. Or its confusing in naming structure from migration tree <-- that is what you _are_ doing aha... so ok.. give your view, as long as it makes better common sense.

red1

hi,

I would prefer a version view instead of db view, i.e version/db-type/sql script.

The version view is more convenient for on going trunk maintenance as it is frequently needed to compare script of different db of the same version. The only benefits I can see for the db-type/version approach is you can check out all migration script for a db-type in one svn co command, something that people rarely need to do. Most of the time, you would need script for 1 to 3 version, i.e using the version driven view, you will usually need just 1 to 3 svn co command to get the db specific script you need, hardly a problem for me.

Finally, unless there is very good reason of doing it, we should avoid such changes that require everyone to do a massive svn update, I means it could be annoying especially for people with slow connection.

Regards,
Low

+1 to reorganize.

I think I prefer version/db/sql -- but either way would work.

Paul

Hi,

> This thread seems to be going the right way, taking ideas from others
> and thinking carefully about impact.

No, this thread is not going the right way - the change was done with just one vote - and it was a negative vote!

Michael said he was going to revert but it's not reverted - so I'm reverting it at this moment.

Now, the "votation" is strange - it looks like we're voting for three different things:

1 - organize better or leave as it is
I see until now all of us who voted are ok with a reorganization

2 - version/db/sql
it's preferred by three voters (Heng Sin, Paul and me)

3 - db/version/sql
it's preferred by two voters (Michael, Trifon)

Redhuan didn't say his preference

The option 3 was committed without expecting more votes - and without considering the votes of the people with more commits here.


Anyways, I'm reverting - I'm pro for a reorg but properly done.


Peacefully,

Carlos Ruiz

Yes Carlos,
I did not state my preference because i agreed with the whole thread and its direction. I voted 'Aye' because i agree with the whole sum of the thread plus the associated tracker.

I say, 'going the right way' is because everyone here is:

1) Participating, saying 'Aye' or 'Nay'
2) Giving their reasons why so.
3) Tracker has a commit and a revert request and an agreement by the committer to revert.

So i like to see that going on.

How Am I Thinking? :-)

red1

Hi Redhuan,

Thanks for the explanation - I have said in past MANY TIMES that reverting is very bad.

I don't like to revert - do you think I like to spend 1 hour reorganizing the things as they were before?
Well - this is the time I'm spending on this issue - and I haven't still finished.

But there are rules here - committing things (not-bugs) need votation and asking for community wisdom - and top committers must have "preferent" vote against just lurkers - etc.

Pushing this type of things is calling for a flame about revert. For me it's just obvious Michael is not following the established rules of trunk and just trying to provoke another flame war.

Anyways I'm acting fairly - following established rules - and following non-finished votation.

I'm really tired on this - people say that I must take care of trunk - just read the PMC role (even with pathetic votation I consider this my responsibility).
But HOW CAN I TAKE CARE of trunk if people rejects to follow SIMPLE RULES.

Please don't push me to revert - just follow the rules - it's something simple.

Anyways - I'm really tired about all this flaming and fighting here.
It's just stupid to keep thinking I can be a PMC just to blame - but not to follow or respect.


Peacefully,

Carlos Ruiz

Ok, that means (i assume) Mike did not count the votes well and proceeded. Now, that will be breaking the voting rule.

"Votes counted or in progress must be in favour of the action before one can proceed to act upon it".

I got a side question. What if Mike did get winning vote. Is not it still open to veto counter vote? I mean by veto is:

1) There is very valid reason from a single argument that the action is fatal.

2) PMC has reason to go against a majority vote because it knows better.

3) CC has reason.

And i mean by go against not necessarily means full opposition. It may be asking for more arguments and delay decision.

On a side note (when i can find time later) I will try to formulate all these into the Project Charter in the open wiki. I will also try to incorporate the other rules you already proposed recently. Its open to voting there if ppl want, otherwise it is pathetically accepted.

Hi Redhuan, thanks you pointed to this topic.

Votation system here has big flaws. I suppose that's something we need to review if we want a better project.
There are "professional" voters here :-) I mean - there are people here that just like to push the +1 button without thinking the consequences, or even understanding what's going on.

Another thing that I've criticized in past - it's just silly to consider valid a votation for a wide open question like "do you like insurance vertical added to the trunk?"
Well - these type of questions are mostly (and obviously) voted with +1 - but, what are the consequences? is it properly written? is documented? is there a maintainer? and so on.

I think we have a big flaw with the current votation process.

And the votes must be different in democracy than meritocracy - AFAIU open source is based on meritocracy, not in democracy.
Again, what does it mean a vote +1 from a lurker that never have seen one line of code on the project?
Or how can we measure the vote of somebody that doesn't help to maintain code - against the vote from a constant maintainer of the project? Whom vote is more trustful?


Anyways - I'm just suggesting - we have what we have until now - but it's good to review things that are not working.


Regards,

Carlos Ruiz

Carlos,
This meritocracy and peerocracy is not new. It was raised by Linus and Jimmy Wales before.

I also on day one ask that democratic majority not be the deciding factor and that a lone counter voice be heard or exhausted before a decision is made. This is for the exact reasons of unequal merit to a probable vote you raised above.

I even ask that Council be not fit for voting for new rules and best to leave to the wisdom of the community. After all, they all have their merits in their own ways.

In Islam there is a principle "Amanah Lil Ahli" - a trusted task to its expert. For trunk probably we can ask someone who shown to be an expert to decide on it. For Accounting perhaps its a very proven accountant. For wiki again a good editor and so on.

My role as pathetic (leader) is to hold the fort and let the bazaar grow. This is my utmost strategy. My 2nd priority is to harness focus and speed forth. But it is 2nd, not 1st. If we cannot be popular and suscribed by the most of the mob we cannot be successful.

It is like a business. If it is good and perfect in all its rules but no one actaully likes it anymore then that business is sinking. We remember Compiere how it was most popular, but its policies are not friendly and it call us unfriendly names, even though we paid and helped its forums.

Thus it is to me more of an issue of priority. There is one other thing i must not forget. It also has to be fun. Now of course i enjoy the project a lot. Otherwise why would i spend hours on end and forgo sleep even just to speed up replies and tutorials to newbies who wouldn't pay me a single cent. It is a passion.

So i understand that the passion can be pain in the ass, if people or users do not listen to us and break things that we provided for them. In that respect i have withheld specific vote and rather vote generally.

But i think humans are reasonable and of sound mind (those that can afford to read this thread up to this excurciating length). They are more human if they believe in their weakness and ask for advice and are willing to understand others first.

Please advice.

red1

I wish to cast the ballot if the box is not closed.

+1 vote to version/db path

I agree because i try to visualise how i or any dummy would want to use the migration folder.

I would go to the version and see that the scripts are there. Fine. Then, like Heng Sin said, i probably want to do stuff like comparing an Oracle script to other DB scripts.

Now in the earlier Mike proposal, i may have to struggle back and forth between higher DB folders and zoom back into the version script. Also i wont be sure if a script is missing and like to peek at the adjoining room.

Quick Mike, play dead.

red1

Hi Redhuan, you and me again philosophizing on this project :-)

There is one thing I want to expand:

> There is one other thing i must not forget. It also has to be fun.

This statement has two sides:

1 - Children having fun break glasses - so we as parents put the glasses far from their hands until them grow and learn how to avoid breaking - and for fun we give them plastic vases instead.
That's what I keep saying about trunk - please play with experimental - but please avoid breaking trunk.
Fixing trunk is not a funny task.

2 - It must be fun for everybody - I'm loosing the fun here - Karsten recently criticized me because of my bad mood.
Yes, this project is becoming boring for me - it's not funny to revert - it's not funny to enforce rules. I'm tired about all of this.

Some time ago I tried to quit silently and go to work alone in branches/stable - and it was funny for some time.
And then people followed me - people that I LIKED there - but also people that I didn't like there, and the fun ended soon.
And now people asked me to return stable to become trunk - and I'm exactly at the same situation. Others having fun - and I'm here like the boring person enforcing rules, etc.

Yes, I know, it will be a lot easier for me (and funnier for everybody) if I simply keep quiet and let the mess grow (like the "do nothing" mantra I don't like).
I don't consider this good for the project. Entropy will consume us.

Who knows? Maybe I was wrong.
Giving the benefit of doubt I preferred to work out of trunk, and wait for the mess to organize alone (negentropy) - but it didn't happen.

And it seems I'm going to do the same again (just that this time I won't be alone - I'll be with people that I like to work with).
I must have fun also here, and recover my good mood.


Regards,

Carlos Ruiz

Hi Redhuan,

> +1 vote to version/db path

Please somebody with better bandwidth than me revert revisions 7620 to 7624 and 7626.

Or change the things to version/db (I suppose it's easier to revert first - and then try to make the changes - there are implications in some scripts and build.xml files)

If somebody with better bandwidth can help me with the revert - thanks a lot!
If not I'll try tomorrow - my revert broke in the 80% of completion.

(Or maybe doing the revert in parts can do the trick)
BTW, Paul and Victor already uploaded migration scripts that need to be moved to the proper directories.


Peacefully,

Carlos Ruiz

Well Carlos, it is not fun if one child does not wants to play anymore because s/he kept been scolded or put into one corner away from the other kids.

For me early on, my greatest motivation of joining such an Open Source project was:

1) Learn alot faster from the gurus.

2) Teach the others so that they thought you are also a guru.

I managed to achieve that though i can say that its never-ending. There will always be new gurus or new stuff to learn and idiots and newbies to teach.

But more importantly i know that i have a lot of catching up to do on (1). (2) is no problem as u can see from my shortened guestbook and status in the bazaar.

So i focus on my core strength, playing the role of social artist rather than guru. I don't really care if the newbie breaks anything as long as everyone plays the game.

Thus i feel very sad if you feel otherwise and wish to confine yourself to another branch (again) with the other gurus. It is as if you do not let me play with the real nice glasses anymore.

How i wish there is some sort of playground where all the kids can play and no one can get hurt.

red1

Since Mike already agreed and may not be able to get to this atm, i can try to revert that.

But first, what is the best way to do that? Any Eclipse friendly commands i can use?

Red1,

> How i wish there is some sort of playground where all the kids can play and no one can get hurt.

There is!

https://sourceforge.net/softwaremap/trove_list.php?form_cat=80

19,801 SF projects that are not enterprise class business systems! ;-)

Have fun, Joel

Firstly, I will do the revert and I will sort out the new commits as well - as previously agreed. I tried to do this last night, but when I got up this morning I see it failed.

Secondly, thanks Carlos for moving the policy discussions to a new thread. This allows us to focus on two questions and two options here:

Question 1: Could the migration directory do with some housekeeping?

So far, I think we are agreed about this point as it will help people understand the directory better and the current structure is only as it is because it grew that way from when we only had one database. Please say if you do not agree with the need/opportunity for some housekeeping.

Question 2: How should it be done?

Option A: database/version/patch.sql - advantages are that svn is recursive and allows a user to check out everything for their target database or just the version or single file. It would be a very unusual setup for a production server to have two databases on it - so a script would usually only want data from one database - hence promoting database to the top level. The exception to this is the developer (who must be experienced in Adempiere already) who has to traverse an additional directory in order to copy the patch to each database type (currently 1)

Option B: version/database/patch.sql - advantage it is easier for developers.

So perhaps we can vote on option A or B - or indeed C or D if anyone has some other ideas. As it stands, I think the issue is whether we structure this to be more intuitive for an experienced developer or for users.

My vote +1 is for Option A.