Menu
▾
▴
Re: [GeoNetwork-devel] db migration from 2.8/2.10 to 3.0 ?
|
From: Landry B. <br...@cr...> - 2015-01-28 15:20:08
|
On 01/28/15 15:52, Jesse Eichar wrote: > I use the migration in geocat regularly. I use the postgis version. > Also the tests test the migration from 2.8 -> 3.0 for h2. In geocat > there are 6000+ metadata 200+ groups, who knows how many users as well > as 10-12 harvesters. This is to say that the migration works in some > cases but it very complicated and no doubt needs more testing. What I > did for testing was: > > * Make a back up of the database > * Put break points in the DatabaseMigration class especially in the > catch clauses. > * Start server > * find error and fix > * replace partially migrated DB with backed up version > * repeat That procedure looks fine when debugging, but in my case after enabling all logging (ie geonetwork logger at DEBUG level) i see zero message from the DatabaseMigration test, which seems to hint that it's not even reached, the error happening much earlier at hibernate initialization.. which also tries to apply some schema updates to the database ? Maybe this is because the new fields are nullable=false but no default value is provided ? As a workaround i tried (on top of a copy of my 2.8 db) to add the missing fields: => alter table Settings add column internal char(1); => alter table Settings add column position int4; => alter table UserGroups add column profile int4; if i do this, the original hibernate error about those 3 fields disappear, but not of the migration scripts are run anyway, and stuff blows later on because the schema doesnt match what hibernate expects. Whatever i do, i get none of the 'webapp (sub)version/database (sub)version' messages in the logs, so something else is deeply wrong. Landry > On Wed, Jan 28, 2015 at 2:36 PM, Landry Breuil <br...@cr... > <mailto:br...@cr...>> wrote: > > On 01/27/15 12:43, Landry Breuil wrote: > > On 01/27/15 12:24, María Arias de Reyna wrote: > >> On Tue, Jan 27, 2015 at 12:17 PM, Landry Breuil <br...@cr... > <mailto:br...@cr...>> wrote: > >>> Hi, > >>> > >>> i know this is still in development, but has anyone tried > starting a > >>> 3.0.0-SNAPSHOT on top of a copy of a production database > running 2.8 or > >>> 2.10 ? Here, GN fails to start with JPA/hibernate errors: > >>> > >>> INFO [jeeves.apphand] - - Z39.50... > >>> ERROR [jeeves.engine] - Raised exception while starting the > application. > >>> Fix the error and restart. > >>> ERROR [jeeves.engine] - Handler : org.fao.geonet.Geonetwork > >>> ERROR [jeeves.engine] - Exception : > >>> > org.springframework.dao.InvalidDataAccessResourceUsageException: could > >>> not extract ResultSet; SQL [n/a]; nested exception is > >>> org.hibernate.exception.SQLGrammarException: could not extract > ResultSet > >>> ..... > >>> Caused by: org.postgresql.util.PSQLException: ERROR: column > >>> setting0_.internal does not exist > >>> ..... > >>> > >>> It seems none of the migration scripts from previous versions > to 3.0.0 > >>> are run - i dont see any 'WARN [geonetwork.database] - > Executing SQL > >>> from:' messages before the actual catalog startup, as it is the > case > >>> when starting from an empty database (which of course works fine). > >>> > >>> version is 2.8.0 in the database. > >> > >> > >> Hi Landry, > >> > >> As you noted, version 3.0.0 is still in development and there are a > >> lot of changes involved. That's why we decided to name it 3.0.0 > >> instead of 2.12.0. This means, it is not an easy migration as it was > >> upgrading between 2.x versions. > >> > >> There are things that are not yet finished but can be worked around. > >> For example, if you are using shapefiles for indexing metadata > >> (instead of PostGIS), I would recommend you to remove shapefiles and > >> make geoNetwork rebuild them again. > > > > Yeah, removing the index causes no issue. > >> > >> About the database, if I were you, I would move first to version > 2.10 > >> (latest stable) and see if migration succeed. If it does, then I > would > >> try to move to 3.0.0. I know that the expected behaviour is to move > >> directly to 3.0.0 and have everything working, but as said, it is > >> still unstable and not all things are polished. > > > > I know, that's why i'm testing the upgrade path in advance to debug > > it... looking a bit more, there's another message which might > give hints > > on why the migration fails: > > > > ERROR [org.hibernate.tool.hbm2ddl.SchemaUpdate] - HHH000388: > > Unsuccessful: alter table Settings add column internal char(1) > not null > > ERROR [org.hibernate.tool.hbm2ddl.SchemaUpdate] - ERROR: column > > "internal" contains null values > > ERROR [org.hibernate.tool.hbm2ddl.SchemaUpdate] - HHH000388: > > Unsuccessful: alter table Settings add column position int4 not null > > ERROR [org.hibernate.tool.hbm2ddl.SchemaUpdate] - ERROR: column > > "position" contains null values > > ERROR [org.hibernate.tool.hbm2ddl.SchemaUpdate] - HHH000388: > > Unsuccessful: alter table UserGroups add column profile int4 not null > > ERROR [org.hibernate.tool.hbm2ddl.SchemaUpdate] - ERROR: column > > "profile" contains null values > > > > I'll try to manually apply the migration scripts one by one, but > i dont > > see how hibernate can at the same time want to add a new empty > column, > > *and* enforcing it to be not null.. unless there should be a default > > value ? At least that's what sql/migrate/v2110/2-migrate-default.sql > > seem to do. maybe hibernate enforces not null in the code 'a bit too > > early' ? > > Manually applying all the sql upgrade scripts in the correct order > produces a working gn3 (minor some annoyances/tweaks, see issue #736) > but it seems the 'auto-migrate-database-version' process is broken atm > in develop head. > > No script is run at all, since hibernate errors out very early in the > startup process, and afair the migration scripts are run later on. > > Did anyone try an actual 2.10 -> 3.0 migration test ? > > -- > Landry Breuil > Mouton a 5 pattes du CRAIG > > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming. The Go Parallel Website, > sponsored by Intel and developed in partnership with Slashdot Media, > is your > hub for all things parallel software development, from weekly thought > leadership blogs to news, videos, case studies, tutorials and more. > Take a > look and join the conversation now. http://goparallel.sourceforge.net/ > _______________________________________________ > GeoNetwork-devel mailing list > Geo...@li... > <mailto:Geo...@li...> > https://lists.sourceforge.net/lists/listinfo/geonetwork-devel > GeoNetwork OpenSource is maintained at > http://sourceforge.net/projects/geonetwork > > -- Landry Breuil Mouton a 5 pattes du CRAIG |