Task #5770

Upgrade SDI to Geonetwork 3.2

Added by Fabio Sinibaldi almost 3 years ago. Updated 10 months ago.

Status:In ProgressStart date:Aug 30, 2018
Priority:UrgentDue date:
Assignee:Fabio Sinibaldi% Done:

98%

Category:Data Management
Sprint:Data Catalogue
Infrastructure:Development, Pre-Production, Production
Milestones:
Duration:

Subtasks

Task #12358: Fix gcube-app.xml template for GN 3.2.1Closed_InfraScience Systems Engineer

Task #12426: Test DM integration with GN 3.2.1ClosedGianpaolo Coro

Task #12427: Check Preproduction environmentClosedGianpaolo Coro

Task #12428: Test DM integration with GN 3.2.1 in PreprodClosedFrancesco Mangiacrapa

Task #12429: Update Preprod to GN 3.2.1ClosedFabio Sinibaldi

VM Creation #12412: 3.2.1 Geonetwork for preprod environmentClosed_InfraScience Systems Engineer

Task #12511: Register GeoNetwork 3.2.1 in preprodClosedFabio Sinibaldi


Related issues

Related to BlueBRIDGE - Project Activity #5679: Publishing geospatial datasets: testing the IRD workflow Closed Apr 26, 2017
Related to D4Science Infrastructure - VM Creation #9558: Updated VM for GN3 Migration Closed Aug 31, 2017
Related to D4Science Infrastructure - VM Creation #9702: GeoNetwork 3 dedicated instance for Tuna Atlas Closed Sep 15, 2017
Blocked by gCube - Feature #5769: GeoNetwork 3.0 support Closed Nov 15, 2016
Blocked by gCube - Feature #16706: geonetwork migration utility to manage public files Closed May 16, 2019

History

#1 Updated by Fabio Sinibaldi almost 3 years ago

#2 Updated by Pasquale Pagano almost 3 years ago

  • Priority changed from Normal to Urgent

#3 Updated by Fabio Sinibaldi over 2 years ago

#4 Updated by Emmanuel Blondel about 2 years ago

#5 Updated by Emmanuel Blondel about 2 years ago

#6 Updated by Emmanuel Blondel almost 2 years ago

@pasquale.pagano@isti.cnr.it @fabio.sinibaldi@isti.cnr.it Any news on that? I see the ticket is set as 'urgent' since 10 month.. In our work on FAO-IRD Tuna Atlas, this became a blocker. Indeed, the HTML display in GN2 is not as powerful as in GN3, and we are not able to visualize HTML metadata views given the richness of metadata (an GN2 internal issue with parsing XML..). We are at now constrained to do an harvesting from a IRD GN3 instance, unfortunately only for testing, and we reached the point where we need ASAP to share links to stakeholders in FAO, IRD and RFMOs.
Can you tell us if we could get a GN3 instance for Tuna Atlas, and if yes, when?
Thank you so much in advance

#7 Updated by Fabio Sinibaldi almost 2 years ago

Hi Emmanuel,
indeed the upgrade of GN took more time then estimated, also because the original development plan and SDI architecture changed according to communities needs. However, we should be able to provide such version from gcube 4.7 which will be released later this month.

#8 Updated by Fabio Sinibaldi almost 2 years ago

@emmanuel.blondel@fao.org we discussed internally on the subject and decided to try to provide the updated version of GN for Tuna Atlas as soon as possible, prior the gcube 4.7 release. I'll update the progress here on this matter.

#9 Updated by Fabio Sinibaldi almost 2 years ago

#10 Updated by Fabio Sinibaldi almost 2 years ago

  • % Done changed from 0 to 50
  • Status changed from New to In Progress

The first configuration and migration tests from tuna atlas VRE's GN to the newer GN 3 instance in development infrastructure were successful.

NB : both marshalling (geoapi) and metadata validation (geonetwork webservice side) were disabled in order to deal with metadata files found in tuna atlas VRE's geonetwork.

#11 Updated by Emmanuel Blondel almost 2 years ago

Thanks Fabio. For your GN tests you can use the geonetwork metadata, but please provide us a empty GN3 instance as we will have anyway to run our workflow to see that publication is running well with Geonetwork 3, and then republish everything with our latest workflow changes. Thank you

#12 Updated by Paul Taconet almost 2 years ago

Hi,
Do you think the GN 3 instance can be installed and ready by the end of next week?
Thanks,
Paul

#13 Updated by Fabio Sinibaldi almost 2 years ago

Hi Paul,
yes. I'm confident that GN 3 for tuna atlas will be ready to be used by the end of the week, probably before. I'm creating the ticket for a new machine right now. Once the service will be ready and configured in our Information System, the previous existing one will be dismissed.

#14 Updated by Paul Taconet almost 2 years ago

Thanks a lot Fabio. Waiting for your update then,
Paul

#15 Updated by Fabio Sinibaldi almost 2 years ago

#16 Updated by Fabio Sinibaldi almost 2 years ago

In order to provide the updated service to Tuna Atlas VRE, incompatibility with our library has been detected in 3.0.x versions.

3.2.1 works well, but 3.2.2 introduced the adoption of CSRF token, which requires additional logic in order to be handled.

#17 Updated by Emmanuel Blondel almost 2 years ago

FAO FI recently funded upgrade of geonetwork-manager to support GN3 REST API with backward compatibility for the other versions (2.6.x and 2.10x series). It was done by our external FAO partner Geosolutions IT (who developed the the geonetwork manager library used in your gCube wappers). The Java API was successfully tested with FAO GEMS tool which relies on geonetwork-manager. A maven release has been done (so you can access it).

I suggest you to upgrade geonetwork-manager if not yet done, to stick on more stable version of GN3 3.0.x, and not to enter to earlier in support the new REST API introduced with GN3.2.x since it's still in beta. If they are any issues related to geonetwork-manager Java client in interaction with GN 3.0.x series, please forward us so we could report them to the originator of geonetwork-manager on which you are mainly relying on

#18 Updated by Fabio Sinibaldi almost 2 years ago

Hi @emmanuel.blondel@fao.org,

The adoption of geonetwork-manager.1.4-SNAPSHOT is as old as this ticket (version was suggested by GeoSolutions team), and previous versions have been used to interact with geonetwork service since its adoption. Problem is that this library doesn't seem to offer ways to deal with users and groups, functionality that we had to implement extending the library provided by GeoSolutions.
The REST API is well documented from 3.2 so we've been able to develop on it, but in 3.0 it's more obscure.

In case I'm missing something, please tell me. It would solve lots of problems.

#19 Updated by Emmanuel Blondel almost 2 years ago

Hi Fabio, you can have a look to this file https://github.com/geonetwork/core-geonetwork/blob/3.0.x/web/src/main/webapp/WEB-INF/config/config-service-xml-api.xml that shed a bit of light on services for groups.

For example, you can retrieve the groups with /geonetwork/srv/eng/xml.group.list?_content_type=xml (GET). This request gives you a way to retrieve the internal group ID given a group name.

In https://github.com/geonetwork/core-geonetwork/blob/3.0.x/web/src/main/webapp/WEB-INF/config/config-ui-metadata.xml there is some information for assigning privileges to metadata with md.privileges.update (here I suspect an issue in geonetwork-manager that use md.privileges instead, that i had to deal with in R geonapi). md.privileges is the GET method to get privileges of a metadata, md.privileges.update is to assign/update privileges.

With this method you can specify the groupId, previously retrieved from above method.

Apparently you can also switch a metadata from one group to another with md.group.update but I didn't try.

I agree with you that it's obscure, and to be honest I don't really like GN3, too opaque,not enough documented and backward compatibility is broken in several features. The UI is not behaving well neither, they switched to AngularJS but badly handle Angular directives.

#20 Updated by Fabio Sinibaldi almost 2 years ago

Thanks a lot @emmanuel.blondel@fao.org!!
With these information it will be easier to adapt the library to this version as well. And yes, I also have noticed a strange behavior on the GUI.

#21 Updated by Fabio Sinibaldi 12 months ago

  • Subject changed from Upgrade SDI to Geonetwork 3.0 to Upgrade SDI to Geonetwork 3.2

The more stable version 3.2.1 has been chosen for the next SDI upgrade. This won't affect dedicated instances endorsed with GN 3.0 unless otherwise requested.

#22 Updated by Fabio Sinibaldi 11 months ago

  • Due date changed from Aug 30, 2018 to Sep 07, 2018

due to changes in a related task

#24 Updated by Fabio Sinibaldi 3 months ago

  • Blocked by Feature #16706: geonetwork migration utility to manage public files added

Also available in: Atom PDF