VM Creation #9702

GeoNetwork 3 dedicated instance for Tuna Atlas

Added by Fabio Sinibaldi over 1 year ago. Updated over 1 year ago.

Status:ClosedStart date:Sep 15, 2017
Priority:UrgentDue date:
Assignee:_InfraScience Systems Engineer% Done:

100%

Sprint:SDI-nodes management
Infrastructure:Production Applications list and version:

geonetwork v 3.2.2

RAM:5 Applications context path:

geonetwork

CPU:2 External storage data disk:

DiskSpace:10 Smartgears auth connector:
Smartgears:Yes data transfer enabled:No
Smartgears Version:LATEST PostgreSQL database:Yes
Application Instructions:

PostgreSQL database needs postgis extensions

Number of VMs:1
Smartgears scopes:

/d4science.research-infrastructures.eu
/d4science.research-infrastructures.eu/gCubeApps
/d4science.research-infrastructures.eu/gCubeApps/FAO_TunaAtlas

Milestones:
Duration:

Description

A VM hosting GeoNetwork 3 is required in Tuna Atlas VRE.
Suggested host name :
geonetwork-tunaatlas-d4s.d4science.org


Related issues

Related to D4Science Infrastructure - Task #5770: Upgrade SDI to Geonetwork 3.2 In Progress Aug 30, 2018
Related to D4Science Infrastructure - VM Creation #9830: New couple of reverse proxies to redirect and (or) balanc... Closed Sep 27, 2017
Blocked by D4Science Infrastructure - Task #9668: Produce ansible role for GeoNetwork 3.x Closed Sep 12, 2017
Copied to D4Science Infrastructure - VM Creation #9779: GeoNetwork 3 dedicated instance for IOTC SS3 Closed Sep 15, 2017

History

#1 Updated by Fabio Sinibaldi over 1 year ago

  • Related to Task #5770: Upgrade SDI to Geonetwork 3.2 added

#2 Updated by Fabio Sinibaldi over 1 year ago

  • Blocked by Task #9668: Produce ansible role for GeoNetwork 3.x added

#3 Updated by Tommaso Piccioli over 1 year ago

  • % Done changed from 0 to 30
  • Status changed from New to In Progress

Host name will be "geonetwork2-tunaatlas.d4science.org"

#4 Updated by Andrea Dell'Amico over 1 year ago

  • RAM changed from 2 to 5
  • % Done changed from 30 to 100
  • Status changed from In Progress to Feedback

http://geonetwork2-tunaatlas.d4science.org/geonetwork/ is available. The ssl certificate will be emitted in the next days.

#5 Updated by Emmanuel Blondel over 1 year ago

Hi @andrea.dellamico@isti.cnr.it , awesome! Could we have the credentials, so we could start performing our GN3 publication tests? Thank you

#6 Updated by Fabio Sinibaldi over 1 year ago

Hi @emmanuel.blondel@fao.org,
as I was trying to register the new instance in the infrastructure, I've encountered an issue with the latest adoption of CSRF token. I didn't expect a similar change between minor versions. We are going to revert to version 3.2.1 today, so I can register it to the infrastructure in order to let users gather credentials via the sdi service.

@andrea.dellamico@isti.cnr.it Let's keep the 3.2.2 ansible role as well, in order to test and integrate this version in development infrastructure.

#7 Updated by Andrea Dell'Amico over 1 year ago

Done. https is also available with a valid certificate.

#8 Updated by Emmanuel Blondel over 1 year ago

Thank you Fabio and Andrea. Fabio, let me know when it is registered, I will use the SDI service then to get the information we need. And thanks a lot for having pushed on GN3 for Tuna Atlas, much appreciated.

#9 Updated by Fabio Sinibaldi over 1 year ago

Instance is registered and credentials are available via sdi-service.

#10 Updated by Fabio Sinibaldi over 1 year ago

@emmanuel.blondel@fao.org can we stop/destroy the previous geonetwork 2 instance?

#11 Updated by Emmanuel Blondel over 1 year ago

No yet please, We still have to run integration tests and the complete workflow on it, hopefully I could start today but I'm in a conference the full day. I will try to launch tests in parallel of the conf. I will let you know when we can safely dismiss the GN2. Thank you for your understanding

#12 Updated by Fabio Sinibaldi over 1 year ago

No problem, sounds reasonable. We'll be waiting for your notification.

#13 Updated by Emmanuel Blondel over 1 year ago

Back! :-) I had a look and it appears that you moved to a slightly recent version (3.2.x series), however i think it's too prematurate, Indeed the GN 3.2.x series introduced a completely new REST API, See http://geonetwork-opensource.org/manuals/trunk/eng/users/overview/change-log/version-3.2.0.html

Some considerations (for Tuna Atlas et generally for the infra SDI):
* GN team tags this new REST API as beta, which reinforces the fact it's too premature to adopt 3.2.x
* Libraries (Java geonetwork-manager, R geonapi, etc) are not supporting this new beta REST API, and probably they will not if it sticks in a beta stage, which blocks us for all our programmatic metadata publication needs.

This means that you if we (for Tuna atlas) and you (for generalization to the entire SDI) want to use Geonetwork 3, we should stick at this stage with GN 3.0.x series for which our REST API clients will work. IMHO, downgrading to 3.0.4 would be the best for all of us. http://geonetwork-opensource.org/manuals/trunk/eng/users/overview/change-log/version-3.0.4.html

Let me know what you think
Cheers

#14 Updated by Fabio Sinibaldi over 1 year ago

Hi,
actually I've tested 3.2.1 by using geonetwork-manager and it worked well (that's what I've been doing in #5770), but if you need 3.0.4 we can work to provide that version.

#15 Updated by Emmanuel Blondel over 1 year ago

yes please downgrade for tuna atlas.

Just to know, which operation did you test with geonetwork-manager?

#16 Updated by Andrea Dell'Amico over 1 year ago

The server is now running version 3.0.4

#17 Updated by Fabio Sinibaldi over 1 year ago

Various issues when trying to configure the instance with our libraries.
I'm trying to solve, otherwise I'm gonna need to configure it manually (it's gonna take a bit).

@emmanuel.blondel@fao.org About testing 3.2.1,
I've been using users and groups apis (which I had to implement because are not covered by it.geosolutions.geonetwork.GNClient interface) and metadata search/ insertion means. For furhter details, these methods are exposed by org.gcube.spatial.data.geonetwork library documented at https://wiki.gcube-system.org/gcube/GeoNetwork_library.

#18 Updated by Fabio Sinibaldi over 1 year ago

I gave up fixing "on the fly", and manually configured both instance and IS.
Double checked the configuration with both sdi-service and geonetwork library.
Read/search operations seem working well (they are needed in order to integrate GeoExplorer and CKAN facilities).

Credentials are available as usual via the sdi-service.
@emmanuel.blondel@fao.org please test the instance and, as agreed before, notify us as soon as we can dismiss the previous one (hosted at previous geonetwork1-tunaatlas.d4science.org ).

#19 Updated by Emmanuel Blondel over 1 year ago

Thanks Fabio, could you please check the WEB-INF config.xml. Normally this configuration is similar to previous versions. I tried it but I get the same issue as for previous GN2 installations:

Open the file WEB-INF/config.xml and look for this:

<service name="xml.metadata.get"> 
        <class name=".services.metadata.Show"> 
                <param name="skipPopularity" value="y" /> 
                <param name="skipInfo" value="y" /> 
        </class> 
</service> 

After changing "" to " name="skipInfo" value="n" />" GeoNetwork should return the "geonet:info"
section in the response from "xml.metadata.get".

This information is required to perform metadata updates

Let me know

#20 Updated by Emmanuel Blondel over 1 year ago

Apologies path should slightly differ in GN3, you would find it in WEB-INF/config/config-service-xml-api.xml (see https://github.com/geonetwork/core-geonetwork/blob/3.0.x/web/src/main/webapp/WEB-INF/config/config-service-xml-api.xml)

We might need to change few other things there.

Let me know when the change is applied

#21 Updated by Roberto Cirillo over 1 year ago

#22 Updated by Fabio Sinibaldi over 1 year ago

Configuration file updated. Now gn info are displayed in xml.metadata.get response messages.

#23 Updated by Emmanuel Blondel over 1 year ago

Hello @fabio.sinibaldi@isti.cnr.it , i've performed tests and at this stage everything goes fine. I've published all resources available in the GN2 instance.

Before you proceed to dismiss the Geonetwork 2 instance (http://geonetwork1-tunaatlas.d4science.org/geonetwork), would it be possible to have an alias better than geonetwork2-tunaatlas.d4science/geonetwork? ideally for us the following link: https://tunaatlas.d4science/geonetwork

I would use this new link to share to key colleagues (especially for some them, link to GN2 had already been shared)

Thanks for your support

#24 Updated by Andrea Dell'Amico over 1 year ago

Emmanuel Blondel wrote:

Before you proceed to dismiss the Geonetwork 2 instance (http://geonetwork1-tunaatlas.d4science.org/geonetwork), would it be possible to have an alias better than geonetwork2-tunaatlas.d4science/geonetwork? ideally for us the following link: https://tunaatlas.d4science/geonetwork

It's doable but a question: do you plan to ask the same for the tunaatlas geoserver? because if so, I'll need to put a reverse proxy before both of them. Otherwise I'll just add a CNAME to the geonetwork server.

#25 Updated by Julien Barde over 1 year ago

Indeed, I think it would make sens to share the same prefix it for the URL of geoserver as well because the goal is to make it easier for users to remember it. It might be worth for other software components of the VREs as well if they can be accessed out of the VRE (Sharelatex, RStudio...)

#26 Updated by Andrea Dell'Amico over 1 year ago

Julien Barde wrote:

Indeed, I think it would make sens to share the same prefix it for the URL of geoserver as well because the goal is to make it easier for users to remember it. It might be worth for other software components of the VREs as well if they can be accessed out of the VRE (Sharelatex, RStudio...)

OK for the geoserver, I can't do that for the services that have / as a context path. They are supposed to be accessed from inside a VRE, anyway.

#27 Updated by Emmanuel Blondel over 1 year ago

Thanks Andrea, let us know when the new URLs are in place (for those apps where you can modify it like geoserver, geonetwork and possibly others). Cheers

#28 Updated by Emmanuel Blondel over 1 year ago

Hello @andrea.dellamico@isti.cnr.it when do you think you could make available the URLs https://tunaatlas-d4science.org/geonetwork and https://tunaatlas-d4science.org/geoserver . We would need this to share the links ASAP to key people in the community. Thanks in advance

#29 Updated by Andrea Dell'Amico over 1 year ago

Emmanuel Blondel wrote:

when do you think you could make available the URLs https://tunaatlas-d4science.org/geonetwork and https://tunaatlas-d4science.org/geoserver . We would need this to share the links ASAP to key people in the community. Thanks in advance

Later today or monday should be available.

#30 Updated by Andrea Dell'Amico over 1 year ago

  • Related to VM Creation #9830: New couple of reverse proxies to redirect and (or) balance some of the services added

#31 Updated by Emmanuel Blondel over 1 year ago

I would also need an access to Geonetwork logs, indeed they are issues related to size of ISO 19110 metadata files, and i would like to inspect logs and to liaise with Geonetwork core team If it appears to be a bug. I suppose I need an SFTP access for this? Thanks in advance

#32 Updated by Andrea Dell'Amico over 1 year ago

Emmanuel Blondel wrote:

I would also need an access to Geonetwork logs, indeed they are issues related to size of ISO 19110 metadata files, and i would like to inspect logs and to liaise with Geonetwork core team If it appears to be a bug. I suppose I need an SFTP access for this? Thanks in advance

You can access all the logs from here: http://geonetwork2-tunaatlas.d4science.org/gcube-logs/

#33 Updated by Andrea Dell'Amico over 1 year ago

http(s)://tunaatlas.d4science.org/{geonetwork,geoserver} is ready.
There's no valid https certificate yet, it will be emitted in the next days.

#34 Updated by Emmanuel Blondel over 1 year ago

Thanks @andrea.dellamico@isti.cnr.it and @fabio.sinibaldi@isti.cnr.it I've tested and everything it's ok. Thanks for your efforts in pushing to GN3. At this stage, considering Fabio's request, I think we can safely dismiss the Geonetwork 2 instance, and close this ticket.

#35 Updated by Andrea Dell'Amico over 1 year ago

  • Status changed from Feedback to Closed

geonetwork1-tunaatlas.d4science.org has been dismissed.

Also available in: Atom PDF