Project

General

Profile

Actions

Feature #21891

closed

Migrate OpenPortal machines to ubuntu 18.04

Added by Tommaso Piccioli over 2 years ago. Updated over 1 year ago.

Status:
Closed
Priority:
High
Assignee:
_InfraScience Systems Engineer
Target version:
Start date:
Jul 30, 2021
Due date:
% Done:

100%

Estimated time:

Description

3 machines

dnet.openportal.isti.cnr.it
backend.openportal.isti.cnr.it
openportal.isti.cnr.it

See #21881


Add

Subtasks


Add

Related issues

Related to ISTI Open Access - Feature #11601: Migrate the OpenPortal services to production machinesClosed_InfraScience Systems EngineerOct 12, 2018

Delete relationActions
Related to ISTI Open Access - Feature #21881: Update the PHP versionClosed_InfraScience Systems EngineerJul 28, 2021

Delete relationActions
Related to ISTI Open Access - Support #22008: Cater for OpenPortal for ISPCClosedLeonardo CandelaSep 15, 2021

Delete relationActions
Related to ISTI Open Access - Incident #22438: FTP for ISPC Open Portal is not reachable from outside Closed_InfraScience Systems EngineerNov 18, 2021

Delete relationActions
Related to ISTI Open Access - Support #23904: Complete the migration to the new machinesClosedTommaso PiccioliSep 22, 2022

Delete relationActions
Actions #1

Updated by Tommaso Piccioli over 2 years ago

  • Related to Feature #11601: Migrate the OpenPortal services to production machines added
Actions #2

Updated by Anonymous over 2 years ago

Actions #3

Updated by Leonardo Candela over 2 years ago

  • Priority changed from Normal to High
Actions #4

Updated by Leonardo Candela over 2 years ago

Updated by Michele Artini over 2 years ago

I add some details about the components of the current machines:

dnet.openportal.isti.cnr.it

  • Java (OpenJDK 1.8.0_171)
  • Tomcat7 (7.0.52-1ubuntu0.16)
  • dnet application (it manages 2 embedded databases, the files are stored in /var/lib/dnet/)

backend.openportal.isti.cnr.it

  • MySQL (5.5.62):
    • Joomla databases: op_beta_joomla, op_ismar_joomla, op_prod_joomla
    • dnet databases: isti_affiliations
  • Solr (5.5.5): the index can be automatically regenerated by dnet
  • Mongo (3.6.9): relevant databases are dnet_logs_isti, mdstore_isti, oaistore_isti

openportal.isti.cnr.it

  • Apache 2, it hosts the Joomla sites, it exposes the content of the /data directory and redirect the calls to https://openportal.isti.cnr.it/oai to the dnet application
  • PHP with modules for mongo and solr
  • FTP server, it permits to manipulate the file under /data, it must be accessible also by the dnet application
  • Joomla (3 instances)

Updated by Andrea Dell'Amico over 2 years ago

Michele Artini wrote in #note-5:

  • Tomcat7 (7.0.52-1ubuntu0.16)

Tomcat's version will be 8.5

Updated by Michele Artini over 2 years ago

There are also some shell scripts under /etc/cron.hourly of openportal.isti.cnr.it . These scripts keep the totals updated on the main page of the portals.

Updated by Andrea Dell'Amico over 2 years ago

It's time to manage everything inside the ansible playbook, I hate having to search servers and find all the manual tweaks.

Updated by Tommaso Piccioli over 2 years ago

  • Status changed from New to In Progress

A new record in the ISPC DNS is needed

openportal.ispc.cnr.it 146.48.87.209

Actions #10

Updated by Andrea Dell'Amico over 2 years ago

Actions #11

Updated by Andrea Dell'Amico over 2 years ago

Actions #12

Updated by Andrea Dell'Amico over 2 years ago

Actions #13

Updated by Andrea Dell'Amico over 2 years ago

Actions #14

Updated by Andrea Dell'Amico over 2 years ago

  • Assignee changed from Tommaso Piccioli to _InfraScience Systems Engineer

Updated by Andrea Dell'Amico over 2 years ago

Michele Artini wrote in #note-7:

There are also some shell scripts under /etc/cron.hourly of openportal.isti.cnr.it . These scripts keep the totals updated on the main page of the portals.

@michele.artini@isti.cnr.it those script should be provisioned by ansible. Where do we can find them?

Updated by Andrea Dell'Amico over 2 years ago

Also: all the problems seems to have been solved.
Can we schedule the migration of the two other portals?

Actions #17

Updated by Andrea Dell'Amico over 2 years ago

  • % Done changed from 0 to 30

Updated by Anonymous over 2 years ago

The scripts in /etc/cron.hourly are specific for each instance of portal. I need to edit them easily.

Updated by Andrea Dell'Amico over 2 years ago

I still don't like manual editing in place, for whatever reason.
We could:

  • Give you the ability to run the ansible playbook
  • You put the cron entries into a git repository, and we use a cron job to pull from it.
Actions #20

Updated by Andrea Dell'Amico over 2 years ago

  • Related to Incident #22438: FTP for ISPC Open Portal is not reachable from outside added

Updated by Michele Artini over 2 years ago

@tommaso.piccioli@isti.cnr.it Could you permit the access to newbackend-openportal.isti.cnr.it (mysql, mongo and solr via https) from openportal.isti.cnr.it?

So, I will reconfigure all the portals to use the new backends and we could dismiss:

  • dnet-openportal.isti.cnr.it (entire machine)
  • backend-openportal.isti.cnr.it (for now only mongo and solr )

Updated by Tommaso Piccioli over 2 years ago

Michele Artini wrote in #note-21:

@tommaso.piccioli@isti.cnr.it Could you permit the access to newbackend-openportal.isti.cnr.it (mysql, mongo and solr via https) from openportal.isti.cnr.it?

done.

Updated by Michele Artini over 2 years ago

I reconfigured the ISTI, ISMAR and BETA portals to be managed by the services hosted on the new machines:

  • newdnet-openportal.isti.cnr.it (dnet aggregator)
  • newbackend.openportal.isti.cnr.it (mongo, solr and mysql)

Probably we could stop the old machines:

  • dnet-openportal.isti.cnr.it
  • backend.openportal.isti.cnr.it

Instead the portals are yet hosted on the old machine (Ubuntu 14.04, PHP 7.2.13) and need to be moved on the new server.

@tommaso.piccioli@isti.cnr.it @andrea.dellamico@isti.cnr.it
In the old server to enable the OAI exporter from the ISTI portal, according to the new mechanism, I commented the following line:

# ProxyPassMatch ^/oai(.*)$ https://dnet-openportal.isti.cnr.it/is/mvc/oai/oai.do$1

of file:

/etc/apache2/sites-enabled/openportal.isti.cnr.it.conf

That line is no longer necessary, you should remove it in the Ansible scripts.

Updated by Tommaso Piccioli over 2 years ago

dnet-openportal.isti.cnr.it and backend.openportal.isti.cnr.it switched off.

Updated by Michele Artini about 2 years ago

The ISTI portal was using the solr API, exposed by the backend.openportal.isti.cnr.it via https, to render some pages, for example: https://www.isti.cnr.it/en/research/publications

Could you reassign the name backend.openportal.isti.cnr.it to newbackend.openportal.isti.cnr.it and configure a proxy to expose the port 8983 via https?

Actions #26

Updated by Michele Artini over 1 year ago

  • Related to Support #23904: Complete the migration to the new machines added

Updated by Michele Artini over 1 year ago

  • Status changed from In Progress to Closed
  • % Done changed from 30 to 100

La migrazione รจ stata completata.

Actions

Also available in: Atom PDF