Feature #9724

DataMiner PoolManager - Improve the configuration file

Added by Giancarlo Panichi over 2 years ago. Updated over 2 years ago.

Status:ClosedStart date:Sep 20, 2017
Priority:ImmediateDue date:
Assignee:Nunzio Andrea Galante% Done:

100%

Category:data-analysis
Sprint:zz - DataMiner PoolManager
Milestones:
Duration:

Description

Currently, the configuration information of DMPoolManager is contained in the file service.propertie.
This file must be corrected and must be similar to:

#YML node file
SVN_REPO = https://svn.d4science.research-infrastructures.eu/gcube/trunk/data-analysis/RConfiguration/RPackagesManagement/

svn.repository = https://svn.d4science.research-infrastructures.eu/gcube
svn.algo.main.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms


#STAGE
STAGE_GHOST = dataminer-dev-ghost.d4science.org
svn.stage.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/dev/algorithms

svn.stage.deps-linux-compiled = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-pre-installed = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-r-blackbox = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt
svn.stage.deps-r = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt
svn.stage.deps-java =
svn.stage.deps-knime-workflow = 
svn.stage.deps-octave = 
svn.stage.deps-python = 
svn.stage.deps-windows-compiled = 

#PROD
PROD_GHOST = 
svn.prod.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/algorithms

svn.prod.deps-linux-compiled = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.prod.deps-pre-installed = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.prod.deps-r-blackbox = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt
svn.prod.deps-r = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt
svn.prod.deps-java = 
svn.prod.deps-knime-workflow =
svn.prod.deps-octave = 
svn.prod.deps-python = 
svn.prod.deps-windows-compiled = 

In the future, this configuration information should be moved in a resource on Information System(IS)


Related issues

Related to D4Science Infrastructure - VM Creation #9920: DataMiner PoolManager - Add a new host for dataminer-ghos... Closed Oct 10, 2017

History

#1 Updated by Nunzio Andrea Galante over 2 years ago

  • Status changed from New to In Progress

#2 Updated by Nunzio Andrea Galante over 2 years ago

Now the configuration file has the following format.
The old ones in preprod and prod environments need to be replaced with this one.
Moreovern the file web.xml will not be touched anymore cause the context is decided runtime according to the kind of call (stage or prod).

#YML node file
SVN_REPO = https://svn.d4science.research-infrastructures.eu/gcube/trunk/data-analysis/RConfiguration/RPackagesManagement/

svn.repository = https://svn.d4science.research-infrastructures.eu/gcube
svn.algo.main.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms

#STAGE
STAGE_GHOST = dataminer-ghost-d.dev.d4science.org
svn.stage.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/dev/algorithms

svn.stage.deps-linux-compiled = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-pre-installed = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-r-blackbox = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt
svn.stage.deps-r = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt
svn.stage.deps-java =
svn.stage.deps-knime-workflow =
svn.stage.deps-octave =
svn.stage.deps-python =
svn.stage.deps-windows-compiled =

#PROD
svn.prod.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/algorithms

Some modifications have been made to the source in order to obtain this new feature.
@roberto.cirillo@isti.cnr.it Since this has already been tested in dev environment, do you think is it still possible to deploy this new version in preprod/prod in this release?

#3 Updated by Nunzio Andrea Galante over 2 years ago

  • % Done changed from 0 to 100
  • Status changed from In Progress to Feedback

#4 Updated by Giancarlo Panichi over 2 years ago

We need to add this properties:

#STAGE
svn.stage.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/dev/software

#PROD
svn.prod.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/software

#5 Updated by Nunzio Andrea Galante over 2 years ago

Giancarlo Panichi wrote:

We need to add this properties:

#STAGE
svn.stage.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/dev/software

#PROD
svn.prod.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/software

Done.
The new format is the following:

#YML node file
SVN_REPO = https://svn.d4science.research-infrastructures.eu/gcube/trunk/data-analysis/RConfiguration/RPackagesManagement/

svn.repository = https://svn.d4science.research-infrastructures.eu/gcube
svn.algo.main.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms


#STAGE
STAGE_GHOST = dataminer-ghost-d.dev.d4science.org
svn.stage.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/dev/algorithms
svn.stage.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/dev/software

svn.stage.deps-linux-compiled = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-pre-installed = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-r-blackbox = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt
svn.stage.deps-r = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt
svn.stage.deps-java =
svn.stage.deps-knime-workflow = 
svn.stage.deps-octave = 
svn.stage.deps-python = 
svn.stage.deps-windows-compiled = 

#PROD
svn.prod.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/algorithms
svn.prod.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/software

#6 Updated by Nunzio Andrea Galante over 2 years ago

The following parameter has been added to the configuration file; it is related to the new algorithms path in the ghost dataminer.

svn.algo.ghost.repo = /home/gcube/wps_algorithms/algorithms/dev/software

@roberto.cirillo@isti.cnr.it pls add it to the ansible configuration.

@g.panichi@isti.cnr.it in dev you can find a version fixed in the bug concerning the fact that the installation is carried out anyway, even if the dependencies list is empty.
Could you please check?

#7 Updated by Roberto Cirillo over 2 years ago

Nunzio Andrea Galante wrote:

The following parameter has been added to the configuration file; it is related to the new algorithms path in the ghost dataminer.

svn.algo.ghost.repo = /home/gcube/wps_algorithms/algorithms/dev/software

@roberto.cirillo@isti.cnr.it pls add it to the ansible configuration.

Done

#8 Updated by Nunzio Andrea Galante over 2 years ago

Since now it will possible to specify more dependencies lists from svn different locations (instead of the actual one), in order to allow the check related to the R dependencies (and for all the other languages then) in a distributed manner.

svn.stage.deps-r = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt

Such new feature will be available from gcube release 4.8.

#9 Updated by Nunzio Andrea Galante over 2 years ago

For completeness compared to the previous post, also the value of the following parameter will be populated with more svn link:

svn.stage.deps-r-blackbox = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt

Moreover, it has been decided that such new version will be available both in in preprod and production environments as soon as possible.

#10 Updated by Roberto Cirillo over 2 years ago

I need to understand what is the complete file for each environment. I'm a bit confused now.
Please could you attach the property file filled for every environment: dev, preprod, proto and prod.
After that I'm going to modify the playbook with the new settings. Thanks.

#11 Updated by Nunzio Andrea Galante over 2 years ago

  • Development configuration file
#YML node file
SVN_REPO = https://svn.d4science.research-infrastructures.eu/gcube/trunk/data-analysis/RConfiguration/RPackagesManagement/

svn.repository = https://svn.d4science.research-infrastructures.eu/gcube
svn.algo.main.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms

#STAGE
STAGE_GHOST=dataminer-ghost-d.dev.d4science.org
svn.stage.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/dev/algorithms
svn.stage.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/dev/software
svn.algo.ghost.repo = /home/gcube/wps_algorithms/algorithms/dev/software

svn.stage.deps-linux-compiled = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-pre-installed = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-r-blackbox = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt
svn.stage.deps-r = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt 
svn.stage.deps-java =
svn.stage.deps-knime-workflow = 
svn.stage.deps-octave = 
svn.stage.deps-python = 
svn.stage.deps-windows-compiled = 

#PROD
svn.prod.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/algorithms
svn.prod.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/software
  • Pre-production configuration file
#YML node file
SVN_REPO = https://svn.d4science.research-infrastructures.eu/gcube/trunk/data-analysis/RConfiguration/RPackagesManagement/

svn.repository = https://svn.d4science.research-infrastructures.eu/gcube
svn.algo.main.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms

#STAGE
STAGE_GHOST=dataminer-ghost-t.pre.d4science.org
svn.stage.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/preprod/algorithms
svn.stage.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/preprod/software
svn.algo.ghost.repo = /home/gcube/wps_algorithms/algorithms/preprod/software

svn.stage.deps-linux-compiled = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-pre-installed = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-r-blackbox = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt
svn.stage.deps-r = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt
svn.stage.deps-java =
svn.stage.deps-knime-workflow = 
svn.stage.deps-octave = 
svn.stage.deps-python = 
svn.stage.deps-windows-compiled = 

#PROD
svn.prod.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/algorithms
svn.prod.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/software
  • Proto configuration file
#YML node file
SVN_REPO = https://svn.d4science.research-infrastructures.eu/gcube/trunk/data-analysis/RConfiguration/RPackagesManagement/

svn.repository = https://svn.d4science.research-infrastructures.eu/gcube
svn.algo.main.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms

#STAGE
STAGE_GHOST=dataminer-proto-ghost.d4science.org
svn.stage.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/proto/algorithms
svn.stage.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/proto/software
svn.algo.ghost.repo = /home/gcube/wps_algorithms/algorithms/proto/software

svn.stage.deps-linux-compiled = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-pre-installed = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-r-blackbox = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt
svn.stage.deps-r = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt 
svn.stage.deps-java =
svn.stage.deps-knime-workflow = 
svn.stage.deps-octave = 
svn.stage.deps-python = 
svn.stage.deps-windows-compiled = 

#PROD
svn.prod.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/algorithms
svn.prod.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/software
  • Prod configuration file
#YML node file
SVN_REPO = https://svn.d4science.research-infrastructures.eu/gcube/trunk/data-analysis/RConfiguration/RPackagesManagement/

svn.repository = https://svn.d4science.research-infrastructures.eu/gcube
svn.algo.main.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms

#STAGE
STAGE_GHOST=dataminer-proto-ghost.d4science.org
svn.stage.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/algorithms
svn.stage.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/software
svn.algo.ghost.repo = /home/gcube/wps_algorithms/algorithms/prod/software

svn.stage.deps-linux-compiled = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-pre-installed = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-r-blackbox = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt
svn.stage.deps-r = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt 
svn.stage.deps-java =
svn.stage.deps-knime-workflow = 
svn.stage.deps-octave = 
svn.stage.deps-python = 
svn.stage.deps-windows-compiled = 

#PROD
svn.prod.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/algorithms
svn.prod.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/software

As far as the production environment concerns, maybe it is the case to have a dedicated ghost, as done for all the the other environments. Currently we use the same of proto.

@g.panichi@isti.cnr.it please double check on the present post, in order to allow @roberto.cirillo@isti.cnr.it to finally automatize the ansible roles for the DataMinerPoolManager Service.

#12 Updated by Roberto Cirillo over 2 years ago

  • Priority changed from Normal to Immediate
  • Assignee changed from Nunzio Andrea Galante to Giancarlo Panichi

@g.panichi@isti.cnr.it do you agree with this settings?

#13 Updated by Giancarlo Panichi over 2 years ago

The prod configuration is not necessary. In the future, the property file will contain a field for the ghost of production.

Proto configuration file

#YML node file
SVN_REPO = https://svn.d4science.research-infrastructures.eu/gcube/trunk/data-analysis/RConfiguration/RPackagesManagement/

svn.repository = https://svn.d4science.research-infrastructures.eu/gcube
svn.algo.main.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms

#STAGE
STAGE_GHOST=dataminer-proto-ghost.d4science.org
svn.stage.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/proto/algorithms
svn.stage.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/proto/software
svn.stage.algo.ghost.repo = /home/gcube/wps_algorithms/algorithms/proto/software

svn.stage.deps-linux-compiled = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-pre-installed = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-r-blackbox = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt
svn.stage.deps-r = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt 
svn.stage.deps-java =
svn.stage.deps-knime-workflow = 
svn.stage.deps-octave = 
svn.stage.deps-python = 
svn.stage.deps-windows-compiled = 

#PROD
PROD_GHOST=dataminer-prod-ghost.d4science.org
svn.prod.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/algorithms
svn.prod.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/software
svn.prod.algo.ghost.repo = /home/gcube/wps_algorithms/algorithms/prod/software

svn.prod.deps-linux-compiled = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.prod.deps-pre-installed = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.prod.deps-r-blackbox = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt
svn.prod.deps-r = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt 
svn.prod.deps-java =
svn.prod.deps-knime-workflow = 
svn.prod.deps-octave = 
svn.prod.deps-python = 
svn.prod.deps-windows-compiled = 

Preprod configuration file

#YML node file
SVN_REPO = https://svn.d4science.research-infrastructures.eu/gcube/trunk/data-analysis/RConfiguration/RPackagesManagement/

svn.repository = https://svn.d4science.research-infrastructures.eu/gcube
svn.algo.main.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms

#STAGE
STAGE_GHOST=dataminer-ghost-t.pre.d4science.org
svn.stage.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/preprod/algorithms
svn.stage.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/preprod/software
svn.stage.algo.ghost.repo = /home/gcube/wps_algorithms/algorithms/preprod/software

svn.stage.deps-linux-compiled = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-pre-installed = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-r-blackbox = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt
svn.stage.deps-r = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt 
svn.stage.deps-java =
svn.stage.deps-knime-workflow = 
svn.stage.deps-octave = 
svn.stage.deps-python = 
svn.stage.deps-windows-compiled = 

#PROD
PROD_GHOST=dataminer-prod-ghost.d4science.org
svn.prod.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/algorithms
svn.prod.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/software
svn.prod.algo.ghost.repo = /home/gcube/wps_algorithms/algorithms/prod/software

svn.prod.deps-linux-compiled = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.prod.deps-pre-installed = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.prod.deps-r-blackbox = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt
svn.prod.deps-r = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt 
svn.prod.deps-java =
svn.prod.deps-knime-workflow = 
svn.prod.deps-octave = 
svn.prod.deps-python = 
svn.prod.deps-windows-compiled = 

Dev configuration file

#YML node file
SVN_REPO = https://svn.d4science.research-infrastructures.eu/gcube/trunk/data-analysis/RConfiguration/RPackagesManagement/

svn.repository = https://svn.d4science.research-infrastructures.eu/gcube
svn.algo.main.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms

#STAGE
STAGE_GHOST=dataminer-ghost-d.dev.d4science.org
svn.stage.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/dev/algorithms
svn.stage.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/dev/software
svn.stage.algo.ghost.repo = /home/gcube/wps_algorithms/algorithms/dev/software

svn.stage.deps-linux-compiled = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-pre-installed = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-r-blackbox = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt
svn.stage.deps-r = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt 
svn.stage.deps-java =
svn.stage.deps-knime-workflow = 
svn.stage.deps-octave = 
svn.stage.deps-python = 
svn.stage.deps-windows-compiled = 

#PROD
PROD_GHOST=dataminer-prod-ghost.d4science.org
svn.prod.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/algorithms
svn.prod.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/software
svn.prod.algo.ghost.repo = /home/gcube/wps_algorithms/algorithms/prod/software

svn.prod.deps-linux-compiled = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.prod.deps-pre-installed = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.prod.deps-r-blackbox = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt
svn.prod.deps-r = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt 
svn.prod.deps-java =
svn.prod.deps-knime-workflow = 
svn.prod.deps-octave = 
svn.prod.deps-python = 
svn.prod.deps-windows-compiled = 

Perhaps so it's clearer, only stage section change.
@roberto.cirillo@isti.cnr.it and @nunzioandrea.galante@eng.it, are we all in agreement?

#14 Updated by Giancarlo Panichi over 2 years ago

  • Related to VM Creation #9920: DataMiner PoolManager - Add a new host for dataminer-ghost in PreProduction environment added

#15 Updated by Roberto Cirillo over 2 years ago

  • Assignee changed from Giancarlo Panichi to Nunzio Andrea Galante

This is a bit different from the previous version posted by @nunzioandrea.galante@eng.it . If @nunzioandrea.galante@eng.it agree with this version, I'm going to update the ansible playbook.

#16 Updated by Nunzio Andrea Galante over 2 years ago

I think the correct version is the one i mentioned in the previous post, cause:

  • for #PROD, do not make sense to have parameters as svn.prod.software.repo, svn.prod.algo.ghost.repo, svn.prod.deps-linux-compiled, svn.prod.deps-pre-installed, svn.prod.deps-r-blackbox, svn.prod.deps-r, svn.prod.deps-java, svn.prod.deps-knime-workflow, svn.prod.deps-octave, svn.prod.deps-python, svn.prod.deps-windows-compiled
  • for @STAGE, according to the latest released version, it is necessary that the name of the parameters is the ones that i wrote (e.g., svn.algo.ghost.repo)

#17 Updated by Nunzio Andrea Galante over 2 years ago

According to the videoconference i had with @g.panichi@isti.cnr.it minutes ago, we can consider as definitive the following three configuration files.
In the actual scenario we do not consider the #PROD parameters proposed by Giancarlo, but they could be useful towards new developments that could be required in the future. For this reason, they should have to be available in the configuration files.

  • Development
#YML node file
SVN_REPO = https://svn.d4science.research-infrastructures.eu/gcube/trunk/data-analysis/RConfiguration/RPackagesManagement/

svn.repository = https://svn.d4science.research-infrastructures.eu/gcube
svn.algo.main.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms

#STAGE
STAGE_GHOST=dataminer-ghost-d.dev.d4science.org
svn.stage.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/dev/algorithms
svn.stage.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/dev/software
svn.stage.algo.ghost.repo = /home/gcube/wps_algorithms/algorithms/dev/software

svn.stage.deps-linux-compiled = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-pre-installed = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-r-blackbox = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt
svn.stage.deps-r = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt 
svn.stage.deps-java =
svn.stage.deps-knime-workflow = 
svn.stage.deps-octave = 
svn.stage.deps-python = 
svn.stage.deps-windows-compiled = 

#PROD
PROD_GHOST=dataminer-prod-ghost.d4science.org
svn.prod.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/algorithms
svn.prod.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/software
svn.prod.algo.ghost.repo = /home/gcube/wps_algorithms/algorithms/prod/software

svn.prod.deps-linux-compiled = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.prod.deps-pre-installed = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.prod.deps-r-blackbox = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt
svn.prod.deps-r = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt 
svn.prod.deps-java =
svn.prod.deps-knime-workflow = 
svn.prod.deps-octave = 
svn.prod.deps-python = 
svn.prod.deps-windows-compiled = 

  • Preproduction
#YML node file
SVN_REPO = https://svn.d4science.research-infrastructures.eu/gcube/trunk/data-analysis/RConfiguration/RPackagesManagement/

svn.repository = https://svn.d4science.research-infrastructures.eu/gcube
svn.algo.main.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms

#STAGE
STAGE_GHOST=dataminer-ghost-t.pre.d4science.org
svn.stage.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/preprod/algorithms
svn.stage.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/preprod/software
svn.stage.algo.ghost.repo = /home/gcube/wps_algorithms/algorithms/preprod/software

svn.stage.deps-linux-compiled = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-pre-installed = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-r-blackbox = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt
svn.stage.deps-r = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt 
svn.stage.deps-java =
svn.stage.deps-knime-workflow = 
svn.stage.deps-octave = 
svn.stage.deps-python = 
svn.stage.deps-windows-compiled = 

#PROD
PROD_GHOST=dataminer-prod-ghost.d4science.org
svn.prod.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/algorithms
svn.prod.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/software
svn.prod.algo.ghost.repo = /home/gcube/wps_algorithms/algorithms/prod/software

svn.prod.deps-linux-compiled = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.prod.deps-pre-installed = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.prod.deps-r-blackbox = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt
svn.prod.deps-r = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt 
svn.prod.deps-java =
svn.prod.deps-knime-workflow = 
svn.prod.deps-octave = 
svn.prod.deps-python = 
svn.prod.deps-windows-compiled = 

  • Proto
#YML node file
SVN_REPO = https://svn.d4science.research-infrastructures.eu/gcube/trunk/data-analysis/RConfiguration/RPackagesManagement/

svn.repository = https://svn.d4science.research-infrastructures.eu/gcube
svn.algo.main.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms

#STAGE
STAGE_GHOST=dataminer-proto-ghost.d4science.org
svn.stage.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/proto/algorithms
svn.stage.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/proto/software
svn.stage.algo.ghost.repo = /home/gcube/wps_algorithms/algorithms/proto/software

svn.stage.deps-linux-compiled = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-pre-installed = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.stage.deps-r-blackbox = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt
svn.stage.deps-r = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt 
svn.stage.deps-java =
svn.stage.deps-knime-workflow = 
svn.stage.deps-octave = 
svn.stage.deps-python = 
svn.stage.deps-windows-compiled = 

#PROD
PROD_GHOST=dataminer-prod-ghost.d4science.org
svn.prod.algorithms-list = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/algorithms
svn.prod.software.repo = /trunk/data-analysis/DataMinerConfiguration/algorithms/prod/software
svn.prod.algo.ghost.repo = /home/gcube/wps_algorithms/algorithms/prod/software

svn.prod.deps-linux-compiled = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.prod.deps-pre-installed = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt
svn.prod.deps-r-blackbox = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt
svn.prod.deps-r = /trunk/data-analysis/RConfiguration/RPackagesManagement/r_deb_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_cran_pkgs.txt, /trunk/data-analysis/RConfiguration/RPackagesManagement/r_github_pkgs.txt 
svn.prod.deps-java =
svn.prod.deps-knime-workflow = 
svn.prod.deps-octave = 
svn.prod.deps-python = 
svn.prod.deps-windows-compiled = 

#18 Updated by Roberto Cirillo over 2 years ago

Thanks @nunzioandrea.galante@eng.it. if it's needed a new service version please add the new service version (when it is ready) to this ticket and after that I'm going to deploy the new version with the new settings, in preproduction environment.

#19 Updated by Nunzio Andrea Galante over 2 years ago

New staging version 155107 released and tested in dev successfully.

#20 Updated by Roberto Cirillo over 2 years ago

The new service has been deployed in preproduction environment too.

#21 Updated by Nunzio Andrea Galante over 2 years ago

Thanks @roberto.cirillo@isti.cnr.it, the service has worked successfully also in preprod environment.

#22 Updated by Nunzio Andrea Galante over 2 years ago

  • Status changed from Feedback to Closed

Also available in: Atom PDF