Support #13024

DataMiner - The service must support the https protocol

Added by Giancarlo Panichi 7 months ago. Updated 3 months ago.

Status:ClosedStart date:Dec 11, 2018
Priority:NormalDue date:Apr 11, 2019
Assignee:Andrea Dell'Amico% Done:

100%

Category:System Application
Sprint:Ansible playbooks and roles
Infrastructure:Development, Pre-Production, Production
Milestones:
Duration: 88

Description

The DataMiner service must support the https protocol.

To support https among other things it is necessary to update the wps config file for example:

~/tomcat/webapps/wps/config$pico wps_config.xml

.....

 <Server protocol="http" hostname="dataminer1-pre.d4science.org" hostport="80" includeDataInputsInResponse="false" computationTimeoutMilliSeconds="259200000" cacheCapabilites="false" webappPath="wps" repoReloadInterval="0.0" minPoolSize="10" maxPoolSize="20" keepAliveSeconds="1000" maxQueuedTasks="100">

...

Screenshot 2018-12-13 at 11.59.03.png (142 KB) Andrea Dell'Amico, Dec 13, 2018 12:01 PM

Screenshot 2018-12-14 at 19.11.59.png (436 KB) Andrea Dell'Amico, Dec 14, 2018 07:13 PM

2956
2962

History

#1 Updated by Andrea Dell'Amico 7 months ago

  • Sprint changed from DataMiner to Ansible playbooks and roles
  • Assignee changed from Lucio Lelii to _InfraScience Systems Engineer
  • Category changed from data-analysis to System Application
  • Project changed from gCube to D4Science Infrastructure
  • Tracker changed from Feature to Support

This is a provisioning activity. The information contained there is also completely redundant, do you know it it's possible to remove it?

#2 Updated by Andrea Dell'Amico 7 months ago

  • Infrastructure Development, Pre-Production, Production added

#3 Updated by Giancarlo Panichi 7 months ago

In any case, do not modify the other attributes so we are sure that everything continues to work as expected.

#4 Updated by Andrea Dell'Amico 7 months ago

  • % Done changed from 0 to 60
  • Status changed from New to In Progress

@g.panichi@isti.cnr.it I changed the script that modifies the host/port/protocol data inside wps_config.yml. Can I test it on the dev dataminers?

#5 Updated by Giancarlo Panichi 7 months ago

Yes @andrea.dellamico@isti.cnr.it , you can do the tests in the dev environment.

#6 Updated by Andrea Dell'Amico 7 months ago

  • % Done changed from 60 to 80

Done. All the dev dataminers have been restarted with the modified configuration.

#7 Updated by Andrea Dell'Amico 7 months ago

I run a test in dev, an error is returned because the application runs the query over http but specifies the 443 port. The protocol is http even when I explicitly set https in the for field (see the screenshot: i removed the token before taking the screenshot, but it was there when I run the test).

#8 Updated by Andrea Dell'Amico 7 months ago

I've found another strangeness: the GetCapabilities response should be generated starting from a template, I read here: https://wiki.52north.org/Geoprocessing/ConfiguringTheWPS. The template is located at wps/config/wpsCapabilitiesSkeleton.xml and the one we are distributing contains:

<?xml version="1.0" encoding="UTF-8"?>
<wps:Capabilities service="WPS" version="1.0.0" xml:lang="en-US" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:wps="http://www.opengis.net/wps/1.0.0" xmlns:ows="http://www.opengis.net/ows/1.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/wps/1.0.0 http://schemas.opengis.net/wps/1.0.0/wpsGetCapabilities_response.xsd" updateSequence="1">
        <ows:ServiceIdentification>
                <ows:Title>52°North WPS ${version}</ows:Title>
                <ows:Abstract>Service based on the 52°North implementation of WPS 1.0.0</ows:Abstract>
                <ows:Keywords>
                        <ows:Keyword>WPS</ows:Keyword>
                        <ows:Keyword>geospatial</ows:Keyword>
                        <ows:Keyword>geoprocessing</ows:Keyword>
                </ows:Keywords>
                <ows:ServiceType>WPS</ows:ServiceType>
                <ows:ServiceTypeVersion>1.0.0</ows:ServiceTypeVersion>
                <ows:Fees>NONE</ows:Fees>
                <ows:AccessConstraints>NONE</ows:AccessConstraints>
        </ows:ServiceIdentification>
        <ows:ServiceProvider>
                <ows:ProviderName>52North</ows:ProviderName>
                <ows:ProviderSite xlink:href="http://www.52north.org/"/>
                <ows:ServiceContact>
                        <ows:IndividualName>Your name</ows:IndividualName>
                        <ows:PositionName>Your position</ows:PositionName>
                        <ows:ContactInfo>
                                <ows:Phone>
                                        <ows:Voice></ows:Voice>
                                        <ows:Facsimile></ows:Facsimile>
                                </ows:Phone>
                                <ows:Address>
                                        <ows:DeliveryPoint></ows:DeliveryPoint>
                                        <ows:City></ows:City>
                                        <ows:AdministrativeArea></ows:AdministrativeArea>
                                        <ows:PostalCode></ows:PostalCode>
                                        <ows:Country></ows:Country>
                                        <ows:ElectronicMailAddress></ows:ElectronicMailAddress>
                                </ows:Address>
                        </ows:ContactInfo>
                </ows:ServiceContact>
        </ows:ServiceProvider>
        <ows:OperationsMetadata>
                <ows:Operation name="GetCapabilities">
                        <ows:DCP>
                                <ows:HTTP>
                                        <ows:Get xlink:href="http://dynamicallygeneratedURL/GeoPS?"/>
                                        <ows:Post xlink:href="http://dynamicallygeneratedURL/GeoPS"/>
                                </ows:HTTP>
                        </ows:DCP>
                </ows:Operation>
                <ows:Operation name="DescribeProcess">
                        <ows:DCP>
                                <ows:HTTP>
                                        <ows:Get xlink:href="http://dynamicallygeneratedURL/GeoPS/GeoPS?"/>
                                        <ows:Post xlink:href="http://dynamicallygeneratedURL/GeoPS/GeoPS"/>
                                </ows:HTTP>
                        </ows:DCP>
                </ows:Operation>
                <ows:Operation name="Execute">
                        <ows:DCP>
                                <ows:HTTP>
                                        <ows:Get xlink:href="http://dynamicallygeneratedURL/GeoPS/GeoPS?"/>
                                        <ows:Post xlink:href="http://dynamicallygeneratedURL/GeoPS/GeoPS"/>
                                </ows:HTTP>
                        </ows:DCP>
                </ows:Operation>
        </ows:OperationsMetadata>

        <wps:Languages>
                <wps:Default>
                        <ows:Language>en-US</ows:Language>
                </wps:Default>
                <wps:Supported>
                        <ows:Language>en-US</ows:Language>
                </wps:Supported>
        </wps:Languages>

</wps:Capabilities>

I assume that we use a different one, but derived from that one, and out template still havehref="http://whatever. Am I correct? If so, just remove the http: occurrences from that file.

#9 Updated by Andrea Dell'Amico 7 months ago

  • Assignee changed from _InfraScience Systems Engineer to Giancarlo Panichi

@g.panichi@isti.cnr.it found in the code the template used to generate the getCapabilities results and he's changing it to provide the correct protocol. I'm reverting the change in my script until the new dataminer version is ready and has been deployed.

#10 Updated by Giancarlo Panichi 7 months ago

  • Assignee changed from Giancarlo Panichi to Andrea Dell'Amico

@andrea.dellamico@isti.cnr.it the code is ready on dataminer2-d-d4s for test.

#11 Updated by Andrea Dell'Amico 7 months ago

The #PROTOCOL# variable is not expanded, see the screenshot.

#12 Updated by Giancarlo Panichi 7 months ago

  • Assignee changed from Giancarlo Panichi to Andrea Dell'Amico

Hi @andrea.dellamico@isti.cnr.it , I solved the problem the code is available again on dataminer2-d-d4s for test.

#13 Updated by Andrea Dell'Amico 7 months ago

  • % Done changed from 80 to 100
  • Status changed from In Progress to Feedback

I confirm that it's working.

#14 Updated by Giancarlo Panichi 7 months ago

Very well, now we need to test as many algorithms as possible to check if even most clients support https redirection.

#15 Updated by Andrea Dell'Amico 7 months ago

  • Assignee changed from Andrea Dell'Amico to Giancarlo Panichi

Mind that there is no redirection at the load balancer or reverse proxy level.

#16 Updated by Andrea Dell'Amico 7 months ago

I suggest that on the test phase in dev/preprod we manually change the wps_config.xml file so that we do not risk to break the production dataminers if we have to provision them.

#17 Updated by Giancarlo Panichi 3 months ago

  • Assignee changed from Giancarlo Panichi to Andrea Dell'Amico
  • Status changed from Feedback to Closed
  • Due date set to Apr 11, 2019

Now, the DataMiner services are able to support both http and https based on the configuration that is chosen during the installation of the service. I close this ticket.

Also available in: Atom PDF