Project WP #686

WP9 - VRE Commons Development [Months: 1-30]

Added by Franco Zoppi over 3 years ago. Updated 3 months ago.

Status:ClosedStart date:Oct 15, 2015
Priority:UrgentDue date:Jan 18, 2018
Assignee:Massimiliano Assante% Done:

97%

Sprint:WP09
Lead beneficiary:1 - CNR Budget Effort (PM):89.00
Milestones:
Duration: 591

Description

Objectives
This work package will develop and release a comprehensive set of facilities enabling the creation of the VREs and the implementation of the services envisaged by WP5-8. In particular, the work package will extend and consolidate the gCube technology by offering a series of software frameworks supporting almost all the phases underlying complete data management over a rich array of data typologies including tabular data, geospatial data, and data expressed using semantic web formats. These software frameworks are conceived to operate by maximising the exploitation of an infrastructure resulting from the federation of existing infrastructures, e.g. they are designed to be capable to rely on a dynamic yet heterogeneous offering of third party facilities – such as storage or computing – when realising their characterising features.

Description of work and role of partners
CNR, UOA, FORTH


Subtasks

Project Task #687: T9.1 Data Storage, Discovery and Access [Months: 1-29]ClosedYannis Marketakis

Project Activity #1061: Requirements collection for Data Storage, Access and Disc...ClosedYannis Marketakis

Project Activity #1062: Inventory of currently existing solutions for T9.1ClosedYannis Marketakis

Task #2246: Defining the requirements a draft design for supporting D...ClosedYannis Marketakis

gCube - Feature #1742: Evaluate transforming the Workspace into a widgetPausedFrancesco Mangiacrapa

gCube - Feature #1442: Integrate OAI-PMH directly to harvesting/indexing mechanismClosedPanagiota Koltsida

gCube - Feature #3914: Develop a portlet for managing oai-pmh sourcesClosedPanagiota Koltsida

gCube - Feature #2546: Endow Workspace Resources Explorer with filtering and dis...ClosedFrancesco Mangiacrapa

Task #1806: Integrate X3ML engine with MatWareClosedYannis Marketakis

Task #3926: MatWare plugins development for GRSF sourcesClosedYannis Marketakis

Task #3927: Matware plugins development for FIRMS data sourceClosedYannis Marketakis

Task #3928: Matware plugins development for RAM data sourceClosedYannis Marketakis

Task #4284: Matware plugins development for FishSource data sourceClosedYannis Marketakis

Task #5524: Update MatWare plugin for FishSourceClosedYannis Marketakis

gCube - Feature #4124: Provide Workspace Explorer with search function and loadi...PausedFrancesco Mangiacrapa

gCube - Feature #4595: Migrate Jackrabbit from Derby to PostgreSQLClosedValentina Marioli

gCube - Bug #4060: Bug saving AccountingItemClosedValentina Marioli

gCube - Bug #4586: HL cannot create ReportTemplateClosedValentina Marioli

gCube - Bug #4779: HL: short links are not generated from getPublicLink methodClosedValentina Marioli

gCube - Bug #4781: HL: application folders are not set as hidden when createdClosedValentina Marioli

gCube - Bug #4780: HL: Error unsharing foldersClosedValentina Marioli

gCube - Feature #2616: Adding examples to HL 2.0 API wikiClosedValentina Marioli

Project Activity #4994: MatWare web application designRejectedYannis Marketakis

gCube - Feature #5661: Refactoring of index and search mechanismNewNikolas Laskaris

gCube - Feature #5662: Index and Search clientClosedNikolas Laskaris

gCube - Feature #1445: Enhance search results visualization NewNikolas Laskaris

gCube - Feature #5129: Enhance index field groupingsClosedNikolas Laskaris

gCube - Feature #6250: Modify search query submission on index to return also th...ClosedEfthymios Theodorakopoulos

gCube - Feature #1440: Remove index ingestion dependency to TreeManager In ProgressNikolas Laskaris

gCube - Feature #1444: Provided faceted search capabilities at the backendClosedNikolas Laskaris

gCube - Feature #1438: Remove Resource Registry dependencyIn ProgressNikolas Laskaris

gCube - Feature #5663: Integrate external sources into gcube index searchIn ProgressNikolas Laskaris

gCube - Feature #5664: Datafusion on search resultsNewNikolas Laskaris

gCube - Feature #5665: Modify index cluster mechanism to run cluster on VOsClosedNikolas Laskaris

gCube - Feature #5667: Implement a datasource/index management PortletClosedEfthymios Theodorakopoulos

gCube - Feature #6159: Extend service's search capabilities of querying individu...ClosedEfthymios Theodorakopoulos

gCube - Feature #5772: Enhance search portletClosedEfthymios Theodorakopoulos

gCube - Feature #6134: Search portlet enhancements / new featuresNewNikolas Laskaris

gCube - Feature #6136: Search Portlet Enhancement - Enable searching on individu...NewEfthymios Theodorakopoulos

gCube - Feature #6137: Search portlet enhancement - Expose faceted search result...ClosedEfthymios Theodorakopoulos

gCube - Task #7888: Configure title and snippet for each datasource from inde...ClosedEfthymios Theodorakopoulos

gCube - Task #7889: Modify index and search backend to take into consideratio...ClosedEfthymios Theodorakopoulos

gCube - Task #7890: Introduce external sources logic on the index/search modulesClosedEfthymios Theodorakopoulos

gCube - Task #7891: Allow adding an external source from index-configurator a...ClosedEfthymios Theodorakopoulos

gCube - Task #7954: Search Portlet: Display the available data sources and al...ClosedEfthymios Theodorakopoulos

gCube - Task #7955: Search Portlet: Display the number of records per data so...ClosedEfthymios Theodorakopoulos

gCube - Task #7956: enhance results display in search portletClosedEfthymios Theodorakopoulos

Task #8278: Update MatWare (fetch) plugins for FishSourceClosedYannis Marketakis

Task #8365: Create a client that harvests the data catalog contents i...ClosedYannis Marketakis

Task #8366: Update components to build on Java 1.8RejectedYannis Marketakis

gCube - Task #8528: Implement an OAI-PMH publisher on top of the Index - Sear...ClosedEfthymios Theodorakopoulos

Task #8715: Publish the indexed Data Sources to the D4Science catalogueRejectedNikolas Laskaris

Task #8786: Enhance MatWare plugin for FIRMS to extract also Observat...ClosedYannis Marketakis

Task #9859: Update MatWare plugin for FishSourceClosedYannis Marketakis

Project Task #688: T9.2 Data Analytics Facilities [Months: 1-29]ClosedGianpaolo Coro

Task #2520: Create an algorithm to publish HTML pages on an Apache se...ClosedGianpaolo Coro

gCube - Feature #2545: Dataminer enhancementsClosedGianpaolo Coro

Task #3078: Enhance SAIClosedGiancarlo Panichi

Task #3081: Workspace enhancement requestsClosedFrancesco Mangiacrapa

D4Science Infrastructure - Task #1854: Tool for usage statistics on DataMinerClosedAndrea Dell'Amico

gCube - Bug #3925: Issue contacting URI Resolver after several Workspace upl...ClosedRoberto Cirillo

Task #3986: Enable WPS interface for the Ichtyop modelClosedGianpaolo Coro

Task #3987: Build an example of data embedding request to WPSClosedGianpaolo Coro

Task #4121: Create an enhanced version of a Statman Algorithm to publ...ClosedGianpaolo Coro

Task #4138: Integrate CCAMLR charts updaterClosedGianpaolo Coro

Task #4171: Support algorithms integration via SAIClosedGianpaolo Coro

Project Activity #4172: Dataminer interface enhancementsClosedGiancarlo Panichi

Project Activity #4185: publishing the "VPA_ICCAT_BFT_E_Retros" algorithm in the ...ClosedRoberto Cirillo

Task #4211: Intersect two layers in VRE using RClosedLevi Westerveld

Project Activity #4195: publishing the "Ichthyop_model_one_by_one" algorithm in t...ClosedGianpaolo Coro

D4Science Infrastructure - Task #4610: Algorithms installation in D4ScienceClosedGianpaolo Coro

Task #4684: Install SPREAD on DataminerClosedEmmanuel Blondel

gCube - Feature #3983: Implement DataMiner client libraryClosedGiancarlo Panichi

gCube - Feature #4713: DataMiner - Add a direct link to the algorithmsClosedGiancarlo Panichi

Task #4745: Shapefiles importing algorithmClosedGianpaolo Coro

Task #4802: Import OSCAR data in the production environmentClosedGianpaolo Coro

Incident #4819: Oscar files not visible on GeoExplorerClosedFrancesco Mangiacrapa

Project Activity #4896: SAI enhancements ClosedGiancarlo Panichi

Project Activity #4899: Dataminer service enhancementsClosedGianpaolo Coro

gCube - Feature #5258: Add link to help on DataminerClosedGiancarlo Panichi

Support #4957: Help for parallization of Fortran, ICCAT step 3 algorithmRejectedJulien Barde

D4Science Infrastructure - Task #5169: Prepare environment to test Generic Worker on DataminerClosedAndrea Dell'Amico

gCube - Feature #5549: Connect to the new Data Transfer Service in DataMinerIn ProgressLucio Lelii

Task #5459: connect virtual workspace with RStudio workspaceClosedValentina Marioli

gCube - Support #5555: Production dependencies update for DataMiner 4.2.0ClosedGianpaolo Coro

D4Science Infrastructure - Task #5590: DataMiner as Generic WorkerClosedAndrea Dell'Amico

D4Science Infrastructure - Incident #5638: Check data availability through URI ResolverRejectedRoberto Cirillo

D4Science Infrastructure - Incident #5645: Service EP Retrieval issueClosedLucio Lelii

D4Science Infrastructure - Incident #5626: SAI does not save filesClosedRoberto Cirillo

Task #5725: CMSY as-a-ServiceClosedGianpaolo Coro

gCube - Feature #5747: WPS Statistical - Migrate to DataMiner ClosedAndrea Dell'Amico

Task #5912: Interact with OBIS through REST APIClosedGianpaolo Coro

Project Activity #5913: CMSY report templateRejectedEnrico Anello

Task #7101: Defining and creating workspaces on Geoserver dynamicallyClosedFabio Sinibaldi

gCube - Task #7102: Transform DataMiner proxy information into Service EP inf...NewRoberto Cirillo

Task #7120: Manage SAI outputClosedGiancarlo Panichi

D4Science Infrastructure - Task #7126: SAI should make user specify the algorithm categoryClosedGiancarlo Panichi

Task #7129: K-Statistics and DataMinerClosedPaolo Scarponi

D4Science Infrastructure - Task #7167: DataMiner interface should report encoded parameters in t...ClosedGianpaolo Coro

Task #7352: Data Miner Installations AlignerClosedPaolo Scarponi

Task #7549: Identify interesting repositories from OpenDOARClosedPanagiota Koltsida

gCube - Bug #7638: oai-pmh harvester portlet static number of records displayClosedNikolas Laskaris

gCube - Bug #7637: OAI-PMH Harvester portlet - Edit buttonClosedNikolas Laskaris

gCube - Task #7639: OAI-PMH harvester portlet enhancementClosedNikolas Laskaris

gCube - Task #7640: OAI-PMH Harvester portlet - Improve Error MessagesClosedNikolas Laskaris

Project Activity #7641: Harvest and Index the following data sources in iSearch VREClosedPanagiota Koltsida

Task #8034: Install tomcat manager on dataminer machinesRejected_InfraScience Systems Engineer

D4Science Infrastructure - Task #8367: Automatize DataMiner TestingClosedLucio Lelii

D4Science Infrastructure - Incident #8942: Check Mono installationClosed_InfraScience Systems Engineer

Task #8952: Make R scripts inherit token and SDI informationClosedLucio Lelii

Task #9274: CSV to NETCDF Conversion SuiteClosedTAHA IMZILEN

Task #9321: DataMiner should work with local IP addressesClosedAndrea Dell'Amico

Task #9660: Activate a VM in the GARR OpenStack instance and configur...ClosedAndrea Dell'Amico

Task #9661: Activate a dataminer instance without public IP address i...ClosedAndrea Dell'Amico

D4Science Infrastructure - Task #9409: Report the computation ID as a metadata in the DataMiner ...NewLucio Lelii

Task #9521: Integrate the CMSY version for NOAA DLM ToolClosedGianpaolo Coro

D4Science Infrastructure - Support #9536: Critical Workspace-SAI issuesClosedValentina Marioli

D4Science Infrastructure - Task #9569: HL: Problem with locksClosedLucio Lelii

D4Science Infrastructure - Task #9570: Metadata information not correctly reported to SAIRejectedValentina Marioli

D4Science Infrastructure - Task #9571: HL: not add extensions to files during the zip creationClosedValentina Marioli

D4Science Infrastructure - Task #9549: Make R scripts inherit user name and VREClosedGianpaolo Coro

Task #9554: Support Vector Machines as a ServiceClosedPaolo Scarponi

D4Science Infrastructure - Task #9583: Represent ASFIS AquaMaps distribution with NetCDF formatClosedPaolo Scarponi

Task #9707: Change OBIS interaction calls in SPDClosedLucio Lelii

Task #9832: Generate climate change and environmental structured data...ClosedGianpaolo Coro

Task #9936: Switch the cloud computing facility for PAIMClosedGianpaolo Coro

D4Science Infrastructure - Task #10003: Solve the issue of Windows line terminators in shell scriptsClosedGianpaolo Coro

Task #10130: Analysis of Climate Change DataClosedGianpaolo Coro

D4Science Infrastructure - Task #10177: Install the CMSY-FAST algorithm in the StockAssessment VREClosedGianpaolo Coro

D4Science Infrastructure - Task #10223: Provide machines for FAO courseClosed_InfraScience Systems Engineer

Task #10475: Rasterize AquaMaps Native 2050 distributionsClosedGianpaolo Coro

D4Science Infrastructure - Task #10479: Cloud Provisioning RequestsClosed_InfraScience Systems Engineer

gCube - Bug #10517: SAI Set MainClosedGiancarlo Panichi

D4Science Infrastructure - Task #10518: Add the CMSY_LEGACY algorithm to the StockAssessment VREClosedPaolo Scarponi

D4Science Infrastructure - Task #10519: Add the CMSY_VECTORIZED algorithm to the StockAssessment VREClosedPaolo Scarponi

Task #10649: Assess the Stock Assessment processesClosedGianpaolo Coro

Task #10705: Add user name to algorithms descriptions in SAI publicationClosedGianpaolo Coro

gCube - Task #10749: DataMiner algorithms sharingClosedGianpaolo Coro

gCube - Task #10750: The algorithms installer should manage a list of users wi...ClosedGianpaolo Coro

gCube - Task #10778: The DataMiner should filter the algorithms depending on t...ClosedLucio Lelii

gCube - Task #10779: The SAI should send information to the Pool Manager about...ClosedGiancarlo Panichi

gCube - Task #10994: SAI should allow deploying in another VREIn ProgressGiancarlo Panichi

D4Science Infrastructure - Task #10813: Make the new Support Vector Machines models available to ...ClosedPaolo Scarponi

D4Science Infrastructure - Task #11047: Install PhotoScan license server on dlib8x.dom0.research-...Closed_InfraScience Systems Engineer

D4Science Infrastructure - Task #11395: Reinstall dlib8x.dom0.research-infrastructures.eu changin...Closed_InfraScience Systems Engineer

Task #11415: Import Argo observation data in the BlueBRIDGE VREsClosedGianpaolo Coro

Project Task #689: T9.3 Data Publishing Facilities [Months: 1-29]ClosedLeonardo Candela

Task #1856: Assess CKAN platform for data publishingClosedFrancesco Mangiacrapa

Task #2265: Install ckanext-harvest - Remote harvesting extension (fo...ClosedAndrea Dell'Amico

Task #2266: Install GeoNetwork harvester for CKANClosedAndrea Dell'Amico

Task #2270: Install ckanext-ldap - LDAP authentication for CKANClosedAndrea Dell'Amico

Task #2278: Install ckanext-pdfview - a plugin for PDF files on CKANClosedAndrea Dell'Amico

Task #2522: Install privatedatasets - Private Datasets extension for ...ClosedAndrea Dell'Amico

Task #2523: Install ckanext-hierarchy plugin for CKANClosedAndrea Dell'Amico

Task #2524: Install ckanext-pages plugin for CKANClosedAndrea Dell'Amico

Task #2525: Install spatial_query and spatial_metadata CKAN extension...ClosedAndrea Dell'Amico

Task #2526: Install Spatial Search WidgetClosedFrancesco Mangiacrapa

Task #2530: Enable CKAN track visits to pages and setup a cron jobClosedAndrea Dell'Amico

Task #2555: Update CKAN CSW Harvester to support Geonetwork loginClosedAndrea Dell'Amico

Task #3135: Uri Resolver enhancements: create a Geonetwork ResolverClosedFrancesco Mangiacrapa

Task #3156: CSW GetRecords does not return "dc:identifier" for EGIP m...ClosedFrancesco Mangiacrapa

Task #3770: Fix the indexing cron jobClosed_InfraScience Systems Engineer

Project Activity #1436: Metadata Management EngineClosedPanagiota Koltsida

Task #4166: Develop a CKAN-based Data CatalogueClosedFrancesco Mangiacrapa

Task #3751: Create Ckan Data Catalogue PortletClosedFrancesco Mangiacrapa

gCube - Feature #4122: Creation of a ckan metadata publisher widgetClosedCostantino Perciante

Task #3645: Hide the navigation section of CKAN through a cookieClosedFrancesco Mangiacrapa

Task #3676: Ckan-connector: add a cookie for hiding the CKAN navigati...ClosedLucio Lelii

Task #4043: Update proxy rules for data-d.d4science.org/uri-resolverClosedAndrea Dell'Amico

gCube - Feature #4140: Create a Ckan utility libraryClosedCostantino Perciante

Task #4184: Create datacatalogue-metadata-discovery libraryClosedFrancesco Mangiacrapa

gCube - Feature #4203: Allow to create ckan organization from the vre definition...ClosedCostantino Perciante

gCube - Feature #4208: Please provide methods to create / delete a ckan organiza...ClosedLucio Lelii

gCube - Feature #4207: Uri Resolver upgrade: it must support new Geonetwork ManagerClosedFrancesco Mangiacrapa

Task #4202: Create 3 New hosts for CKAN production instanceClosed_InfraScience Systems Engineer

gCube - Feature #4233: Embedding ckan-metadata-publisher-widget into gcube-ckan-...ClosedCostantino Perciante

gCube - Feature #4234: Please allow dev (and dev2, dev5) machines to access ckan...Closed_InfraScience Systems Engineer

Task #4236: GeoExplorer integration with Geonetwork libraryClosedFrancesco Mangiacrapa

Task #4250: Geonetwork Resolver upgrade: it must return only "private...ClosedFrancesco Mangiacrapa

Task #4336: Create 2 New hosts for CKAN production: SoBigData and Blu...Closed_InfraScience Systems Engineer

Task #4827: Nagios monitoring for CKANClosed_InfraScience Systems Engineer

gCube - Feature #4331: Enhance the gcube-ckan-datacatalog portlet headerClosedCostantino Perciante

gCube - Feature #4330: Integrate ckan-publisher-widget into the workspaceClosedCostantino Perciante

gCube - Feature #4393: Improve the ckan license listClosedCostantino Perciante

Task #4455: Metadata Publisher Widget: should integrate CKAN Dataset ...RejectedCostantino Perciante

gCube - Feature #4470: Modify the product.ini ckan file in production, in order ...ClosedAndrea Dell'Amico

gCube - Feature #4529: Create a customized my Organizations page ClosedCostantino Perciante

gCube - Feature #4582: Liferay hook to remove ckan token on portal logoutRejectedCostantino Perciante

Task #4596: Check Ckan configurations on HTTPS: blocked loading mixed...ClosedAndrea Dell'Amico

Task #4641: Ckan HTTPS: workaround to download CKAN resources and avo...ClosedFrancesco Mangiacrapa

Task #4704: Nginx configuration: Error: Permission denied to access p...ClosedFrancesco Mangiacrapa

Task #4770: Create a custom theme "CKAN D4Science Theme" for CKANClosedFrancesco Mangiacrapa

Task #4831: Update licenses for products under the open-science organ...ClosedCostantino Perciante

Task #4832: Update licenses for products under the imarine organizati...ClosedCostantino Perciante

Task #4835: Can we add fileidentifier for Thredds layers?ClosedGianpaolo Coro

Task #4848: CKAN optimization: Consider the run of a nightly cron in ...ClosedFrancesco Mangiacrapa

Task #4849: Please install CKAN extension for managing datasets relat...Closed_InfraScience Systems Engineer

Task #4851: Customize CKAN Template: show resource links for public m...ClosedFrancesco Mangiacrapa

Task #4920: CKAN Metadata Profile advancements: provide vocabulary fi...ClosedFrancesco Mangiacrapa

Task #4921: Home Library advancements: provide methods to copy folder...ClosedValentina Marioli

Task #4923: Improve ckan publisher widget to handle listbox multisele...ClosedCostantino Perciante

gCube - Feature #4925: Improve ckan portlet flexibilityClosedCostantino Perciante

Task #4928: CKAN template enhancements: improve organization view upd...ClosedFrancesco Mangiacrapa

Task #5044: Update ckan-widget to support new metadata model (v.2)ClosedCostantino Perciante

Task #5088: Performing copy of workspace data into user's private cat...ClosedCostantino Perciante

Task #5119: CKAN role update: restrict CKAN permissions for Editor vi...ClosedFrancesco Mangiacrapa

Task #5134: Ckan library must support group creation and products' as...ClosedCostantino Perciante

Task #5184: Ckan-publisher-widget: create group for profile and assig...ClosedCostantino Perciante

Task #5196: CKAN role update: add/remove a dataset to a group must be...ClosedFrancesco Mangiacrapa

Task #5379: Update CKAN classes to fix 302 URL redirectClosedFrancesco Mangiacrapa

Task #5423: CKAN: Update ansible playbook in order to integrate new f...ClosedAndrea Dell'Amico

Task #5455: Check encoding used on Geonetwork, Geoserver and CKAN mac...Closed_InfraScience Systems Engineer

Task #5564: Data Catalogue Portlet improvements: Get Link per dataset RejectedFrancesco Mangiacrapa

Task #5615: Perform a ckan logout when browser tab closingClosedFrancesco Mangiacrapa

Task #5622: Move the smartgears ckan instance to a dedicated tomcat i...ClosedAndrea Dell'Amico

Task #5640: CKAN over HTTPs configurationsClosedAndrea Dell'Amico

Task #5742: Update the CKAN extension d4science_theme integrating cka...ClosedFrancesco Mangiacrapa

Task #5749: Upgrade CKAN in dev environment to a recently release (v....ClosedAndrea Dell'Amico

Task #5806: Install google analytics extension for CKANClosedAndrea Dell'Amico

Task #5817: Install plugin Google Analytics Report: ga-reportClosedAndrea Dell'Amico

Task #5891: CKAN Portlet: use Google Analytics Cross-domain trackingClosedFrancesco Mangiacrapa

Task #6067: Install ckanext-googleanalytics extension on productionClosedAndrea Dell'Amico

Task #6119: Provide CatalogueResolver: get/resolve a link to a CKAN E...ClosedFrancesco Mangiacrapa

Task #6392: CKAN Harvester: must be patched for allowing the use of d...ClosedFrancesco Mangiacrapa

Task #6492: Catalogue Resolver: "improve"/"build better" public URLs ...ClosedFrancesco Mangiacrapa

Task #6546: Python urllib3 misconfiguration between ckan-d-d4s and ck...ClosedAndrea Dell'Amico

Task #6563: Data catalogue improvementsClosedCostantino Perciante

Task #6564: SoBigData catalogue in sobigdata.d4science.org gateway ne...ClosedMassimiliano Assante

Task #6566: Remove catalogue's banner when user is not logged inClosedCostantino Perciante

Task #6567: Data catalogue portlet: investigate why the iframe render...ClosedCostantino Perciante

Task #6568: SoBigData catalogue: remove/hide map widgetClosedFrancesco Mangiacrapa

Task #6569: Catalogue's private products: always show title and descr...ClosedFrancesco Mangiacrapa

Task #6570: Enhance publication widgetClosedCostantino Perciante

Task #6572: Data catalogues: give access to Activity Stream and Recen...ClosedAndrea Dell'Amico

Task #6574: Product's resources enhancements ClosedFrancesco Mangiacrapa

Task #7207: Catalogue widget: support single file publication from Wo...ClosedCostantino Perciante

Task #6917: Publication widget: enable publication of a single fileClosedCostantino Perciante

Task #6952: Catalogue Resolver: update on the fly the Application Pro...ClosedFrancesco Mangiacrapa

Task #6962: CKAN production.ini: change privatedatasets configurationsClosedAndrea Dell'Amico

Task #7027: Ckan Harvester patch: update a private package setting "s...ClosedFrancesco Mangiacrapa

Task #7028: CKAN ansible playbook: supervisorctl restartClosedAndrea Dell'Amico

Task #7188: Add Dataset/Product facility: show up only to user 'sysad...ClosedFrancesco Mangiacrapa

D4Science Infrastructure - Task #3980: Configure geonetwork users in order to integrate CKANClosedFabio Sinibaldi

D4Science Infrastructure - Task #4919: Enhance raster data metadata for the data catalogueClosedGianpaolo Coro

Project Activity #5001: Provide for GRSF Data Catalogue ClosedFrancesco Mangiacrapa

Project Activity #5009: Create 1 New host for CKAN production instance: GRSFClosedAndrea Dell'Amico

Project Activity #5045: Create a new CKAN instance: ckan-grsfClosedAndrea Dell'Amico

Project Activity #5086: CKAN behaviour enhancements: add a hierarchical view for ...ClosedFrancesco Mangiacrapa

Task #5462: Creation of a web service to allow GRSF products publishingClosedCostantino Perciante

Task #5477: Customize CKAN template for GRSFClosedFrancesco Mangiacrapa

Task #5478: Create static Groups on CKAN for GRSFClosedFrancesco Mangiacrapa

Task #5498: Creation of the wiki page for GRSF Data CatalogueClosedCostantino Perciante

Task #5650: Allow ckan to accept repeated fieldsClosedFrancesco Mangiacrapa

Task #5889: GRSF management data catalogue: offer products' status ch...ClosedCostantino Perciante

Task #5890: Provide web service information to handle GRSF records st...ClosedYannis Marketakis

Project Activity #6088: Create a new CKAN instance: ckan-grsfadminClosedAndrea Dell'Amico

gCube - Feature #5443: Create workspace on-the-fly on GeoServerRejectedFabio Sinibaldi

Task #5659: Improve ckan-widget with a resource pickerClosedCostantino Perciante

Task #5756: Improve ckan widget: let the user choose the groups of th...ClosedCostantino Perciante

Task #5600: Implementation of grsf-services-coreClosedYannis Marketakis

Task #6259: Update GRSF-services-updater service to support direct pu...ClosedYannis Marketakis

Task #6655: Creation of a custom widget for GRSF management facilitiesClosedCostantino Perciante

Task #6660: Implementation of a patch service for retrieving Fishery ...ClosedYannis Marketakis

Task #7541: Improvements on gCube Resource Catalogue (Release 4.4)ClosedFrancesco Mangiacrapa

Task #7496: Ckan Geonetwork Harvester patch: update a private package...ClosedFrancesco Mangiacrapa

Task #7324: Install plugin ckanext-geoview: Geospatial viewer for CKA...ClosedAndrea Dell'Amico

Task #7351: Metadata Profile enhancement: add new capabilitiesClosedFrancesco Mangiacrapa

Task #7441: Install ckanext-dcat: linked data and RDF features for CKANClosedAndrea Dell'Amico

Task #7544: Metadata profiles have to be renderd as Types, not as GroupsClosedFrancesco Mangiacrapa

Task #7415: Create a web service for supporting catalogue publication...ClosedCostantino Perciante

Task #7616: Porting gcubedatacatalogue-metadata-discovery library to ...ClosedFrancesco Mangiacrapa

Task #7722: Provide for "share link"ClosedCostantino Perciante

Task #7871: Metadata Profile v3: improvementsRejectedFrancesco Mangiacrapa

Task #7933: Move publisher widget to new metadata profiles schema (v.3)ClosedCostantino Perciante

Task #7985: Ckan Theme Plugin enhancements: porting to Metadata Profi...ClosedFrancesco Mangiacrapa

Task #8032: Ckan D4STheme Plugin enhancements: required access to pri...ClosedFrancesco Mangiacrapa

Task #8267: Medatada Discovery Library: definition of namespaces and ...ClosedFrancesco Mangiacrapa

Task #8286: Ckan D4STheme Plugin enhancements: display items grouped ...ClosedFrancesco Mangiacrapa

Task #8287: CKAN ansible playbook: new configurationsClosed_InfraScience Systems Engineer

Task #8319: Install ckanext-dcat to the productionClosed_InfraScience Systems Engineer

Task #8320: Improvements on gCube Resource Catalogue (Release 4.5)ClosedCostantino Perciante

Task #7440: Ckan harvester investigation: how to optimize memory usageClosedFrancesco Mangiacrapa

Task #5807: Install OAI-PMH harvester for CKANClosedAndrea Dell'Amico

Task #8571: Plese improve metadata model with "propagateUp" propertyClosedFrancesco Mangiacrapa

Task #8298: Create a script to set Types (as system:type) to ckan items ClosedCostantino Perciante

Task #8662: Add a patch for ckanext-geonetwork via ansibleClosedFrancesco Mangiacrapa

Task #8726: Ckan Geonetwork Harvester: add snippet to set type as "Da...ClosedFrancesco Mangiacrapa

Task #8748: Ckan D4STheme Plugin enhancement: create a dedicated page...ClosedFrancesco Mangiacrapa

Task #8888: Restore ckan-bb before June 2, 2017, 10:00 PMClosed_InfraScience Systems Engineer

Task #8925: Create a script in order to permanently delete all datase...ClosedFrancesco Mangiacrapa

Task #8970: Switch ckan-sobigdata.d4science.org to ckan-sobigdata2.d4...Closed_InfraScience Systems Engineer

Task #9053: Improvements on gCube Resource Catalogue (Release 4.6)ClosedFrancesco Mangiacrapa

Task #9007: Catalogue WS: avoid transparent namespaces handlingClosedCostantino Perciante

Task #9026: CKAN ansible playbook enhancement: deploy language for ho...Closed_InfraScience Systems Engineer

Task #9015: Ckan harvester enhancement: set "Item URL" via harvester ...ClosedFrancesco Mangiacrapa

Project Activity #8964: Data Catalogue enhancement: provide a "method" to visuali...ClosedFrancesco Mangiacrapa

Task #10086: Catalogue portlet refactoringClosedCostantino Perciante

Project Activity #10472: View Single VRE in Catalogue ClosedCostantino Perciante

Task #8425: Dataset pages time loading is high in production env: inv...Closed_InfraScience Systems Engineer

Task #6561: A small pool connections nginx/apache/ckan causes Timeout...ClosedFrancesco Mangiacrapa

Task #8199: Check that every item has an item's url fieldClosedFrancesco Mangiacrapa

Task #6985: Liferay snippet: perform ckan logout in case of liferay l...ClosedFrancesco Mangiacrapa

Task #5521: CKAN update roles/permissions: a public dataset must prov...RejectedFrancesco Mangiacrapa

Task #4830: Improve metadata stored in GeonetworkClosedCostantino Perciante

Task #9076: Catalogue Widget: manage workspace "Url" file type properlyClosedCostantino Perciante

Task #9094: CKAN D4S: purge all datasetsClosedFrancesco Mangiacrapa

Task #8879: Ckan harvester: set up configuration for GN Tuna AtlasClosedFrancesco Mangiacrapa

Task #9091: Data Catalogue enhancement: force harvesting of ISO 19139...ClosedFrancesco Mangiacrapa

Task #9130: SE plugin: Notify VRE Managers/Catalogue Admins about Cat...RejectedCostantino Perciante

Task #9400: Ckan library: add support for harvestersClosedCostantino Perciante

Task #8817: Install OAI-PMH Plugin for CKANClosedFrancesco Mangiacrapa

Task #9298: Configure OAI-PMH CKAN plugin Closed_InfraScience Systems Engineer

Task #9315: Create email addresses for Catalogue managersClosedTommaso Piccioli

Task #9299: ckan-d-d4s core error End of script output before headers...ClosedAndrea Dell'Amico

Task #9372: How-to install ckan OAI plugin in productionClosedAndrea Dell'Amico

Task #9145: Fix resource format (mime type) in ckan util libraryClosedCostantino Perciante

Task #9281: CKAN harvesting improvements: assign the system:type acco...ClosedFrancesco Mangiacrapa

gCube - Task #9467: Ckan Resources: 'created' field is set to unknownClosedFrancesco Mangiacrapa

Task #9728: CKAN provisioning enhancement: create an ansible module f...Closed_InfraScience Systems Engineer

Task #10056: CKAN template enhancement: resources belonging to public ...ClosedFrancesco Mangiacrapa

gCube - Task #10070: Catalogue Resolver enhancement: resolve public/private it...ClosedFrancesco Mangiacrapa

Task #10383: gCube Metadata Profile: add field repeatability definitio...ClosedFrancesco Mangiacrapa

Task #10392: Data Catalogue: create a portlet to provide a widget faci...ClosedMassimiliano Assante

Project Activity #9201: Publication & Harvesting of ISO 19119 service metadata ap...ClosedGianpaolo Coro

Project Task #9131: Inventory mappings between WPS 1.0.0 (and OWS) vs. ISO191...ClosedEmmanuel Blondel

Project Task #9202: Test CKAN Harvesting of GN ISO19119 service metadataClosedFrancesco Mangiacrapa

Task #9312: Improve CKAN template for dataset pagesClosedFrancesco Mangiacrapa

Project Activity #9314: Improve CKAN GN Harvesting to support links to GN metadat...ClosedFrancesco Mangiacrapa

Project Activity #9379: Reinforce CKAN GN harvester to deal with Author and Manin...ClosedFrancesco Mangiacrapa

Task #9975: Improve ASFIS AquaMaps distribution metadata ClosedPasquale Pagano

Task #10520: GRSF-services-core updatesClosedYannis Marketakis

Project Activity #10572: Prioritize information about species, and assessment area...ClosedYannis Marketakis

StocksAndFisheriesKB - Task #11464: Update stock endpoint for GRSF-Publication-ServiceClosedYannis Marketakis

Task #11468: Publish algorithms in the catalogueClosedFrancesco Mangiacrapa

gCube - Feature #3929: Implement a liferay hook to avoid mutiple emails when a u...ClosedPanagiota Koltsida


Related issues

Related to BlueBRIDGE - Key Performance Indicator #973: Blue Commons: Number of new/enhanced Service and Library Closed Oct 12, 2015 Feb 28, 2018
Related to BlueBRIDGE - Risk #978: 2. Depending technologies / specifications get abandoned Closed Oct 12, 2015 Feb 28, 2018
Related to BlueBRIDGE - Risk #980: 4. Delivered systems miss functional or non-functional (p... Closed Oct 12, 2015 Feb 28, 2018
Related to BlueBRIDGE - Risk #982: 6. A partner / body underperforms, defaults or faces othe... Closed Oct 12, 2015 Feb 28, 2018
Related to BlueBRIDGE - Risk #983: 7. Domain or project’s concepts become obsolete or irrele... Closed Oct 12, 2015 Feb 28, 2018
Related to BlueBRIDGE - Risk #984: 8. Delays in production of deliverables, be it interim on... Closed Oct 12, 2015 Feb 28, 2018
Related to BlueBRIDGE - Risk #986: 10. Lack of required competences for the completion of pr... Closed Oct 12, 2015 Feb 28, 2018
Related to gCube - Feature #1653: Redesign and Reimplement USers Management portlet Closed Mar 31, 2016
Related to gCube - Feature #1319: Liferay Migration from 6.0 to 6.2 Closed Jan 08, 2016 Jul 22, 2016
Related to BlueBRIDGE - Task #2120: Provide tools to modify VREs presentation text and to sub... Closed Feb 03, 2016 Mar 18, 2016
Related to gCube - Feature #2484: Reimplement Roles Management portlet Closed Mar 07, 2016
Related to gCube - Feature #2867: Create an new portlet for managing Site-Teams per VRE Closed Mar 22, 2016
Related to gCube - Feature #3972: Allow users to login to portal with different e-mail addr... Closed May 10, 2016
Related to gCube - Feature #3974: Create a portlet where users can declare their additional... Closed May 10, 2016

History

#1 Updated by Franco Zoppi over 3 years ago

  • Start date changed from Sep 12, 2015 to Sep 01, 2015

due to changes in a related task

#2 Updated by Franco Zoppi over 3 years ago

  • Due date changed from Feb 28, 2018 to Jan 31, 2018

due to changes in a related task

#3 Updated by Franco Zoppi over 3 years ago

  • Status changed from New to In Progress

#4 Updated by Massimiliano Assante over 3 years ago

  • Assignee set to Massimiliano Assante

#5 Updated by Massimiliano Assante over 3 years ago

M1 September 2015 Activity

Summary of the discussions occurred during the KO meeting:

For Data Access and Storage (T9.1) the Task Leader proposed to re-use already existing solutions provided from the gCube infrastructures and enhance them according to the needs. While for Data Discovery a possible architecture for assisting the discovery of datasets was presented to the audience. The involved partners agreed to discuss the details and the actual plan of the Task in a forth-coming face-2-face meeting (BlueCommons Meeting was scheduled, 3-5 November, Athens).
An overview of the D4Science techniques for Data Analytics (T9.2) and of the users and stakeholders currently using them was shown by the Task Leader at the BlueBRIDGE Kick-off, along with a short-term plan. The involved partners agreed on enhance the current set of facilities offered by the Statistical Manager and by the Tabular Data Manager services of the D4Science Infrastructure.
For Data Publishing a presentation from the WP9 Leader followed by a discussion aiming at clarifying the goal of the task and the potential partners’ contribution occurred at the BlueBRIDGE Kick-off. From this it emerged that gCube technology already has some of the expected facilities yet effort is needed to complete and complement them with new ones thus to fully meet the expectations. The involved partners agreed that the data produced by D4Science services or the data federated by the infrastructure should be published in an Infrastructure Data Catalogue and to further discuss this in the upcoming BlueCommons Meeting (BlueCommons Meeting was scheduled, 3-5 November, Athens).

List of planned actions for the first 6 months reported by the Task Leaders:

T9.1

  • Inventory of already existing solutions from the gCube infrastructure that could/should be reused in BlueBRIDGE [Oct'2015]
  • Specify the requirements (also identify if current solutions are not enough for the problem at hand) [Oct-Nov'2015]
  • Design new functionalities [Nov-Dec'2015]
  • Start the implementation of the new functionalities [Dec'2015-Feb'2015]

T9.2

  • Publication of all the StatMan algorithms via WPS [Oct. ’15->Jun ‘16]
  • Production of NetCDF files out of geospatial data in D4Science [Mar ‘16]
  • Estimating best practices of infra usage, stock assessment, biodiversity analysis [Aug ‘16]
  • Enhancing standard description of datasets: SDMX datasets import and export [Feb ‘16]
  • Enhancing StatMan parallel processing capabilities [Oct ’15->Sep ‘16]
  • Integration of BB community models: EwE, ICCAT, FIN, UoA [Sept ’15->2017]
  • Connecting SeaDataNet DIVA interpolation [Jan ‘16]
  • Enhancing social networking to manage experiment-specific discussions [Feb ‘16]
  • Automatic reports production from users’ interactions [Jun ‘16]
  • Supporting offline/online e-learning and courses [Sep ’15->2017]

T9.3

  • to develop a project-wide shared understanding of data publishing and formulate major requirements;
  • to start collecting information on data and other VRE products to be published by liaising with WP5-8;
  • to liaise with WP2 to develop the BlueBRIDGE Data Management Plan;
  • to deeply inspect gCube services dealing with data publishing to identify strength and weaknesses;
  • to further discuss data publishing in a technical meeting (Nov. ‘15);

List of activities performed beyond the KO meeting (T9.2 Only):

  • Developing and releasing an algorithm to publish raster data in the infrastructure. This activity also involved enhancements on the Data Transfer services.
  • Developing an authentication method for the WPS interface to the StatMan algorithms that was compliant with external clients and software (e.g. QGIS, Chrome etc.).
  • Integrating two algorithms developed by Universidade Federal de Mato Grosso (Brazil) which normalise precipitation data
  • Refining a Time Series Forecasting process and applying it to Indian Ocean Tuna Commission data
  • The Taverna software is supporting WPS, the Taverna team is not going support the CNR team with assessment and testing activity. Nevertheless, this assessment has been substituted by a WPS service testing using Quantum GIS.

#6 Updated by Pasquale Pagano over 3 years ago

#7 Updated by Pasquale Pagano over 3 years ago

  • Related to Risk #978: 2. Depending technologies / specifications get abandoned added

#8 Updated by Pasquale Pagano over 3 years ago

  • Related to Risk #980: 4. Delivered systems miss functional or non-functional (performance, scalability, etc.) requirements added

#9 Updated by Pasquale Pagano over 3 years ago

  • Related to Risk #982: 6. A partner / body underperforms, defaults or faces other severe operation issues added

#10 Updated by Pasquale Pagano over 3 years ago

  • Related to Risk #983: 7. Domain or project’s concepts become obsolete or irrelevant during project’s lifetime added

#11 Updated by Pasquale Pagano over 3 years ago

  • Related to Risk #984: 8. Delays in production of deliverables, be it interim ones (with dependencies) or final ones added

#12 Updated by Pasquale Pagano over 3 years ago

  • Related to Risk #986: 10. Lack of required competences for the completion of project’s tasks added

#13 Updated by Massimiliano Assante over 3 years ago

M2 October 2015 Activity

In T9.1 FORTH worked on 2 parallel activities, completing the inventory on the gCube existing solutions for Data Storage, Discovery and Access (as planned) and analysing the functional requirements for this task expected functionalities. Both activities were presented and discussed at the BlueCommons Meeting held in Athens at the very beginning of November. Within this task CNR enhanced the HomeLibrary component, which stands at the base of the user's Workspace. In particular there were performance improvements and enhancements in searching in specific folders, locking and, user's file history tracking.

For the Data Analysis task (T9.2) most of the work was performed on the Statistical Manager part.
Specifically:

  • a WPS connector for R was developed to exploit and run BlueBRIDGE statistical manager algorithms from R. This is particularly important as it makes the D4Science Infrastructure resources accessible through the tools that are commonly used ( R ) by our reference communities of practice (CoP).
  • an algorithm to produce uniform environmental maps from satellite or in situ data was developed. This algorithm exploit the SeaDataNet DIVA interpolation web-service and was successfully tested with Copernicus data from ESA. This work is at the crossroad of T9.2 for the analytics part and T10.1 (Integrate Computing Infrastructure Resources) for the fact that we exploit part of the computing resources offered by Seadatanet.
  • an algorithm to produce GRID-CWP codes from longitude and latitude columns in a table was completed and discussed with FAO.
  • The porting of Vessels Transmitted Information algorithms onto the Statistical Manager started and is planned to be completed by the end of 2015.

For the Data Publishing task (T9.3) an activity leading to developing of a common understanding of Data Publishing started.
This includes:

  • discussions aiming at analysing it by using a data-type centric occurred at the BlueCommons Meeting held in Athens;
  • discussions occurred with WP2 to support the formulation of the Data Management Plan;
  • discussions on the commonalities and differences between the needs of data publishing and the activity leading to the development of the data catalogue.

Within this task NKUA redesigned the gCube OAI-PMH provider in order to support different data models and performed a survey on technologies and frameworks used to build metadata repositories and how these can be used in gCube.

#14 Updated by George Kakaletris over 3 years ago

  • Related to Task #1340: Move CITE's OpenSource code into a separate publicly accessible repository added

#15 Updated by George Kakaletris over 3 years ago

  • Related to Feature #1440: Remove index ingestion dependency to TreeManager added

#16 Updated by George Kakaletris over 3 years ago

  • Related to Feature #1438: Remove Resource Registry dependency added

#17 Updated by George Kakaletris over 3 years ago

  • Related to deleted (Feature #1440: Remove index ingestion dependency to TreeManager )

#18 Updated by Massimiliano Assante over 3 years ago

M3 November 2015 Activity

In T9.1 the partners worked on the facets identification and their description. This activity is core for the design of the new Resource Model, a first draft of the facets has already been identified and can be found in the BlueBRIDGE WIKI. See https://support.d4science.org/projects/bluebridge/wiki/On_Resources_Facets_Catalogues_and_the_Forthcoming_Information_System (#1310).
Also:

  • Several enhancements, bugs and new features of the Home Library have been carried out (#894, #710, #519, #724, #725, #739, #1059, #1118).
  • CNR started enhancing the GisViewer (#1283) to make it compliant with non WMS Standard layers (the one of Thredds, a raster webmap server)

For the Data Analysis task (T9.2) most of the work was performed on the Statistical Manager part.

  • The Statistical Manager connector to the SeaDataNet interpolation service activity has been finalised and the component is under integration.
  • The porting and development of the old VTI algorithms onto the Statistical Manager has been completed, so far only 1 out of 5 of these algorithms is available in production but the other ones are going to be present in the next release. A design and * implementation of a new system to allow community scientists to upload algorithms by themselves has started #1451

For the Data Publishing task (T9.3) UOA continued working on the analysis of "metadata engines", namely DSpace and Fedora and CNR on an analysis of the data publishing practices promoted by generalist scientific data repositories, which is still ongoing.

#19 Updated by Massimiliano Assante over 3 years ago

M4 December 2015 Activity

In T9.1 (Data Access and Storage) the partners worked on the facets identification and their description. In particular during December FORTH worked on the description of a Dataset using Facets in order to understand if the identified facets (https://support.d4science.org/projects/bluebridge/wiki/Facets) are enough and appropriate to describe this resource. See also https://support.d4science.org/projects/bluebridge/wiki/On_Resources_Facets_Catalogues_and_the_Forthcoming_Information_System (#1310).

For the Data Analysis task (T9.2) most of the work was performed on the Statistical Manager part.

  • The Statistical Manager connector to the SeaDataNet interpolation service has been released and a video has been produced (http://goo.gl/vI2tRZ), which demonstrates how to process Copernicus data and publish/share a map on the e-Infrastructure #843.
  • A new version of the Statistical Manager has been completed and tested, which enhances performance and solves several bugs
  • Activity on the integration of the EwE stock assessment algorithms has continued, IRD, CNR and ENG agreed on Integrate .Net algorithms on the BlueBRIDGE computing platform. ENG developed a first version of the EwE wrapper which is currently under evaluation.
  • DataMiner, the gCube service providing state-of-the art DataMining algorithms and ecological modelling approaches under the Web Processing Service (WPS) standard has been installed in the production environment and is now available.

For the Data Publishing task (T9.3):

  • UOA continued working on the analysis of "metadata engines", but there are no report available yet;
  • the analysis of data publishing practices promoted by generalist repositories, continued and early findings have been reported and summarised.
  • an analysis of the CKAN open source technology for supporting data publishing started in CNR. It is planned to deploy such a technology and start populating it;
  • FORTH started integration activities of MatWare with X3ML engine for supporting the transformation needs towards the construction of a global registry of stocks and fisheries.

#20 Updated by Massimiliano Assante over 3 years ago

  • Related to deleted (Feature #1438: Remove Resource Registry dependency)

#21 Updated by Massimiliano Assante over 3 years ago

  • Related to deleted (Task #1340: Move CITE's OpenSource code into a separate publicly accessible repository)

#22 Updated by Massimiliano Assante over 3 years ago

M5 January 2016 Activity

In T9.1 (Data Access and Storage) FORTH finalised the work on the facets identification for the resource type Dataset.
A detailed description with emphasis on the weaknesses was presented at the 1st TCOM. The results of this activity can be found at the following link. https://support.d4science.org/projects/bluebridge/wiki/Resource_Profiles#section-6
At the end of the month, CNR started the design, study and implementation of a full-text search over posts and comments of the social part. It was decided to adopt Elastic (Formerly Elasticsearch) as enabling technology. More information available at #2091

For the Data Analysis task (T9.2)

  • The work on the Statistics Algorithm Importer (SAI), the interface allowing scientists to upload algorithms by themselves, continued #1451. The plan is to roll this out within the gCube release (3.10.1).
  • The work on the integration of the Ecopath with Ecosim (EwE) stock assessment algorithms has been completed and is available in the development environment. The plan is to roll this out within the next gCube release (3.11.0).
  • IRD requirements coming from WP5:
  • import Sardara-produced NetCDF files in the infrastructure - is underway, these files needs to be prepared for publishing by processing them with a dedicated Statistical Manager algorithm, see #1983
  • Integration and experimentation of the VPA Workflow by IRD, Ifremer and ICCAT was started #1903

Also, one algorithm to retrieve information from FishBase tables was provided via WPS and experimented with IRD #1777

For the Data Publishing task (T9.3)

  • import Sardara-produced NetCDF files in the infrastructure - is underway, in particular it was decided to publish these NedCDF files in the D4Science THREDDS Data Server (TDS) instance. After the preparation phase performed by the Statistical Manager. #1983
  • an instance of the CKAN open source technology has been created and it is under testing;
  • the analysis of generalist data repositories wrt the support they offer for data publishing has been completed, 5 repositories have been analysed and the results of this analysis have been written in a paper currently under review #1706.
  • the integration of MatWare with X3ML engine for supporting the transformation needs towards the construction of a global registry of stocks and fisheries from FORTH is progressing #1806.
  • NKUA work on a Metadata Management Engine is progressing, in particular the Fedora Repository 4.x has been tested but without encouraging results and the api is "unstable". Currently MongoDB is being tested under this particular usage pattern. #1436

#23 Updated by Massimiliano Assante over 3 years ago

  • Related to Feature #1653: Redesign and Reimplement USers Management portlet added

#24 Updated by Massimiliano Assante over 3 years ago

  • Related to Feature #1319: Liferay Migration from 6.0 to 6.2 added

#25 Updated by Massimiliano Assante about 3 years ago

M6 February 2016 Activity

In T9.1 (Data Access and Storage) CNR worked on the implementation of full-text search support over posts and comments of the social part. This activity has been completed and included in the next gCube Release 3.11.0. More information available at #2091. Several improvements have been delivered to the Workspace and Messages applications.
FORTH worked on the definition of the requirements for supporting data discovery services by exploiting semantic knowledge bases, this activity is still on going #2246.

For the Data Analysis task (T9.2) CNR worked on:

  • finalising the Statistical Algorithms Importer (SAI) #1451
  • porting of (IRD Ifremer and ICCAT) Virtual Population Analysis (VPA) Workflow for Stocks assessment in the Statistical Manager platform (#1903)
  • releasing a more efficient version of StatMan (2.0) in the production environment #1222
  • investigating with FAO and IRD possible strategies to transform geospatially explicit datasets into NetCDF files #2230.

Also, The Knitr (allowing flexible and fast dynamic report generation with R) compiler was released in the production environment and is available as an algorithm in the Statistical Manager #1825. Knit is exploited also as one workflow step of the Virtual Population Analysis (VPA) Workflow for Stocks assessment.

For the Data Publishing task (T9.3)

  • CNR continued working on CKAN (#1856) by analysing existing plugins/extensions aiming at enlarging its capabilities, specifically an initial set of extensions to be installed have been identified (e.g. to interface with GeoNetwork and LDAP, to deal with PDF)
  • An analysis of the first results of the RDA working group on Metadata started, https://rd-alliance.org/metadata-p7-tokyo.html
  • NKUA work on a Metadata Management Engine is progressing (#1436), NKUA clarified that the engine is meant to aggregate metadata for the sake of discovery
  • FORTH integration of MatWare with X3ML engine for supporting the transformation needs towards the construction of a global registry of stocks and fisheries is progressing but at the time of writing we haven't received any feedback on this, although the due date was few days ago.

#26 Updated by Gianpaolo Coro about 3 years ago

  • Due date changed from Jan 31, 2018 to Mar 09, 2016

due to changes in a related task

#27 Updated by Gianpaolo Coro about 3 years ago

  • Start date changed from Sep 01, 2015 to Oct 19, 2015
  • Due date changed from Nov 06, 2015 to Mar 09, 2016

due to changes in a related task

#28 Updated by Massimiliano Assante about 3 years ago

  • Related to Task #2120: Provide tools to modify VREs presentation text and to subscribe participants added

#29 Updated by Massimiliano Assante about 3 years ago

  • Related to Feature #2484: Reimplement Roles Management portlet added

#30 Updated by Panagiota Koltsida about 3 years ago

  • Related to Feature #2867: Create an new portlet for managing Site-Teams per VRE added

#31 Updated by Leonardo Candela about 3 years ago

  • Due date changed from Nov 06, 2015 to Mar 09, 2016

due to changes in a related task

#32 Updated by Leonardo Candela about 3 years ago

  • Start date changed from Oct 19, 2015 to Sep 01, 2015
  • Due date changed from Nov 06, 2015 to Jan 31, 2018

due to changes in a related task

#33 Updated by Massimiliano Assante about 3 years ago

M7 March 2016 Activity

In T9.1 (Data Access and Storage) FORTH finalised the definition of the requirements and a draft design for supporting Data(set) Discovery (Task #2246) in a document (http://goo.gl/WltYey) that CNR is reviewing in order to provide feedback. Also, improvements have been requested to the Workspace Resources Explorer web application, in order to equip it with filtering and display features to better support the DataMiner use case (Feature #2546).

For the Data Analysis task (T9.2) CNR worked on:

  • The IRD VPA Workflow for stock assessment was integrated using the Statistical Algorithm Importer (#1451) to demonstrate the feasibility of running this process in the infrastructure (#1903). In particular CNR shared with IRD staff the Statistical Algorithms Importer (SAI) project used to generate the currently running VPA Workflow algorithm.
  • CNR released a new version of the Statistical Manager and Dataminer in the production environment, and increased the number of executable algorithms.
  • Dataminer was equipped with functions to manage provenance using the infrastructure Workspace.
  • CNR finished releasing a process to download and extract GIS information from the infrastructure as ESRI GRID files (ASCII format) #2229

For the Data Publishing task (T9.3) CNR worked on the CKAN exploitation / assessment (#1856), in particular:

  • 9 CKAN plugins have been selected and installed on the CKAN Instance of the development infrastructure(see #1856).
  • a mechanism enabling to harvest both public and private datasets from a GeoNetwork instance has been developed and tested in the development infrastructure.
  • an harvesting test has been performed and produced 13,247 datasets published into a test installation of CKAN.

#34 Updated by Massimiliano Assante about 3 years ago

M8 April 2016 Activity

In T9.1 (Data Access and Storage and Discovery) the activities revolved around:

  • Investigation on the requirements for supporting data discovery services by exploiting semantic knowledge bases (#2246)
  • Started the activities for enhancing the functionalities of Workspace Resources Explorer (#2546)

For the Data Analysis task (T9.2) CNR worked on:

  • investigating the possibility to integrate the e-Infrastructure computational facilities with the OpenCPU technology. This will allow Web developers to simply integrate javascript libraries to interact with Dataminer #3787
  • developing a wrapper portlet for RStudio that allows users to work directly with this IDE without passing through TabMan (as the previous approach required) #3169
  • supporting the BlueBRIDGE users' and developers' community with tools and discussions to manage charts annotations #3153 and publication of NetCDF files #1983

Also, CNR continued developing the porting of the StatMan interface onto DataMiner #2521 and on developing an algorithm to publish HTML pages on a public website hosted by the infrastructure, was released on the BlueBRIDGE portal #2520

For the Data Publishing task (T9.3), the assessment and exploitation of CKAN technology continued:

  • a ckan-connector has been created and installed (#3063) to support automatic login via token and automatic user creation on ckan in case of first login
  • the GeoNetwork resolver has been reinforced to use Geonetwork MEF services (#3791);
  • the development of a specific portlet started (#3751). This portlet should integrate the CKAN instance in a VRE, e.g. by discovering the ckan-connector from IS and contact it passing the gcube-token; by displaying the redirect to "ckan-d-d4s.d4science.org" returned by ckan-connector in IFrame object;

The procedure enabling to publish geospatial raster data (Raster data publisher algorithm) has extended to publish all kinds of NetCDF files as well as NCML files (#3138).

#35 Updated by Leonardo Candela about 3 years ago

  • Start date changed from Sep 01, 2015 to Oct 19, 2015
  • Due date changed from Jan 31, 2018 to Mar 31, 2016

due to changes in a related task

#36 Updated by Leonardo Candela about 3 years ago

  • Start date changed from Oct 19, 2015 to Oct 15, 2015
  • Due date changed from Mar 31, 2016 to Jun 30, 2016

due to changes in a related task

#37 Updated by Panagiota Koltsida about 3 years ago

  • Related to Feature #3972: Allow users to login to portal with different e-mail addresses under the same account added

#38 Updated by Panagiota Koltsida about 3 years ago

  • Related to Feature #3974: Create a portlet where users can declare their additional e-mail addresses added

#39 Updated by Massimiliano Assante almost 3 years ago

M9 May 2016 Activity

In T9.1 (Data Access and Storage and Discovery) FORTH worked on Matware plugins for supporting the transformation of data for the construction of GRSF (#3926).
CNR and NKUA worked on the development of new facilities and portlets for the gCube framework (#3914). http://next.d4science.org

For the Data Analysis task (T9.2) CNR worked on:

  • CNR worked to make the RStudio environment directly accessible from the portal, using the same authorization methods of the portal and generally of the infrastructure #3169 #2516
  • The OpenCPU service was integrated with Dataminer, in order to provide a javascript connection to the infrastructure algorithms. This was achieved by porting the Dataminer-WPS-R-connector on OpenCPU. FAO is currently integrating an algorithm to demonstrate charts production on CCAMLR (Antarctic) data #3787 using this technology. This activity came after a sequence of discussions with FAO and IRD about the best technology to use #3153
  • Large effort was spent on the refinement of Dataminer, to let it communicate with the Workspace #2545, in order to manage provenance, and on the porting of the StatMan Web interface to Dataminer #2521
  • The Ichtyop model was ported on StatMan and was also endowed with a WPS interface, by porting it on Dataminer #3986

For the Data Publishing task (T9.3), the assessment and exploitation of CKAN technology continued, in particular:

  • the development of a publishing widget (to enact metadata feeding) started (#4122);
  • the development of a portlet interfacing with the data catalogue started (#3751);
  • a model for mapping gCube concepts (namely context) to CKAN concepts (namely hierarchies of Organizations and Groups) has been defined;

On May 24th a WP9 telco among partners occurred. Agenda and Minute in the related wiki page: https://support.d4science.org/projects/bluebridge/wiki/WP920160524Telco

#40 Updated by Massimiliano Assante almost 3 years ago

M10 June 2016 Activity

In T9.1 (Data Access and Storage and Discovery) FORTH worked on Matware plugins for supporting the transformation of data for the construction of GRSF (#3926). In particular MatWare plugins for FIRMS were updated #3927, new plugins for RAM #3928 have been created, and plugins for FishSource #4284) are ongoing (#3926).
CNR worked on several new functionalities of Workspace (#2546). NKUA finalised the development of the new UsersManagement Portlet and social login hooks.

For the Data Analysis task (T9.2) CNR helped in the development of a connection between a FAO Web application and Dataminer for a CCAMLR demo tool #3787. It also worked on the porting of the StatMan interface on Dataminer (activity is completed) #2521

CNR provided support to a number of WP5/WP6/WP7 related issues about algorithms development (e.g. #4211, #3563, #3158, #1777, #3835, #4211, #2289, #2344)

For the Data Publishing task (T9.3), the activity focused on the exploitation and enhancement of the CKAN technology to implement a Data Catalogue (#4166). In particular:

  • a widget enabling to publish "products" in CKAN has been developed (#4122);
    • this has been integrated in the Workspace for publishing ;
    • this has been integrated in the Data Portlet (#4233)
  • the development of a specific Data Portlet continued (#3751), e.g.
    • added proper management of log out,
    • portlet header has been reinforced (#4331),
    • a custom version of My Organisations page has been created (#4529);
  • a revised version of the GeoExplorer component has been released to deal with the changes on spatial datasets publishing (#4236);
  • the production-oriented deployment scenario has been created (#4202);
    • a new deployment strategy envisaging the deployment of a Catalogue per VO plus a Catalogue for the entire infrastructure has been defined (#4336);
  • a presentation on Data catalogue has been prepared for the TCom;

#41 Updated by Massimiliano Assante over 2 years ago

M11 and M12 July and August 2016 Activity

In T9.1 (Data Access and Storage and Discovery) FORTH worked Development of MatWare plugins for supporting the construction of the GRSF KB (#3926); In particular the Matware plugins for FIRMS data source were completed (#3927) while Matware plugins for RAM and FishSource data sources are still ongoing (#3928, #4284).

NKUA finalised the development of a new OAI-PMH harvester meant to be working in the new gCube Search Framework NKUA is currently working on (based on Elastic search). In order to
manage and monitor the available oai-pmh sources a portlet is being developed as well by NKUA, currently under testing (#3914). Both the Harvester and the portlet are going to be part of the next gCube 4.1.0 release.

CNR studied and experimented the feasibility to migrate from Apache JackRabbit 2 (current workspace service repository) to Apache Jackrabbit 3 (Oak) #4595. The Preprod portal has been updated to Oak with new HomeLibrary libraries and a new HomeLibrary webapp. CNR also worked on bug fixing and resolving incidents on the Messaging and Workspace services #2747

For the Data Analysis task (T9.2) 30 algorithms circa were provided by the BlueBRIDGE community were integrated with Dataminer using the Statistical Algorithm Importer #4610. Moreover:

  • The OSCAR NASA data were ported to the production environment to be used in drifting experiments #4802 and a parallelised version of the IRD Ichthyop model was implemented and integrated with Dataminer #4195.
  • An algorithm able to import Shapefiles within the infrastructure was finished and proposed to Grid-A for experiments #4745 and an active support was given to Grid-A and FAO by CNR in the development of a script to intersect WFS layers #4211
  • The ICCAT Blue Fin Tuna algorithms were ported to the production environment #4185

In addition, the following developments were completed:

  • An operation to rapidly link algorithms and get their user interface was developed #4713
  • The Dataminer interface now manages polygons drawing #4172
  • The StatMan interface was completely ported on Dataminer #2521

For the Data Publishing task (T9.3) the first version of the data catalog (based on the CKAN technology) has been released and made available in the infrastructure. Specifically:

  • an instance of CKAN has been deployed to act as the BlueBRIDGE data catalogue (#4336) and added to the monitoring infrastructure (#4827);
  • the set of supported licenses has been revised (#4393 #4470 #4831 #4832);
  • the CKAN metadata publisher facility has been released (#4122);

In addition, the following developments were completed to support the above mentioned facilities:

  • the CKAN data catalogue portlet has been enhanced and released, e.g. added further links (#4331);
  • the management of product to product relationship is progressing, some misbehaviour has been identified (#4455);
  • GeoNetowrk users configuration has been reconsidered to enable the harvesting from CKAN (#3980);

#42 Updated by Yannis Marketakis over 2 years ago

  • Due date changed from Jul 15, 2016 to Sep 20, 2016

due to changes in a related task

#43 Updated by Massimiliano Assante over 2 years ago

M13 September 2016 Activity

In T9.1 (Data Access and Storage and Discovery) FORTH finalised the development and testing of MatWare plugins for supporting the construction of the GRSF KB (#3928, #4284) and started the design of a MatWare web application aiming at supporting the users in the construction and refreshing the GRSF KB.

NKUA enhanced the Index and search facilities related to the new gCube Search Framework NKUA is currently working on (based on Elastic search).

CNR continued working on the feasibility to migrate from Apache JackRabbit 2 (current workspace service repository) to Apache Jackrabbit 3 (Oak) #4595. Moreover:

  • The Storage Manager was ported to the new Authorization 2 Framework (#4945) and a new feature allowing to temporary store files in a volatile area was implemented and released (#4839)
  • The HomeLibrary, a feature allowing Guest Access to Workspace Folders (ready-only mode) has been developed and released. The Workspace history keeps track of the information about these public access. (Feature #3782). HomeLibrary provides now support for the infrastructure Data Catalogue, capability for publishing workspace files on gCube Data Catalogue were developed and released.

For the Data Analysis task (T9.2) :

  • SPREAD has been ported on Dataminer service #4684 and the Dataminer portlet has been enhanced by adding new features such as bounding boxes selection using interactive maps, time and date selection panels #4172
  • The ICHTHYOP algorithm has been ported to the production environment #4195 and The Ecopath with Ecosym algorithm has been equipped with a feature to alert a user about the job status via notification email #4924
  • Support to the installation and enhancement of algorithms has been provided #4610 #4957

For the Data Publishing task (T9.3)

The activity focuses on continuing the exploitation and enhancement of the CKAN technology to realise a Data catalogue (#4166). Specifically, 3 (specific catalogue) roles have been introduced (Catalogue_Member, Catalogue_Editor, and Catalogue_Admin roles (#5119)), the the CKAN portlet multiscope capability was enhanced, the gCube publishing widgets we have been enhanced and a procedure for cleaning problematic tags for datasets harvested from GeoNetwork (#4830) is in place.
Moreover, the CKAN technology has been deployed to support the development of the Global Record of Stocks and Fisheries (GRSF) Catalogue (#5009 #5045);

#44 Updated by Yannis Marketakis over 2 years ago

  • Due date changed from Oct 28, 2016 to Oct 31, 2016

due to changes in a related task

#45 Updated by Gianpaolo Coro over 2 years ago

  • Due date changed from Oct 31, 2016 to Nov 11, 2016

due to changes in a related task

#46 Updated by Massimiliano Assante over 2 years ago

M14 October 2016 Activity

An activity for the preparation and for participating to the TCom meeting (Pisa, 18-21 October) was performed in the period, this activity spans the whole WP (members);

In T9.1 (Data Access and Storage and Discovery) FORTH finalised the development activities of MatWare plugins (in particular FishSource plugin (#4284)).
CNR extended the StorageManager service to support objects stored in the volatile area (#5186).
NKUA released the Index and search facilities related to the new gCube Search Framework and a series of portlets for exploiting such functionality. It is difficult to report more details about this activity as it seems the status of the tickets related are not up-to-date (some tickets are still in on progress status but they were released in gCube 4.1, e.g. #3914, #1438, #5129)

For the Data Analysis task (T9.2) :

About 30 new algorithms have been installed in the production environment #4610, an activity to connect RStudio with the Workspace has started #5459 as well as one to use a new DataTransfer service in computations #5549.

The rest of the activities were focused on enhancing the DataMiner service and on the Statistical Algorithm Importer (SAI) service, in particular:

  • The GenericWorker for Cloud computations has been ported to DataMiner #5590 #5169
  • An online helper has been added to DataMiner #5258
  • Logs are now produced out of all the SAI algorithms #4899
  • A DataMiner client library is under development #3983

For the Data Publishing task (T9.3)

The activity focuses on 2 main tasks:

  • continuing the exploitation and enhancement of CKAN technology to realise the Resource Catalogue (#4166)
    • Groups management has been revised, e.g. facility made available to selected roles only (#5196), groups creation made automatic (#5134 #5184)
    • Roles management has been reinforced, e.g. (#5119)
    • Persist workspace items whenever they are published (#5088 #4921)
    • fixed some CKAN issues, e.g. 302 redirect (#5379)
    • the ansible script for the deployment of the patched version of CKAN was revised (#5423)
    • reconciled the charset encodings and locale used by the machines hosting GN, GeoServer and CKAN (#5455)
  • customising the Resource Catalogue for the GRSF exploitation case (#5001)
    • a first version of a REST service enabling to publish stock and fishery records has been developed (#5462)
    • a wiki page documenting the activity and the major decisions was created (#5498)
    • a set of groups for organising the records have been created (#5478)

#47 Updated by Gianpaolo Coro over 2 years ago

  • Due date changed from Nov 11, 2016 to Dec 09, 2016

due to changes in a related task

#48 Updated by Massimiliano Assante over 2 years ago

M15 November 2016 Activity

In T9.1 (Data Access and Storage and Discovery) during the period the Home Library (HL) and Workspace portlet were enhanced by adding HL support for Windows default uncompression program when downloading an entire folder in zip format(#5623), and the downloading of an entire folder in zip format has been enhanced in Workspace portlet by means of WebSocket support so as to avoid browser's timeout in case of very large folders requiring lot of time for compression. Both features were deployed with gCube 4.1 rollout of November.

NKUA performed various refactorings in the search system and the index mechanism (#5661).

For the Data Analysis task (T9.2) :

  • CMSY (Oceana version) has been integrated as-a-Service #5725 and a report template has been defined based on the CMSY process #5913
  • The FishBase requests have been all moved to DataMiner #5747
  • Dataminer now uses several Generic Worker, also from the EGI infrastructure #5590 #5169
  • The Workspace has been integrated with the RStudio environment #5459 this means that one can access workspace files and folder by using R functions into RStudio. This feature was added to gCube 4.2

For the Data Publishing task (T9.3) the activity focuses on the continuation of the two main tasks:

  • continuing the exploitation and enhancement of CKAN technology to realise the Resource Catalogue (#4166)
    • improved CKAN publishing widget: select the group (#5756), select files and level of nesting (#5659);
    • support for repeatable fields added (#5650)
    • installed accounting facilities, namely Google Analytics (#5806 #5817 #5891)
    • released a revised version of CKAN customised for D4Science / BlueBRIDGE (#5742)
    • reinforced the CKAN logout facility (#5615)
  • customising the Resource Catalogue for the GRSF exploitation case (#5001)
    • FORTH started the development of the service publishing records into GRSF Catalogue (#5600)
    • the deployment of a new instance of the catalogue for the curation phase (i.e. GRSF_Admin) was started (#6088);
    • a facility for managing records status change has been developed (#5889)
    • a specific template has been created (#5477)
    • the REST service enabling FORTH to push records into the catalogue has been released (#5462)

#49 Updated by Yannis Marketakis over 2 years ago

  • Due date changed from Dec 09, 2016 to Dec 21, 2016

due to changes in a related task

#50 Updated by Yannis Marketakis over 2 years ago

  • Due date changed from Dec 21, 2016 to Jan 30, 2017

due to changes in a related task

#51 Updated by Massimiliano Assante over 2 years ago

M16 December 2016 and M17 January Activity

In T9.1 (Data Access and Storage and Discovery):

FORTH finalized the work on MatWare plugins for fetching data from remote sources, so that they can be transformed and ingested into the GRSF KB (#3926).
NKUA performed various refactorings in the search system and the index mechanism (#5661). The main rationale of this activity is to improve and facilities and the portability of the index/search.
CNR worked on fixing several incidents with the workspace portlet (#6536, #6130, #6168, #6214) and to enhancements of the Home library for supporting files versioning (#5165)
All the partners worked for the preparation for the 1st periodic review.

For the Data Analysis task (T9.2) :

  • The interaction between the Workspace and RStudio has been released on production #5459 this means that one can access workspace files and folder by using R functions into RStudio.
  • Issues related to infrastructure resources retrieval by DataMiner have been solved #5645
  • The DataMiner client library has been released, to be used by other services like TabMan #3983

For the Data Publishing task (T9.3) the activity focuses on the continuation of the two main tasks:

  • Resource Catalogue development:

    • enhanced the access to product resources’, e.g. inform the user that is needed to access a VRE for accessing the resource (#6574);
    • made publicly available the name & description of “private” products (#6569);
    • removed the parts on Activity Stream and Recent Activity (#6572);
    • added tag clouds for formats, groups, tags (#6572);
    • reduced the time needed to load the catalogue portlet (#6567);
  • Support for the Global Record of Stock and Fisheries (GRSF):

    • created a custom widget for GRSF Management (#6655);
    • reconciling links between GRSF records in KB and Catalogue records (#6660);
    • GRSF-service-updater (to update records state and republish) has been finalised (#6259);

#52 Updated by Gianpaolo Coro over 2 years ago

  • Due date changed from Jan 30, 2017 to Feb 20, 2017

due to changes in a related task

#53 Updated by Gianpaolo Coro over 2 years ago

  • Due date changed from Feb 20, 2017 to Mar 24, 2017

due to changes in a related task

#54 Updated by Massimiliano Assante about 2 years ago

M18 February and M19 March Activity

In T9.1 (Data Access and Storage and Discovery):

  • CNR worked on delivering the Versioning feature to the Workspace application, in particular the Home Library service was enhanced to offer such functionality and the portlet to support it via GUI. The versioning feature has been added as part of the gCube release 4.3, rolled-out mid march.
  • FORTH has prepared a proposal for the semantic enrichment of gCube Data Catalogue that presented and discussed it in the 4th TCom at ICES HQ (https://goo.gl/0OX5hH)
  • NKUA updated the search framework for offering more functionalities and enhanced preview of results (#5772, #6134). In particular various enhancements have been performed in the search system and the index mechanism. The main rationale of this activity is to improve and facilities and the portability of the index/search.

For the Data Analysis task (T9.2):

  • Support to WP7 activities has been given:
    • CNR enhanced and released a shapefiles publication process with option to indicate a GIS workspace for the generated map #7101
    • Activity to support the FAO T7.3 use case for the pufferfish spread in the Mediterranean Sea, continued with important results #7149
  • A Big Data generation process has been started to produce Taxonomic Authority Files from WoRMS #7865
  • Minor enhancements have been delivered to the Data Miner and Statistical algorithm Importer applications. #7167, #7126, #7120

For the Data Publishing task (T9.3):

The activity focused on the continuation of two main tasks, Resource Catalogue development and GRSF, in particular for the Resource Catalogue enhancements were delivered to the Catalogue publication widget, to the Catalogue item pages, to the Catalogue Resolver and to the Catalogue Harvester. (#6570)(#6574) (#6952) (#6569)(#7188)

Moreover, support for single files publishing has been added (#6917 #7207);

Specifically for GRSF:

  • a widget for GRSF records admin has been developed (#6655)
  • customised the service and the template, e.g. “product” -> “record” (#7052 #7055 #7059 #7060 #7061)

#55 Updated by Costantino Perciante about 2 years ago

  • Due date changed from Mar 24, 2017 to Apr 30, 2017

due to changes in a related task

#56 Updated by Yannis Marketakis about 2 years ago

  • Due date changed from Apr 30, 2017 to May 05, 2017

due to changes in a related task

#57 Updated by Yannis Marketakis about 2 years ago

  • Due date changed from May 05, 2017 to May 12, 2017

due to changes in a related task

#58 Updated by Yannis Marketakis about 2 years ago

  • Due date changed from May 12, 2017 to May 16, 2017

due to changes in a related task

#59 Updated by Massimiliano Assante about 2 years ago

M20 April Activity

In T9.1 (Data Access and Storage and Discovery):

  • NKUA performed various refactorings in the search system and the index mechanism (#5661) for improving the search facilities and the portability of the index/search.
  • FORTH updated Matware plugins for fetching data from FishSource (for the purposes of GRSF) with respect to the new JSON contents of the data source (#8278).
  • CNR worked on the clustering of the workspace repository, this implies replace the current DB (Derby) with PostgreSQL and to migrate all the current data in the repo. Also, the OAI-PMH Harvester portlet was analysed for enhancements #7640, in particular to harvest and index a number of data sources for the iSearch VRE #7641 #7549

For the Data Analysis task (T9.2):

  • Activity to support the FAO T7.3 use case for the pufferfish spread in the Mediterranean Sea was presented at FAO of the UN HQ in a seminar #7149
  • An R process to invoke BiOnym was enhanced, which allows FAO to evaluate the match between ASFIS and WoRMS #6508.
  • A new syncing process was deployed to synchronise several DataMiner instances. This process is used to rapidly install an algorithm in the production VREs #7352

For the Data Publishing task (T9.3):

The activity focused mainly on Resource catalogue development, in particular mechanisms for resource profile ( #7351 #7616 #7871 ) specification and for decoupling profile specifications from fields specification have been developed. The RESTful web service and the related widget for publishing in the catalogue have been enhanced by supporting V3 metadata profiles #7415.

In addition:

  • the capability of exporting catalogue metadata in the Resource Description Framework (RDF) format is now available
  • the CKAN technology was modified to :
    • deal with attribute namespaces #8286
    • add the notion of metadata type / product type #7544
    • to make private items searchable #7496
    • support for "share link" added #7722
    • replacing the tooltip-based approach with a pop-up-based one to inform users on how to access "private" items

#60 Updated by Yannis Marketakis almost 2 years ago

  • Due date changed from May 16, 2017 to Jun 02, 2017

due to changes in a related task

#61 Updated by Massimiliano Assante almost 2 years ago

M21 May Activity

In T9.1 (Data Access and Storage and Discovery):

  • Refactoring the search system and the index mechanism (#5661) for improving the search facilities and the portability of the index/search. More specifically the activities focused on the new clients for search and index (#5662). Furthermore an OAI-PMH publisher service has been implemented that expose the indexed data sources records with respect to the OAI-PMH specification (#8528).
  • Implementing a harvester for the RDF contents of the Data catalog (#8365).
  • Identifying the datasets - the ones that are already indexed - that could be published in the data catalogue (#8715).
  • Updating the functionality of several MatWare plugins, based on the new requirements/recommendations and the updated data sources (#8278, #8786).

For the Data Analysis task (T9.2):

  • Enhancements on DataMiner were performed in order to automatise the platform tests #8367 and the deployment process of the service #8034
  • Interaction with FAO to make taxonomic searches in the WoRMS repository through BiOnym has gone forward, involving researchers from Univ. of Washington #7865
  • The process used to estimate marine invasive species is under evaluation by an expert, who periodically interacts with the Task working group #7149
  • A process to generate NeTCDF-CF files out of CSV files is under development and will be presented at the next TCOM in June #2230

For the Data Publishing task (T9.3):

The activity focused on the Resource catalogue development:

  • improvement of the resource profile model with a propagateUp property for groups (#8751)
  • creation of an ad hoc page for 'types' associated to items (#8748)
  • 'type' information has been defined and set for items into the ckan-bb catalogue (#8298)
  • a bug of the geonetwork extension of the catalogue has been fixed (#8662)
  • support for time dependent and independent fields for GRSF publisher service added (#8720)

#62 Updated by Massimiliano Assante almost 2 years ago

M22 June Activity

In T9.1 (Data Access and Storage and Discovery):

  • definition of the process for publishing the indexed data sources into the D4Science Resource Catalog (#8715).
  • a new proposal for enhancing the data discovery facilities of the resource catalogue through the semantic enrichment and exploitation of the catalogue entries has been presented and discussed in the 5th Tcom (https://goo.gl/cUYmMP).

For the Data Analysis task (T9.2):

A re-engineering of DataMiner service in ongoing, in order to also meet automatic deployment requirements #8975 #9084 #5559. Enhancement have been delivered to the service: the new Data Transfer service is now connected #5549 and an algorithm to transform CSV files into NetCDF files has been implemented #2230

Moreover, in this task an R Studio benchmark process has been developed to implement restrictions to resources usage by users #6941 and a discussion is ongoing on how to make R scripts inherit information from the e-Infrastructure #8952

For the Data Publishing task (T9.3):

The activity focused on the Resource catalogue development:

The Catalogue REST Service has been improved to explicitly support namespaces (#9007). An investigation has been performed to understand the high load time for catalogue item pages (#8425). Other activities regarded:

  • the publishing of workspace items of type URL to make them a Resource URL (#9076)
  • removing duplicates erroneously created by harvesting (#9094)
  • reinforcing the support for OAI-PMH for exposing catalogue items (#8817)
  • harvesting GN objects in FAO Tuna Atlas VRE (#8879)

Activity related to GRSF development support (see #8719):

  • the REST service for publishing GRSF records has been reinforced to deal with time-dependent and time-independent information;

#63 Updated by Massimiliano Assante over 1 year ago

M23 and M24 July & August 2017 Activity

In T9.1 (Data Access and Storage and Discovery):

  • Within this task in July CNR successfully migrated the workspace content repository to a new powerful backend, the activity required several tasks being coordinated and nearly 2 months planning before being actually executed, #8562

For the Data Analysis task (T9.2):

Most of the activities have concentrated on the engineering of several DataMiner facilities:

  • Publishing the live logs through nginx #9329
  • Automatisation of testing #8367
  • Enhancing the engineering of the Service #8975 #8952
  • Making SAI more robust to users' interactions #9328
  • Automatisation of the SAI-integrated processes #5557 #9450

  • Four processes to transform CSV files into NetCDF files have been released, which will be suggested to the AquaMaps Consortium as a means to publish their data #2230 #9274

  • A new CMSY version developed by NOAA and FAO has been integrated with DataMiner #9521

  • Support Vector Machines are now part of the general-purpose machine learning tools provided by DataMiner #9554

For the Data Publishing task (T9.3):

The activity focused on the Resource catalogue development, in particular:

  • GeoNetwork catalogue harvesting was improved to properly dealing with object native type (#9281)
  • Installing, configuring and developing the "native ckan" OAI-PMH plug-in (#8817 #9298 #9315). The plugin was also enhanced with proper queries #9299
  • Reinforcing the ansible playbook to deal with languages (#9026) and Mime type management fixing (#9145)

#64 Updated by Yannis Marketakis over 1 year ago

  • Due date changed from Jun 02, 2017 to Oct 13, 2017

due to changes in a related task

#65 Updated by Massimiliano Assante over 1 year ago

M25 September 2017 Activity

In T9.1 (Data Access and Storage and Discovery) a THREDDS synchronization feature has been integrated with Workspace (Feature #9760) and a data transfer geotk plugin has been implemented d #8772. Both components were released in gCube 4.7.0

For the Data Analysis task support to the invasive puffer fish prediction use case has been given. The activity has been validated both by FAO and two external scientists. A scientific paper has been written and sent to the Ecological Modelling Journal #7149;

The remaining effort has been spent at making the new version of DataMiner available in the production VREs, including all the enhancements developed in the last months. In particular:

  • A first version of the automatic algorithm deployment system has been released and installed in the production environment #5557;
  • Inheritance of infrastructure information for R Scripts has been enhanced: DataMiner now passes information about the user name, the VRE, and the user token #9549 #8952
  • Maps of climate change have been produced by processing environmental data from several sources #9832
  • Connection to OBIS has been enhanced in the Absence Records Estimation algorithm, through the passage to the OBIS API v2 #9699
  • CNR has been also spending effort to make 11,600 AquaMaps Native distributions, hosted by the e-Infrastructure, available as NetCDF files and to propose a transformation service to the AquaMaps Consortium #9583

For the Data Publishing task (T9.3):

The BlueBRIDGE Catalogue has been revised in its structure, specifically:

It has been enriched with new item types (each of which having its own metadata format). It now contains 4 types (the number of items present is reported in brackets):

  • Dataset (51234)
  • ResearchObject (1)
  • Service (5)
  • VirtualResearchEnvironment (54), in particular all the VREs present in BlueBRIDGE have been published in the catalogue. With actual links pointing to the VRE public pages.

The Organisations have been revised as well, they now represent the entities that can publish in the catalogue and support an Organisations Hierarchy.
The Groups that were empty have been now exploited with categories coming from GeoNetwork topic category (gmd:MD_TopicCategoryCode) such as biota, oceans, boundaries, environment etc.

#66 Updated by Gianpaolo Coro over 1 year ago

  • Due date changed from Oct 13, 2017 to Nov 15, 2017

due to changes in a related task

#67 Updated by Massimiliano Assante over 1 year ago

M26 October 2017 Activity

In T9.1 (Data Access and Storage and Discovery) the MatWare plugin for fetching data from the second version of the FishSource API has been updated (#9859). The file copy operation of the Home Library has been revised (#10239).

For the Data Analysis task (T9.2):

  • Long-term climatic forecasts have been analysed, as published by AquaMaps #9832 and NASA #9973, with a final production of 11 environmental parameters time series that give insight about global climate change under different scenarios of greenhouse gases emission and resources exploitation.
    • A time series analysis on different ocean areas at global scale has revealed interesting shared properties and similarities between these locations #10130. The resulting data can be of support to general Blue Growth analyses. This will be discussed at the "Expert Meeting on Climate Change and Fisheries in the Mediterranean and Black Sea" on December.
    • Enhancements and support have been provided for Statistical Algorithm Importer, e.g. in the management of black boxes when using the Windows OS #10003
    • The transformation and publication of 11,600 AquaMaps Native distributions into NetCDF files has been completed #9583
    • A new version of the CMSY Stock Assessment algorithm has been installed, which is 10 times faster than the legacy algorithm, although with coarser results #10177

For the Data Publishing task (T9.3):

  • reinforced the GeoNetwork harvester
  • to collect information on author and maintainer (#9379)
  • to add links to GeoNetwork metadata (#9314)
  • the catalogue UX had been enhanced by means of the browse by Types, Organisations, Groups feature #10056

#68 Updated by Massimiliano Assante over 1 year ago

M27 November 2017 Activity

In T9.1 (Data Access and Storage and Discovery) the activity focused on publishing the indexed Data Sources to the D4Science catalogue (#8715) and on a proposal for enhancing the mappings between CKAN and RDF DCAT for exporting enriched RDF descriptions for the resource catalog https://support.d4science.org/issues/10347

For the Data Analysis task (T9.2):

A transformation process to rasterize and publish the AquaMaps Native 2050 distributions through THREDDS is ongoing #10475, and a fast version of CMSY developed by NOAA has been integrated with the DataMiner and provided to the FAO Stock Assessment course #10177.
Moreover, the Climate Change data forecasts hosted by BlueBRIDGE have been enriched with NASA precipitations data between 1950 and 2100 #9832 #9973 and enhancements to the Statistical Algorithm Importer application have been made to meet users' requirements #10187

For the Data Publishing task (T9.3):

TBA

#69 Updated by Gianpaolo Coro over 1 year ago

  • Due date changed from Jan 31, 2018 to Feb 15, 2018

due to changes in a related task

#70 Updated by Gianpaolo Coro over 1 year ago

  • Due date changed from Feb 15, 2018 to Jan 31, 2018

due to changes in a related task

#71 Updated by Massimiliano Assante over 1 year ago

M28 December 2017 Activity

In T9.1 (Data Access and Storage and Discovery) the activity focused on carrying out the required changes in the resource catalog and the index API for enabling the retrieval and publishing of the index data in the resource catalog (related tickets: #8715, #10596, #10725, #10735)

For the Data Analysis task (T9.2):

  • Enhancements to the SAI interface have been added to enhance user experience #10187. In particular, adjustments like (i) hiding the software creation process in the publication button, (ii) reordering the interface tabs, (iii) managing Workspace locking issues #10199 etc. were made after collecting feedback from users;
  • The Support Vector Machines training and projection processes have been fixed and enhanced to meet requirements of ecological modelling problems #10813
  • A plan to add private sharing of algorithms between users in a VRE has been developed and started, which impacts several services of the infrastructure but has large beneficial impact on the infrastructure users #10749
  • Indexing of the DataMiner algorithms on the GeoNetwork is being investigated #10704
  • Assessment of the Stock Assessment processes integrated in BlueBRIDGE for the WECAFC case has been conducted #10649

For the Data Publishing task (T9.3):

  • Proposal for enhancing the mappings between CKAN and RDF DCAT for exporting enriched RDF descriptions for the resource catalog (#10347)

#72 Updated by Massimiliano Assante about 1 year ago

M29 and M30 January and February 2018 Activity

In T9.1 (Data Access and Storage and Discovery) the activity focused on Carrying out the required changes in the resource catalog and the index API for enabling the retrieval and publishing of the index data in the resource catalog (related tickets: #8715, #10735)

For the Data Analysis task (T9.2):

  • Enhancements to the SAI interface have been added as a continuation of the activity of December #10994 #10187, e.g. the publisher's user name is now reported in the algorithm description #10779, public/private algorithms sharing is enabled #10778 #10750 #10705, work-in-progress regards deploying an algorithm in "another" VRE from SAI #10994
  • The way to index DataMiner algorithms on GeoNetwork is still under study with some options explored during these months #10704
  • An algorithm to automatically build virtual meshes and Web applications from a set of photos has been developed and used in a course #11047 #11205, as a way to liaise with other projects and set the bases of possible new projects.

#73 Updated by Pasquale Pagano about 1 year ago

  • Status changed from In Progress to Closed

#74 Updated by Francesco Mangiacrapa 10 months ago

  • Due date changed from Jan 31, 2018 to Jan 18, 2018

due to changes in a related task

Also available in: Atom PDF