Project Task #633
Project WP #629: WP4 - VREs Deployment and Operation [Months: 1-30]
T4.4 Software integration and release [Months: 1-29]
|Status:||Closed||Start date:||Oct 09, 2015|
|Priority:||Urgent||Due date:||Mar 30, 2018|
|Assignee:||Maria Di Girolamo||% Done:|
|Lead beneficiary:||3 - ENG||Participants:||1 - CNR, 4 - UOA|
Task Leader: ENG; Participants: CNR, UOA;
This task is in charge of managing the release process of the software maintained and enhanced in WP4-WP10 work packages. In particular, this task (i) defines release procedures, (ii) establishes the release plan; (iii) coordinates the release process; (iv) operates the tools required to support the release activities; (v) keeps the software under continuous integration; (vi) performs functional validation of released web components (i.e. portlets); (vii) validates the software documentation; (viii) takes care of the distribution of the software. This task largely benefits from the practices established and experience gained within D4Science. In particular,the task supports a continuous integration process in combination with automated unit test written through the practice of test-driven development. This solution allows integrating software developed by a distributed developer’ community while avoiding one developer’s work-in-progress breaking another developers activity. Through the combination of Unit and Integration test with additional static and dynamic tests, measures, and profile performance the quality of software can be improved and the time taken to deliver it can be reduced. The entire process leverages state of the art solutions and technologies for continuous integration (e.g. Jenkins). When integrated, the software is distributed and operated via common toolbox, such as Docker, in order to avoid the usual issues characterized by conflicts, dependencies and inconsistent environment. Moreover by packaging software, dependencies, and configuration together, the portability of the applications in any environment of the hybrid data infrastructure is ensured.
#6 Updated by Gabriele Giammatteo almost 4 years ago
September 2015 Activity
* some tools updated and/or configured: Redmine (CNR), new Distribution Site (CNR), ETICS (ENG)
* Integration of gCube 3.9.0 started (ALL)
* New release procedure defined and being tested in gCube 3.9.0 (ENG, CNR)
* Update of the Integration and Distribution Wiki to reflect new procedures and tools (CNR, ENG)
* Jenkins evaluation started. An instance deployed @ENG and initial tests carried out. To be continued in next months (ENG)
* rethink documentation review procedure (in iMarine it has not been effective)
* replace BTRT builds with ETICS remote builds (starting from org.gcube.HEAD) as much as possible
* carry out tests with Jenkins to understand how to use it in gCube release process
* migration to Git. Set-up repositories, update integration procedures and tools
* by M5 GitHub to be used to publish source code after releases
#13 Updated by Gabriele Giammatteo over 3 years ago
October 2015 Activities
* Release gCube 3.9.0 has been integrated with 133 components added/updated in 17 different subsystem
* Release Procedure and Documentation procedures have been updated to fit the new tools and needs of BlueBRIDGE project. Tools (Redmine, ETICS) and documentation (gCube wiki) have been updated accordingly
* Deliverable D4.1 based on the gCube wiki has been reviewed and delivered to the commission
* A Jenkins instance has been configured at ENG infrastructure and configured to build some gCube components as preliminary tests to switch to the new integration system
#52 Updated by Gabriele Giammatteo over 3 years ago
November 2015 Activities¶
- a new procedure for portlet functional test has been defined. The required templates to describe and execute tests have been produced. A request to developers to provide functional tests for their portlet has been sent
- gCube 3.9.0 has been closed. In total 134 components have been updated/added to the release. In total the integration of the release took 64 days
- a change to maven-builder has been implemented in order to generate new verion numbers that includes also SVN revision and timestamp. This will simplify deployment phase of artifacts
- status and plan review of gCube components has been completed
- a total of 33 abandoned components have been removed from gCube 3.10.0 and gcube.HEAD
- A new building server has been provided by CNR to build org.gcube.HEAD. The daily builds of gCube HEAD appers in BTRT (the same of the releases)
- new fields for reporting documentation status of released components has been added to release tickets
#55 Updated by Gabriele Giammatteo over 3 years ago
December 2015 Activities¶
- integration of gCube 3.10.0 continued with regular builds every day
- new components have been removed from the release because they resulted abandoned and not used. In total, 42 components have been removed so far from release gCube 3.10.0
- ENG produced a list of components that need to be re-factored because they depends on obsolete components. This list has been shared with partners and will be kept updated release by release
- modifications to the build/release tools have been updated to generate Maven artifacts that include svn revision number in the file name. However, the new versioning system will be used in gCube 3.11.0
- All partners contributed to specify portlet functional tests. They are available in the public workspace of iMarine portal (https://goo.gl/F0fNFX). Tests have been specified for 18 portlets out of 26
#57 Updated by Gabriele Giammatteo over 3 years ago
Feb 2016 Activities¶
- release gCube 3.10.0 has been closed and published on gCube website
- a maintenance release started (org.gcube.3.10.0-1) to fix a bug in logs configuration.
- an intial procedure for the publication of released source code on GitHub has been set-up (gCube 3.10.0 first release published)
- activity to evaluate if and how is possible to generate the fullgar of Ant components started
- definition of a procedure to update the README and LICENSE files of all gCube components started
- Work started on gCube 3.11.0. In this release gCore-dependant and SmartGears-dependant components will be split in two different releases. An initial division has been done and a list of components to move produced.
#58 Updated by Gabriele Giammatteo over 3 years ago
Mar 2016 Activities¶
- new format for distro files (README, LICENSE, INSTALL, ...) defined and documented on the wiki
- distro files of all components have been updated in gCube 3.10.1 by all technical partners
- integration of gCube 3.10.1 continued during this month to include the update of distro files
- During the integration of gCube 3.10.1 a mechanism to automatically generate in ETICS source packages for all components has been developed (they will be used for the publication of the releases on GitHub)
- integration of gCube 3.11.0 started. Since this release, gCore-based and SmartGears-based components will be released separately. This needed an extra-effort to "divide" the components in two separate releases analysing the dependencies
#67 Updated by Gabriele Giammatteo over 3 years ago
Apr 2016 Activities¶
- integration of gCube 3.11.0 and gCube 3.11.0-gcore releases continued during April.
- identification and (when possible) re-factoring of gCore based components has been performed. Components not refactored has been moved in gCube 3.11.0-gcore
- during the integration of these releases the updating of the README and LICENSE files continued.
#68 Updated by Gabriele Giammatteo about 3 years ago
May 2016 Activities¶
- Release gCube 3.11.0 has been integrated and released at the end of May. This release was split in two different projects (for integration purposes)
- org.gcube.3.11.0: 234 components updated (14 new + 73 upd + 147 drp) on 24 subsystems;
- org.gcube.3.11.0-gcore: 329 components updated (3 new + 28 upd + 298 drp) on 20 subsystems;
#70 Updated by Gabriele Giammatteo about 3 years ago
June 2016 Activities¶
- integration of gCube 4 release started in the last half of the month
- org.gcube.HEAD release has been split in org.gcube.HEAD and org.gcube.HEAD-gcore as already done for gCube 3.11
- the functional testing procedure and test plans have been reviewed and updated: gCube 4 will ship a major update of the portal technology (i.e. Liferay 6.2) and it will be critical to properly test all portlets.
- An updated the list of the FT portlets (https://goo.gl/aAz5Gh), test plans (https://i-marine.d4science.org/group/data-e-infrastructure-gateway/workspace) and testers (https://wiki.gcube-system.org/gcube/Role_Tester) have been released.
- activities on migration to Git and publication of artifacts on Zenodo continued and interim results have been presented at the BlueBRIDGE 2nd TCOM
#72 Updated by Gabriele Giammatteo almost 3 years ago
July/August 2016 Activity¶
gCube 4.0.0 release has been integrated, tested and deployed during July. The release included 40 new component and 134 updated components (https://wiki.gcube-system.org/gcube/Software_Integration_and_Distribution:_Release_Log).
This release included a major upgrade of the portal technology at Liferay 6.2. All portlets have been migrated to the new version. Technical partners CNR, ENG, UoA carried out functioanl testing of portlets (https://wiki.gcube-system.org/gcube/FTMasterTablegCube4.0.0).
Git migration and Zenodo publication activities progressed but still ongoing.
ETICS has been updated to keep the configurations synchronised with changes on svn. This should reduce the effort needed to manually keep information in ETICS up-to-date.
#77 Updated by Gabriele Giammatteo almost 3 years ago
September 2016 Activity¶
- preparation of gCube 4.1.0 release started: nightly builds have been activated and monitored and functional testing procedure has been setup.
- Main changes in this release are: Authorization 2.0, SmartGears 2.0, new GeoAnalytics and InvestmentAnalysis subsystems. This release integration and testing will continue and completed during October 2016.
for Zenodo publication activity, the model and the format for metadata has been decided. The same metadata will be published also in the Data Catalogue (#5114).
distribution Wiki has been updated to reflect changes in testing procedures.
editing of deliverable D4.2 started
#78 Updated by Gabriele Giammatteo almost 3 years ago
October 2016 Activity¶
- integration, testing and roll-out of gCube 4.1.0. The release has been finalized and the production infrastructure upgraded at the end of the month. The release involved, 46 new components, 170 updated components and 2 new subsystems: investment-analysis and geoanalytics;
- production of D4.2 on the BlueBRIDGE wiki. Most of the content is available, but it is not yet finalized;
- preparation of 3rd TCOM discussions and presentations;
#91 Updated by Gabriele Giammatteo over 2 years ago
November 2016 Activity¶
- gCube 4.1.1 maintenance release was opened (#5816) on November 21st (and closed the day after) to fix a critical issue in the NewFeed portlet;
- Started the integration of gCube 4.2.0 (#5058 ; #5060). Upgrade of the infrastructure with this release is planned on Dec 12nd;
- Preparation for the functional testing of gCube 4.2.0 (they will start on December 1st);
- Deliverable D4.2 finalized (#635; https://goo.gl/Gi0sgy);
- Zenodo Publisher completed (#3923);
#96 Updated by Gabriele Giammatteo over 2 years ago
December 2016 and January 2017 Activity¶
- gCube4.2.0 ( #5058 and #5060) has been closed: . In total 121 components have been updated/added to the release and the integration of the release took 19 days.
- Modified the root-workaspace for the TestPlan: gCube and not BlueCommons: https://goo.gl/bRtrI0. In this way all developers and testers can access it. 39 portlets pass the test: https://wiki.gcube-system.org/gcube/FTMasterTablegCube4.2.0.
- Deliverable D4.1 resubmitted and D4.2 submitted: https://goo.gl/7cxqgI;
- preparation of 1st review meeting;
- release gCube 4.3 started: #6243 (#6244);
- review and modifications to the release and testing #6509.
#100 Updated by Gabriele Giammatteo over 2 years ago
February 2017 Activities¶
- gCube 4.2.1 maintenance release integrated and delivered. It tooks 8 days in total and fixed few issues on portal components and the data catalogue
- new testing procedure reviewed to improve the activities of the developers and tester. Redmine will replace Excel and Wiki tables to organize the procedure (#7063)
- upgrade of Wiki to reflect changes in the procedures
- gCube 4.3.0 integration phase finalized. Tests will start on March
#110 Updated by Gabriele Giammatteo over 2 years ago
March 2017 Activity¶
gCube 4.4.0 started: introducing from start to this release a new procedure to improve and simplify the release cycle.
- New release plan are provided in order to improve the development, integration and test activities (refer to the wiki documentation for more details: https://wiki.gcube-system.org/gcube/Major/Minor_Release_Cycle_procedure#Release_Integration).
- All components are grouped in 3 macro area: enabling, common apps and community apps.
- Based on this subdivision was possible to fix time releases (with intermediary deadline, the Release Gant is provided in the wiki documentation : https://wiki.gcube-system.org/gcube/Major/Minor_Release_Cycle_procedure#Introduction) to maintain for all next releases: one release should last for six weeks; no pauses between releases; different deadlines during the same release that involves different group of components ; continuous pre-prod upgrade and testing: as soon as a component build successfully it must be deployed on pre-prod and then in production
- Re-starting with the deleting of all components abandoned or not used.
- Etics improvement about the identification of the dismissed components. as reported in the wiki documentation : https://wiki.gcube-system.org/gcube/ETICS#Dismissed_Modules.
- Introduced in the RedMine Track Issue a new field about the identification of the enabling, common and community Apps and subsystem (identified as Grouping) and other improvements as automatic identification of the subsystem manager
- Re-assignment of the portlets based on the rules reported in the wiki documentation: https://wiki.gcube-system.org/gcube/Functional_Testing#Mandatory_Recommendations_and_Rules
- Partecipation ,preparation and presentation of the WP4 activities to the 4th TCOM: https://support.d4science.org/projects/bluebridge/wiki/4th_TCom_Meeting_07-10_March_2017_-_Copenaghen_(Denmark)
#112 Updated by Gabriele Giammatteo over 2 years ago
April 2017 Activity¶
integration and testing of gCube 4.4.0. The release has been closed on April 30th
- extensive clean-up of the release (100 obsolete components removed). It allowed to reduce building times of ~ 2 hours
- 95 components updated, 9 components new
Migration to Java 8 continued:
- upgrade of development infrastructure completed (excepting for few gHN un-upgradable nodes)
- set-up of the building environment to build with Java 8
#118 Updated by Gabriele Giammatteo about 2 years ago
May 2017 Activity¶
- release gCube 4.5.0 started on May, 2nd (Smartgear #8211, gCore #8212, functional tests #8213)
- continue the clean-up of the obsolete components
- Migration to Java 8 started
- upgrade and testing of development infrastructure completed, excepting for few gHN un-upgradable nodes (#6309)
- integration of gCube switched to Java 8 for Smartgears release in ETICS (while gCore components are still in Java 7)
- integration tools have been updated to allow fine grain control on which Java version to use for the compilation
- wiki updated with migration issues and guidelines :https://wiki.gcube-system.org/gcube/GCube_Java_8_migration_guide
#124 Updated by Gabriele Giammatteo about 2 years ago
June 2017 Activity¶
- release gCube 4.6.0 started on June, 12nd (Smartgear #8545, gCore #8811, functional tests #8814). It should end on July 20th.
- continue the clean-up of the obsolete components and started to contact the developer to ask if possible to upgrade to Java8 their configuration component as discussed during the last 5th TCOM (https://goo.gl/koMlsC)
- Continues the Git-Hub migration activities. As announced in the last 5th TCOM ENG started with the migration of their own components to resolve and analyze and describe the migration steps to share with the rest of the partners
#125 Updated by Gabriele Giammatteo almost 2 years ago
July/August 2017 Activity¶
- gCube 4.6.0 integration started on June, 12nd (Smartgear #8545, gCore #8811, functional tests #8814). Due to the serious problem with the ETICS portal, the ENG team set-up a temporary building environment and executed manually the builds of the components. Since this required considerable effort, the release plan has been reviewed and it has been decided to keep only a subset of the expected components in the release (higher priority was assigned to the Community Apps components)
- gCube 4.6.0 has been closed on the 25th of July and all components excluded have been postponed to gCube 4.6.1 (end of August)
- at the end of July, issues with ETICS have been solved and the status of the configurations have been synchronized with the status of the release
- gCube 4.6.1 integration started on July 25th (Smartgear #9255, gCore #9390, functional tests #9399) in ETICS. This release contains all the components excluded from gCube 4.6.0 (enabling and common components)
- Functional tests on gCube 4.6.1 will start on 1st of September and it will be deployed in production on Sept 11st
- release 4.7.0 started on 28th August (Smartgear #9397, gCore #9398, functional tests #9399)
#130 Updated by Gabriele Giammatteo almost 2 years ago
September 2017 Activity¶
gCube 4.6.1 integration of all componets excluded from the #4.6.0 continues and was closed on September 19 (Smartgear #9255, gCore #9390, functional tests #9297). In total 82 (13 new, 63 updated and 3 removed) components modified and 17 subsystem updated. The closure of the release has been delayed (originally planned for Sept, 4th) due to integration issues on some components (#9679).
gCube 4.7.0 integration started on 28 August (Smartgear #9397, gCore #9398, functional tests #9774). All components planned in this release are integrated. The functional test started on the last week of September and will be closed on October 4th, the roll-out is on October 5th.
#137 Updated by Gabriele Giammatteo almost 2 years ago
October 2017 Activity¶
- mainteneance release gCube 4.7.1 (from 16th to 18th October) has been integrated to fix multiple user groups assignment and email formatting in the user management portlet (#9961)
- gCube 4.8.0 release integration started on October 11th (Smartgear #9723, gCore #9884, functional tests #9885). All components of the three groups can be implemented and integrated. The release should be closed on November 21th
- a new configuration has been added in the gCube Maven configuration to create uberjar artifacts (a jar that embeds its dependencies) during the building process (#10089). A similar approach is being implemented for the servicearchives. This allows to reduce errors and non-compliant configurations in the building of gCube software artifacts
#138 Updated by Gabriele Giammatteo over 1 year ago
November 2017 Activity¶
- integration of gCube 4.8.0. The release has been closed on November 29th with one week delay. This release contains: 4 new components, 47 updated components in 18 different subsystems
- integration of gCube 4.9.0 started on November 29th. It will be closed on December, the 19th
#141 Updated by Gabriele Giammatteo over 1 year ago
December 2017 Activity¶
- integration of gCube 4.9.0 finished and the release has been closed on December 19th 2017. This release contains: 3 new components and 35 updated components in 11 different subsystems
- release gCube 4.10.0 started on 18 December 2017. This will be the latest BlueBridge project release and will be closed on February 12nd
- TOC for D4.4 has been produced