D4.3 - Open Science Visualisation Technologies¶
- D4.3 - Open Science Visualisation Technologies
- Data Visualisation in Aginfra+
- Structure of the deliverable
- Visualization Framework as a whole
- Visualization Toolkit
- Source Code
Data visualisation is ‘the representation and presentation of data to facilitate understanding’ (Kirk, 2016: 19), the act of taking information / data and placing it into a visual context adding meaning into complicated datasets so that their message is clear and concise, thus allowing scientists, researchers and in general end-users to detect patterns, trends, and outliers in groups of data. Different types of visualisations, from simple to complex ones, exist and can be exploited based on what fits best for the type of data.
A successful visualisation, enables users to quickly find desired information in reliable and easily understood ways. This is an important goal for the tools being developed as part of this work.
Data Visualisation in Aginfra+¶
Aginfra+ project involves communities and scientists conducting research in the agri-food sector. These communities have different needs and requirements with respect to the data visualisation, as they are working on different domains, analysing different types of data and targeting different results and outcomes.
Aiming on satisfying the needs of these 3 communities (WP5-Agro-climatic and Economic Modelling Community, WP6-Food Safety Risk Assessment Community, WP7-Food Security Community), while also offering generic visualisation facilities, work package 4 has developed the Visualisation Framework Toolbox offering a number of specific and generic tools and libraries.
Following an agile process, the work has evolved under the following principles:
a. Identification of needs and requirements, and interpretation to specifications and thus presentation features,
b. Experimentation on the above outcomes through the exploitation of state of the art technologies and selection of the most appropriate technologies,
c. Implementation of features from the product list in the planned sprint
d. Collaboration with the specific communities, iterating on the implemented features and receiving feedback
c. Release of the tested features / tools using specific cycles
D4.1: Technical Specifications Report accumulated and included the preliminary needs as initially reported by the communities in the mid of the first year, describing the architecture of the visualisation framework covering the requirements gathered until that time. It consists of three areas, namely the backend services (business logic), the front end services and the enabling services (external supporting ones), all of them being part of the framework.
The design followed, allows extensions and additions of new components making feasible to adopt to additional needs and requirements. D4.1 acts as the basis of the work achieved and being documents in D4.3 deliverable report.
Structure of the deliverable¶
The D4.3 deliverable is an online document that evolves during the time in order to always contain the complete list of the Visualisation Technologies, tools and frameworks that are available in Aginfra+. Since this deliverable has 3 iterations, this section will contain the main updates and changes for each covering period.
- The first version of the deliverable D4.3 (presentation layer: Open Science Visualization Technologies) was aligned to the level of maturity of the three communities' evolvement. For the presentation layer, on one hand we experimented on a variety of technologies, indicatively NASA WWW, OpenLayers, Google Charts, and the ChartJS and D3.js, and on the other hand we provided a first small-scale demonstration on specific visualization capabilities fitting specific derived research queries. During the first period (M12) we focused on offering geospatial visualization capabilities by exploiting the already available VRE tool (Geoanalytics Platform) for WP5 community.
- Generic visualization components have been offered to all three communities. They include bar charts, 3D scatter plots, line charts, steps, splines, etc, and they are constantly updated according to the individual communities needs.
Until the second iteration of the deliverable, due in M24 of the project the following updates took place:
* Specific Visualisation Components for the Food Security community have been implemented and made available. These include:
* Interactive visualization of Plant Height and Plant Height with images for each point.
* HeatMap: Spatial Visualization of plant height from PhenoArch ZA17 experiment
* 3D Scatter for plant variables
* Mind Map application (extended and improved from an existing open source project)
The third iteration of the deliverable contains the final list of tools and libraries for visualising data. It contains all the improvements and updates for all the components developed or extended during the project period. It worth mentioning that during the covered period a new experimental application has been developed as part of the visualisation toolkit, named Network Graph Tool that is capable of visualisating the movement of specific networks. This work has been requested by the WP6- Food Safety Risk Assessment community.
In addition, new components have been designed, developed and delivered during the last period, until M33. These components are:
* Agrodatacube Dashboard Application
* Network Graph Movement Application
* Plant Height Visualization with multiple images support
Visualization Framework as a whole¶
The Data Visualization Framework has been designed and implemented with the vision to support the visualization of diverse data types, in different formats and with various visualizations. Aiming to be extensible in an easy way, its architecture follows a modular design, separating the data storage, the business logic and the front end / User Interface components.
The implemented architecture of the framework is available below:
It is developed using cutting edge open source technologies, including the several visualization libraries (d3.js, chart.js, ploty.js and world wind map). All the visualization elements are available as React widgets, while the back end service is accessible through a REST API.
Generic Visualization Tools¶
A mind map is an easy way to brainstorm thoughts organically without worrying about order and structure. It allows you to visually structure your ideas to help with analysis and recall. A mind map is a diagram for representing tasks, words, concepts, or items linked to and arranged around a central concept or subject using a non-linear graphical layout that allows the user to build an intuitive framework around a central concept.
An open source solution has been extended and made available as a native VRE application, allowing users to easily create mind maps, having the option to either save/open them to/from the local computer, browser's local storage or into the Aginfra+ workspace supporting sharing among the VRE users.
Generic Visualization Charts¶
A number of generic visualization types are offered through the visualization frameworks including:
1.Spline / Scatter / Bar / Line / Step
14.World Wind Map
An example of a generic chart is shown below:
Generic visualization example concerning salmonella concentration
Another example of a 3D Scatter, visualizing the plant variables can be seen below:
In the data world, geospatial data is one of the most increasing type of data, offering high capabilities to scientists and researchers. This makes geospatial visualization of high importance. The Geospatial Visualization tool in the Aginfra+ is based on the existing, open source platform named "Geoanalytics Platform".
The Geoanalytics Platform is a Web GIS solution that facilitates analysts and scientists to visualize and manage geospatial information. Furthermore, it enables them to run complex analysis over an area of interest and extract valuable information based on various factors.
In the context of the Aginfra+ project the Geoanalytics platform has been extended and improved in order to support the additional requirements of the three communities.
A brief overview of the platform is provided below, while for detailed information a dedicated user guide exists (LINK HERE)
The Geoanalytics platform consists of a number of sub-systems offering:
-Project definition, management and sharing
-Exploration of existing geospatial datasets
-Geospatial layer and attribute visualization
-Geospatial datasets import, management and update
The platform utilizes primarily open (OGC) standards, allowing it to seamlessly integrate into the offered VREs, but also from external clients through its OGC published APIs
The following data types / protocols are natively supported for importing datasets.
- WFS interface
- GeoTIFF raster image
- External layer as reference from an existing Geoserver
- NetCDF files
Customized Visualization Tools¶
Agrodatacube Dashboard Application¶
The Agrodatacube Dashboard Application has been exclusively developed for the Agro-climatic community. It is conceived as an interactive dashboard where users can select the parcels and retrieve information about their crops. Information is displayed in a table format and in addition in a number of different graphs.
Furthermore the users have the option to execute a simulation crop analysis, for the supported fields only, and visualize the results of the simulation.
It is based on the AgroDataCube Service (https://agrodatacube.wur.nl/), that is currently supports parcels from the Netherlands.
Plant Height Visualization¶
The Food Security component has expressed the need for visualizing the height of plants at a specific time. In addition, it was desirable to be able to display the image of the plant for a selected point, as well as to be able to select multiple points in order to visually compare the different images.
A specific type of visualizing and with a specific configuration was made available to support this case. An example can be seen below:
Plant Variables HeatMap Visualization¶
A usable and interactive way to visualize data from a greenhouse is a spatial visualization. The position of every plant is known and the user can select a specific date and a heatmap is presented indicating the value of the variable for each plant in the greenhouse. This visualization is available and can be seen below:
Network Graph Evolution Application¶
There might be changes in the structure of a network (e.g. supply chain networks, patent citation networks, or twitter topic networks) that can only be captured by having a time-resolved visualization, i.e. a kind of movie. In the Food Risk Assessment community, it is important to visualize the changes of such a network based on specific criteria and a date range and display it in a movie.
A first approach on such a visualization has been implemented. It supports a specific data schema in json format and can dynamically support filters on the available metadata and dates.
Detailed documentation on the visualization framework and the Geospatial platform can be found in the dedicated pages
- Visualization Framework: https://support.d4science.org/projects/aginfraplus_wiki/wiki/Graph_Visualisation_documentation
- Geospatial Platform: https://support.d4science.org/projects/aginfraplus_wiki/wiki/Geopatial_Visualization_Documentation
Visualization framework is an open source framework and its source code can be found at: https://github.com/madgik/VisualisationFramework
It is available under the EUPL V1.2 license. For more information visit: https://joinup.ec.europa.eu/sites/default/files/custom-page/attachment/eupl_v1.2_en.pdf