D4.3 - Open Science Visualisation Technologies

Introduction

Data visualisation is ‘the representation and presentation of data to facilitate understanding’ (Kirk, 2016: 19), the act of taking information / data and placing it into a visual context adding meaning into complicated datasets so that their message is clear and concise, thus allowing scientists, researchers and in general end-users to detect patterns, trends, and outliers in groups of data. Different types of visualisations, from simple to complex ones, exist and can be exploited based on what fits best for the type of data.
A successful visualisation, enables users to quickly find desired information in reliable and easily understood ways. This is an important goal for the tools being developed as part of this work.

Data Visualisation in Aginfra+

Aginfra+ project involves communities and scientists conducting research in the agri-food sector. These communities have different needs and requirements with respect to the data visualisation, as they are working on different domains, analysing different types of data and targeting different results and outcomes.
Aiming on satisfying the needs of these 3 communities (WP5-Agro-climatic and Economic Modelling Community, WP6-Food Safety Risk Assessment Community, WP7-Food Security Community), while also offering generic visualisation facilities, work package 4 has developed the Visualisation Framework Toolbox offering a number of specific and generic tools and libraries.

Following an agile process, the work has evolved under the following principles:
a. Identification of needs and requirements, and interpretation to specifications and thus presentation features,
b. Experimentation on the above outcomes through the exploitation of state of the art technologies and selection of the most appropriate technologies,
c. Implementation of features from the product list in the planned sprint
d. Collaboration with the specific communities, iterating on the implemented features and receiving feedback
c. Release of the tested features / tools using specific cycles

D4.1: Technical Specifications Report accumulated and included the preliminary needs as initially reported by the communities in the mid of the first year, describing the architecture of the visualisation framework covering the requirements gathered until that time. It consists of three areas, namely the backend services (business logic), the front end services and the enabling services (external supporting ones), all of them being part of the framework.
The design followed, allows extensions and additions of new components making feasible to adopt to additional needs and requirements. D4.1 acts as the basis of the work achieved and being documents in D4.3 deliverable report.

Structure of the deliverable

The D4.3 deliverable is an online document that evolves during the time in order to always contain the complete list of the Visualisation Technologies, tools and frameworks that are available in Aginfra+. Since this deliverable has 3 iterations, this section will contain the main updates and changes for each covering period.

M12

  • The first version of the deliverable D4.3 (presentation layer: Open Science Visualization Technologies) was aligned to the level of maturity of the three communities' evolvement. For the presentation layer, on one hand we experimented on a variety of technologies, indicatively NASA WWW, OpenLayers, Google Charts, and the ChartJS and D3.js, and on the other hand we provided a first small-scale demonstration on specific visualization capabilities fitting specific derived research queries. During the first period (M12) we focused on offering geospatial visualization capabilities by exploiting the already available VRE tool (Geoanalytics Platform) for WP5 community.
  • Generic visualization components have been offered to all three communities. They include bar charts, 3D scatter plots, line charts, steps, splines, etc, and they are constantly updated according to the individual communities needs.

M24

Until the second iteration of the deliverable, due in M24 of the project the following updates took place:
* Specific Visualisation Components for the Food Security community have been implemented and made available. These include:
* Interactive visualization of Plant Height and Plant Height with images for each point.
* HeatMap: Spatial Visualization of plant height from PhenoArch ZA17 experiment
* 3D Scatter for plant variables
* Mind Map application (extended and improved from an existing open source project)

M33

The third iteration of the deliverable contains the final list of tools and libraries for visualising data. It contains all the improvements and updates for all the components developed or extended during the project period. It worth mentioning that during the covered period a new experimental application has been developed as part of the visualisation toolkit, named Network Graph Tool that is capable of visualisating the movement of specific networks. This work has been requested by the WP6- Food Safety Risk Assessment community.

In addition, new components have been designed, developed and delivered during the last period, until M33. These components are:
* Agrodatacube Dashboard Application
* Network Graph Movement Application
* Plant Height Visualization with multiple images support

Visualization Framework as a whole

The Data Visualization Framework has been designed and implemented with the vision to support the visualization of diverse data types, in different formats and with various visualizations. Aiming to be extensible in an easy way, its architecture follows a modular design, separating the data storage, the business logic and the front end / User Interface components.
The implemented architecture of the framework is available below:

visualisation_framework_arch.png

It is developed using cutting edge open source technologies, including the several visualization libraries (d3.js, chart.js, ploty.js and world wind map). All the visualization elements are available as React widgets, while the back end service is accessible through a REST API.

Visualization Toolkit

Generic Visualization Tools

Mind Map

A mind map is an easy way to brainstorm thoughts organically without worrying about order and structure. It allows you to visually structure your ideas to help with analysis and recall. A mind map is a diagram for representing tasks, words, concepts, or items linked to and arranged around a central concept or subject using a non-linear graphical layout that allows the user to build an intuitive framework around a central concept.
An open source solution has been extended and made available as a native VRE application, allowing users to easily create mind maps, having the option to either save/open them to/from the local computer, browser's local storage or into the Aginfra+ workspace supporting sharing among the VRE users.

mindmap-aginfra-communities.png

Generic Visualization Charts

A number of generic visualization types are offered through the visualization frameworks including:
1.Spline / Scatter / Bar / Line / Step
3.Pie
4.Doughnut
5.Polar
6.3D Scatter
10.Graph
11.Mind Map
12.Tree
13.Map
14.World Wind Map
15.Table

An example of a generic chart is shown below:

Visualization of Salmonella concentration example within WP6
Generic visualization example concerning salmonella concentration

Another example of a 3D Scatter, visualizing the plant variables can be seen below:
WP7_plant-variables_UoA_14062018.png

Geospatial Visualization

In the data world, geospatial data is one of the most increasing type of data, offering high capabilities to scientists and researchers. This makes geospatial visualization of high importance. The Geospatial Visualization tool in the Aginfra+ is based on the existing, open source platform named "Geoanalytics Platform".
The Geoanalytics Platform is a Web GIS solution that facilitates analysts and scientists to visualize and manage geospatial information. Furthermore, it enables them to run complex analysis over an area of interest and extract valuable information based on various factors.

In the context of the Aginfra+ project the Geoanalytics platform has been extended and improved in order to support the additional requirements of the three communities.

A brief overview of the platform is provided below, while for detailed information a dedicated user guide exists (LINK HERE)
The Geoanalytics platform consists of a number of sub-systems offering:
-Project definition, management and sharing
-Exploration of existing geospatial datasets
-Geospatial layer and attribute visualization
-Geospatial datasets import, management and update

The platform utilizes primarily open (OGC) standards, allowing it to seamlessly integrate into the offered VREs, but also from external clients through its OGC published APIs

The following data types / protocols are natively supported for importing datasets.
- TSV
- WFS interface
- Shapefile
- GeoTIFF raster image
- External layer as reference from an existing Geoserver
- NetCDF files

07 - Layer Details.png

Customized Visualization Tools

Agrodatacube Dashboard Application

The Agrodatacube Dashboard Application has been exclusively developed for the Agro-climatic community. It is conceived as an interactive dashboard where users can select the parcels and retrieve information about their crops. Information is displayed in a table format and in addition in a number of different graphs.
Furthermore the users have the option to execute a simulation crop analysis, for the supported fields only, and visualize the results of the simulation.

It is based on the AgroDataCube Service (https://agrodatacube.wur.nl/), that is currently supports parcels from the Netherlands.

agrodatacube-dashboard-main.png

Plant Height Visualization

The Food Security component has expressed the need for visualizing the height of plants at a specific time. In addition, it was desirable to be able to display the image of the plant for a selected point, as well as to be able to select multiple points in order to visually compare the different images.
A specific type of visualizing and with a specific configuration was made available to support this case. An example can be seen below:

plant-height-with-images.png

Plant Variables HeatMap Visualization

A usable and interactive way to visualize data from a greenhouse is a spatial visualization. The position of every plant is known and the user can select a specific date and a heatmap is presented indicating the value of the variable for each plant in the greenhouse. This visualization is available and can be seen below:

heatmap.png

Network Graph Evolution Application

There might be changes in the structure of a network (e.g. supply chain networks, patent citation networks, or twitter topic networks) that can only be captured by having a time-resolved visualization, i.e. a kind of movie. In the Food Risk Assessment community, it is important to visualize the changes of such a network based on specific criteria and a date range and display it in a movie.
A first approach on such a visualization has been implemented. It supports a specific data schema in json format and can dynamically support filters on the available metadata and dates.

network_graph_full_view.png

Documentation

Detailed documentation on the visualization framework and the Geospatial platform can be found in the dedicated pages

Source Code

Visualization framework is an open source framework and its source code can be found at: https://github.com/madgik/VisualisationFramework
It is available under the EUPL V1.2 license. For more information visit: https://joinup.ec.europa.eu/sites/default/files/custom-page/attachment/eupl_v1.2_en.pdf

References

NASA WWW: https://worldwind.arc.nasa.gov/
OpenLayers: https://openlayers.org/
Chart.js: http://www.chartjs.org/
Plotly: https://plot.ly/
D3.js:https://d3js.org/

mindmap-aginfra-communities.png (59.8 KB) Panagiota Koltsida, Oct 04, 2019 09:53 AM

WP7_plant-variables_UoA_14062018.png (65.9 KB) Panagiota Koltsida, Oct 04, 2019 10:59 AM

07 - Layer Details.png (1.3 MB) Panagiota Koltsida, Oct 04, 2019 11:27 AM

agrodatacube-dashboard-main.png (703 KB) Panagiota Koltsida, Oct 04, 2019 01:03 PM

plant-height-with-images.png (228 KB) Panagiota Koltsida, Oct 04, 2019 01:14 PM

heatmap.png (75.1 KB) Panagiota Koltsida, Oct 04, 2019 01:24 PM

network-move-application.png (113 KB) Panagiota Koltsida, Oct 04, 2019 03:09 PM

visualisation_framework_arch.png (147 KB) Panagiota Koltsida, Oct 04, 2019 03:36 PM

network_graph_full_view.png (202 KB) Panagiota Koltsida, Feb 11, 2020 08:22 AM