3.3 Assess FAIRness: tools and practices

Description

In this unit you will learn to:

Choose appropriate tools to assess the FAIRness of data and other research outputs

Learning resources

Presentation Slides

Tools for assessing FAIRness

Given the flexibility of the FAIR principles, it is usually difficult to assess FAIRness in a systematic way. Furthermore, as principles, they are open to interpretation and this interpretation may require expert support to ensure that the FAIR principles are still being applied correctly. The FAIR principles are also different in nature, which has a direct impact on the FAIRness assessment. Some correspond to strictly technical aspects that can be both automatically implemented and verified. Others correspond to contextual and community standards that require a human evaluation of their implementation.

In this context, the support of FAIR implementations is based on two main types of tools: manual and automatic self-assessment tools, in addition to the training that provides knowledge about FAIRification. The former allow data owners to analyse their datasets according to the FAIR principles, sometimes with a rough estimate of the level of FAIRness. The latter takes as input a URL to a dataset of any type and tests the characteristics of the dataset according to the FAIR criteria. In the case of automatic evaluation, given the room for interpretation of some principles, it is common for different automatic tools to give different FAIRness results.

These different tools should therefore be considered, in the current state of the art, as supporting tools, useful not to obtain a final and fully exhaustive result, but as a good indication of the general level of FAIRness.

FAIR Data Self Assessment Tool

The Australian Research Data Commons (ARDC) has developed a FAIR self-assessment tool of the manual type. It is presented as a web form with questions to be answered and contextual information to help users assess the level of FAIRness of their resources. The self-assessment process results in a global measurement through a bar line.

https://ardc.edu.au/resource/fair-data-self-assessment-tool/

FAIR-Checker

Developed by the French Institute of Bioinformatics, the FAIR-checker is an automated FAIRness evaluation tool. Given a URL of a dataset, the tool analyses the RDF metadata and collects information from various data aggregators. The result of the analysis is a radar chart where each of the four FAIR principles is given a score. A comprehensive table also shows the detailed results of the analysis.

https://fair-checker.france-bioinformatique.fr/

F-UJI

Similar to FAIR-checker, F-UJI is another automated self-assessment tool. Although the programming behind the tool is different, it is used in the same way. F-UJI analyses a URL by collecting all the metadata associated with the record and testing it against the detailed FAIR principles. The result consists of a global percentage of FAIRness for the four FAIR principles, the score achieved for each principle, and the corresponding level in each case. Note that although F-UJI and FAIR-checker use the same list of detailed FAIR principles, they don't check in exactly the same way, resulting in a more extensive list of checks by F-UJI.

https://www.f-uji.net/

FAIR-Aware tool

The FAIR-Aware tool (formerly SATISFYD) is a manual self-assessment tool. Like the others, it is discipline agnostic and provides a series of ten questions presented in a web form. Rather than listing all the detailed FAIR principles, it adapts its content to the needs and experience of researchers. Indeed, the tool focuses on raising awareness of the FAIR principles rather than on their concrete implementation. For each question, it provides a scale against which the data owner can measure their willingness to improve specific implementations of the FAIR Principles.

https://fairaware.dans.knaw.nl/