Overview
Quality assurance is a fundamental part of mature service provision, and during such quality assurance processes, assessment of compliance with a variety of norms and standards as well as end user expectations are typically involved. Compliance can be evaluated or assessed in formal or informal ways, and the results can be adjudicated internally, or by way of external review or audit - these are all variations of the same fundamental process.
Compliance assessment should be an integral part of e-infrastructure provision in general, and specifically should apply to quality assurance in the field of research data management - ensuring that the systems, services, and functionality provided to the community meet the relevant norms and expectations. While relevant norms and associated quality assurance frameworks exist for digital systems in general (ISO 27001, ITIL, etc.), compliance assessment in respect of other considerations in the landscape is relatively poorly developed.
The Compliance Assessment Toolkit (CAT), while aimed at the operational assessment of EOSC PID Policy compliance (and derivations based on the policy) was purposely designed and implemented to be reusable in many contexts, including current and emerging sets of quality assurance needs in the landscape. These cover, but are not limited to, aspects such as community expectations (FAIR, TRUST, and CARE), EOSC alignment (node entry requirements, interoperability frameworks, rules of participation), and formal specifications (e.g. participation in AAI federations). The CAT is operationally available for PID Policy compliance assessments, and work on beta releases for FAIR and AAI federation assessments are under way.
The Compliance Assessment Toolkit provides a platform for the standardised encoding, recording, publication, and querying of compliance. Long-term persistence of assessments is also supported. The toolkit links assessment to best practices, guidance, specifications, and recommendations, assisting end users not only to determine a level of performance against a benchmark, but also to identify steps towards improvement. The operational instance of CAT - focused on PID Policy Compliance - allows casual users to search for and find assessments, obtain PID selection and appraisal support via the PID Knowledge Base, and integrate read-only API services into external systems. Validated users can perform, edit, and publish assessments, and curators (administrators) can assist end users and institutions with the development and deployment of custom PID policies. These can, in turn, be used to validate the compliance of institutions and services against the custom policy.
The CAT is expected to have a significant impact on evaluation, appraisal, and assessment-related tasks and requirements in the context of EOSC. There is a pervasive problem in the research landscape, in that alignment, compliance, or fulfilment are often claimed without any public evidence. It is, for example, common to claim FAIR compliance, alignment with TRUST principles, or support for Open Science without any verifiable evidence that such claims are based on a consistent and repeatable process of assessment. CAT addresses this by standardising and harmonising compliance assessment, and by making a public record of compliance available for the long term. In doing so, it directly supports quality assurance in EOSC and beyond.
