May 2020 Allotrope Connect

Public Session Day 3, May 13th: Abstracts

​

10:00am - HDF Group: Highly Scalable Data Service (HSDS): an open source reference implementation of HDF5 in the cloud

Authors:

Aleksandar Jelenak, Senior Informatics Architect, HDF Group

John Readey, Senior Architect, HDF Group

​

Abstract: The Hierarchical Data Format (HDF) implements a model for managing and storing data. The model includes an abstract data model and storage model (the data format), and libraries to implement the abstract model and map the storage model to different storage mechanisms. HDF5 is the current open-source library that provides users a programming interface to a concrete implementation of the abstract models and serves as the basis for the Allotrope Data Format (ADF). 

​

When HDF5 was released, it was designed to address current and anticipated requirements of modern systems, applications, and storage mechanisms. Unfortunately, this did not include object storage and access as a web service. In order to meet the growing expansion and interest by the HDF community into the cloud, The HDF Group has implemented the existing HDF data model into a new, REST-based web service for HDF5 data stores. This new implementation is called the Highly Scalable Data Service (HSDS). 

​

HSDS is an open-source solution for reading and writing HDF data using a storage schema designed to work well with object storage (e.g. AWS S3, Azure Blobs) or POSIX-based storage. Developed to make large datasets accessible in a manner that’s both fast and cost-effective, HSDS stores HDF5 files as objects, but provides the functionality traditionally offered by the HDF5 library as accessible by any HTTP client. This presentation provides an overview of HSDS including how data can be stored in either a POSIX files system or using object-based storage such as AWS S3, Azure Blob Storage, or OpenIO. In addition, we will discuss how HSDS can be deployed on a single machine using Docker or on a cluster using Kubernetes (or AKS on Microsoft Azure).
 

10:30am - Mestrelab: NMR Converter: Exciting next steps

Authors: 

Guy Desmarquets, Senior Vice President, Global Sales & Strategic Partnership, Mestrelab
Mark Dixon, Senior Account Manager & Business Development, USA & Canada, Mestrelab

​

Abstract: Following the Tabular Data Model approach, a converter of Bruker NMR format to ADF (B2ADF) has been created. This was a first step which generated expectation for the next steps, the key one being developing a Varian (Agilent) NMR format to ADF converter. The state of development and the challenges encountered for developing the V2ADF are presented and discussed during this talk. Finally we will discuss the future next step about reconciling the Bruker & Varian-specific metadata models to come up with a “universal” NMR model for those terms where it makes sense.

11:00am - Allotrope Product Team: From Modeling to Validation - Interoperable ADF, ADM & AFO

Authors: 

Amnon Ptashek, Technical Director, Allotrope Foundation

Ben Wooford-Lim, Senior Engineer, Allotrope Foundation

​

Abstract: Interoperability is a fundamental key to transform the acquisition, exchange, and management of laboratory data throughout its complete lifecycle. Using semantic technology, the Allotrope Data Format (ADF) and the associated Allotrope Data Model (ADM) together with a controlled vocabulary provide a mechanism to model simple as well as rich use case scenarios. The presentation covers the technical aspects of the modeling process from the fundamental structure of the file format and models all the way to validation and the associated tools. It covers the technology in a high level but at the same token dives and zooms into the structure of the fundamentals such as RDF, Allotrope Ontology, ADF composition, ADM and its Manifest, Tabular versus Graph modeling, Shape and SHACL Validation.

11:30am - Allotrope Product Team: From Modeling to Production - ADMs, WGs, Tools, SDLC, Support Maintenance

Authors: 

Amnon Ptashek, Technical Director, Allotrope Foundation

Matthew Fox, Product Director, Allotrope Foundation

​

Allotrope modelling is structured around a well defined processes and infrastructure to enable community members to materialize the technology and bring it into production. The presentation covers the guidelines and the governance process, the SDLC infrastructure, the support and maintenance, and  the list of approved models and the ones in the pipeline. Validation and ADF inspection tools will be presented. In addition a walkthrough the Aggregation Model development  will be demonstrated.