Structural Representation Information
This page describes a number of ways in which Structural Representation Information may be created.
Last updated
This page describes a number of ways in which Structural Representation Information may be created.
Last updated
A Data Object may be described down to the bit level in a text document, by which we mean they are meant for human use rather than a computer being able to use it (this may change as computers using deep learning to truly understand the text).
As an example the Internet Engineering Task Force standards, on which much of the Internet is based, is documented in RFCs. For example the RFC describing ASCII format for Network Interchange is RFC20, which includes a table, constructed using text characters, to show how ASCII alphanumeric characters and a number of control characters should be encoded in bits.
Other RFCs include complex diagrams constructed using ASCII characters, for example RFC5755.
Other text descriptions are for example PDF documents such as for SOHO, or indeed paper documents in the case of older instruments.
Several formal description languages are available which allow the description of Data Objects down to the bit level in a way which can be used by computers.
This is a CCSDS and ISO standard language which allows data to be described. There is a support website for EAST Based Access Tools – the BEST toolkit, and other tools supported by the French Space Agency. An example complex structure for a communications package which can be described by EAST is shown below.
CDPP (Plasma Physics Data Centre) uses EAST to describe and provide access to and combine many types of data, and provides many examples.
DRB is is an Open Source Java application programming interface for reading, writing and processing heterogeneous data.
DFDL is a modeling language from the Open Grid Forum that allows description of text, dense binary, and legacy data formats in a vendor- neutral declarative manner. DFDL is an extension to the XML Schema Description Language (XSD).
DFDL is a way of describing the data. It is not a data format. DFDL should be able to describe many data formats, including:
Textual and binary
Commercial record-oriented
Scientific and numeric
Modern and legacy
Industry standards
An open source interpreter is available as Apache Daffodil
DFDL is used to describe various scientific data files, particularly those used in Earth Observation, by ESA in the Standard Archive Format for Europe (SAFE).
On ingestion LABDRIVE uses a number of tools (see https://public.docs.libnova.com/labdrive/api/#/PRONOM) that enable the automatic identification of the file format of a particular file, typically by examining file signatures, usually the first few bytes of the file.
The file formats are identified using PRONOM identifiers, however PRONOM provides very little information about the format as illustrated in the following.
The format identification allows the file to be displayed but does not normally identify the various components to which semantics may be attached.
the BEST toolkit, and other tools
