Configure metadata

Understanding item's metadata

Metadata is data that describes other data. "Meta" here means "an underlying definition or description." Metadata summarizes basic information about your data, which can make finding and working with your data easier.

Every item in a LABDRIVE data container has a unique item identifier and, by using this unique identifier, the platform is capable of associating metadata to it.

Metadata can be used for searching (even using complex queries), exported or consumed by other systems. See Search and Queries for more details.

Metadata can be associated to the objects:

• Manually, using the LABDRIVE Management Interface (for a single item or in bulk),

• Loaded from a CSV, XML or JSON file,

• Using the LABDRIVE API,

• Using the LABDRIVE Lambda Functions (you can define code functions that will do rich and advanced extraction and processing to your metadata)

There are some metadata types like the file name, associated events or the container an item is in, that the platform is creating and maintaining for you, while there are metadata types that you can easily add to your objects using multiple methods.

All metadata in LABDRIVE is indexed and searchable.

How metadata is organized

Metadata fields that you would like to have assigned to your items are called Descriptors in LABDRIVE.

Descriptors are grouped into Metadata Schemas. For instance, when two departments with different metadata needs share a single LABDRIVE instance, each department can create its own schema. Eg: the Physics lab will have "Experiment ID", "Date" and "Creator" in their "Physics" metadata schema while the Biology department will have "Specimen name" and "Date" as their metadata schema. When the schema is assigned to a data container, they can use their own fields to describe the content they place in.

Each Descriptor has a:

• Name: that is shown in the management interface and helps users to easily identify them.

• Description: that is shown in the management interface and helps users to understand context or instructions on how to fill certain descriptors.

• IECode (Import/export code): that is used as a field identifier for the API and functions, as a friendly name for the Descriptor.

• Data type: is the type of data the field accepts:

  • String,

  • Multi-line string,

  • Date,

  • Linked field (includes a link to another item or to any HTTP external resource)

  • or Value list (enumerated list of possible values)

• Category: Descriptors can be grouped visually in the Management interface. Category defines this grouping. Eg: Create a "Descriptive" category and create "Title " and "Author" inside. Create a "Rights" category and create the "Usage rights description" and "License" inside.

Field types are not yet validated when using the API (eg: introducing a value for a Value list type descriptor that is not in the list of possible values). It is the user responsibility to use them properly.

Configuring metadata descriptors

Using the web interface

To create a metadata schema:

1.- Go to Configuration and then to Object metadata

2. In the submenu, select Categories first and use the Add Category button to create a category. You will need to have at least one for the next step.

3. Go to Configuration and then to Object metadata (again), but this time select Metadata instead.

4. Create a new schema using Add Schema. Give it a meaningful name related to the content you are going to use or the type of metadata that it will contain. Eg: "High energy experiments" or "Dublin Core".

5. Select the option Show Descriptors for the one you have created.

6. A new button Add Descriptor will appear to create your new descriptor. Select it.

7. Fill in the form with the definition for the descriptor (see above for a description of each field in the descriptor):

• Name.

• Description.

• IECode (Import/export code).

• Data type:

  • String,

  • Multi-line string,

  • Date,

  • Linked field (includes a link to another item or to any HTTP external resource)

  • or Value list (enumerated list of possible values)

• Category: Select the category you created previously.

Select OK to save your new descriptor.

8. Continue creating the descriptors you need.

Now, when you create a new container, LABDRIVE will ask you what is the declared metadata schema you want to use:

If you add an item to your container, you can right-click it, go to the Metadata tab, and you will find all descriptors you have created:

Using the API

API examples here are shown just for illustrative purposes. Check the LABDRIVE API documentation for additional information and all available methods.

The platform provides great flexibility on managing metadata, and allows the user to assign metadata to the containers and/or to the objects (folder/files). Metadata schemas (set of fields) can be different for containers and for files, as they usually have distict needs.

Container level-metadata schemas

To create a container-level metadata schema using the API:

1. Sign in to the LABDRIVE Management Interface

2. Obtain your LABDRIVE API key selecting your name and then Access Methods:

3. Use the following method to create the new metadata schema:

    curl --request POST \
    --url "$your_platform_url/api/containers/metadata/schema" \
    --header 'Content-Type: application/json' \
    --header "authorization: Bearer $your_platform_api_key" \
    --data '{
      "name": "New metadata schema",
      "description": "Schema description"
    }'

LABDRIVE answers you with the identifier of the created schema:

Alternatively, if you already created the schema, it is possible to get a list of them using:

curl --request GET \
      --url "$your_platform_url/api/containers/metadata/schema" \
      --header 'Content-Type: application/json' \
      --header "authorization: Bearer $your_platform_api_key" 

4. Metadata fields are grouped into categories (to group them visually when using the Management Interface). As every metadata field needs to belong to one category, we need to create them in advance using the following method:

curl --request POST \
      --url "$your_platform_url/api/containers/metadata/category" \
      --header 'Content-Type: application/json' \
      --header "authorization: Bearer $your_platform_api_key" \
      --data '{
        "name": "New metadata category",
        "collapse": "no"
      }'

LABDRIVE answers with the category id:

Alternatively, if you already created the category, it is possible to get a list of them using:

curl --request GET \
      --url "$your_platform_url/api/containers/metadata/category" \
      --header 'Content-Type: application/json' \
      --header "authorization: Bearer $your_platform_api_key"

5. Now, using the schema identifier and the metadata category identifier, you can create the item metadata field:

curl --request POST \
     --url "$your_platform_url/api/containers/metadata/schema/5/descriptor" \
     --header 'Content-Type: application/json' \
     --header "authorization: Bearer $your_platform_api_key" \
     --data '{
"name": "New descriptor", 
"description": "Additional description for this field",
"iecode": "new_descriptor",
"container_metadata_category_id": 4,
"data_type": "STRING"
}'

The data_type parameter indicates the type of value the field contains, and possible values for it are:

• STRING

• LONGSTRING (for a multi-line text box)

• NUMBER

• DATE (in YYYY-MM-DD format)

• BOOL (true or false)

• LINK_FIELDS (a link to other system file)

• ENUM (the management interface offers a selection for the possible values for the field. The API DOES NOT ENFORCE OR VALIDATE THEM)

For using the ENUM, you should include an additional parameter in the query with the possible values using "enum_values": [ "Value 1", "Value 2" ]

Object (folder/file)-level metadata schema

In the same way as in the previous section, it is possible to create object-level metadata schemas programatically (usually, simply replacing the "containers" by "objects" in the method):

To create a metadata schema, you can use:

 curl --request POST \
      --url "$your_platform_url/api/objects/metadata/schema" \
      --header 'Content-Type: application/json' \
      --header "authorization: Bearer $your_platform_api_key" \
      --data '{
        "name": "New object metadata schema",
        "description": "Schema description"
      }'

To list existing schemas,

curl --request GET \
        --url "$your_platform_url/api/objects/metadata/schema" \
        --header 'Content-Type: application/json' \
        --header "authorization: Bearer $your_platform_api_key"

To create a category, you can use:

curl --request POST \
        --url "$your_platform_url/api/objects/metadata/category" \
        --header 'Content-Type: application/json' \
        --header "authorization: Bearer $your_platform_api_key" \
        --data '{
          "name": "Tier 4",
          "collapse": "no"
        }'

To list existing categories, you can use:

curl --request GET \
        --url "$your_platform_url/api/objects/metadata/category" \
        --header 'Content-Type: application/json' \
        --header "authorization: Bearer $your_platform_api_key"

To create a new metadata field, you can use:

curl --request POST \
       --url "$your_platform_url/api/objects/metadata/schema/1/descriptor" \
       --header 'Content-Type: application/json' \
       --header "authorization: Bearer $your_platform_api_key" \
       --data '{
  "name": "New descriptor",
  "description": "Additional description for this field",
  "iecode": "new_descriptor",
  "metadata_schema_category_id": 1,
  "data_type": "STRING"
  }'

Load metadata schema in bulk

When creating a few metadata fields, using the Management Interface to create them is the easier way. But when your metadata schema has multiple fields, you can also load them in bulk.

The platform allows you load them using a XML file or a JSON file. Let's see how this is done using the JSON option:

1. Log in to your platform with a user account with administrative credentials.

2. Go to your Configuration menu

3. Select the Object metadata section and then Metadata

4. Select the Import metadata option:

5. Upload your file with the following format:

{
    "id": "", // Add to existing object metadata schema
    "name": "",
    "description": "",
    "descriptors": [
        {
            "name" : "",
            "iecode": "",
            "metadata_schema_category_id": "",
            "description": "",
            "data_type": "",
            "enum_values": ""
        }
    ]
}

For the data_type option, the following values are considered:

• STRING: One-line string field

• LONGSTRING: Multi-line string field

• ENUM: List of values

• DATE

• NUMBER

• LINK_FIELDS: Link to other folder/file or external HTTP links.

When using the ENUM data type, you can also specify the possible values for the field:

    {
        "name": "Publisher",
        "iecode": "DC.PUBLISHER",
        "metadata_schema_category_id": "1",
        "description": null,
        "data_type": "ENUM",
        "enum_values": "[\"Value1\",\"Value2\",\"Value3\"]"
    },

If you want to make a copy of an existing schema as a base for a new one, you can EXPORT the one you would like to use as the template as a JSON file and IMPORT it as a new one once you have made the desired changes in the JSON file.

BUT: It is important that you remove the "id" parameter from the JSON/XML file you load back to the system. If not, the platform will give you an error, as it will understand you want to add fields to the existing schema.

LABDRIVE support for the OAIS Information Model is described in LABDRIVE support for OAIS Conformance.