Introduction to metadata

LABDRIVE allows users to add metadata to any file, folder or data container preserved in the system. Functions can be used to automatically generate metadata on ingest, or it can be added manually, or both.

Metadata can be used for searching (even using complex queries), exported or consumed by other systems. Metadata can be associated with the objects in the following ways:

• Manually, using the LABDRIVE Management Interface (for a single item or in bulk),

• Loaded from a CSV, XML or JSON file,

• Using the LABDRIVE API,

• Using the LABDRIVE Lambda Functions (you can define code functions that will do rich and advanced extraction and processing to your metadata)

• Using the LABDRIVE Python library in conjunction with a Jupyter Notebook.

Add metadata using the Management Interface

Add metadata for a single item manually

1. Locate the data container you would like to add metadata to using the Containers menu section or by searching. This guide assumes metadata is properly configured for the data container. See Configuration\Metadata for more details or Working with data containers to see how to create them.

2. Select Check-in in case you are not checked in the container and you have the check-in/out enabled for the data container.

3. In the data container page, choose Explore content:

4. Right click the item you would like to add metadata to and select Properties

5. In the dialog, select the Metadata tab, and the metadata fields that are part of the metadata schema linked to your container are shown. You can add or remove metadata from them. When you have finished, do not forget to select Save.

Add metadata to multiple items in bulk

It is possible to add metadata to multiple items at once.

1. Locate the data container you would like to add metadata to using the Containers menu section or by searching. This guide assumes metadata is properly configured for the data container. See Configuration\Metadata for more details or Working with data containers to see how to create them.

2. Select Check-in in case you are not checked in the container and you have the check-in/out enabled for the data container.

3. In the data container page, choose Explore content:

4. Select multiple items using your mouse to click-and-drag a box around the files or folders you want to select. You can also use Ctrl and Shift and use the Select all, Select none or Invert selection in the file browser top bar.

You can also filter the files in the folder you are looking at by file name, for instance, for selecting all XML or JPG files:

5. When the items to which you want to apply metadata have been selected, select the Function Bulk Metadata Editor in your functions side bar. If you have folders selected, LABDRIVE will also apply your metadata to the files contained in them:

Exporting and Importing metadata

Object's metadata can be exported in XML, JSON or CSV format. To export metadata, select the object, right click and select Properties. Then select the Metadata tab. The export option is at the bottom of the form, in the Download button:

For instance, exporting it in JSON format produces the following output:

For importing metadata back, select the Import metadata option:

Metadata recommendations and safe characters

XML Related Object Key Constraints

As specified by the XML standard on end-of-line handling, the following special characters will be normalized and replaced by their equivalent SML entity code when they are inserted within XML tags:

Special Character	XML replacement
'	'
"	"
&	&
<	&lt;
>	&gt;
\r	&#13; or &#x0D;
\n	&#10; or &#x0A;

Reserved symbols and codes

The platform uses a pre-defined set of characters as prefixes across the different functions and services offered. Therefore, it is recommended to avoid these combinations and reserve them for the required functions and events run within the platform:

Set of Characters	Usage
[|] (pipe between square brackets)	Separator for more than one metadata value (i.e., Author 1[|]Author 2[|]Author 3)
[|||] (three pipes between square brackets)	Setting a link metadata value (API only) (i.e., type:%type%[|||]description:%description%[|||]link:%link%)
.. (two consecutive periods)	Multilanguage prefix (Transfer Connector) (i.e., es..title, es..author)

Add metadata using the API

When users are uploading files to the platform, search results may not show the new file for a while. Consider this in your code when you are uploading and immediately after the upload, you need to search for the uploaded file or alter its metadata. See Tips when working with the API.

API examples here are just illustrative. Check the LABDRIVE API documentation for additional information and all available methods.

1. Sign in to the LABDRIVE Management Interface

2. Obtain your LABDRIVE API key selecting your name and then Access Methods:

3. To change metadata for an item using the LABDRIVE API, you need to know the unique file identifier (LABDRIVE File id) for it. You can do that by using the API search methods. LABDRIVE API for searching is rich on features and it accepts complex queries.

In this guide we are going to search for the items matching a file name:

Curl

curl --request POST \
  --url "$your_labdrive_url/api/file/elastic" \
  --header "Authorization: Bearer $your_labdrive_api_key" \
  --data '{
  "must": [
    {
      "term": {
        "filename.keyword": "my_filename.txt"
      }
    },
    {
      "term": {
        "container_id": 100
      }
    }
  ]
}'

Use:

• url: Your LABDRIVE address

• header: Your LABDRIVE API Token (add Bearer prefix)

• filename.keyword: The filename you are searching for

  • It is important to add ".keyword" to this field, to allow for exact matches. LABDRIVE will not provide accurate results for the "filename" field without the ".keyword"

• container_id: (optional) The id of the container to filter by.

4. With your unique file identifier (LABDRIVE File id) you can list your item's metadata:

Curl

curl --request GET \
  --url "$your_labdrive_url/api/file/<FileID>/metadata" \
  --header "authorization: Bearer $your_labdrive_api_key"

Use:

• url: Your LABDRIVE address

• ID: The LABDRIFE File ID from the previous step

• header: Your LABDRIVE API Token (add Bearer prefix)

Or you can update it: If you would like to set the metadata field title to "Experiment result 188/12", you can do it in the following way:

Curl

    curl --request PUT \
    --url "$your_labdrive_url/api/file/<FileID>/metadata" \
    --header 'Content-Type: application/json' \
    --header "authorization: Bearer $your_labdrive_api_key" \
    --data '{
      "metadata": [
   	      {
   		        "iecode": "title",
   		        "value": "Experiment result 188/12",
   		        "action": "replace"
   	      }
       ]
    }'

You can also add multiple values to a descriptor by sending an array of values:

Curl

    curl --request PUT \
    --url "$your_labdrive_url/api/file/<FileID>/metadata" \
    --header 'Content-Type: application/json' \
    --header "authorization: Bearer $your_labdrive_api_key" \
    --data '{
      "metadata": [
   	      {
   		       "iecode": "dc.creator",
   		       "value": ["Emmanuelle Charpentier", "Jennifer Doudna"],
   		       "action": "replace"
   	      }
       ]
    }'

Use:

• url: Your LABDRIVE address

• ID: The LABDRIFE File ID from the previous step

• header: Your LABDRIVE API Token (add Bearer prefix)

• iecode: The metadata field to edit

• value: The metadata value to edit (string or array of strings)

• action: replace (remove existing value/s and add the new ones), add (add a new one, leaving the existing value/s) or delete (will clear any value)

CAUTION: If you include no action parameter IN ANY of the request parameters, LABDRIVE will remove EVERY metadata field value from the object and will just add the last one you included (not just the one you are including in your request). This is designed for the case in which you want to make a single query and replace every metadata field in the item.

This means that if the file has title="new dataset" and author="Mike" and you launch a method without specifying an action with author="David", the author will be set to "David", but the title will be removed.

Metadata which is specified in the OAIS Reference Model Information Model is described in LABDRIVE support for OAIS Conformance.