Ingestion

PreviousCatalog – Object group detail NextIngestion – Creating a new ingestion job

Last updated 3 years ago

Was this helpful?

Ingestion

The main ingestion page is used for:

Creating a new ingestion job, with the link Create new ingestion job, shown in the upper part of the page.
Getting a list of all ingestion jobs by execution status: not started, running, finished, waiting for user intervention, and unrecoverable.

All listings allow access to the detail page of each ingestion job. Jobs not started or waiting for user intervention may also have a button for Start or Re-start.

All ingestion jobs lists have a filter system before their header that filters the results shown by number and date range.

The main aspects of an ingestion job in LIBSAFE are:

They are considered to be historical processes, meaning that their registry cannot be deleted once created.
For jobs finished successfully, the content of the temporary ingestion folder is deleted. It is not recommended that you place in this directory an only copy of the object to be ingested.
There cannot be ‘retries’ of ingestion jobs. A job that fails, after user intervention, is really turned into a new job.
Ingestion jobs waiting for user intervention are jobs that have failed in the sanitizer, check and explorer steps. The files in the objects may be corrected and the job be restarted (actually, a copy of the files is made to a new job, and this one is executed). Once the new job is restarted, the failed job disappears from the general lists of ingestion jobs.
Unrecoverable ingestion jobs are those that have failed during the archiving and audit steps. These errors are related to technical aspects of the platform and require the intervention of a system administrator. The user should in any case check to guarantee that the platform is complying with all necessary conditions for the ingestion processes; the most frequent error being the lack of space in the storage groups (disks). It is important that the system administrator takes care, in the periodical deleting of files in the temporary interchange storage space, that failed objects do not occupy that space permanently.

It is possible and recommended to keep track of the result of ingestion jobs in an automatic way, by configuring the alarm system (only preservation administrators).

PreviousCatalog – Object group detail NextIngestion – Creating a new ingestion job

Last updated 3 years ago

Was this helpful?

The main ingestion page is used for:

Creating a new ingestion job, with the link Create new ingestion job, shown in the upper part of the page.
Getting a list of all ingestion jobs by execution status: not started, running, finished, waiting for user intervention, and unrecoverable.

All listings allow access to the detail page of each ingestion job. Jobs not started or waiting for user intervention may also have a button for Start or Re-start.

All ingestion jobs lists have a filter system before their header that filters the results shown by number and date range.

The main aspects of an ingestion job in LIBSAFE are:

They are considered to be historical processes, meaning that their registry cannot be deleted once created.
For jobs finished successfully, the content of the temporary ingestion folder is deleted. It is not recommended that you place in this directory an only copy of the object to be ingested.
There cannot be ‘retries’ of ingestion jobs. A job that fails, after user intervention, is really turned into a new job.
Ingestion jobs waiting for user intervention are jobs that have failed in the sanitizer, check and explorer steps. The files in the objects may be corrected and the job be restarted (actually, a copy of the files is made to a new job, and this one is executed). Once the new job is restarted, the failed job disappears from the general lists of ingestion jobs.
Unrecoverable ingestion jobs are those that have failed during the archiving and audit steps. These errors are related to technical aspects of the platform and require the intervention of a system administrator. The user should in any case check to guarantee that the platform is complying with all necessary conditions for the ingestion processes; the most frequent error being the lack of space in the storage groups (disks). It is important that the system administrator takes care, in the periodical deleting of files in the temporary interchange storage space, that failed objects do not occupy that space permanently.

It is possible and recommended to keep track of the result of ingestion jobs in an automatic way, by configuring the alarm system (only preservation administrators).