LIBNOVA DOCS Home
Pipelines module
Pipelines module
  • Pipelines Module Definition
    • Pipelines Module
  • Benefits and importance of the Pipelines module
  • Pipeline Metadata Definition (Metadata Schemas)
  • Creating a new Pipeline
  • Pipelines details
  • Target selection and object filtering (Triggers)
  • Execution/launching and review restrictions
  • Steps
  • Monitoring the progress of your pipeline tasks (Jobs)
  • Filtering jobs
  • Jobs and API access
  • Running a pipeline on a given container
  • Branching
  • Wait for user step
  • Automating your pipeline (using a function)
  • Stopping a running job
  • Pipeline Details
Powered by GitBook
On this page

Pipeline Metadata Definition (Metadata Schemas)

Last updated 7 months ago

Before we create any new pipeline, we have to define its metadata.

These metadata schemas have no relation with the ones we have in our objects or containers.

Pipeline metadata is independent.

Pipeline metadata definition refers to the structured information that describes and tracks various aspects of data as it flows through a data processing pipeline. It includes details such as data sources, transformations, processing steps, and output destinations.

Metadata helps ensure data consistency, traceability, and compliance with standards, enabling efficient data management, monitoring, and troubleshooting. Pipeline metadata is essential for understanding the data's journey, ensuring quality, and facilitating reproducibility in complex data workflows.

The following shows you where these can be included as part of one or several Metadata Schemas: