Functions
The platform's Functions let you run code in the platform, in response to certain events or triggers, without needing any client-side script.
Functions are useful when you want the platform to behave in a specific way in response to internal or external events, or when you want to add your own code to be executed on demand (typically by yourself), in the same way Macros work for MS Excel, for instance.
With the Platform's Functions, you just upload your code and add the triggers that you would like it to execute. Users are able to define data container-level Functions that are executed on certain events:
CRUD Functions for files and metadata: When files are created, read, updated or deleted.
E.g. every time you upload one of your astrophysics-related files to a certain data container:
E.g.: Every time you upload one of your photographs:
Extract the date from the file name to the metadata catalogue, so it is searchable using the web interface or the API,
Calculate each file integrity and
Tag images that contain faces
Periodic functions: Every minute, hour and day.
E.g.: Webhooks to other systems
Executed-by-demand functions: When the user selects files using the GUI or launched using the API.
E.g.: For fetch.txt files containing a manifest of files to download, make the platform to download them and place them inside the container.
How to build a Function?
When building a platform function, two aspects are usually defined:
The interface or triggers that the function will consider: When the function is going to be presented to the user, or how the platform will execute it.
The code that the function will execute
Let's build a sample function step by step:
Declaring the function in the interface
1. Go to Configuration and then to Functions:
2. Select Add function:
3. Give your function a meaningful name and description (both elements will be shown in the interface to the users, so make it sure you keep it short and concise). Enable it.
4. Go to the triggers section and select the events that you would like to execute the function.
You have three main types of triggers:
On demand: The function is executed when the user selects "RUN" in the interface (or when the function is called using the API):
Automatically: On item (folder/files) create, delete, metadata change or storage policy change: The platform will automatically execute the function when, for instance, a new file is created. This is useful to automate certain actions.
Periodically: The platform will execute the function every minute, day, week, etc, as defined in a cron string.
Select the type of trigger carefully following the instructions in the platform interface.
It is possible to select more than one trigger for the same function (when the function is executed, it receives the trigger type that has activated it, so you will be able to manage in your code how to proceed on each case).
Be extra careful when selected triggers like "On item create" as, if you don't properly adjust it, it can generate huge workloads and problems. If you are in doubt, ask LIBNOVA.
5. Go to the parameters section and define the parameters that the function will accept. These parameters are only for on-demand functions, in which the user will be requested to provide certain parameters when calling the function.
You have multiple types of parameters to choose from:
6. Add your code.
7. Select Create
How to code a Function?
Functions workflow
When you create a Function, the platform will precompile it, package it in a Docker container and deploy one or more instances (depending on the parameter Replicas in the Configuration tab).
This means that your functions are always running and ready to accept your workload. There is no warm-up when launching them.
When an execution request is received, the platform creates a Job and sends a message to a queue for execution. Any of the instances/replicas that are running will catch the request and start processing it. One instance/replica is capable of executing only one request at a given time. When it finishes, it starts working in the next one.
Function build/deploy cycle
Under the Code tab, you can create and edit your function. When you have added your code, simply select Save. This will initiate the compilation of the function, that usually takes a few minutes.
To see what is happening, you can select See function build log in the right top corner:
If everything goes fine, you will see a green Ready message:
This means that your function has been compiled, and that the number of Replicas specified are running.
If your code contains errors, the platform will set it in error. You can scroll down in the message to have a hint on the problem:
Function's code structure
Let's create a sample function.
First, you should initialize your function, initializing your components and importing any required dependency. Second, the handler function, that will contain a try block:
When the function is called, the handler method is the first that is called:
and under it, you have your logic.
It is important that you enclose all your logic in a try block. If you don't do so, unexpected things will happen when you have errors in your code (the function will restart every time you build it, re-using the same job id, which will create repeated entries, etc. In other words, a mess you will usually want to avoid). Always remember to add it.
Sample function source code
Here you have the full code of a working function that demonstrates the use of the functionalities. We have added many comments explaining every step:
The code you have in lines 42, 45 and 50 produces the following output in the function log:
The code you have in lines 136 to 140, produces the following output in the function log:
When something goes wrong in your code, the exception handling in line 146 will start, producing this output in the function log, that will provide a hing on the cause:
If you want to test this function, simply create a new function, paste your code, provide a name and create the following parameters:
Save it, and you will see it appearing in your container's side bar:
Launching a function using the API
When a function trigger is ON_DEMAND, you can also launch it programatically. For instance, if you would like to launch the previous function, you first need the function id. To get the functions and their ids, you can use:
This will present you the list of available functions:
Then, if you want to launch it over a given file (check Search using the API to get the file id of the file you want to execute the function over) you should use:
It will return the job id the function has been assigned to:
And finally, you can programatically check the status of a job using:
It is also possible to retrieve the function log using:
And its assets:
Last updated
Was this helpful?