Automations

Automations are jobs that you can schedule on a regular basis (e.g. every night, every hour). There are automations to manipulate data, to send emails, to run any Python script... A Chain-Script automation can be used to execute several "child" automation one after the other. An automation can also be triggered by an API call or from a button within a dashboard.

On any automation page:

• The first tab lets you configure the automation.
• The Execution tab lets you run the script or schedule it.
• The Usage tab shows you how your automation is used.
• The Wehhook tab lets you configure an URL you can call to trigger the execution of the automation from an API call.

Data-related automations

• ETL-Step: To Extract data from a datasource, Transform it and Load the result in a datawarehouse (i.e. a storage). This is called an ETL step.
• Pivot data: to pivot a table and store the result in another table.
• FTP import: to import CSV or XLS files from a (S)FTP into a storage.
• S3 import: to import CSV or XLS files from a S3 bucket.
• Sync datasource to storage: to copy all the content from a datasource into a storage.
• Concatenate sources to storage: to concatenate the content of several datasources in a single storage.
• SQL Query to storage: to store the result of a SQL query on your database into a storage.

Send PDF report by email

This job generates a dashboard as a PDF file (one page per tab) and send it by email to a list of recipients. Recipients do not need to be Serenytics users.

Send data as XLSX/CSV file by email

With this job, you can define a dataset and export as an XLSX or CSV file attached to an email.

In the file section, you can use a template to insert a date in the filename.

Here are some examples:

To generate the date of the day (in UTC timezone)
data_{{DATE_YYYY_MM_DD}}.xlsx

To do more custom formatting, use the function format_now(format,timezone):
data_{{format_now("YYYY---MM---DD")}}.csv
data_{{format_now("YYYY---MM---DD HH:mm", "Europe/Paris")}}.csv

Export data as XLSX/CSV file to SFTP/S3

Generate an export (XLSX or CSV) and drop it to a SFTP or S3 bucket.

In the file section, you can use a template to insert a date in the filename.

Here are some examples:

To generate the date of the day (in UTC timezone)
data_{{DATE_YYYY_MM_DD}}.xlsx

To do more custom formatting, use the function format_now(format,timezone):
data_{{format_now("YYYY---MM---DD")}}.csv
data_{{format_now("YYYY---MM---DD HH:mm", "Europe/Paris")}}.csv

Alarm

To send an email when a value is above a given threshold.

Chain automation

When executed, a chain automation will trigger all its "child" automation, one after the other.

If a "child" automation fails, the chain automation will stop and be marked as failed.

Python script automation

A python script lets you run any Python code, within Serenytics. See here for full documentation.

Other automations

• Refresh cache: to load the cache of a given dashboard (to reduce loading time for users)
• Reload users metadata: to load users' metadata from a datasource (usually an XLS file).

Run-as-user field for automations

When an automation is run, it must decide which user credentials will be used. This is very important as a script using credentials from a user with large permissions will have access to many datasources.

If the automation is run from the Studio interface by clicking on 'Run Now' button, the automation will be run using the credentials of the connected user.

If the automation is run automatically from a scheduling, the automation uses the credentials of the automation field run-as-user. When a user creates a new automation, he becomes the default value for run-as-user. If this field is not set (i.e. the user has been deleted), the script uses the credentials of the first admin user of the organization.

This field can be modified from the Advanced tab in the script configuration (only admin users can modify this field).

Allow all users to run this script field

In some cases, you want to allow all the logged user of your organization to run a script. This is only possible if you enable this option (by default, only studio users who are authorized on the automation's folder can run it). For example, this is required if you add a button in a dashboard triggering this script (using the function btn_run_automation).

Credentials and Allow all users to run this script option

When this option is activated, the automation will always be run with the credentials of the user defined in the field run-as-user. If you are using a Python script, you can retrieve the user who triggered the script in the dict provided by client.script_args in the field triggered_by_user.

Mutex

When a mutex is configured for a set of scripts, it ensures these scripts will never run in parallel. This is particularly useful when your scripts are doing reads and writes on the same data and are triggered from dashboards buttons (in this case, you have no control over their executions' order).

When several scripts share the same mutex name, if they are run at the same time, one of them will run first, and the others will wait. Once the first one is finished, the second one will run and so on.

An easy way to test this option is to create two Python scripts. The first one executes a time.sleep(10) and the second one is a simple print("hello world"). When both scripts are executed simultaneously, the second script will be executed almost immediately. If you define a mutex for them and trigger them at the same time (starting with the waiting script), the second one will be executed once the first one is finished (so it will be in waiting status for at least 10 seconds).