Job Scheduling

This section describes a mechanism for scheduling and executing data pipelines created in SQL or Python.

Functions such as load_data in SQL and @morph.load_data() in Python allow multiple processes to be built as a pipeline. Scheduled execution of this pipeline can automate recurring tasks such as daily sales totals.

By opening the Jobs tab, you can check the currently set jobs and register new jobs.

You can check the logs and execution results when the job is executed by opening the detailed page.

Job scheduling settings

Job scheduling should be configured for the last function in the pipeline. The load_data function also executes the function to be loaded at runtime, so any data pipeline can be executed periodically by specifying the last function in the data pipeline.

The execution date and time can be set flexibly, such as ‘every day’ or ‘only on weekdays’. The execution time can also be set in 15-minute increments.