Adding a Task that Runs a Job in a Structured Pipeline

Follow the instructions below to add a task that runs a job in a structured pipeline.

  1. In the Administration application, expand the Tools menu and click Workflow Manager. Anzo displays the Workflows screen, which lists any existing workflows. For example:

  2. Expand the workflow that you want to add a task to. For example:

  3. Click Add Task. The Create Task dialog box is displayed:

  4. At the top of the dialog box, click the drop-down list and select ETL Load Service to set up a task that runs a structured job. The dialog box presents the options that are valid for ETL Load Service Tasks:

  5. Configure the task by completing the following fields as needed:
    • Load Service Name: This field specifies the name for the task.
    • Target Job: This field specifies the job that this task should run. Click the drop-down list and select the desired job.
    • ETL Engine Config: This field specifies the ETL engine to use for publishing the job.
    • Keep Last N-Datasets: This field specifies the number of file-based linked data sets (FLDS) from this pipeline to retain on disk before deleting the oldest ones.
    • Load Threshold: This field specifies the percentage of the job that must complete successfully for the ingestion to be considered a success.
    • Job Execution Timeout: This field specifies the number of milliseconds to wait for job executions to run to completion.
    • Number Status Checks: This field specifies the number of times to perform a status check on the job run.
    • Wait Time Between Status Checks: This field specifies the number of milliseconds to wait between status checks.
    • Include Preceding Stages: This field indicates whether to run the all of the stages in the publishing process (generate, compile, deploy, and run) or whether to complete the run step only. Selecting Include Preceding Stages runs all of the steps,
    • Index: This field specifies a numeric value that represents the order in which this task should run in the workflow.
  6. Click Create to add the task to the workflow. For example, the image below shows a workflow with one Task.

You can repeat this process to add tasks that run additional jobs.

Related Topics