Get Started with Plans
Plans is an orchestration tool. Orchestration is a set of functionalities that supports the scheduled execution of task sequences in the Alteryx Analytics Cloud. These tasks could be external processes, data transformation jobs, HTTP requests, and more.
The sequence of tasks can be gated based on success or failure criteria and can be restarted if needed where the plan failed.
Overview
In many scenarios, data pipelines have dependencies in them. Data sourced from one dataset must be cleaned and delivered for use in another data pipeline. When a pipeline is built, it needs to be scheduled for periodic execution. Ideally, when the execution completes, downstream stakeholders must be informed that the pipeline has succeeded or failed in execution.
In AAC, plans provide the mechanism for building these robust data pipelines within the application, in which sequences of tasks can be executed to deliver more diverse datasets. As a result of the success or failure of these executions, AAC can deliver messages via email, Slack or any other platform with public API.
Terms
Term | Description |
|---|---|
plan | A plan is a sequence of tasks that are executed from the platform or on assets to which you have access. To orchestrate tasks, you build a plan. A plan can be scheduled for execution, triggered manually, or invoked via API. |
trigger | A task is executed based on a trigger. A trigger is a condition under which a task is executed. In many cases, the trigger for a task is based on the schedule for the plan. |
task | A task is a unit of execution in the platform. |
snapshot | A snapshot of the plan is captured, and the plan is executed against this snapshot. For more information on snapshots, see "Plan execution" below. |
Tip
As one of your plan tasks, you can configure an HTTP message, which allows you to trigger endpoints in AAC itself. So, you can configure your plan executions to trigger a wide range of activities within the product itself.
Example
The following example shows how to create a plan that executes the following tasks:
Execute the task that performs the initial cleaning of your data. The results data is saved to a known location.
Execute the task that transforms the cleaned data for downstream uses.
Based on the outcome of step 2:
On success: Send a success message to a Slack channel.
On failure: Send a failure message to a Slack channel.
In Plan View, this plan might look like the following:
Figure: A plan with a sequence of tasks
After creating the above plan, you can perform test runs.
If the tests are successful, you can schedule the plan for periodic execution.
Task Types
The following tasks are available:
Workflow task: Execute one of your Designer Cloud workflows as part of a plan that you create in AAC. Go to Plan View for Workflow Tasks.
Desktop Workflow task: Orchestrate the execution of your Designer Desktop workflows that were uploaded to AAC. Go to Plan View for Desktop Workflow Tasks
Report task: Create and export a report from the data you've prepped and blended in AAC. Go to Plan View for Report Tasks.
Flow task: Generate all of the defined output objects for a flow. Go to Plan View for Flow Tasks.
ML Prediction task: Make predictions on new data with your Machine Learning project. Go to Plan View for ML Prediction Tasks.
ML Upload task: Upload new training data to your Machine Learning project. Go to Plan View for ML Upload Tasks.
HTTP task: Execute a task over HTTP protocol. Go to Plan View for HTTP Tasks.
Slack task: Send a message from AAC to a specified Slack channel. Go to Plan View for Slack Tasks.
Location Intelligence task: Orchestrate your Location Intelligence projects with other tasks. For more information, go to Plan View for Location Intelligence Tasks.
Data Exchange Between Tasks
You can interact with tasks in Plans by directly using the output of an upstream workflow task as the input for a downstream task, enabling seamless overrides.
Supported Exchanges:
Designer Cloud → Machine Learning Predict
HTTP → Workflow task
Workflow task → HTTP
Workflow task → Workflow task
The schema of the override output should be the same as the input you are overriding.
Limitation:
For the Workflow task to Workflow task exchange, the upstream output must be in CSV format, while the downstream input can be in any format.
Merging Tasks
To create additional task execution logic, connect the output of multiple upstream tasks to the input of a single task. The resulting task of a merge waits until all upstream connected tasks complete before executing. The merge also passes relevant workflow outputs or task metadata from all merged upstream tasks. For example, if you connect 2 Workflow tasks to a Slack task, you can use the metadata (for example, workflow duration or end time) from both workflows in your Slack message.
For more information on how to create and manage your plans, go to Plan Tasks.
Overview of Orchestration
Orchestration is a set of functionalities that supports the scheduled execution of task sequences in the Alteryx Analytics Cloud. These tasks could be external processes, data transformation jobs, HTTP requests, and more.
In the following sections, you can review short summaries of specific features and explore more detailed information on them.
Limitations
You cannot specify parameter overrides to be applied to plans specifically.
Plans inherit parameter values from the objects referenced in the plan's tasks.
If overrides are applied to parameters inside of assets in a plan, those overrides are passed to the plan at the time of task execution.
Limited Parameterization Support
Plans only supports parameterization for Trifacta Classic Flows at this time.
Basic Task
You create a plan and schedule it using the following basic tasks.
Create the plan. A plan is a container for definition of the tasks, triggers, and other objects. See Manage Plans.
In Plan View, you specify the objects that are part of your plan. See Build Plans.
Schedule: The schedule defines the set of triggers that queue the plan for execution.
Trigger: A trigger defines the schedule and frequency at which the plan is executed. A plan can have multiple triggers (e.g. monthly versus weekly executions).
Task(s): Next, you specify the tasks that are executed in order.
As needed, you can apply override values to any flow parameters. These overrides are applied during a plan run. For more information, see Manage Plans Parameters.
Limited Parameterization Support
Plans only supports parameterization for Trifacta Classic Flows at this time.
To test:
Select Run now.
To track progress, select the Runs link.
In the Run Details page, you can track the progress.
The first task is executed and completes, before the second task is started.
Individual tasks are executed as separate jobs, which you can track through the Job History page. See Job History Page.
When the plan has completed, you can verify the results through the Job details page. See Job Details Page.
If you are satisfied with the plan definition and your test run, the plan will execute according to the scheduled trigger.
Plan Scheduling
Through the Plan View page, you can configure the scheduled executions of the plan. Plan schedules are defined using triggers.
These schedules are independent of schedules for other asset types.
You cannot create schedules for individual tasks.
For more information, go to Schedules.
Plan Execution
When a plan is triggered for execution, a snapshot of the plan is taken. This snapshot is used to execute the plan. Tasks are executed in the sequence listed in Plan View.
Important notes:
At the flow level, you can define webhooks and email notifications that are triggered based on the successful generation of outputs. When you execute a plan containing an output with one of these messages, the message is triggered and delivered to stakeholders.
Note
Webhook messages and email notifications cannot be directly triggered based on a plan's execution. However, you can create HTTP-based tasks to send messages based on a plan task's execution.
Tip
When a flow email notification is triggered through a plan, the internal identifier for the plan is included in the email.
Enable
Enable the following setting:
Plans feature
Plan sharing, import, and export must also be enabled.
Logging
For more information on debugging plans, see Diagnose Failed Plan Runs.