Problem Statement
Is it possible to Trigger Azure Data Factory (ADF) pipeline from MSFT Data Factory, as there is no out-of-the-box Activity for Azure Data Factory?
Prerequisites
- Azure Data Factory
- MSFT Data Factory
- Organizational Account /Service Principal
- APIs used
Solution
Note. To trigger an ADF pipeline, the Organizational Account / Service Principal must have the below minimum permission on the ADF.
/ Microsoft.DataFactory/factories/pipelines/createrun/action
1. Create a pipeline in MSFT Fabric Data Factory and add the parameters below to the pipeline.
2. Drag and drop the Web activity with the below configurations.
Where create a new connection in web activity with organizational account authentication and appropriate base uri and token audience, click sign in to proceed.
Where
Relative
3. The Trigger ADF API call is asynchronous. Hence, we do not know whether the pipeline has actually been completed. The successful execution of the web activity doesn’t mean that the ADF pipeline was a success. To check the status of the Pipeline, add another web activity with the below config.
Use the same connection as created in #2.
4. We have to add a polling pattern to periodically check on the status of the trigger until it is complete. We start with an until activity. In the settings of the until loop, we set the expression so that the loop executes until the output of the above web activity is not equal to "Failed" or "Succeeded".
Expression
@or(
equals(activity('Get Pipeline Details').output.status, 'Failed'),
equals(activity('Get Pipeline Details').output.status, 'Succeeded')
)
Within Until Activity
Output