Get File details triggering the Azure Data Factory Pipeline / Synapse via Storage Event Trigger

Problem Statement

In the case of Storage Event trigger framework within Synapse / ADF pipelines, there can be scenarios wherein the pipeline can be triggered on arrival on any file within a blob path.

So in such scenarios, is it possible to get the details of the file ( file name, file path ) that led to the pipeline execution.

Prerequisites

  1. Synapse / Data Factory

Solution

The storage event trigger captures the folder path and file name of the blob into the properties @triggerBody().folderPath and @triggerBody().fileName. To use the values of these properties in a pipeline, you must map the properties to pipeline parameters.

  1. Add parameters to the Pipeline, say FileName and FilePath

Get File details triggering the Azure Data Factory Pipeline / Synapse

2. Create the Storage Event trigger, and a form will Pop up as below:

Get File details triggering the Azure Data Factory Pipeline / Synapse

Fill all necessary details and click on Continue.

3. Click Continue after validating the details in Data Preview mode, and a new page pops up asking for details for the parameter values.

Get File details triggering the Azure Data Factory Pipeline / Synapse

After mapping the properties to parameters, we can access the values captured by the trigger through the @pipeline().parameters.parameterName expression throughout the pipeline.

Result

Pipeline

Get File details triggering the Azure Data Factory Pipeline / Synapse

Input Blob

Get File details triggering the Azure Data Factory Pipeline / Synapse

Output

Get File details triggering the Azure Data Factory Pipeline / Synapse

Get File details triggering the Azure Data Factory Pipeline / Synapse


Similar Articles