Azure Azure Data Factory Mapping Data Flow

Highlight

With Azure Data Factory Mapping Data Flow, you can create fast and scalable on-demand transformations by using visual user interface. In just minutes you can leverage power of Spark with not a single line of code written.

Intro

In this episode I give you introduction to what Mapping Data Flow for Data Factory and how can it solve your day to day ETL challenges. In a short demo I will consume data from blob storage, transform movie data, aggregate it and save multiple outputs back to blob storage.

Sample code and data: https://github.com/MarczakIO/Azure4Everyone/tree/master/AzureDataFactory-MappingDataFlow-Intro

Agenda

In today episode I will cover

1. What is Mapping Data Flow for Data Factory?
2. Creating data flow pipelines
3. Consuming data from blob storage
4. Transformation of data
5. Aggregations

Video

Next steps for you after watching the video

1. Azure Data Factory introduction video
   • https://youtu.be/EpDkxTHAhOs
2. Check mapping data flow documentation
   • https://docs.microsoft.com/en-us/azure/data-factory/concepts-data-flow-overview?WT.mc_id=AZ-MVP-5003556s
3. Helpful tips and samples
   • https://github.com/kromerm/adfdataflowdocs/blob/master/data-flow-expression-samples.md