Azure Data Factory is a data-integration service that allows user to create data-driven workflows on the cloud for automating data movement and data transformation. Data Factory is an ETL (Extract Transform Load) tool on Cloud. Data Factory helps to deliver extraction, transformation, and loading processes within the cloud. It helps user to create a transform process on the structured or unstructured raw data that is why users can analyze the data and use processed data to provide actionable business decisions. Raw and unorganized big data is often stored in relational, non-relational, and other storage systems. However, raw data doesn't have the proper context to provide meaningful intuition to analysts, data scientists. So, big data requires a service that can refine these enormous stores of raw data into actionable business insights. And here Azure Data Factory that managed service which built for these complex hybrid extract-transform-load (ETL), extract-load-transform (ELT), and projects for data integration.
ETL process follows four steps:
- Connect & Collect
- Transform
- Publish
- Monitor
Components of Data Factory:
- Pipelines
- Activities
- Datasets
- Linked Services
- Data Flows
- Integration Runtimes
Advantages:
- User can create and schedule data-driven workflows (called pipelines).
- User can build complex ETL (extract-transform-load) processes that transform data visually.
- User can publish transformed data to data stores.
- Raw data can be transformed into meaningful data stores and data lakes for accurate business decisions.
