Azure Data Factory Functions with your ERP system

Requirement

The client required multiple integrations between its ERP Systems with Oracle for their cloud ERP implementation, using Azure. These processes were manual and took up significant effort from the business.

Our Solution

There are two resources that play a vital role in this integration

Data Factory
Azure Functions

Data Factory

It extracts data from these ERP Systems and drops it into the Azure SQL database. A scheduled process triggers the pipelines and extracts the data from the database and sends it to Azure Functions. The Data Factory also takes care of logging the pipeline runs and email notifications in case of a failure.

Azure Functions

It receives data through an HTTP POST request, retrieves the credentials for Oracle from the key vault, and sends the data through a SOAP API request. If there is an error in Oracle, ‘Azure functions’ retrieves the error from there and sends it to the Data Factory as a response.

Advantages to our approach

‘Azure Functions’ is serverless that scales to meet demand
‘Azure Functions’ saves cost because it is billed by the exact amount of time used by the application, and runs only when required (triggered by the Data Factory)
‘Azure Functions’ can scale to handle huge surges of traffic
Using ‘Data Factory’, we can invoke pipelines with on-demand and trigger-based scheduling
We can visually monitor pipeline activity with logging and pipeline history, and track error sources