Recro.io - Azure Data Engineer - SQL/Python (3-8 yrs)

Job Description :

Responsibilities :

- Build and optimize ETL pipelines using Azure Data Factory.

- Develop data models and Information Model objects(PO, Invoice, Vendor, etc. ).

- Write optimized SQL queries with advanced functions (CTEs, Window Functions).

- Implement Spark optimizations(partitioning, AQE, large datasets).

- Troubleshoot and fix pipeline and code issues.

- Handle large-scale data ingestion and transformations.

Requirements :

- SQL Expert : Window functions, CTEs, recursive queries.

- Python : Data structures, and optimizations for large datasets.

- Azure Data Factory : Pipeline orchestration, performance tuning, parameterization.

- Apache Spark : DAG, partitioning, job/stage/task hierarchy, AQE.

- SAP data experience.

- Experience in scenario-based problem-solving.