Data Analyst (5-8 yrs)

Job Brief :

We have an urgent requirement of a Data Analyst with 5 to 8 years of experience.

Key Responsibilities :

- Maintain and further develop the current ETL pipeline

- Design, develop, and maintain ETL processes to extract data from source systems, transform it into the required format, and load it into the target databases or data warehouses.

- Create and optimize data pipelines for real-time, batch, and near-real-time data processing.

- Write complex SQL queries, scripts, and stored procedures for data manipulation and transformation.

Data Integration :

- Integrate data from multiple sources (e.g., APIs, flat files, databases, cloud platforms) into centralized storage.

- Ensure that the data is properly transformed according to business logic and validation rules.

Data Quality & Validation :

- Implement data quality checks and validation routines to ensure the accuracy and reliability of the data.

- Detects and resolves data discrepancies, inconsistencies, or errors.

Data Optimization & Performance :

- Monitor ETL jobs for performance and efficiency. Optimize ETL processes to reduce processing time and resource usage.

- Troubleshoot and resolve performance issues related to ETL pipelines.

Technical Expertise :

- Our current pipeline is a combination of Python and SQL. Should have expertise in both.

- Experience with relational databases like SQL Server, PostgreSQL

- Experience with cloud platforms like AWS ( Athena, Kubernetes )

Preferred skills :

- Design, implement, and manage CI/CD pipelines

- Automate manual processes related to software build, testing, deployment, and configuration management.

- Deploy and manage applications on cloud platforms like AWS

- Build and maintain scalable, resilient cloud-based infrastructure using services like EC2, S3, Kubernetes, and Docker.

- Implement container orchestration solutions with Kubernetes, Docker