Data Engineer / Software Engineer

Job Description:

Position: Data Engineer

Shift Timings: 1 PM to 10 PM

Years of Experience: 5+ Years

Job Location: Anywhere

We are looking for a talented Data Engineer with expertise in AWS to join our data team. In this role, you will design, build, and maintain robust data pipelines and architectures to support our data-driven decision-making processes. You will work closely with data analysts and scientists to ensure data integrity and accessibility, optimizing data workflows in a cloud environment. A strong focus on automated testing is essential to ensure the reliability and performance of our solutions.

Key Responsibilities:

• Understands data architecture and overall business requirements around the data platform. Collects key data requirements from product/business groups on a regular basis.
• Accountable for data strategy on how the organization collects, processes, uses, governs, and stores data.
• Owns data engineering life cycle including research, proof of concept, architecture, development, testing, deployment, and maintenance of data platform.
• Data collection through pipelines from various sources, including databases, APIs, partners, and external providers.
• Standardize data by converting different data formats from sources to a standard format and cleanse it by removing inconsistencies.
• Design and develop ETL or ELT, cleansing, aggregation, enrichment, and making data usable for analytics.
• Make consumable data available on high-throughput data stores and build standard data consumption patterns (APIs, flat file, Database tables etc.) for users and systems to fetch data at scale.
• Add instrumentation across data pipelines for monitoring and alerting on internal problems before they result in user-visible outages.
• Build processes and diagnostic tools to troubleshoot, maintain, and optimize data processes and respond to customer and production incidents.
• Implements end-to-end automated CICD practices for building, scanning, packaging, testing, and deploying on a data platform, with the ability to continuously deliver secure solutions at scale.
• Adopt continuous learning of modern data engineering practices. Maintain industry standards through incremental adoption of new technology and best practices.
• Create and continuously maintain high-quality, up-to-date documentation of data processes, data dictionaries, and transformations to meet audit and compliance standards.

Minimum Qualifications and Education:

• 3+ years of experience working in AWS/Public Cloud-based infrastructure.
• Basic understanding and hands-on skills in Terraform/IaC.
• 3-5 years of strong knowledge and experience in Data pipelines through modern cloud computing. (Databricks, Glue, Lambda etc)
• 3-5 years of hands-on experience in Data storage and warehousing through cloud storage (S3, managed warehousing, Postgres, RDS, Redshift, etc)
• 3+ years of Hands-on experience in Data analysis, coding, and testing. (Python, Spark, Scala, Shell)
• 2+ years of experience creating data architecture diagrams, dictionaries, documentation, and test plans.
• Experience with basic data monitoring and alerting patterns and tools.

Interested can share your CV's to Swapna.mallipedi@aapc.com or Indiajobs@aapc.com.

All the very best for your role search