Prama AI - Senior Data Engineer - Python/PySpark (5-10 yrs)

Location : Ahmedabad |Hybrid

Benefits : 5 Day Working | Compatible Salary | Career Growth | Flexible working | Potential Remote Opportunity | Medical Insurance

Must-Have :

- 5+ years of experience with AWS tech stack (S3, Glue, Redshift, Athena, Lambda, CloudWatch, SQS, IAM roles, CloudTrail).

- 3-5 years of SQL, Python & Pyspark programming experience.

- Experience working with ETL Tools.

- Experience with CDC mechanisms for database sources.

- Experience building distributed architecture-based systems, especially handling large data volumes and real-time distribution.

- Initiative and problem-solving skills when working independently.

- Expertise in building high-performance, highly scalable, cloud-based applications.

- Experience with SQL and No-SQL databases.

- Good collaboration and communication skills, highly self-driven, and take ownership.

- Experience in Writing well-documented, Clean, and Effective codes is a must.

Good to have :

- AWS Cloud Certifications.

- Knowledge and experience in designing and developing RESTful services.

- 1-3 years of experience in DBT with Data Modeling, Airflow, MWAA, SQL, Jinja templating, and packages/macros to build robust, performant, and reliable data transformation and feature extraction pipelines.

- 1-2 years of experience in Airbyte building ingestion modules for streaming, batch.

- Good experience building Real-Time streaming data pipelines with Kafka, Kinesis etc.

- Familiarity with Big Data Design Patterns, modeling, and architecture.

- Working knowledge of DevOps methodologies, including designing CI/CD pipelines.

- Good understanding of Data warehousing & Data Lake solutions concepts.

Role & Responsibilities :

- Create and maintain scalable, robust AWS architecture.

- Develop API-based, CDC, batch, and real-time data pipelines for structured and unstructured datasets.

- Enable integration with third-party systems as needed.

- Ensure solutions are repeatable and scalable across the organization.

- Work with client teams to gather requirements, develop solutions, and deploy them.

- Provide robust solution documentation for a wide audience.

- Collaborate with data professionals to bring applications to life, meeting business needs.

- Prioritize data protection and cloud security in all deliverables.