AWS Data Engineer

Requirements: Proficient in AWS (Glue, Lambda, S3, DynamoDB, etc.), relational/NoSQL DBs (MySQL, MongoDB), data pipelines (Airflow), stream-processing (Spark, Flink), and scripting languages (Python, PySpark, Scala). Skilled in EC2, EMR, RDS, Redshift.

Responsibilities:

Data Ingestion:

Designing and implementing data ingestion pipelines using tools like AWS Glue, AWS Kinesis, or AWS S3.
Extracting data from various sources, including databases, APIs, and files.
Transforming and cleaning data to ensure consistency and quality.

Data Storage:
Selecting appropriate storage solutions, such as Amazon S3, Amazon Redshift, or Amazon DynamoDB.
Optimizing data storage for performance and cost-effectiveness.
Implementing data retention policies and backup strategies.

Data Processing:
Using tools like AWS Glue, AWS EMR, or AWS Lambda to process and transform data.
Applying data engineering techniques, such as ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform).
Building data pipelines to automate data processing tasks.

Data Warehousing and Analytics:
Designing and building data warehouses and data marts.
Creating data models and schemas to support business intelligence and analytics.
Optimizing query performance and data access.

Data Security and Governance:
Implementing security measures to protect sensitive data.
Enforcing data governance policies and standards.
Monitoring data quality and integrity.

Location - Remote, Hyderabad,Ahmedabad,pune,chennai,kolkata.