Simform Solutions - Data Engineer - ETL/PySpark (3-7 yrs)

We are looking for a Data Engineer who is passionate about managing large amounts of data, eager to learn and accept challenges, and who is committed to delivering exceptional results. If you are a team player, with a positive attitude and a desire to make a difference, then we want to hear from you.

Roles and Responsibilities :

- Experience working with AWS service for data ETL

- 3-7 years of Experience in developing data ingestion, data processing, and analytical pipelines for big data, relational databases, NoSQL, and data warehouse solutions

- Experience in designing data models effectively to manage optimal performance.

- Designing, developing, monitoring, and maintaining end-to-end data pipelines

- Should have extensive working knowledge of handling around 100GB of data in data ingestion, data processing, and pipeline creation.

- Experience with designing and implementing complex high throughput distributed batch and/or real-time solutions using one of AWS Glue, Apache Kafka, AWS Kinesis, Spark, Flink and Apache Airflow

- Extensive hands-on experience implementing data migration, data processing, ETL, or ELT pipeline development and monitoring.

- Strong programming skills in Python, SQL skills for data analysis and reporting

- Should know about any Data Warehouse like Redshift, Snowflake, and BigQuery

- Proven ability to work independently, mentor peers and meet tight deadlines

- Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases like Postgres, SQL Server, MySQL

- Should know about NoSQL Database like MongoDB, Cassandra, Neptune

- Experience in writing complex queries, functions, store procedures, execution plans, Performance Tuning and Query Optimization, Indexing, Partitioning, and Denormalization

- Should be well versed with migration strategies by manual and automated processes

- Excellent interpersonal, communication skills (both verbal and written), and ability to interact effectively

- Should attend client calls independently

- AWS certification would be a big plus