Emorphis Technologies - Senior Data Engineer - AWS (7-14 yrs)

Primary Responsibilities :

- Data Pipeline Development : Independently design, build, and monitor robust data pipelines and Data marts that cater to various business needs. Ensure data is extracted, transformed, and loaded (ETL) efficiently.

- Data Extraction for Analysis/Model Training : Utilize Enterprise Data Warehouse (EDW) to query and extract relevant data sets for comprehensive analysis and machine learning model training purposes.

- Proficiency with SQL : Demonstrate expert-level skills in writing and optimizing complex SQL queries to manage and manipulate large datasets, ensuring high performance and accuracy.

- Programming Skills : Proficient in Python and PySpark for data processing tasks and automation. Capable of managing database operations in PostgreSQL.

- AWS Services Expertise : Hands-on experience with key AWS services such as AWS Glue for ETL tasks, AWS SageMaker for machine learning, AWS S3 for storage, and SNS for notifications. Leverage these tools to enhance data engineering solutions.

- Excel Proficiency : Skilled in utilizing Microsoft Excel for data analysis, reporting, and visualization.

Additional Skills (Nice to Have) :

- Dashboarding & Visualization : Capable of creating dynamic and insightful dashboards using H2O Wave app to visualize data trends, patterns, and insights effectively.

- Quick Analysis : Adept at performing rapid analysis to identify high-risk agents, pin codes, products, partners, or to find data distributions. Provide actionable insights based on these analyses.

- Data Deduplication : Conduct deduplication operations to identify and merge duplicate records using fuzzy logic or regular expressions, ensuring data integrity and consistency.

- Automation : Skilled in automating repetitive data engineering tasks using Python scripts, improving efficiency and reducing manual effort.