Data engineer

We are looking for an experienced data engineer to join our team. You will use various methods to transform raw data into useful data systems.
Should have strong analytical skills and the ability to combine data from different sources.
Data engineer Mandatory Skills Java/Python, Pyspark, AWS Glue, Airflow, Lambda, Athena, S3, Kinesis, Unit Testing using Pytest.
Responsibilities
Analyze and organize raw data.
Build data systems and pipelines.
Evaluate business needs and objectives.
Interpret trends and patterns.
Conduct complex data analysis and report on results.
Prepare data for prescriptive and predictive modelling.
Build algorithms and prototypes.
Combine raw information from different sources.
Explore ways to enhance data quality and reliability.
Identify opportunities for data acquisition.
Develop analytical tools and programs.
Collaborate with data scientists and architects on several projects.

At least minimum 8 years of experience as a data engineer or in a similar role
Strong organizational, problem-solving, and critical thinking skills; Strong documentation skills
Knowledge of programming languages Java or Python
Cluster Computing frameworks Proficiency in Spark/Pyspark
AWS Data Services Proficiency in Lake formation, Glue ETL (or) EMR, S3, Glue Catalog, Athena, Kinesis (or) MSK, Airflow (or) Lambda + Step Functions + Event Bridge
AWS Data Security Good Understanding of security concepts such as Lake formation, IAM, Service roles, Encryption, KMS, Secrets Manager
Technical expertise with data models, data mining, and segmentation techniques
Hands-on experience with SQL database design
Great numerical and analytical skills
Data engineering certification (e.g IBM Certified Data Engineer) is a plus