Senior Data Engineer

Role & responsibilities:

This role is accountable for running day to day operations of Data Platform in Azure / AWS Databricks NCS. Data Engineer is accountable for ongoing Development, Enhancement support and maintenance data availability and data quality, performance enhancement and stability of the system.

• Minimum 8 years of experience in data analytics field.
• Experience with Azure/AWS Databricks.
• Experience in building and optimizing data pipelines, architectures and data sets.
• Excellent experience in Scala or Python, PySpark and SQL.
• Ability to troubleshoot and optimize complex queries on the Spark platform.
• Knowledgeable on structured and unstructured data design / modelling, data access and data storage techniques.
• Expertise in designing and deploying data applications on cloud solutions, such as Azure or AWS.
• Hands on experience in performance tuning and optimizing code running in Databricks environment.
• Demonstrated analytical and problem-solving skills particularly those that apply to a big data environment.
• Data Engineering expertise in Big Data Management using Cloudera, Python, Hive, Scala, Data Warehouse, Data Lake, AWS, Azure