Assistant/ Deputy Manager -Data Engineer

Role :-

• Design, implementation, and management of data platform, data pipelines to help fulfill data access and aggregation functionality for users in cloud environment.

Responsibilities :-

• Designing, constructing, developing, installing, testing and maintaining architectures of large-scale data processing systems.
• Experiment with various data models to optimize the search queries and come up with the proposals.
• Build data pipelines that capture, process and store organized data into large databases which would be ready for consumption.
• Expertly use Query programming languages such as SQL and functional programming languages such as Python, for mining and storing data into databases.
• Perform ETL tasks on the Big Data, using highly-scalable algorithms written in Scala / Java / PySpark so that in can utilize the power distributed computing.
• Have end- to- end responsibility for leading projects focused on extracting, merging, analyzing and managing large sets of data across multiple, disparate databases
• Be able to transform unstructured raw data in to formats suitable for statistical modeling, visualization and machine learning environments.
• Establish methodologies for quickly rolling out new data analysis capabilities for standalone data- driven products and service to support our associates.
• Be able to understand varies data sources and may be able to implement automated data quality and auditing through standard tools and custom code.
• Demonstrate a deep knowledge of and ability to operationalize, leading data technologies and best practices
• Be responsible for maintaining project plans, clean code, and well- written documentation
• Be able to work in teams and collaborate with stakeholders todefine requirements
• Make decisions independently on analytical problems & methods.
• Be able to identify and suggest novel areas of future work for themselves or the team
• Be able to work in a globally distributed team in an Agile/ Scrum approach

Competency Requirements :-

• Hands-on data engineering or other data- intensive development experience
• Experience processing large amounts of structured and unstructured data
• Advanced knowledge of programming languages such as Python /Java
• Experience building scalable data models and performing complex relational databases queries using SQL (Oracle, MySQL), etc.
• Experience with Big Data tools like Spark, Hadoop, Amazon EMR.
• Experience working on Cloud platforms like AWS,IBM,Azure including EMR, RDS, EC2, Vault, ECS, S3, EFS, etc.

Desired but not essential :-

• Experience in IoT time series data domain
• Exposure to automotive data

Educational Qualification :-

• B.Tech/M.Tech