Data Engineer

• Develop and modify Spark jobs using Scala for various tasks
• Work with Kinesis streams using Spark Streaming to join and aggregate huge data sets and integrate with third-party data sources
• Build and ship new features to optimize services to suit evolving business requirements
• Work as an active and engaged owner of data infrastructures
• Gain a thorough understanding of all aspects of company business
• Maintain high coding standards and promote the same by providing code review and feedback to collaborators
• Diagnose and debug identified issues, problems, and errors across various systems
• Work closely with Data Science, Product, Research, and Engineering teams to define a development roadmap
• Help to gather and organize requirements when underspecified

Job Requirements:

• Bachelor s/Master s degree in Engineering, Computer Science (or equivalent experience)
• At least 3+ years of relevant experience as a data engineer
• Must be able to work with modern technologies like Scala
• Should possess a deep understanding of Spark development
• A thorough understanding of AWS-based development is essential
• Experience in ingesting, processing, storing, analyzing, and working in a big-data environment
• Familiarity with Athena, S3 data lake, and Parquet will be helpful
• Knowledge of Spark streaming with Kinesis will be beneficial
• Ability to optimize jobs for performance and development cost is required