Big Data Engineer - P50

What you will do

• Build a data platform using big data processing and warehousing technologies.
• Build, maintain, and ideate on frameworks and tools required to ingest, store, process and serve data for global scale, data-intensive systems.
• Build data pipelines to publish reports and to power dashboards for ALM clients.
• Collaborate with cross-functional teams to determine key interfaces for your projects and build them with quality and agility.
• Do analysis and present key findings, insights and concepts to key influencers and leaders to help them make data driven decisions on business and product roadmap.
• Support Machine Learning engineers and Data Scientists with their dataset needs for training the ML or statistical models and evaluating them.
• Be aware of the latest open-source projects and technological breakthroughs in the data engineering space and explore them to build better and more effective solutions.
• Drive innovation through research and experimentation, encouraging an environment where new ideas can thrive.
• Mentor junior engineers on the team

• A bachelors degree in computer science or related fields.
• 8-12 years of experience as a server-side developer, with at least 4 years in Data Engineering, with a consistent track record of designing, implementing, and delivering large scale, high-quality solutions.
• Solid understanding and programming skill in languages such as Java or Python and frameworks and libraries based on them with a focus on data engineering.
• Strong expertise and familiarity with various Big Data technologies and frameworks including Delta Lake, Apache Hive, Datahub, Apache Spark, Apache Flink, Apache Storm, Trino, and AWS EMR.
• Strong familiarity with the concepts of RDBMS, Data Lake, Data Warehouse, Data Lakehouse, and Medallion Architecture.
• Knowledge of at least one workflow orchestration tools such as Airflow or Oozie.
• Experience working with ML engineers, Data Scientists, product and engineering leadership on their data and analysis needs.
• Experience working with the DevOps teams to drive operationally excellent infrastructure.
• Excellent problem-solving skills, with a consistent track record of delivering innovative solutions.
• Ability to thrive in a collaborative, inclusive, and diverse workplace, embracing different perspectives and ideas.
• Being comfortable with the ambiguity and having the ability to adapt to evolving priorities.
• Be a role model and a mentor for the team by doing diligent code and design reviews, by doing knowledge sharing sessions and showcasing sound technical decision making in day-to-day work.
• Being ahead of the curve by familiarizing yourself with the latest and the best in the technological landscape and turning them into effective solutions for our customers.