Lead I - Data Engineering

This role requires proficiency in data pipeline development including coding and testing data pipelines for ingesting wrangling transforming and joining data from various sources. Must be skilled in ETL tools such as Informatica Glue Databricks and DataProc with coding expertise in Python PySpark and SQL. Works independently and has a deep understanding of data warehousing solutions including Snowflake BigQuery Lakehouse and Delta Lake. Capable of calculating costs and understanding performance issues related to data solutions.

Outcomes:

Measures of Outcomes:

Outputs Expected:

Code Development:

Documentation:

Configuration:

Testing:

Domain Relevance:

Project Management:

Defect Management:

Estimation:

Knowledge Management:

Release Management:

Design Contribution:

Customer Interface:

Team Management:

Certifications:

Skill Examples:

Knowledge Examples:

Knowledge Examples

Seeking a Big Data Engineer with 4-6 years of experience to play a key role in building, managing, and evolving our big data pipelines. Youll leverage your data engineering expertise to not only onboard customers but also contribute to critical initiatives that advance our technology stack. This role offers the opportunity to directly improve product capabilities while expanding your skill set with cutting-edge technologies. Responsibilities:- Collaborate closely with Technical Leaders to devise and build the right solutions. Participate in design discussions and brainstorming sessions to select, integrate, and maintain tools and frameworks required to solve Big Data problems. Implement/Maintain systems to cleanse, process, and analyze large data sets using distributed processing tools like Airflow and Spark. Ability to learn quickly and adapt to new technologies Experience working with software and support teams in multiple regions Contribute insights and suggestions as part of our continuous improvement. Ability to work effectively in a fast-paced, collaborative environment. Required Qualifications 4 to 6 years of experience in Data engineering. Strong computer science background and knowledge of software and product development methodologies. In-depth understanding of the Big Data ecosystem including processing frameworks like Spark, Hadoop. and the file types they deal with. Experience with ETL and Data pipeline orchestration tools like Apache Airflow, dbt, etc. Excellent coding skills in Python, Java or Scala, SQL. Experience with Git and build tools like Gradle/Maven/SBT. Experience/Understanding with data warehouse platforms. Experience working on cloud platforms (like AWS,GCP,Azure) Strong understanding of object-oriented design, data structures, algorithms, profiling, and optimization. Excellent communication and collaboration skills