Lead II

You have a deep interest and passion for technology. You love writing and owning code and enjoy working with people who challenge you at every stage. You possess strong problem-solving, analytical, decision-making, and excellent communication skills. You are self-driven and motivated, thriving in a fast-paced, results-driven agile environment with varied responsibilities.

• Develop data pipelines, frameworks, applications, and APIs using industry best practices, adapting to new methodologies for increased flexibility and agility.
• Drive Proof of Concept (POC) and Proof of Technology (POT) evaluations.
• Build data systems and data quality monitoring processes to ensure the health of big data systems.
• Review KPIs, monitoring, and alarms for successful production system performance.
• Collaborate with stakeholders to understand business and reporting needs, proposing and developing creative data solutions to enhance insights, automate processes, and add value.
• Influence and educate the team by sharing your experience, ideas, and learning.
• Stay updated with the latest development tools, technology trends, patterns, and methodologies, sharing knowledge with key stakeholders.
• Publish and update technical architecture and user/process documentation.

• Education: Bachelors degree in Computer Science or a related technical field.
• Experience: 4+ years of relevant experience in Scala/Python (PySpark), distributed databases, and Kafka, with solid hands-on multi-threading and functional programming expertise.
• Technical Skills:
  • Strong understanding of CS fundamentals, data structures, algorithms, and problem-solving.
  • Hands-on experience in SQL and query optimization.
  • Experience building frameworks for data ingestion and consumption patterns.
  • Proficiency with orchestration tools like Airflow (preferred), Automic, or Autosys.
  • Hands-on experience in data processing and data manipulation.
  • Expertise with GCP cloud technologies, including GCS, DataProc, DPaaS, BigQuery, and Hive.
  • Experience working with streaming data via Kafka or structured streaming.
  • Exposure to Lambda Architecture.
  • Familiarity with visualization tools for data reporting, such as Tableau, Power BI, or Looker.
• Soft Skills:
  • Excellent communication and collaboration skills.
  • Strong appreciation for productivity and helping others work more efficiently.

Scala/Python (PySpark), Big Data Knowledge, Scripting (Shell/Python), GCP Data Tech Stack, Airflow, CI/CD, Kafka, Distributed Database, SQL/NoSQL .