Data Engineer - Spark/Python (5-10 yrs)

Job Description :

- 5-10 years of meaningful experience in data engineering working on building ETL pipelines, big data tech like spark.

- Very good expertise in algorithms and data structures

- Excellent in at least one programming language: Python, Go, R

- Hands-on expertise big data technologies such as Apache Spark, Hadoop, message queues(Kafka), Apache Airflow, Azure ML, Elastic Search

- Databases: Hands-on expertise in data modeling, database schema definition, query optimization, usage of relevant data stores (blob storage, data warehouse, MongoDB, Timescale DB),

- Hands-on expertise working on analytics platforms such as Azure synapse or equivalent

- Good understanding of performance analysis and optimizations

- Hands-on experience in at least one cloud platform - Azure(preferred), AWS, or GCP

- B.E/B.Tech/M.E/M.Tech/M.S in Computer/Electronics Engineering or equivalent degrees with experience

Nice to have :

- Knowledge of data visualization tools such as Power BI, Tableau

- Working knowledge of the "Internet of Things" desired

- Knowledge of internet-scale cloud-based distributed systems

- Good understanding of MQTT, Web Sockets, Azure Service Bus, Apache Pulsar

- Good knowledge of Domain-Driven Design, Test Driven Development

- Experience developing in an Agile / SCRUM Environment

Soft skills :

- Self-driven, results-oriented, collaborative, comfortable with a fast-paced, always-on, highly ambiguous start-up like environment

- Good with rapid problem solving, a solid technical background, and a passion for continuous learning and improvement

- Good verbal and written communication skills