Lifesight Technologies - Data Engineer - Python/PySpark (3-5 yrs)

Job Description :

Lifesight is growing rapidly and seeking a strong Data Engineer to be a key member of the Data and Business Intelligence organization with a focus on deep data engineering projects. You will be joining as one of the few initial data engineers as part of the data platform team in our Bengaluru office. You will have an opportunity to help define our technical strategy and data engineering team culture in India.

You will design and build data platforms and services while managing our data infrastructure in cloud environments that fuel strategic business decisions across Lifesight products. A successful candidate will be a self-starter, who drives excellence, is ready to jump into a variety of big data technologies and frameworks, and can coordinate and collaborate with other engineers, as well as mentor other engineers in the team.

Responsibilities :

- Build quality data solutions and refine existing diverse datasets to simplified models encouraging self service.

- Build data pipelines that optimize data quality and are resilient to poor-quality data sources.

- Low-level systems debugging, performance measurement, and optimization on large production clusters.

- Maintain and support existing platforms and evolve to newer technology stacks and architectures.

Requirements :

- 3+ years of professional experience as a data or software engineer.

- Proficiency in Python and pyspark.

- Deep understanding of Apache Spark, Spark tuning, creating RDDs, and building data frames.

- Create Java/ Scala Spark jobs for data transformation and aggregation.

- Experience in big data technologies like HDFS, Hive, Kafka, Spark, Airflow, Presto, etc.

- Experience working with various file formats like Parquet, Avro, etc for large volumes of data

- Experience with one or more NoSQL databases.

- Experience with AWS, GCP.