Senior Data Engineer - ETL/Python (8-10 yrs)

Job Description :

We are seeking a skilled and experienced Data Engineer with expertise in Python, data warehousing, ETL (Extract, Transform, Load), and SQL. As a Data Engineer, you will play a crucial role in designing, developing, and maintaining our data infrastructure to support the organization's data needs. You will collaborate with cross-functional teams to ensure the effective collection, storage, and retrieval of data for analytics and reporting purposes.

Responsibilities :

1 Work with product managers and data analysts to build data products that power our business.

2 Build and maintain data analytics reports, ETL /ELT pipelines, data lakes, and streaming solutions.

3 Analyze new data sources and work with stakeholders to understand the impact of integrating new data into existing pipelines and models

4 Develop & maintain critical data pipelines, using tools such as Apache Airflow, to ensure highly accurate and reliable business reporting

5 Perform ad-hoc analysis and provide insights to stakeholders within the organization such as Marketing, Growth & Sales teams to drive prioritization.

6 Create data visualizations and analytic reports using various tools such as Tableau

7 Optimize database queries to improve and maintain performance on very large data sets.

8 Work closely with the business and analysis team for the deliverables.

Requirements :

1. 8+ years of relevant experience. Masters or Bachelors degree in CS/ML/AI or equivalent discipline.

2. Expertise with coding with Pyspark, Pandas and visualizations with Tableau.

3. Scheduling/orchestrating pipelines using Databricks and Airflow.

4. Expertise with data querying (Snowflake, SQL, Redshift, Spark) and analyzing billions of rows of data points.

5. Familiarity with SQL, especially within cloud-based data warehouses like Snowflake, Google Big Query, and Amazon Redshift

6. Strong understanding of relational DB such as MySQL/PostgreSQL, document-based storage systems such as MongoDB or CouchDB, key-value stores (Memcached, Redis), Column-oriented stores (HBase, Cassandra), graph-oriented stores; their distinct advantages, disadvantages, and trade-offs.

7. Experience building machine learning, statistical, and analytical model, and tuning parameters.

8. Experience designing experiments and extracting insights.

9. Knowledge of various ETL/ ELT techniques and frameworks

If you are a motivated end to end Data Analyst with a passion for building robust and scalable data solutions, we invite you to apply and contribute to our dynamic and innovative team.