Upload Button Icon Add office photos
Engaged Employer

i

This company page is being actively managed by Egen Team. If you also belong to the team, you can get access from here

Egen Verified Tick

Compare button icon Compare button icon Compare

Filter interviews by

Clear (1)

Egen Data Engineer Interview Questions and Answers

Updated 12 Apr 2024

Egen Data Engineer Interview Experiences

1 interview found

Data Engineer Interview Questions & Answers

user image Anonymous

posted on 12 Apr 2024

Interview experience
5
Excellent
Difficulty level
Moderate
Process Duration
Less than 2 weeks
Result
Selected Selected

I applied via LinkedIn and was interviewed before Apr 2023. There were 3 interview rounds.

Data Engineer Jobs at Egen

View all

Interview questions from similar companies

Interview experience
4
Good
Difficulty level
Moderate
Process Duration
Less than 2 weeks
Result
No response

I applied via Naukri.com and was interviewed in Aug 2024. There were 2 interview rounds.

Round 1 - Technical 

(12 Questions)

  • Q1. Tell me about yourself and Project
  • Ans. 

    I am a Senior Data Engineer with experience in developing data pipelines and optimizing data storage for various projects.

    • Developed data pipelines using Apache Spark for real-time data processing

    • Optimized data storage using technologies like Hadoop and AWS S3

    • Worked on a project to analyze customer behavior and improve marketing strategies

  • Answered by AI
  • Q2. What was you day-to-day job in your project
  • Ans. 

    My day-to-day job in the project involved designing and implementing data pipelines, optimizing data workflows, and collaborating with cross-functional teams.

    • Designing and implementing data pipelines to extract, transform, and load data from various sources

    • Optimizing data workflows to improve efficiency and performance

    • Collaborating with cross-functional teams including data scientists, analysts, and business stakeholde...

  • Answered by AI
  • Q3. Spark Architecture
  • Q4. How DAG handle Fault tolerance?
  • Ans. 

    DAGs handle fault tolerance by rerunning failed tasks and maintaining task dependencies.

    • DAGs rerun failed tasks automatically to ensure completion.

    • DAGs maintain task dependencies to ensure proper sequencing.

    • DAGs can be configured to retry failed tasks a certain number of times before marking them as failed.

  • Answered by AI
  • Q5. What is shuffling? How to Handle Shuffling?
  • Ans. 

    Shuffling is the process of redistributing data across partitions in a distributed computing environment.

    • Shuffling is necessary when data needs to be grouped or aggregated across different partitions.

    • It can be handled efficiently by minimizing the amount of data being shuffled and optimizing the partitioning strategy.

    • Techniques like partitioning, combiners, and reducers can help reduce the amount of shuffling in MapRed

  • Answered by AI
  • Q6. What is the difference between repartition and Coelsce?
  • Ans. 

    Repartition increases or decreases the number of partitions in a DataFrame, while Coalesce only decreases the number of partitions.

    • Repartition can increase or decrease the number of partitions in a DataFrame, leading to a shuffle of data across the cluster.

    • Coalesce only decreases the number of partitions in a DataFrame without performing a full shuffle, making it more efficient than repartition.

    • Repartition is typically...

  • Answered by AI
  • Q7. How do you handle Incremental data?
  • Ans. 

    Incremental data is handled by identifying new data since the last update and merging it with existing data.

    • Identify new data since last update

    • Merge new data with existing data

    • Update data warehouse or database with incremental changes

  • Answered by AI
  • Q8. What is SCD ??
  • Ans. 

    SCD stands for Slowly Changing Dimension, a concept in data warehousing to track changes in data over time.

    • SCD is used to maintain historical data in a data warehouse.

    • There are three types of SCD - Type 1, Type 2, and Type 3.

    • Type 1 SCD overwrites old data with new data.

    • Type 2 SCD creates a new record for each change, preserving history.

    • Type 3 SCD maintains both old and new values in the same record.

    • SCD is important for...

  • Answered by AI
  • Q9. Scenerio based questions related to Spark ?
  • Q10. Two SQL Codes and Two Python codes like reverse a string ?
  • Ans. 

    Reverse a string using SQL and Python codes.

    • In SQL, use the REVERSE function to reverse a string.

    • In Python, use slicing with a step of -1 to reverse a string.

  • Answered by AI
  • Q11. Find top 5 countries with highest population in Spark and SQL
  • Ans. 

    Use Spark and SQL to find the top 5 countries with the highest population.

    • Use Spark to load the data and perform data processing.

    • Use SQL queries to group by country and sum the population.

    • Order the results in descending order and limit to top 5.

    • Example: SELECT country, SUM(population) AS total_population FROM table_name GROUP BY country ORDER BY total_population DESC LIMIT 5

  • Answered by AI
  • Q12. Using two tables find the different records for different joins
  • Ans. 

    To find different records for different joins using two tables

    • Use the SQL query to perform different joins like INNER JOIN, LEFT JOIN, RIGHT JOIN, and FULL JOIN

    • Identify the key columns in both tables to join on

    • Select the columns from both tables and use WHERE clause to filter out the different records

  • Answered by AI
Round 2 - One-on-one 

(7 Questions)

  • Q1. What is a catalyst optimiser? How it works?
  • Ans. 

    A catalyst optimizer is a query optimization tool used in Apache Spark to improve performance by generating an optimal query plan.

    • Catalyst optimizer is a rule-based query optimization framework in Apache Spark.

    • It leverages rules to transform the logical query plan into a more optimized physical plan.

    • The optimizer applies various optimization techniques like predicate pushdown, constant folding, and join reordering.

    • By o...

  • Answered by AI
  • Q2. Tell me about the optimization you used in your project.
  • Ans. 

    Used query optimization techniques to improve performance in database queries.

    • Utilized indexing to speed up search queries.

    • Implemented query caching to reduce redundant database calls.

    • Optimized SQL queries by restructuring joins and subqueries.

    • Utilized database partitioning to improve query performance.

    • Used query profiling tools to identify and optimize slow queries.

  • Answered by AI
  • Q3. Pyspark question related to merging two schemas?
  • Q4. What is the best approach to finding whether the data frame is empty or not?
  • Ans. 

    Use the len() function to check the length of the data frame.

    • Use len() function to get the number of rows in the data frame.

    • If the length is 0, then the data frame is empty.

    • Example: if len(df) == 0: print('Data frame is empty')

  • Answered by AI
  • Q5. Spark Architecture
  • Q6. How do you decide on cores and worker nodes?
  • Ans. 

    Cores and worker nodes are decided based on the workload requirements and scalability needs of the data processing system.

    • Consider the size and complexity of the data being processed

    • Evaluate the processing speed and memory requirements of the tasks

    • Take into account the parallelism and concurrency needed for efficient data processing

    • Monitor the system performance and adjust cores and worker nodes as needed

  • Answered by AI
  • Q7. What happens when we enforce schema ?
  • Ans. 

    Enforcing schema ensures that data conforms to a predefined structure and rules.

    • Ensures data integrity by validating incoming data against predefined schema

    • Helps in maintaining consistency and accuracy of data

    • Prevents data corruption and errors in data processing

    • Can lead to rejection of data that does not adhere to the schema

  • Answered by AI

Interview Preparation Tips

Topics to prepare for Persistent Systems Senior Data Engineer interview:
  • SQL
  • Pyspark
  • Python
  • Spark
  • Database
Interview preparation tips for other job seekers - Be prepared with Spark core concepts and SQL Coding

Skills evaluated in this interview

Interview experience
4
Good
Difficulty level
Moderate
Process Duration
Less than 2 weeks
Result
Selected Selected

I applied via Naukri.com and was interviewed before Jun 2023. There were 3 interview rounds.

Skills evaluated in this interview

Interview experience
2
Poor
Difficulty level
Moderate
Process Duration
2-4 weeks
Result
Selected Selected

I applied via Recruitment Consulltant and was interviewed before Nov 2022. There were 3 interview rounds.

Interview experience
3
Average
Difficulty level
-
Process Duration
-
Result
-
Interview experience
4
Good
Difficulty level
-
Process Duration
-
Result
-
Round 1 - Coding Test 

Python, spark, pyspark, sql coding questions

Round 2 - HR 

(2 Questions)

  • Q1. How much are you expecting?
  • Ans. 

    I am expecting a competitive salary based on my experience, skills, and the market rate for Senior Data Engineers.

    • I am looking for a salary that reflects my expertise in data engineering and aligns with industry standards.

    • I am open to discussing compensation based on the responsibilities and expectations of the role.

    • I have researched the average salary range for Senior Data Engineers in this location and industry.

    • I am ...

  • Answered by AI
  • Q2. What position do you need?
  • Ans. 

    I need the position of Senior Data Engineer.

    • Position of Senior Data Engineer

    • Experience in data engineering

    • Strong programming skills in languages like Python, SQL, etc.

    • Experience with big data technologies like Hadoop, Spark, etc.

  • Answered by AI
Interview experience
4
Good
Difficulty level
-
Process Duration
-
Result
-

Interview Preparation Tips

Interview preparation tips for other job seekers - SQL weightage is more, be thorough with that. Databricks project related question.
Interview experience
1
Bad
Difficulty level
-
Process Duration
-
Result
-

Interview Preparation Tips

Interview preparation tips for other job seekers - I interviewed at HCL in april, 2024. There are two technical rounds. Mostly questions were on spark, python and sql. I cleared both the round. HR asked for documents for offer release. I shared as it was requested.

Now there is NO offer nor HR is responding even after dropping 1000 emails. This is frustrating as I am not sure what happened with my application.

Interview Questionnaire 

3 Questions

  • Q1. Pair sum problem
  • Q2. Python basics
  • Q3. Pandas dataframe related basics question

Interview Preparation Tips

Interview preparation tips for other job seekers - Easy and brush up basics of pyrhon and practice some programming questions
Interview experience
1
Bad
Difficulty level
Moderate
Process Duration
Less than 2 weeks
Result
Selected Selected

I applied via Referral and was interviewed before Feb 2023. There was 1 interview round.

Contribute & help others!
anonymous
You can choose to be anonymous

Egen Interview FAQs

How many rounds are there in Egen Data Engineer interview?
Egen interview process usually has 3 rounds. The most common rounds in the Egen interview process are Technical and Coding Test.
How to prepare for Egen Data Engineer interview?
Go through your CV in detail and study all the technologies mentioned in your CV. Prepare at least two technologies or languages in depth if you are appearing for a technical interview at Egen. The most common topics and skills that interviewers at Egen expect are SQL, ETL, Data Engineering, Data Modeling and Data Pipeline.
What are the top questions asked in Egen Data Engineer interview?

Some of the top questions asked at the Egen Data Engineer interview -

  1. Some python related quest...read more
  2. Some SQL related quest...read more
  3. Some python problem to so...read more

Recently Viewed

SALARIES

Tarento Technologies

SALARIES

AST Corporation

JOBS

Contec Global

No Jobs

SALARIES

Planit Testing India

JOBS

Maropost

No Jobs

SALARIES

Motifworks

LIST OF COMPANIES

Nitor Infotech, an Ascendion company

Overview

JOBS

AST Corporation

No Jobs

SALARIES

AST Corporation

SALARIES

AST Corporation

Tell us how to improve this page.

Egen Data Engineer Interview Process

based on 1 interview

Interview experience

5
  
Excellent
View more
Egen Data Engineer Salary
based on 6 salaries
₹6 L/yr - ₹25.1 L/yr
33% more than the average Data Engineer Salary in India
View more details

Egen Data Engineer Reviews and Ratings

based on 2 reviews

4.0/5

Rating in categories

4.4

Skill development

4.0

Work-life balance

4.0

Salary

4.2

Job security

5.0

Company culture

4.0

Promotions

3.6

Work satisfaction

Explore 2 Reviews and Ratings
Talent Acquisition Specialist
12 salaries
unlock blur

₹2.9 L/yr - ₹9.6 L/yr

Associate Application Engineer
10 salaries
unlock blur

₹11 L/yr - ₹11 L/yr

Software Engineer
7 salaries
unlock blur

₹12.7 L/yr - ₹20 L/yr

Data Engineer
6 salaries
unlock blur

₹6 L/yr - ₹25.1 L/yr

Senior Software Engineer
5 salaries
unlock blur

₹10 L/yr - ₹33 L/yr

Explore more salaries
Compare Egen with

TCS

3.7
Compare

Infosys

3.6
Compare

Wipro

3.7
Compare

HCLTech

3.5
Compare
Did you find this page helpful?
Yes No
write
Share an Interview