Add office photos
i
eClerx
Compare
Filter interviews by
eClerx Data Engineer Interview Questions and Answers
eClerx Data Engineer Interview Experiences
1 interview found
Manjiri Manikrao
(2 Questions)
- Q1. Technical questions on AWS tools
- Q2. SQL query related question (Joins)
Data Engineer Jobs at eClerx
Interview questions from similar companies
I applied via Naukri.com and was interviewed in Oct 2024. There were 2 interview rounds.
(7 Questions)
- Q1. How do you optimize SQL queries?
- Ans.
Optimizing SQL queries involves using indexes, avoiding unnecessary joins, and optimizing the query structure.
Use indexes on columns frequently used in WHERE clauses
Avoid using SELECT * and only retrieve necessary columns
Optimize joins by using INNER JOIN instead of OUTER JOIN when possible
Use EXPLAIN to analyze query performance and make necessary adjustments
- Q2. How do you do performance optimization in Spark. Tell how you did it in you project.
- Ans.
Performance optimization in Spark involves tuning configurations, optimizing code, and utilizing caching.
Tune Spark configurations such as executor memory, number of executors, and shuffle partitions.
Optimize code by reducing unnecessary shuffles, using efficient transformations, and avoiding unnecessary data movements.
Utilize caching to store intermediate results in memory and avoid recomputation.
Example: In my projec...
- Q3. What is SparkContext and SparkSession?
- Ans.
SparkContext is the main entry point for Spark functionality, while SparkSession is the entry point for Spark SQL.
SparkContext is the entry point for low-level API functionality in Spark.
SparkSession is the entry point for Spark SQL functionality.
SparkContext is used to create RDDs (Resilient Distributed Datasets) in Spark.
SparkSession provides a unified entry point for reading data from various sources and performing
- Q4. When a spark job is submitted, what happens at backend. Explain the flow.
- Ans.
When a spark job is submitted, various steps are executed at the backend to process the job.
The job is submitted to the Spark driver program.
The driver program communicates with the cluster manager to request resources.
The cluster manager allocates resources (CPU, memory) to the job.
The driver program creates DAG (Directed Acyclic Graph) of the job stages and tasks.
Tasks are then scheduled and executed on worker nodes ...
- Q5. Calculate second highest salary using SQL as well as pyspark.
- Ans.
Calculate second highest salary using SQL and pyspark
Use SQL query with ORDER BY and LIMIT to get the second highest salary
In pyspark, use orderBy() and take() functions to achieve the same result
- Q6. 2 types of modes for Spark architecture ?
- Ans.
The two types of modes for Spark architecture are standalone mode and cluster mode.
Standalone mode: Spark runs on a single machine with a single JVM and is suitable for development and testing.
Cluster mode: Spark runs on a cluster of machines managed by a cluster manager like YARN or Mesos for production workloads.
- Q7. If you want very less latency - which is better standalone or client mode?
- Ans.
Client mode is better for very less latency due to direct communication with the cluster.
Client mode allows direct communication with the cluster, reducing latency.
Standalone mode requires an additional layer of communication, increasing latency.
Client mode is preferred for real-time applications where low latency is crucial.
(2 Questions)
- Q1. Scenario based. Write SQL and pyspark code for a dataset.
- Q2. If you have to find latest record based on latest timestamp in a table for a particular customer(table is having history) , how will you do it. Self join and nested query will be expensive. Optimized query...
Interview Preparation Tips
- SQL
- pyspark
- ETL
Skills evaluated in this interview
I applied via LinkedIn and was interviewed in Nov 2024. There was 1 interview round.
(2 Questions)
- Q1. How to Ensure Data loss in ETL pipeline
- Q2. Why to spin up Dataproc cluster when there is serverless batch job exists.
Data Engineer Interview Questions & Answers
Virtusa Consulting Services
Anonymous
posted on 11 Dec 2024
(2 Questions)
- Q1. What is PySpark, and can you explain its features and uses?
- Ans.
PySpark is a Python API for Apache Spark, used for big data processing and analytics.
PySpark is a Python API for Apache Spark, a fast and general-purpose cluster computing system.
It allows for easy integration with Python libraries and provides high-level APIs in Python.
PySpark can be used for processing large datasets, machine learning, real-time data streaming, and more.
It supports various data sources such as HDFS, ...
- Q2. What is the difference between PySpark and Python?
- Ans.
PySpark is a Python API for Apache Spark, while Python is a general-purpose programming language.
PySpark is specifically designed for big data processing using Spark, while Python is a versatile programming language used for various applications.
PySpark allows for distributed computing and parallel processing, while Python is primarily used for sequential programming.
PySpark provides libraries and tools for working wit...
I applied via Campus Placement and was interviewed in Oct 2024. There was 1 interview round.
(2 Questions)
- Q1. Write code in regular expression to remove the special characters
- Ans.
Use regular expression to remove special characters from a string
Use the regex pattern [^a-zA-Z0-9\s] to match any character that is not a letter, digit, or whitespace
Use the replace() function in your programming language to replace the matched special characters with an empty string
Example: input string 'Hello! How are you?' will become 'Hello How are you' after removing special characters
- Q2. Questions on resume
I applied via Naukri.com and was interviewed in Nov 2024. There was 1 interview round.
(2 Questions)
- Q1. Spark Architecture
- Q2. Cache vs persist, lazy evaluation
(2 Questions)
- Q1. What is Databricks, Internals of it, Optimization technique
- Ans.
Databricks is a unified analytics platform that provides a collaborative environment for data scientists, engineers, and analysts.
Databricks is built on top of Apache Spark, providing a unified platform for data engineering, data science, and business analytics.
Internals of Databricks include a cluster manager, job scheduler, and workspace for collaboration.
Optimization techniques in Databricks include query optimizati...
- Q2. SQL questions of joins,group by
(2 Questions)
- Q1. Scenario based azure data factory questions
- Q2. Project structres, pyspark dataframes related
Interview Preparation Tips
Skills evaluated in this interview
Data Engineer Interview Questions & Answers
NTT Data Information Processing Services
Anonymous
posted on 9 Dec 2024
Selected
I applied via LinkedIn and was interviewed in Jun 2024. There were 3 interview rounds.
General question around data engineering
(3 Questions)
- Q1. Different apache tech and tools
- Q2. What is snowflake
- Q3. Coding questions on python
(2 Questions)
- Q1. About location preference
- Q2. Behavioural questions
Interview Preparation Tips
(2 Questions)
- Q1. Python list comprehension
- Q2. SQL- product with second highest total price
- Ans.
Use SQL query to find the product with the second highest total price.
Use the ORDER BY clause to sort the products by total price in descending order
Use the LIMIT clause to select the second row after sorting
Skills evaluated in this interview
Selected
I applied via Job Portal and was interviewed in Aug 2024. There were 3 interview rounds.
Its mandatory test even for experience people
(1 Question)
- Q1. Related to technology
(1 Question)
- Q1. Very good discussion towards work culture, salary and all
eClerx Interview FAQs
Tell us how to improve this page.
eClerx Interviews By Designations
- eClerx Financial Analyst Interview Questions
- eClerx Analyst Interview Questions
- eClerx Senior Analyst Interview Questions
- eClerx Associate Process Manager Interview Questions
- eClerx Data Analyst Interview Questions
- eClerx Processing Manager Interview Questions
- eClerx Senior Financial Analyst Interview Questions
- eClerx KYC Analyst Interview Questions
- Show more
Interview Questions for Popular Designations
- Senior Data Engineer Interview Questions
- Big Data Engineer Interview Questions
- Software Engineer Interview Questions
- Senior Software Engineer Interview Questions
- Database Admin Interview Questions
- Data Analyst Interview Questions
- System Engineer Interview Questions
- Associate Software Engineer Interview Questions
- Show more
Data Engineer Interview Questions from Similar Companies
Fast track your campus placements
15-20 Yrs
₹ 40-55 LPA
|
Senior Analyst
5.4k
salaries
|  ₹2 L/yr - ₹8 L/yr |
|
Financial Analyst
4k
salaries
| ₹1.2 L/yr - ₹4.8 L/yr |
|
Analyst
4k
salaries
| ₹1 L/yr - ₹6.5 L/yr |
|
Associate Process Manager
2.4k
salaries
| ₹3.8 L/yr - ₹14.5 L/yr |
|
Processing Manager
1.7k
salaries
| ₹6 L/yr - ₹20 L/yr |
Genpact
WNS
TCS
Infosys
Calculate your in-hand salary
- Home >
- Interviews >
- eClerx Interview Questions >
- eClerx Data Engineer Interview Questions
