NTT Data Senior Data Engineer Interview Questions and Answers

Q1. All questions based on databricks and some pyspark , python , SQL only.
Add your answer
Q2. Learn windows function implementation in databricks note book.
Add your answer

Q1. This round about your salary discussion around.
Add your answer

Q1. Windows function-related questions.
Add your answer
Q2. Join Related Questions.
Add your answer

Q1. Join related Questions
Add your answer
Q2. Subqueries related queations.
Add your answer

Python and sql tasks

Q1. Project related questions
Add your answer
Q2. Coding questions on pyspark windows
Add your answer

Q1. Managerial discussions, mostly in and around the previous projects
Add your answer

Q1. Coding in pyspark
Ans. 
Pyspark is a Python library for big data processing using Spark framework.
• Pyspark is used for processing large datasets in parallel.

• It provides APIs for data manipulation, querying, and analysis.

• Example: Using pyspark to read a CSV file and perform data transformations.
Answered by AI
Add your answer
Q2. Databricks optimisation technique
Ans. 
Databricks optimisation techniques improve performance and efficiency of data processing on the Databricks platform.
• Use cluster sizing and autoscaling to optimize resource allocation based on workload

• Leverage Databricks Delta for optimized data storage and processing

• Utilize caching and persisting data to reduce computation time

• Optimize queries by using appropriate indexing and partitioning strategies
Answered by AI
Add your answer
Q3. Aqe details in databricks
Ans. 
Databricks is a unified data analytics platform that provides a collaborative environment for data engineers.
• Databricks is built on top of Apache Spark and provides a workspace for data engineering tasks.

• It allows for easy integration with various data sources and tools for data processing.

• Databricks provides features like notebooks, clusters, and libraries for efficient data engineering workflows.
Answered by AI
Add your answer

Q1. More questions on spark and java
Add your answer
Q2. More questions on sql
Add your answer

Q1. Over all project architecture
Add your answer
Q2. Sample data and its transformations
Ans. 
Sample data and its transformations
• Sample data can be in the form of CSV, JSON, or database tables

• Transformations include cleaning, filtering, aggregating, and joining data

• Examples: converting date formats, removing duplicates, calculating averages
Answered by AI
Add your answer

Q1. Why are u leaving current job
Ans. 
Seeking new challenges and opportunities for growth in a more dynamic environment.
• Looking for new challenges and opportunities for growth

• Seeking a more dynamic work environment

• Interested in expanding skill set and knowledge

• Want to work on more innovative projects
Answered by AI
Add your answer

Q1. What is scd type 2?
Ans. 
SCD type 2 is a method used in data warehousing to track historical changes by creating a new record for each change.
• SCD type 2 stands for Slowly Changing Dimension type 2

• It involves creating a new record in the dimension table whenever there is a change in the data

• The old record is marked as inactive and the new record is marked as current

• It allows for historical tracking of changes in data over time

• Example: If a cust...read more
Answered by AI
Add your answer
Q2. Pyspark question read CSV from folder and add column in each csv file and write it to different location.
Add your answer

Q1. Explain about yourself.
Ans. 
I am a Senior Data Engineer with 5+ years of experience in designing and implementing data pipelines for large-scale projects.
• Experienced in ETL processes and data warehousing

• Proficient in programming languages like Python, SQL, and Java

• Skilled in working with big data technologies such as Hadoop, Spark, and Kafka

• Strong understanding of data modeling and database management

• Excellent problem-solving and communication sk
Answered by AI
Add your answer
Q2. Explain about your current project.
Ans. 
Developing a real-time data processing system for analyzing customer behavior on e-commerce platform.
• Utilizing Apache Kafka for real-time data streaming

• Implementing Spark for data processing and analysis

• Creating machine learning models for customer segmentation

• Integrating with Elasticsearch for data indexing and search functionality
Answered by AI
Add your answer
Q3. Big data concept
Add your answer
Q4. Spark related questions
Add your answer

Spark Optimization, Transformation, DLT, DL, Data Governance
Python
SQL