KPMG India Data Engineer Interview Questions, Process, and Tips

Q1. Pyspark question were asked
Add your answer
Q2. Sql questions were asked
Add your answer

Q1. SCD type 2, snowflake pipe
Add your answer

Q1. It's question on collect list - it should be a straight question instead it was asked like comma separated and in a vague way.
Add your answer
Q2. Few questions on triggers, integration runtime
Add your answer

Q1. Python Dictionary operations
Add your answer
Q2. Write function to check if number is an Armstrong Number
Ans. 
Function to check if a number is an Armstrong Number
• An Armstrong Number is a number that is equal to the sum of its own digits raised to the power of the number of digits

• To check if a number is an Armstrong Number, we need to calculate the sum of each digit raised to the power of the number of digits

• If the sum is equal to the original number, then it is an Armstrong Number
Answered by AI
Add your answer
Q3. How to initiate Sparkcontext
Ans. 
To initiate Sparkcontext, create a SparkConf object and pass it to SparkContext constructor.
• Create a SparkConf object with app name and master URL

• Pass the SparkConf object to SparkContext constructor

• Example: conf = SparkConf().setAppName('myApp').setMaster('local[*]') sc = SparkContext(conf=conf)

• Stop SparkContext using sc.stop()
Answered by AI
Add your answer
Q4. RDDs vs DataFrames. Which is better and why
Ans. 
DataFrames are better than RDDs due to their optimized performance and ease of use.
• DataFrames are optimized for better performance than RDDs.

• DataFrames have a schema, making it easier to work with structured data.

• DataFrames support SQL queries and can be used with Spark SQL.

• RDDs are more low-level and require more manual optimization.

• RDDs are useful for unstructured data or when fine-grained control is needed.
Answered by AI
Add your answer

It was a MCQ test to intepret codes and its outcomes.

Q1. Given a dictionary, find out the greatest number for same key in Python.
Ans. 
Find the greatest number for same key in a Python dictionary.
• Use max() function with key parameter to find the maximum value for each key in the dictionary.

• Iterate through the dictionary and apply max() function on each key.

• If the dictionary is nested, use recursion to iterate through all the keys.
Answered by AI
Add your answer
Q2. Write Pyspark code to read csv file and show top 10 records.
Ans. 
Pyspark code to read csv file and show top 10 records.
• Import the necessary libraries

• Create a SparkSession

• Read the CSV file using the SparkSession

• Display the top 10 records using the show() method
Answered by AI
View 1 more answer
Q3. Write pyspark code to change column name, divide one column by another column.
Ans. 
Pyspark code to change column name and divide one column by another column.
• Use 'withColumnRenamed' method to change column name

• Use 'withColumn' method to divide one column by another column

• Example: df = df.withColumnRenamed('old_col_name', 'new_col_name').withColumn('new_col_name', df['col1']/df['col2'])
Answered by AI
Add your answer
Q4. What are the optimization techniques applied in pyspark code?
Ans. 
Optimization techniques in PySpark code include partitioning, caching, and using broadcast variables.
• Partitioning data based on key columns to optimize join operations

• Caching frequently accessed data in memory to avoid recomputation

• Using broadcast variables to efficiently share small data across nodes

• Using appropriate data types and avoiding unnecessary type conversions

• Avoiding shuffling of data by using appropriate tr...read more
Answered by AI
View 2 more answers

Q1. How do you handle changing schema from source. What are the common issues faced in hadoop and how did you resolve it?
Ans. 
Handling changing schema from source in Hadoop
• Use schema evolution techniques like Avro or Parquet to handle schema changes

• Implement a flexible ETL pipeline that can handle schema changes

• Use tools like Apache NiFi to dynamically adjust schema during ingestion

• Common issues include data loss, data corruption, and performance degradation

• Resolve issues by implementing proper testing, monitoring, and backup strategies
Answered by AI
Add your answer

Q1. Mostly it was salary negotiation
Add your answer

Q1. What is integration run time in adf
Ans. 
Integration run time in ADF is a compute infrastructure used to run activities in Azure Data Factory pipelines.
• Integration run time is a managed compute infrastructure in Azure Data Factory.

• It is used to run activities within pipelines, such as data movement or data transformation tasks.

• Integration run time can be auto-scaled based on the workload requirements.

• It supports various data integration scenarios, including b...read more
Answered by AI
Add your answer
Q2. How do you copy data from on-premise to azure cloud
Ans. 
Data can be copied from on-premise to Azure cloud using various methods like Azure Data Factory, Azure Storage Explorer, Azure Data Migration Service, etc.
• Use Azure Data Factory to create data pipelines for moving data from on-premise to Azure cloud

• Utilize Azure Storage Explorer to manually copy data from on-premise to Azure Blob Storage

• Leverage Azure Data Migration Service for migrating large volumes of data from on-p...read more
Answered by AI
Add your answer

Q1. How many members do you have in your family
Ans. 
I have 4 members in my family including my parents, my sibling, and myself.
• I have 2 parents

• I have 1 sibling

• I am included in the count
Answered by AI
Add your answer
Q2. Are ready to relocate
Ans. 
Yes, I am open to relocating for the right opportunity.
• I am willing to relocate for the right job opportunity

• I am open to exploring new locations and experiences

• I understand the importance of being flexible in the job market
Answered by AI
Add your answer