Deutsche Bank Data Engineer Interview Questions and Answers

Q1. Load 60 GB of csv file and perform transformation.
Ans. 
Use distributed computing framework like Apache Spark to efficiently load and transform the large CSV file.
• Use Apache Spark to distribute the workload across multiple nodes for faster processing.

• Utilize Spark's DataFrame API to read the CSV file and apply transformations.

• Consider partitioning the data to optimize performance.

• Use Spark SQL for complex transformations and aggregations.

• Leverage in-memory processing capabi
Answered by AI
Add your answer

The first round which is online assessment for eliminating candidates was conducted consist of medium level aptitude questions there were 3 sub sec have duration of 20 15 15 minutes and questions 15 10 10 respectively and you cannot move to further section there is time constraints for each then followed by 15 minutes psychometric section having 50 questions that's all for screening round the total duration was 65 min.
Total 24 candidates were selected from this round out of 200 plus

Q1. What motivates to you for this job ?
Add your answer
Q2. What are the risks you have taken in previous role?
Add your answer
Q3. How do you handle the situation when your opinion differ from your senior members in your team?
Add your answer
Q4. Provide a approach to develop ETL pipeline where csv file is dropped in S3 and transform through airflow and load to snowflake.
Ans. 
Develop ETL pipeline using Airflow to process CSV files in S3 and load data into Snowflake.
• Set up an S3 sensor in Airflow to detect when a new CSV file is dropped in the specified bucket.

• Create a custom Python operator in Airflow to read the CSV file from S3, perform necessary transformations, and load data into Snowflake.

• Use SnowflakeHook in Airflow to establish connection with Snowflake and execute SQL queries to loa...read more
Answered by AI
Add your answer

Q1. Airflow SQL Python DWH
Add your answer

Q1. Airflow Pyspark SQL AWS
Add your answer

Q1. Relocation , tech stack
Add your answer

Q1. What is an index and what does it do
Ans. 
An index is a data structure that improves the speed of data retrieval operations in a database by allowing quick access to specific data.
• Indexes are used to quickly locate data without having to search every row in a database table.

• They are created on specific columns in a table to speed up queries that filter or sort by those columns.

• Examples of indexes include primary keys, unique keys, and non-unique indexes.
Answered by AI
Add your answer

Q1. Tell me a time you resolve conflict
Add your answer

Q1. More related towards Hadoop, Spark, scenario based competency questions
Add your answer

Q1. Code performance, optimization, batch processing
Add your answer