NeenOpal Intelligent Solutions Data Engineer Interview Questions and Answers

20 Questions in 20 min

Q1. To measure exactly 9 minutes using the 4-minute and 7-minute sand timers, follow these steps:

1. Start both the 4-minute and 7-minute timers at the same time.
2. When the 4-minute timer runs out, flip it immediately. At this point, 4 minutes have passed, and there are 3 minutes left on the 7-minute timer.
3. When the 7-minute timer runs out, 3 minutes have passed since you flipped the 4-minute timer, totaling 7 minutes. At this point, the 4-minute timer has been running for 3 minutes (1 minute left).
4. When the 4-minute timer runs out again (after 1 more minute), a total of 8 minutes has passed.
5. Finally, flip the 4-minute timer one last time. Wait for 1 more minute, which will give you a total of 9 minutes.

Q2. The SQL query provided will select all customers who do not have any orders. It retrieves customer IDs from the `Customers` table where the `customer_id` is not present in the `Orders` table.

Q3. The first SQL query retrieves the second oldest customer from the `Customers` table by selecting their `first_name` and `age`, ordering the results by age in descending order, and skipping the oldest customer with an offset of 1.

The second SQL query counts how many orders each customer has placed and selects customers who have placed more than one order, grouping the results by `customer_id` and including their `first_name`.

Q1. THey ask about your experience and projects youve worked on create table fact_product ( order_date varchar, order_id varchar, product_id varchar, quantity int, price int ); insert into fact_pr...read more
Add your answer

Q1. List vs tuple difference
Ans. 
List is mutable, tuple is immutable in Python.
• List can be modified after creation, tuple cannot be modified

• List uses square brackets [], tuple uses parentheses ()

• List is used for collections of items that may need to be changed, tuple is used for fixed collections
Answered by AI
Add your answer
Q2. How to removes duplicates in spark
Ans. 
Use dropDuplicates() function in Spark to remove duplicates from a DataFrame.
• Use dropDuplicates() function on the DataFrame to remove duplicates based on all columns

• Specify columns to check for duplicates by passing them as arguments to dropDuplicates() function

• Example: df.dropDuplicates() or df.dropDuplicates(['col1', 'col2'])
Answered by AI
Add your answer