IPG Mediabrands Data Engineer Interview Questions and Answers

Q1. Difference between union and union all in SQL?
Ans. 
Union combines and removes duplicates, Union All combines without removing duplicates.
• Union combines result sets and removes duplicates

• Union All combines result sets without removing duplicates

• Union is slower than Union All as it involves removing duplicates

• Example: SELECT column1 FROM table1 UNION SELECT column1 FROM table2;

• Example: SELECT column1 FROM table1 UNION ALL SELECT column1 FROM table2;
Answered by AI
Add your answer
Q2. How to show top 5 in pandas?
Ans. 
To show top 5 in pandas, use the nlargest() function.
• Use the nlargest() function with the 'n' parameter set to 5 to get the top 5 values in a pandas DataFrame.

• For example: df['column_name'].nlargest(5) will return the top 5 values in the specified column.
Answered by AI
Add your answer

Q1. If we have 3 numerical columns, which chart is a better representation these column?
Ans. 
A scatter plot is a better representation for 3 numerical columns.
• Use a scatter plot to show the relationship between the numerical columns.

• Scatter plots are effective for visualizing correlations and patterns in data.

• Each point on the plot represents a data point with values from all 3 columns.
Answered by AI
Add your answer

Q1. Talk about yourself
Add your answer
Q2. What is your professional experience
Add your answer

Simple SQL questions

They gave me case study in interview based on that I had provide the solution.

Q1. They asked question regarding SQL, statistics and some project related questions
Add your answer

Q1. Tell me about Projects done during course
Add your answer

Q1. Dot net related stuffs
Add your answer
Q2. Dependency injection
Add your answer

Built an app using dot net and angular

Q1. SQL and tableau
Add your answer

Online test on sql join sub query

Q1. Final tech round
Add your answer

Q1. Spark Context vs Spark Session
Ans. 
Spark Context is the entry point to any Spark functionality while Spark Session is a unified entry point for Spark 2.0+.
• Spark Context is the old entry point to Spark functionality.

• Spark Session is a unified entry point for Spark 2.0+.

• Spark Context is used to create RDDs, accumulators and broadcast variables.

• Spark Session is used to create DataFrames, execute SQL queries and read data from external sources.
Answered by AI
Add your answer
Q2. Repartitioning vs Coalesce
Ans. 
Repartitioning increases partitions while Coalesce reduces partitions.
• Repartitioning shuffles data across the cluster and can be used to increase parallelism.

• Coalesce merges partitions without shuffling data and can be used to reduce overhead.

• Repartitioning is expensive and should be used sparingly.

• Coalesce is faster but may not be as effective as repartitioning in increasing parallelism.

• Both can be used to optimize da
Answered by AI
Add your answer
Q3. Sql query to find Second Highest Salary
Ans. 
Sql query to find Second Highest Salary
• Use ORDER BY and LIMIT to select the second highest salary

• Use subquery to select the maximum salary and exclude it from the result set

• Handle cases where there are ties for the highest salary
Answered by AI
Add your answer
Q4. Internal Working of Spark
Ans. 
Spark is a distributed computing engine that processes large datasets in parallel across a cluster of computers.
• Spark uses a master-slave architecture with a driver program that coordinates tasks across worker nodes.

• Data is stored in Resilient Distributed Datasets (RDDs) that can be cached in memory for faster processing.

• Spark supports multiple programming languages including Java, Scala, and Python.

• Spark can be used f...read more
Answered by AI
Add your answer
Q5. What is Braodcast Join
Ans. 
Broadcast Join is a technique used in distributed computing to optimize join operations.
• Broadcast Join is used when one table is small enough to fit in memory of all nodes in a cluster.

• The smaller table is broadcasted to all nodes in the cluster, reducing network traffic.

• Broadcast Join is faster than other join techniques when used appropriately.

• Example: Joining a small reference table with a large fact table in a data
Answered by AI
Add your answer

Q1. Basic concepts of core java
Add your answer
Q2. DSA , collections , map, set, list
Add your answer

Q1. Salary discussion
Add your answer
Q2. Roles and responsibilities
Add your answer