TCS Data Engineer Interview Questions, Process, and Tips

Q1. What is an internal and external table in Hive
Ans. 
Internal tables store data within Hive's warehouse directory while external tables store data outside of it.
• Internal tables are managed by Hive and are deleted when the table is dropped

• External tables are not managed by Hive and data is not deleted when the table is dropped

• Internal tables are faster for querying as data is stored within Hive's warehouse directory

• External tables are useful for sharing data between diffe...read more
Answered by AI
View 2 more answers
Q2. What is partition and coalesce
Ans. 
Partitioning is dividing a large dataset into smaller, manageable parts. Coalescing is merging small partitions into larger ones.
• Partitioning is useful for parallel processing and optimizing query performance.

• Coalescing reduces the number of partitions and can improve query performance.

• In Spark, partitioning can be done based on a specific column or by specifying the number of partitions.

• Coalescing can be used to reduc...read more
Answered by AI
Add your answer
Q3. What is repartition and bucketing
Ans. 
Repartitioning and bucketing are techniques used in Apache Spark to optimize data processing.
• Repartitioning is the process of redistributing data across partitions to optimize parallelism and improve performance.

• Bucketing is a technique used to organize data into more manageable and efficient groups based on a specific column or set of columns.

• Repartitioning and bucketing can be used together to further optimize data p...read more
Answered by AI
Add your answer
Q4. What is the window function
Ans. 
Window function is a SQL function that performs a calculation across a set of rows that are related to the current row.
• Window functions are used to calculate running totals, moving averages, and other calculations that depend on the order of rows.

• They allow you to perform calculations on a subset of rows within a larger result set.

• Examples of window functions include ROW_NUMBER, RANK, DENSE_RANK, and NTILE.

• Window funct...read more
Answered by AI
Add your answer
Q5. What is an anonymous function
Ans. 
An anonymous function is a function without a name.
• Also known as lambda functions or closures

• Can be used as arguments to higher-order functions

• Can be defined inline without a separate declaration

• Example: lambda x: x**2 defines a function that squares its input
Answered by AI
Add your answer
Q6. What is view in SQL and dense and dense rank
Ans. 
View is a virtual table created from a SQL query. Dense rank assigns a unique rank to each row in a result set.
• A view is a saved SQL query that can be used as a table

• Dense rank assigns a unique rank to each row in a result set, with no gaps between the ranks

• Dense rank is used to rank rows based on a specific column or set of columns

• Example: SELECT * FROM my_view WHERE column_name = 'value'

• Example: SELECT column_name, D...read more
Answered by AI
Add your answer

Q1. Difference between rank and dense_rank, Left vs Left anti join
Ans. 
Rank assigns unique ranks to rows, while dense_rank handles ties by assigning the same rank to tied rows. Left join includes all rows from the left table and matching rows from the right table, while left anti join includes only rows from the left table that do not have a match in the right table.
• Rank assigns unique ranks to rows based on the specified order, while dense_rank handles ties by assigning the same rank to ...read more
Answered by AI
Add your answer
Q2. Python list comprehension, SQL query
Add your answer

Q1. Project related questions
Add your answer

Q1. Questions from pyspark - theoritical
Add your answer
Q2. Pyspark basic coding
Add your answer
Q3. SQL easy level coding question
Add your answer

Q1. 2 nd round was managerial. The interviewer asked about projects worked and assessed how I fit for the company
Add your answer

Focus of quantitative maths and aptitude a bit more

Q1. Some basic questions on your comfortable programming knowledge
Add your answer
Q2. Some behaviour oriented questions
Add your answer
Q3. Some role related questions
Add your answer

Q1. Some behaviour oriented
Add your answer
Q2. Salary discussion
Add your answer