Infogain Senior Data Engineer Interview Questions and Answers

Q1. What is Databricks, and how does it differ from Azure Data Factory?
Add your answer

It was pair programming round where we need to attempt a couple of Spark Scenario along with the Interviewer. You will have a boiler plate code with some functionalities to be filled up. You will be assessed on writing clean and extensible code and test cases.

Q1. Questions will be based on the Project where they try to assess the depth of understading you have on your project, data quality checks, handling failure scenarios, reasoning behind choice of tech stack
Add your answer

Python and sql questions

Q1. About spark, data factory, Databricks etc
Add your answer
Q2. Need explain about these tools and technologies
Add your answer

Q1. Mainly they asked about Denodo Developer, Snowflake, SQL
Add your answer

Q1. Architecture of Denodo , big data engineering , business uses
Add your answer

Q1. Salary discussion
Add your answer

Q1. Architecture of spark
Ans. 
Spark is a distributed computing framework that provides in-memory processing capabilities for big data analytics.
• Spark has a master-slave architecture with a central coordinator called the Driver and distributed workers called Executors.

• It uses Resilient Distributed Datasets (RDDs) for fault-tolerant distributed data processing.

• Spark supports various data sources like HDFS, Cassandra, HBase, and S3 for input and outpu...read more
Answered by AI
Add your answer
Q2. SQL code for situations
Ans. 
SQL code for handling various situations in data analysis
• Use CASE statements for conditional logic

• Use COALESCE function to handle NULL values

• Use GROUP BY and HAVING clauses for aggregating data

• Use subqueries for complex filtering or calculations
Answered by AI
Add your answer

Q1. Manager round- describe about self and where do you see yourself in 5 years
Add your answer

Q1. Create spark dataframe
Ans. 
To create a Spark DataFrame, use the createDataFrame() method.
• Import the necessary libraries

• Create a list of tuples or a dictionary containing the data

• Create a schema for the DataFrame

• Use the createDataFrame() method to create the DataFrame
Answered by AI
Add your answer

Hackerearth, advanced sql queries on joins, string literals, sql conceptual MCQ