Genpact Catastrophe Modeling Analyst Interview Questions and Answers

Q1. Project question like how connected to s3 bucket through pyspark,HDFS basic commands likhe copy etc ,how and where created hive table (if used in your project).
Add your answer
Q2. Share variables in pyspark like broadcast and accumulator..(i will suggest go through pyspark official documentation once)
Add your answer
Q3. Sql joins- , how to read file in pyspark, job scheduling means resource allocation within and across spark..(ans- can go through job scheduling of pyspark documentaion )
Add your answer

General question around data engineering

Q1. Different apache tech and tools
Add your answer
Q2. What is snowflake
Add your answer
Q3. Coding questions on python
Add your answer

Q1. About location preference
Add your answer
Q2. Behavioural questions
Add your answer

Q1. Python SQL question
Add your answer

Q1. More on Project side
Add your answer

Q1. Salary Discussion
Add your answer

Q1. Data engineer roles and resposibilities
Add your answer

Q1. Explain your project structure ?
Ans. 
My project structure follows a modular approach with separate folders for data ingestion, processing, and storage.
• Separate folders for data ingestion, processing, and storage

• Use of version control for tracking changes

• Documentation for each module and process

• Unit tests for data pipelines

• Logging and monitoring for tracking data flow
Answered by AI
Add your answer
Q2. What is mean by dag ?
Ans. 
DAG stands for Directed Acyclic Graph, a data structure used to represent dependencies between tasks in a workflow.
• DAG is a collection of nodes connected by directed edges, where each edge represents a dependency between tasks.

• It is acyclic, meaning there are no cycles or loops in the graph.

• DAGs are commonly used in data processing pipelines to ensure tasks are executed in the correct order.

• Example: In a DAG representi...read more
Answered by AI
Add your answer

Q1. Mostly on cloud tools
Add your answer

Q1. What is the difference between supervised and unsupervised learning?
Ans. 
Supervised learning uses labeled data to train the model, while unsupervised learning uses unlabeled data.
• Supervised learning requires a target variable to be predicted, while unsupervised learning does not.

• In supervised learning, the model learns from labeled training data, whereas in unsupervised learning, the model finds patterns in unlabeled data.

• Examples of supervised learning include regression and classification...read more
Answered by AI
Add your answer
Q2. Elaborate concepts of Object Oriented Programming in Python.
Ans. 
Object Oriented Programming in Python focuses on creating classes and objects to organize code and data.
• Python supports classes, objects, inheritance, polymorphism, and encapsulation.

• Classes are blueprints for creating objects, which are instances of classes.

• Inheritance allows a class to inherit attributes and methods from another class.

• Polymorphism enables objects to be treated as instances of their parent class.

• Encap...read more
Answered by AI
Add your answer
Q3. How to find delta between two tables in SQL?
Ans. 
To find delta between two tables in SQL, use the EXCEPT or MINUS operator.
• Use the EXCEPT operator in SQL to return rows from the first table that do not exist in the second table.

• Use the MINUS operator in SQL to return distinct rows from the first table that do not exist in the second table.
Answered by AI
View 1 more answer
Q4. Illustrate exception handling in python.
Ans. 
Exception handling in Python allows for graceful handling of errors and preventing program crashes.
• Use try-except blocks to catch and handle exceptions.

• Multiple except blocks can be used to handle different types of exceptions.

• Finally block can be used to execute code regardless of whether an exception was raised or not.

• Custom exceptions can be defined by creating a new class that inherits from the built-in Exception c
Answered by AI
Add your answer
Q5. Give an example of decorators in Python?
Ans. 
Decorators in Python are functions that modify the behavior of other functions.
• Decorators are defined using the @decorator_name syntax before the function definition.

• They can be used for logging, timing, authentication, etc.

• Example: @staticmethod decorator in Python makes a method static.
Answered by AI
Add your answer

Q1. Architecture of Spark
Ans. 
Spark is a distributed computing system that provides an interface for programming clusters with implicit data parallelism.
• Spark is built on the concept of Resilient Distributed Datasets (RDDs), which are fault-tolerant collections of objects.

• It supports various programming languages such as Scala, Java, Python, and R.

• Spark provides high-level APIs for distributed data processing, including transformations and actions.

• ...read more
Answered by AI
Add your answer
Q2. What is Lazy evaluation in spark
Ans. 
Lazy evaluation is a strategy used by Spark to delay the execution of transformations until an action is called.
• Lazy evaluation improves performance by optimizing the execution plan

• Transformations in Spark are not executed immediately, but rather recorded as a lineage graph

• Actions trigger the execution of the transformations and produce a result

• Lazy evaluation allows Spark to optimize the execution plan by combining an...read more
Answered by AI
Add your answer
Q3. Difference between Left join and inner join
Ans. 
Left join returns all records from the left table and the matching records from the right table.
• Inner join returns only the matching records from both tables.

• Left join includes all records from the left table, even if there are no matches in the right table.

• Inner join excludes the non-matching records from both tables.

• Left join is used to retrieve all records from one table and the matching records from another table.

• I...read more
Answered by AI
Add your answer