BNY Senior Data Management Analyst Interview Questions, Process, and Tips

Q1. What is lazy evaluation in spark.
Ans. 
Lazy evaluation in Spark delays the execution of transformations until an action is called.
• Lazy evaluation allows Spark to optimize the execution plan by combining multiple transformations into a single stage.

• Transformations are not executed immediately, but are stored as a directed acyclic graph (DAG) of operations.

• Actions trigger the execution of the DAG and produce results.

• Example: map() and filter() are transformat...read more
Answered by AI
Add your answer
Q2. What is mapreduce
Ans. 
MapReduce is a programming model and processing technique for parallel and distributed computing.
• MapReduce is used to process large datasets in parallel across a distributed cluster of computers.

• It consists of two main functions - Map function for processing key/value pairs and Reduce function for aggregating the results.

• Popularly used in big data processing frameworks like Hadoop for tasks like data sorting, searching...read more
Answered by AI
Add your answer

Q1. What is skewness and skewd tables
Ans. 
Skewness is a measure of asymmetry in a distribution. Skewed tables are tables with imbalanced data distribution.
• Skewness is a statistical measure that describes the asymmetry of the data distribution around the mean.

• Positive skewness indicates a longer tail on the right side of the distribution, while negative skewness indicates a longer tail on the left side.

• Skewed tables in data engineering refer to tables with imba...read more
Answered by AI
Add your answer

Q1. What is spark and explain working
Ans. 
Spark is a distributed computing framework designed for big data processing.
• Spark is built around the concept of Resilient Distributed Datasets (RDDs) which allow for fault-tolerant parallel processing of data.

• It provides high-level APIs in Java, Scala, Python, and R for ease of use.

• Spark can run on top of Hadoop, Mesos, Kubernetes, or in standalone mode.

• It includes modules for SQL, streaming, machine learning, and gra...read more
Answered by AI
Add your answer

Q1. Explain error handling in PySpark
Ans. 
Error handling in PySpark involves using try-except blocks and logging to handle exceptions and errors.
• Use try-except blocks to catch and handle exceptions in PySpark code

• Utilize logging to record errors and exceptions for debugging purposes

• Consider using the .option('mode', 'PERMISSIVE') method to handle corrupt records in data processing
Answered by AI
Add your answer

Q1. Data Warehousing related questions
Add your answer

Q1. Data Modelling related questions
Add your answer

Q1. Asked about skillsets & tools used in the current job. I was asked basic questions on SQL & Tableau
Add your answer

Detailed interview on SQL, Tableau & Alteryx

Three sections of apti, ML and Case study

Q1. Factors to account for ATM placements in city?
Ans. 
Factors such as foot traffic, proximity to banks, crime rates, and demographics should be considered for ATM placements in a city.
• Foot traffic in the area

• Proximity to banks or financial institutions

• Crime rates in the neighborhood

• Demographics of the area (income levels, age groups)

• Accessibility and visibility of the location

• Local regulations and zoning laws

• Availability of power and network connections

• Competition from ot...read more
Answered by AI
Add your answer

Q1. Money you want for this position
Add your answer
Q2. Why to choose this position and not another
Add your answer

2 questions based on Algorithm and logic

Q1. Questions on basic python, hadoop, sql etc
Add your answer

Asked questions from python and SQL

Q1. What are tupples
Ans. 
Tuples are ordered collections of elements, similar to lists but immutable.
• Tuples are created using parentheses ()

• Elements in a tuple can be of different data types

• Tuples are immutable, meaning their elements cannot be changed once created
Answered by AI
Add your answer
Q2. What is credit risk
Ans. 
Credit risk refers to the potential loss that a lender may incur due to a borrower's failure to repay a loan or meet their financial obligations.
• Credit risk is the risk of default on a debt that may arise from a borrower failing to make required payments.

• It is a key consideration for banks and financial institutions when lending money.

• Factors that contribute to credit risk include the borrower's credit history, financi...read more
Answered by AI
Add your answer