KPI Partners Senior Engineer Interview Questions and Answers

Q1. Sql Questions, Pyspark..
Add your answer

Q1. Sql joins,dense rank and rank,window function
Add your answer

Q1. What is Informatica tool
Ans. 
Informatica is a data integration tool used for ETL (Extract, Transform, Load) processes in data engineering.
• Informatica is used for extracting data from various sources like databases, flat files, etc.

• It can transform the data according to business rules and load it into a target data warehouse or database.

• Informatica provides a visual interface for designing ETL workflows and monitoring data integration processes.

• It ...read more
Answered by AI
Add your answer
Q2. What is Datastage tool
Ans. 
Datastage is an ETL tool used for extracting, transforming, and loading data from various sources to a target destination.
• Datastage is part of the IBM Information Server suite.

• It provides a graphical interface to design and run data integration jobs.

• Datastage supports parallel processing for high performance.

• It can connect to a variety of data sources such as databases, flat files, and web services.

• Datastage jobs can b...read more
Answered by AI
Add your answer

Q1. Basic que on sql join
Add your answer
Q2. Python quetions
Add your answer

Q1. Spark and airflow related questions
Add your answer
Q2. AWS services and it's related questions
Add your answer

Basics of SQL, Python

Experience based questions, SQL and Python

Q1. Tell me about the projects you worked on
Ans. 
I have worked on projects involving building data pipelines, optimizing data storage, and developing machine learning models.
• Built data pipelines using Apache Spark and Airflow

• Optimized data storage by implementing partitioning and indexing strategies

• Developed machine learning models for predictive analytics
Answered by AI
Add your answer
Q2. Based on experience some questions related to projects.
Add your answer

1hour,Time speed distance

1hour,sql,python,algebra,Average

Q1. Components of Databricks
Ans. 
Databricks is a unified data analytics platform that includes components like Databricks Workspace, Databricks Runtime, and Databricks Delta.
• Databricks Workspace: Collaborative environment for data science and engineering teams.

• Databricks Runtime: Optimized Apache Spark cluster for data processing.

• Databricks Delta: Unified data management system for data lakes.
Answered by AI
Add your answer
Q2. How to read a json file
Ans. 
To read a JSON file, use a programming language's built-in functions or libraries to parse the file and extract the data.
• Use a programming language like Python, Java, or JavaScript to read the JSON file.

• Import libraries like json in Python or json-simple in Java to parse the JSON data.

• Use functions like json.load() in Python to load the JSON file and convert it into a dictionary or object.

• Access the data in the JSON fi...read more
Answered by AI
Add your answer
Q3. Second highest salary SQL
Ans. 
To find the second highest salary in SQL, use the MAX function with a subquery or the LIMIT clause.
• Use the MAX function with a subquery to find the highest salary first, then use a WHERE clause to exclude it and find the second highest salary.

• Alternatively, use the LIMIT clause to select the second highest salary directly.

• Make sure to handle cases where there may be ties for the highest salary.
Answered by AI
Add your answer
Q4. How to configure the spark while creating the cluster
Ans. 
Spark cluster configuration involves setting up memory, cores, and other parameters for optimal performance.
• Specify the number of executors and executor memory

• Set the number of cores per executor

• Adjust the driver memory based on the application requirements

• Configure shuffle partitions for efficient data processing

• Enable dynamic allocation for better resource utilization
Answered by AI
Add your answer