Metyis Lead Data Engineer Interview Questions and Answers

Q1. Given a DataFrame df with columns 'A', 'B','C' how would you group the data by the values in column 'A' and calculate the mean of column 'B' for each group, while also summing the values in column 'C' ?
Ans. 
Group data by column 'A', calculate mean of column 'B' and sum values in column 'C' for each group.
• Use groupby() function in pandas to group data by column 'A'

• Apply mean() function on column 'B' and sum() function on column 'C' for each group

• Example: df.groupby('A').agg({'B':'mean', 'C':'sum'})
Answered by AI
Add your answer
Q2. Explain the difference deepcopy() and copy() methods in Python's copy module. Provide a scenario where you would use deepcopy() over copy().
Ans. 
deepcopy() creates a new object with completely independent copies of nested objects, while copy() creates a shallow copy.
• deepcopy() creates a new object and recursively copies all nested objects, while copy() creates a shallow copy of the top-level object only.

• Use deepcopy() when you need to create a deep copy of an object with nested structures, to avoid any references to the original object.

• Use copy() when you only ...read more
Answered by AI
Add your answer
Q3. Discuss the concept of Python decorators and provide an example of how you would use decorators to measure the execution time of a function.
Ans. 
Python decorators are functions that modify the behavior of other functions. They are commonly used for adding functionality to existing functions without modifying their code.
• Decorators are defined using the @ symbol followed by the decorator function name.

• They can be used to measure the execution time of a function by wrapping the function with a timer decorator.

• Example: def timer(func): def wrapper(*args, **kwargs...read more
Answered by AI
Add your answer

Q1. What are the various data warehousing techniques used, and can you explain them in detail?
Ans. 
Various data warehousing techniques include dimensional modeling, star schema, snowflake schema, and data vault.
• Dimensional modeling involves organizing data into facts and dimensions to facilitate easy querying and analysis.

• Star schema is a type of dimensional modeling where a central fact table is connected to multiple dimension tables.

• Snowflake schema is an extension of star schema where dimension tables are normali...read more
Answered by AI
Add your answer
Q2. How has your analytics work contributed to the organization?
Ans. 
My analytics work has helped the organization make data-driven decisions, improve operational efficiency, and identify new opportunities for growth.
• Developed data models and algorithms to optimize business processes

• Generated insights from large datasets to drive strategic decision-making

• Identified trends and patterns to improve customer experience and retention

• Implemented data governance policies to ensure data quality
Answered by AI
Add your answer

Q1. How would you respond in various situations?
Ans. 
I would respond in various situations by remaining calm, assessing the situation, and providing a thoughtful and strategic solution.
• Remain calm and composed

• Assess the situation thoroughly

• Provide a thoughtful and strategic solution

• Communicate effectively with all parties involved
Answered by AI
Add your answer
Q2. What is more important career , team
Ans. 
Both career and team are important, but ultimately career growth should be prioritized.
• Career growth is essential for personal development and achieving professional goals.

• A strong team can support career growth by providing mentorship, collaboration, and opportunities for learning.

• Balancing career and team dynamics is key to long-term success in any role.
Answered by AI
Add your answer

Q1. Questions related to the current and past projects.
Add your answer
Q2. Aws related questions.
Add your answer
Q3. Data warehousing basics
Add your answer

Q1. Explain about the roles and responsibilities.
Add your answer
Q2. Discussion about the role in Genpact.
Add your answer

Q1. Asked about expectated salary
Add your answer
Q2. Why are you switching
Add your answer

Q1. Asked about data tools other than AWS services
Add your answer
Q2. When did you use HUDI and Iceberg
Ans. 
I have used HUDI and Iceberg in my previous project for managing large-scale data lakes efficiently.
• Implemented HUDI for incremental data ingestion and managing large datasets in real-time

• Utilized Iceberg for efficient table management and data versioning

• Integrated HUDI and Iceberg with Apache Spark for processing and querying data
Answered by AI
Add your answer

Q1. What is dba, explain the architecture of dba
Ans. 
DBA stands for Database Administrator. The architecture of DBA involves managing and maintaining databases to ensure data integrity and security.
• DBA is responsible for installing, configuring, and upgrading database software.

• They monitor database performance and troubleshoot issues.

• DBA designs and implements backup and recovery strategies to prevent data loss.

• They also manage user access and security permissions within...read more
Answered by AI
Add your answer
Q2. How you will maintain the database and what are the types of backup
Ans. 
Maintaining the database involves regular monitoring, performance tuning, applying patches, and ensuring backups are taken regularly.
• Regularly monitor database performance and usage

• Perform routine maintenance tasks such as applying patches and updates

• Take regular backups to ensure data integrity and disaster recovery

• Implement security measures to protect the database from unauthorized access

• Optimize database performanc
Answered by AI
Add your answer

Q1. Difference between join and lookup
Ans. 
Join is used to combine rows from two or more tables based on a related column, while lookup is used to retrieve data from a reference table based on a matching key.
• Join combines rows from multiple tables based on a related column

• Lookup retrieves data from a reference table based on a matching key

• Join can result in duplicate rows if there are multiple matches, while lookup returns only the first matching row

• Join is use...read more
Answered by AI
Add your answer
Q2. Diffrence between fact table and dimension table
Ans. 
Fact table contains quantitative data and measures, while dimension table contains descriptive attributes.
• Fact table contains numerical data that can be aggregated (e.g. sales revenue, quantity sold)

• Dimension table contains descriptive attributes for analysis (e.g. product name, customer details)

• Fact table is typically normalized, while dimension table is denormalized for faster queries

• Fact table is usually larger in s
Answered by AI
Add your answer
Q3. Sed command to display last before line
Ans. 
Use sed command to display the line before a specific pattern
• Use 'sed -n '/pattern/{g;1!p;};h' file.txt' to display the line before the pattern

• Replace 'pattern' with the specific pattern you are looking for

• This command will display the line before the pattern in the file
Answered by AI
Add your answer

Q1. Window function coding test
Ans. 
Window function coding test involves using window functions in SQL to perform calculations within a specified window of rows.
• Understand the syntax and usage of window functions in SQL

• Use window functions like ROW_NUMBER(), RANK(), DENSE_RANK(), etc. to perform calculations

• Specify the window frame using PARTITION BY and ORDER BY clauses

• Practice writing queries with window functions to get comfortable with their usage
Answered by AI
Add your answer
Q2. Explain azure data factory
Ans. 
Azure Data Factory is a cloud-based data integration service that allows you to create, schedule, and manage data pipelines.
• Azure Data Factory is used to move and transform data from various sources to destinations.

• It supports data integration processes like ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform).

• You can create data pipelines using a visual interface in Azure Data Factory.

• It can connect to on...read more
Answered by AI
Add your answer

Q1. What is data vault
Ans. 
Data Vault is a modeling methodology for designing highly scalable and flexible data warehouses.
• Data Vault focuses on long-term historical data storage

• It consists of three main components: Hubs, Links, and Satellites

• Hubs represent business entities, Links represent relationships between entities, and Satellites store attributes of entities

• Data Vault allows for easy scalability and adaptability to changing business requ
Answered by AI
Add your answer
Q2. What is lambda architecture
Ans. 
Lambda architecture is a data processing architecture designed to handle massive quantities of data by using both batch and stream processing methods.
• Combines batch processing layer, speed layer, and serving layer

• Batch layer processes historical data in large batches

• Speed layer processes real-time data

• Serving layer merges results from batch and speed layers for querying

• Example: Apache Hadoop for batch processing, Apach
Answered by AI
Add your answer

Q1. Whether have onsite exposure
Ans. 
Yes, I have onsite exposure in previous roles.
• I have worked onsite at various client locations to gather requirements and implement solutions.

• I have experience collaborating with cross-functional teams in person.

• I have conducted onsite training sessions for end users on data architecture best practices.

• I have participated in onsite data migration projects.

• I have worked onsite to troubleshoot and resolve data-related is
Answered by AI
Add your answer

Q1. Data Modeling Concepts, SQL Query
Add your answer
Q2. Data Governance, Data Lineage, ETL process,
Add your answer

SQL Scripts to write and also also asked to design an data model of my choice in Telecom Domain