Lentra AI Data Engineer Interview Questions and Answers

Q1. How to build data pipeline
Ans. 
Building a data pipeline involves extracting, transforming, and loading data from various sources to a destination for analysis.
• Identify data sources and determine the data to be collected

• Extract data from sources using tools like Apache NiFi or Apache Kafka

• Transform data using tools like Apache Spark or Python scripts

• Load data into a destination such as a data warehouse or database

• Schedule and automate the pipeline fo...read more
Answered by AI
Add your answer
Q2. IICS
Add your answer

Q1. What is data bricks
Ans. 
Data bricks is a unified analytics platform that provides a collaborative environment for data scientists, engineers, and analysts.
• Data bricks simplifies the process of building data pipelines and training machine learning models.

• It allows for easy integration with various data sources and tools, such as Apache Spark and Delta Lake.

• Data bricks provides a scalable and secure platform for processing big data and running ...read more
Answered by AI
Add your answer
Q2. How do you optimize your code?
Ans. 
Optimizing code involves identifying bottlenecks, improving algorithms, using efficient data structures, and minimizing resource usage.
• Identify and eliminate bottlenecks in the code by profiling and analyzing performance.

• Improve algorithms by using more efficient techniques and data structures.

• Use appropriate data structures like hash maps, sets, and arrays to optimize memory usage and access times.

• Minimize resource us...read more
Answered by AI
Add your answer
Q3. What is SQL window function?
Ans. 
SQL window function is used to perform calculations across a set of table rows related to the current row.
• Window functions operate on a set of rows related to the current row

• They can be used to calculate running totals, moving averages, rank, etc.

• Examples include ROW_NUMBER(), RANK(), SUM() OVER(), etc.
Answered by AI
Add your answer

Q1. Salary expectations?
Add your answer
Q2. When can you join
Add your answer

Q1. What activities you have used in data factory?
Ans. 
I have used activities such as Copy Data, Execute Pipeline, Lookup, and Data Flow in Data Factory.
• Copy Data activity is used to copy data from a source to a destination.

• Execute Pipeline activity is used to trigger another pipeline within the same or different Data Factory.

• Lookup activity is used to retrieve data from a specified dataset or table.

• Data Flow activity is used for data transformation and processing.
Answered by AI
Add your answer
Q2. How will you execute second notebook from first notebook?
Ans. 
To execute a second notebook from the first notebook, you can use the %run magic command in Jupyter Notebook.
• Use the %run magic command followed by the path to the second notebook in the first notebook.

• Ensure that the second notebook is in the same directory or provide the full path to the notebook.

• Make sure to save any changes in the second notebook before executing it from the first notebook.
Answered by AI
Add your answer
Q3. Difference between data lake storage and blob storage?
Ans. 
Data lake storage is optimized for big data analytics and can store structured, semi-structured, and unstructured data. Blob storage is for unstructured data only.
• Data lake storage is designed for big data analytics and can handle structured, semi-structured, and unstructured data

• Blob storage is optimized for storing unstructured data like images, videos, documents, etc.

• Data lake storage allows for complex queries and ...read more
Answered by AI
Add your answer

Q1. Python, sql, datawarehousing concepts, GCP
Add your answer

Q1. Copy activity in ADF
Add your answer
Q2. Delta table, unity catalog , delta live table in Azure databricks
Add your answer

Q1. Copy activity, Lookup, get metadata, if else, for each activity in ADF
Add your answer
Q2. Conceptual ETL Questions like coalesce and repartition, cache, persist etc.
Add your answer

Half hour with spark python azure databricks

Q1. Architecture databricks
Add your answer
Q2. Sql related questions
Add your answer

Q1. Tell me about yourself, second highest salary , question related to coleasce function
Add your answer