PBI Analytics Azure Data Engineer Interview Questions and Answers

Q1. How would you create a pipeline for ADLS to SQL data movement?
Add your answer
Q2. How would you create a pipeline from REST API to ADLS? What is there are 8 million rows of records?
Add your answer
Q3. IF data needs filtering, joining and aggregation, how would you do it with ADF?
Add your answer
Q4. Explain medallion architecture.
Add your answer
Q5. Explain medallion with databricks
Add your answer

Q1. Basic questions and salary expectation.
Add your answer

Q1. 2 Python Questions based on Strong and List manipulation.
Add your answer
Q2. 2 Sql Questions
Add your answer
Q3. 1 Pyspark Question
Add your answer
Q4. What is service principal, ADF questions...
Add your answer

Q1. Introduction. Question on last company why left.
Add your answer
Q2. Salary Negotiation.
Add your answer

Q1. SCD type 1 and SCD type 2 in databircks
Add your answer
Q2. How to pass parameters form ADF to ADB
Add your answer

Sql queries and knowledge of different syntaxes

Q1. Find the student with marks greater than 80 in all subjects
Ans. 
Filter students with marks greater than 80 in all subjects
• Iterate through each student's marks in all subjects

• Check if all marks are greater than 80 for a student

• Return the student if all marks are greater than 80
Answered by AI
Add your answer
Q2. Write the syntax to define the schema of a file for loading.
Ans. 
Syntax to define schema of a file for loading
• Use CREATE EXTERNAL TABLE statement in SQL

• Specify column names and data types in the schema definition

• Example: CREATE EXTERNAL TABLE MyTable (col1 INT, col2 STRING) USING CSV
Answered by AI
Add your answer

Q1. Activities used in ADF
Ans. 
Activities in Azure Data Factory (ADF) are the building blocks of a pipeline and perform various tasks like data movement, data transformation, and data orchestration.
• Activities can be used to copy data from one location to another (Copy Activity)

• Activities can be used to transform data using mapping data flows (Data Flow Activity)

• Activities can be used to run custom code or scripts (Custom Activity)

• Activities can be u...read more
Answered by AI
Add your answer
Q2. Dataframes in pyspark
Ans. 
Dataframes in pyspark are distributed collections of data organized into named columns.
• Dataframes are similar to tables in a relational database, with rows and columns.

• They can be created from various data sources like CSV, JSON, Parquet, etc.

• Dataframes support SQL queries and transformations using PySpark functions.

• Example: df = spark.read.csv('file.csv')
Answered by AI
Add your answer

Q1. Managerial Questions
Add your answer
Q2. About project roles and resposibilities
Add your answer

Q1. How are you connecting your onPerm from Azure?
Ans. 
I connect onPrem to Azure using Azure ExpressRoute or VPN Gateway.
• Use Azure ExpressRoute for private connection through a dedicated connection.

• Set up a VPN Gateway for secure connection over the internet.

• Ensure proper network configurations and security settings.

• Use Azure Virtual Network Gateway to establish the connection.

• Consider using Azure Site-to-Site VPN for connecting onPremises network to Azure Virtual Network.
Answered by AI
Add your answer
Q2. What is Autoloader in Databricks?
Ans. 
Autoloader in Databricks is a feature that automatically loads new data files as they arrive in a specified directory.
• Autoloader monitors a specified directory for new data files and loads them into a Databricks table.

• It supports various file formats such as CSV, JSON, Parquet, Avro, and ORC.

• Autoloader simplifies the process of ingesting streaming data into Databricks without the need for manual intervention.

• It can be ...read more
Answered by AI
Add your answer
Q3. How do you normalize your Json data
Ans. 
Json data normalization involves structuring data to eliminate redundancy and improve efficiency.
• Identify repeating groups of data

• Create separate tables for each group

• Establish relationships between tables using foreign keys

• Eliminate redundant data by referencing shared values
Answered by AI
Add your answer
Q4. How do you read from Kafka?
Add your answer

Q1. Basics of adf adb
Add your answer
Q2. What is IR in adf pipe line
Ans. 
IR in ADF pipeline stands for Integration Runtime, which is a compute infrastructure used by Azure Data Factory to provide data integration capabilities across different network environments.
• IR in ADF pipeline is responsible for executing activities within the pipeline.

• It can be configured to run in different modes such as Azure, Self-hosted, and SSIS.

• Integration Runtime allows data movement between on-premises and clo...read more
Answered by AI
Add your answer

Q1. Brief about your projects Be much prompt with your roles and responsibilities. Good to know SQL better
Add your answer

Q1. Salary negotiation and about relocation
Add your answer