Infinite Computer Solutions Data Engineer Interview Questions and Answers

Q1. What Azure solutions have you worked with?
Ans. 
I have worked with Azure Data Factory, Azure Databricks, and Azure SQL Database.
• Azure Data Factory for data integration and orchestration

• Azure Databricks for big data processing and analytics

• Azure SQL Database for relational database management
Answered by AI
Add your answer

Q1. Azure related services
Add your answer
Q2. Etl concept in details
Add your answer

Its mandatory test even for experience people

Q1. Related to technology
Add your answer

Q1. Very good discussion towards work culture, salary and all
Add your answer

Q1. Windows functions in sql
Ans. 
Window functions in SQL are used to perform calculations across a set of table rows related to the current row.
• Window functions are used to calculate values based on a specific subset of rows within a table.

• They allow for ranking, aggregation, and other calculations without grouping the rows.

• Examples of window functions include ROW_NUMBER(), RANK(), and SUM() OVER().
Answered by AI
Add your answer
Q2. Delta lake from Databricks
Ans. 
Delta Lake is an open-source storage layer that brings ACID transactions to Apache Spark and big data workloads.
• Delta Lake is built on top of Apache Spark and provides ACID transactions for big data processing.

• It allows for schema enforcement and evolution, data versioning, and time travel queries.

• Delta Lake is compatible with popular data science and machine learning libraries like TensorFlow and PyTorch.
Answered by AI
Add your answer

Q1. Spark Optimizations
Add your answer

Q1. Tell me about your current project? Have you used any AWS services?
Add your answer
Q2. Can you explain about EMR andEC2 instance?
Ans. 
EMR is a managed Hadoop framework for processing large amounts of data, while EC2 is a scalable virtual server in AWS.
• EMR stands for Elastic MapReduce and is a managed Hadoop framework for processing large amounts of data.

• EC2 stands for Elastic Compute Cloud and is a scalable virtual server in Amazon Web Services (AWS).

• EMR allows for easy provisioning and scaling of Hadoop clusters, while EC2 provides resizable compute...read more
Answered by AI
Add your answer
Q3. What type of schemas did you use for your project. (Star schema, Snowflake Schema)
Ans. 
I have experience working with both Star and Snowflake schemas in my projects.
• Star schema is a denormalized schema where one central fact table is connected to multiple dimension tables.

• Snowflake schema is a normalized schema where dimension tables are further normalized into sub-dimension tables.

• Used Star schema for simpler, smaller datasets where performance is a priority.

• Used Snowflake schema for complex, larger dat...read more
Answered by AI
Add your answer
Q4. Have you used python, pyspark in your projects?
Ans. 
Yes, I have used Python and PySpark in my projects for data engineering tasks.
• I have used Python for data manipulation, analysis, and visualization.

• I have used PySpark for big data processing and distributed computing.

• I have experience in writing PySpark jobs to process large datasets efficiently.
Answered by AI
Add your answer
Q5. Do you have any experience with serverless schema?
Ans. 
Yes, I have experience with serverless schema.
• I have worked with AWS Lambda to build serverless applications.

• I have experience using serverless frameworks like Serverless Framework or AWS SAM.

• I have designed and implemented serverless architectures using services like AWS API Gateway and AWS DynamoDB.
Answered by AI
Add your answer

Q1. Pyspark coding questions
Add your answer
Q2. Data Modelling question
Add your answer
Q3. Sql coding questions
Add your answer
Q4. Python coding questions
Add your answer

Q1. Based on the previous projects and cloud technologies
Add your answer

Q1. Questions on Bigquery
Add your answer
Q2. Data ware house Migration questions
Add your answer
Q3. Airflow scheduling questions
Add your answer