Artefact Senior Data Engineer Interview Questions and Answers

Q1. What is difference between dynamic data frame and spark data frame in aws glue job? can we change dynamic dataframe to spark datafrmae?
Ans. 
Dynamic data frame in AWS Glue job is a dynamically generated data frame, while Spark data frame is specifically created using Spark APIs.
• Dynamic data frame is generated dynamically at runtime based on the data source and schema, while Spark data frame is explicitly created using Spark APIs.

• Dynamic data frame is more flexible but may have performance implications compared to Spark data frame.

• You can convert a dynamic d...read more
Answered by AI
Add your answer

Q1. How does Django apply migrations to the database?
Ans. 
Django applies migrations to the database using the 'manage.py migrate' command.
• Django tracks changes to models and generates migration files accordingly.

• The 'manage.py makemigrations' command creates migration files based on model changes.

• The 'manage.py migrate' command applies the generated migration files to the database.

• Migrations help keep the database schema in sync with the changes in Django models.
Answered by AI
Add your answer
Q2. What is hoisting in javascript?
Ans. 
Hoisting in JavaScript is the behavior where variable and function declarations are moved to the top of their containing scope during the compilation phase.
• Variable declarations are hoisted to the top of their scope, but not their assignments.

• Function declarations are fully hoisted, meaning they can be called before they are declared.

• Hoisting can lead to unexpected behavior if not understood properly.
Answered by AI
Add your answer

Create a full stack application in 3 days.

Set of questions on english , aptitude , all are at easy level

Sql basics and some query questions

Q1. Why do u want to join zs ; what do u know about zs; where do u see urself after 5 years ; question on real life scenarios ;
Add your answer
Q2. Question on real life scenarios
Add your answer

Q1. Salary expectation
Add your answer
Q2. When can u join
Add your answer

Q1. What is Linear regression
Add your answer
Q2. What li list and tuple
Add your answer
Q3. Previous company project experience
Add your answer

Q1. Project related discussions
Add your answer
Q2. Meduim level SQl and DSA
Add your answer

Q1. This was data modelling round
Add your answer
Q2. Design a uber data model
Ans. 
Uber data model design for efficient storage and retrieval of ride-related information.
• Create tables for users, drivers, rides, payments, and ratings

• Include attributes like user_id, driver_id, ride_id, payment_id, rating_id, timestamp, location, fare, etc.

• Establish relationships between tables using foreign keys

• Implement indexing for faster query performance
Answered by AI
Add your answer

Three sections are there 1) Aptitude Test 2) SQL 3) DSA

Q1. What is DSA , sorting , difference between array and linked list
Ans. 
DSA stands for Data Structures and Algorithms. Sorting is the process of arranging data in a particular order. Array is a data structure that stores elements of the same data type in contiguous memory locations, while linked list is a data structure that stores elements in nodes with pointers to the next node.
• DSA stands for Data Structures and Algorithms

• Sorting is the process of arranging data in a particular order

• Arra...read more
Answered by AI
Add your answer
Q2. Written a SQL query
Add your answer

Q1. Coding question like add numbers
Add your answer
Q2. Experience on your project
Ans. 
I have experience working on various data analysis projects, including market research, customer segmentation, and predictive modeling.
• Developed predictive models to forecast customer behavior and optimize marketing strategies

• Conducted market research to identify trends and opportunities for growth

• Performed customer segmentation analysis to target specific demographics with personalized marketing campaigns
Answered by AI
Add your answer

Q1. What is pyspark architecture
Ans. 
PySpark architecture is a distributed computing framework that combines Python and Spark to process big data.
• PySpark architecture consists of a driver program, cluster manager, and worker nodes.

• The driver program is responsible for creating SparkContext, which connects to the cluster manager.

• Cluster manager allocates resources and schedules tasks on worker nodes.

• Worker nodes execute the tasks and return results to the ...read more
Answered by AI
Add your answer
Q2. What is skewed partitioning
Ans. 
Skewed partitioning is when data is not evenly distributed across partitions, leading to performance issues.
• Skewed partitioning can occur when a key column has a few values that are much more common than others.

• It can lead to uneven processing and resource utilization in distributed systems like Hadoop or Spark.

• To address skewed partitioning, techniques like data skew detection, data skew handling, and data skew preven
Answered by AI
Add your answer