Photon Interactive AWS Data Engineer Interview Questions and Answers

Q1. SQL problem window function
Add your answer
Q2. SQL code like join and scenerio
Add your answer

Q1. Design round for adf pipeline
Ans. 
Designing an ADF pipeline for data processing
• Identify data sources and destinations

• Define data transformations and processing steps

• Consider scheduling and monitoring requirements

• Utilize ADF activities like Copy Data, Data Flow, and Databricks

• Implement error handling and logging mechanisms
Answered by AI
Add your answer
Q2. Azure synapses and adf adb
Add your answer

Q1. Expected ctc and current ctc negotiations
Ans. 
Discussing expected and current salary for negotiation purposes.
• Be honest about your current salary and provide a realistic expectation for your desired salary.

• Highlight your skills and experience that justify your desired salary.

• Be open to negotiation and willing to discuss other benefits besides salary.

• Research industry standards and salary ranges for similar positions to support your negotiation.

• Focus on the value y...read more
Answered by AI
Add your answer
Q2. Relocation and remote work until ofc open for Pune location
Add your answer

Q1. Azure Scenario based questions
Add your answer
Q2. Pyspark Coding based questions
Add your answer

Q1. ADF, Databricks related question
Add your answer
Q2. Spark Performance problem and scenarios
Ans. 
Spark performance problems can arise due to inefficient code, data skew, resource constraints, and improper configuration.
• Inefficient code can lead to slow performance, such as using collect() on large datasets.

• Data skew can cause uneven distribution of data across partitions, impacting processing time.

• Resource constraints like insufficient memory or CPU can result in slow Spark jobs.

• Improper configuration settings, su...read more
Answered by AI
Add your answer

Q1. What are accumulators in spark?
Ans. 
Accumulators are shared variables that are updated by worker nodes and can be used for aggregating information across tasks.
• Accumulators are used for implementing counters and sums in Spark.

• They are only updated by worker nodes and are read-only by the driver program.

• Accumulators are useful for debugging and monitoring purposes.

• Example: counting the number of errors encountered during processing.
Answered by AI
Add your answer
Q2. Explain spark architecture
Ans. 
Spark architecture is a distributed computing framework that consists of a driver program, cluster manager, and worker nodes.
• Spark architecture includes a driver program that manages the execution of the Spark application.

• It also includes a cluster manager that allocates resources and schedules tasks on worker nodes.

• Worker nodes are responsible for executing the tasks and storing data in memory or disk.

• Spark architectu...read more
Answered by AI
Add your answer
Q3. Write a query to find duplicate data using SQL
Ans. 
Query to find duplicate data using SQL
• Use GROUP BY and HAVING clause to identify duplicate records

• Select columns to check for duplicates

• Use COUNT() function to count occurrences of each record
Answered by AI
Add your answer

Q1. What is pub/sub?
Ans. 
Pub/sub is a messaging pattern where senders (publishers) of messages do not program the messages to be sent directly to specific receivers (subscribers).
• Pub/sub stands for publish/subscribe.

• Publishers send messages to a topic, and subscribers receive messages from that topic.

• It allows for decoupling of components in a system, enabling scalability and flexibility.

• Examples include Apache Kafka, Google Cloud Pub/Sub, and
Answered by AI
Add your answer
Q2. What services have you used in GCP
Ans. 
I have used services like BigQuery, Dataflow, Pub/Sub, and Cloud Storage in GCP.
• BigQuery for data warehousing and analytics

• Dataflow for real-time data processing

• Pub/Sub for messaging and event ingestion

• Cloud Storage for storing data and files
Answered by AI
Add your answer

Q1. Basicn details to check for qualifications
Add your answer

Q1. About my projects
Add your answer

Q1. More details about ML models
Add your answer

Based on SQL , statistics , python , cognitive

Q1. Based on AI/Ml and based on cv
Add your answer
Q2. Based on projects
Add your answer

Q1. How to handle toxic work culture?
Ans. 
Address toxic work culture by open communication, setting boundaries, seeking support, and considering leaving if necessary.
• Open communication with colleagues and management about issues

• Set boundaries to protect your mental and emotional well-being

• Seek support from HR, a mentor, or a therapist if needed

• Consider leaving the toxic work environment if the situation does not improve
Answered by AI
Add your answer
Q2. 5 strength and weakness
Add your answer

Q1. Self introduction
Add your answer

Maths grammar & communication

Q1. Typeing speed skills check same use paragraph
Add your answer
Q2. Typing speed 10 minutes
Add your answer

Q1. Move on next round easy
Add your answer

You're like this job opportunity

Q1. Incremental load in pyspark
Ans. 
Incremental load in pyspark refers to loading only new or updated data into a dataset without reloading the entire dataset.
• Use the 'delta' function in pyspark to perform incremental loads by specifying the 'mergeSchema' option.

• Utilize the 'partitionBy' function to optimize incremental loads by partitioning the data based on specific columns.

• Implement a logic to identify new or updated records based on timestamps or uni...read more
Answered by AI
Add your answer
Q2. Drop duplicates
Add your answer

Q1. Tableau interview questioins
Add your answer
Q2. SQl based quetions
Add your answer