Aidetic Software Interview Questions and Answers

Initially, I began with a medium-level question on LeetCode regarding the longest unique substring. Following that, I was asked about fundamental concepts in data engineering and two SQL questions: one to find the second highest employee salary and the other to determine the highest marks of students in each section.
here are all the questions
You have a dataset with employees, salaries, bonuses, and departments. How
would you calculate the total compensation for each department while
considering that bonuses are applied only to salaries above $50,000?




second highest sal in each department

select empId, dense_rank() over (partition by department order by salary decs) as rnk from employees_table where rank =2




in a streaming job processing event data, how would you handle late-arriving
data to ensure its included in calculations but doesnt delay real-time insights?"

Write PySpark logic to calculate the daily average sales amount for each product
category from a sales dataset spanning years. Consider that the df contains
'category', 'sales_amount', 'sale_date'


sales_df = sales_df.groupBy("category", sale_date).agg({"avg":"sales_amount"}).withColumnRenamed("avg(sales_amount), "Avg_sales_amount"")




-----/

You have a dataset with employees' salaries, bonuses, and departments. How
would you calculate the total compensation for each department while
considering that bonuses are applied only to salaries above $50,000?




in a streaming job processing event data, how would you handle late-arriving
data to ensure its included in calculations but doesnt delay real-time insights?"

Write PySpark logic to calculate the daily average sales amount for each product
category from a sales dataset spanning years. Consider that the df contains 'category', 'sales_amount', 'sale_date'

Q1. No coding but a lot of technical questions on architecture mainly on data bricks , medellion architecture how you work with small files problem optimize keyword z indexing, adls2 most spark based scenario ...read more
Add your answer

It about python basic question on string manipulation

Q1. Tell me about pyspark arcitecture
Ans. 
PySpark architecture is a distributed computing framework that allows for processing large datasets using Python.
• PySpark is built on top of Apache Spark, a fast and general-purpose cluster computing system.

• It utilizes a master-slave architecture with a driver program that communicates with a cluster manager to distribute tasks to worker nodes.

• PySpark supports various data sources such as HDFS, Apache Hive, and Apache H...read more
Answered by AI
Add your answer
Q2. Design lld for streaming
Ans. 
Designing a low-level design for streaming data processing system
• Use a distributed system like Apache Kafka for real-time data streaming

• Implement data processing pipelines using technologies like Apache Spark or Flink

• Ensure fault tolerance and scalability by using a microservices architecture

• Leverage cloud services like AWS Kinesis or Google Cloud Dataflow for managed streaming solutions
Answered by AI
Add your answer

Q1. What is the code to determine and print a happy number?
Ans. 
A happy number is a number which eventually reaches 1 when replaced by the sum of the square of each digit.
• Create a function to determine if a number is happy by repeatedly squaring the digits and summing them until the result is 1 or a cycle is detected.

• Use a set to keep track of seen numbers to detect cycles.

• Example: For number 19, the process would be 1^2 + 9^2 = 82, 8^2 + 2^2 = 68, 6^2 + 8^2 = 100, 1^2 + 0^2 + 0^2 ...read more
Answered by AI
Add your answer
Q2. What is the transformer architecture in the context of neural networks?
Ans. 
Transformer architecture is a type of neural network architecture commonly used in natural language processing tasks.
• Utilizes self-attention mechanism to weigh the importance of different words in a sentence

• Consists of encoder and decoder layers for tasks like machine translation

• Introduced by the paper 'Attention is All You Need' by Vaswani et al.

• Popular implementations include BERT, GPT, and TransformerXL
Answered by AI
Add your answer

Q1. Asked questions related to spark, python and sql
Add your answer
Q2. Some advanced questions related to spark and big data
Add your answer