Netizen Engineering Data Engineer Interview Questions and Answers

Q1. Spark related questions
Add your answer
Q2. Previous projects done
Ans. 
Developed a real-time data processing system for a retail company to optimize inventory management.
• Designed and implemented data pipelines to ingest, process, and analyze large volumes of data

• Utilized technologies like Apache Kafka, Spark, and Hadoop for real-time processing

• Built dashboards and reports for stakeholders to monitor inventory levels and make data-driven decisions
Answered by AI
Add your answer

Q1. 1- Tell me about yourself 2- Spark questions (Architecture, Dag, Lineage,Broadcast, Accumulator, Repartition,coalesce,cache,persist,etc) 3- Basic Pyspark coding problem 4- Python coding problem(Array) 5- ...read more
Add your answer

Q1. 1- Tell me about yourself 2- Spark architecture 3- Parquet vs avro 4- catalyst optimiser 5- external table vs managed table vs cluster table 6- GIL in python 7- Memory management in python 8- Data skewnes...read more
Add your answer

General Aptitude test

Q1. On SQL functions
Add your answer
Q2. Python coding questions
Add your answer

Q1. Cloud Computing
Add your answer
Q2. Spark Programming
Add your answer

Q1. Python, sql, aws
Add your answer

Q1. Aws, architecture and design
Add your answer

Q1. Cultural fit and compensation
Add your answer

Q1. Normalisation of database, views,stored procedures
Ans. 
Normalization is a process of organizing data in a database to reduce redundancy and improve data integrity.
• Normalization involves breaking down a table into smaller tables and defining relationships between them.

• It helps in reducing data redundancy and inconsistencies.

• Views are virtual tables that are created based on the result of a query. They can be used to simplify complex queries.

• Stored procedures are precompiled...read more
Answered by AI
Add your answer

Q1. Duplicate in list
Ans. 
Find duplicates in a list of elements.
• Iterate through the list and keep track of elements seen so far

• If an element is already in the seen elements, it is a duplicate

• Use a hash set to efficiently check for duplicates
Answered by AI
Add your answer
Q2. Anagram in list
Ans. 
Check if any two strings in the list are anagrams of each other
• Iterate through the list and for each string, sort the characters and check if it already exists in a hashmap

• If it exists, then the current string is an anagram of the existing string

• Return true if anagrams are found, false otherwise
Answered by AI
Add your answer

Coding round with a python question and SQL query

Q1. Technical question and a coding question in python, medium level
Add your answer