Filter interviews by
Clear (1)
I applied via Campus Placement and was interviewed before Oct 2022. There were 3 interview rounds.
Basic coding was asked
Top trending discussions
posted on 14 Dec 2024
I was interviewed in Nov 2024.
Use 'hdfs diskbalancer' command to check disk utilisation and health in Hadoop
Run 'hdfs diskbalancer -report' to get a report on disk utilisation
Use 'hdfs diskbalancer -plan <path>' to generate a plan for balancing disk usage
Check the Hadoop logs for any disk health issues
Spark Architecture consists of Driver, Cluster Manager, and Executors. Driver manages the execution of Spark jobs.
Driver: Manages the execution of Spark jobs, converts user code into tasks, and coordinates with Cluster Manager.
Cluster Manager: Manages resources across the cluster and allocates resources to Spark applications.
Executors: Execute tasks assigned by the Driver and store data in memory or disk for further pr...
Optimization techniques in Spark improve performance and efficiency of data processing.
Partitioning data to distribute workload evenly
Caching frequently accessed data in memory
Using broadcast variables for small lookup tables
Avoiding shuffling operations whenever possible
Tuning memory settings and garbage collection parameters
I am unable to provide this information as it is confidential.
Confidential information about salaries in previous organizations should not be disclosed.
It is important to respect the privacy and confidentiality of past employers.
Discussing specific salary details may not be appropriate in a professional setting.
To create a pivot table in SQL from a non-pivot table, you can use the CASE statement with aggregate functions.
Use the CASE statement to categorize data into columns
Apply aggregate functions like SUM, COUNT, AVG, etc. to calculate values for each category
Group the data by the columns you want to pivot on
Creating triggers in a database involves defining the trigger, specifying the event that will activate it, and writing the code to be executed.
Define the trigger using the CREATE TRIGGER statement
Specify the event that will activate the trigger (e.g. INSERT, UPDATE, DELETE)
Write the code or actions to be executed when the trigger is activated
Test the trigger to ensure it functions as intended
posted on 11 Jun 2024
I applied via Job Portal and was interviewed in May 2024. There was 1 interview round.
PySpark architecture is based on the Apache Spark architecture, with additional components for Python integration.
PySpark architecture includes Spark Core, Spark SQL, Spark Streaming, MLlib, and GraphX.
It allows Python developers to interact with Spark using PySpark API.
PySpark architecture enables distributed processing of large datasets using RDDs and DataFrames.
It leverages the power of in-memory processing for fast...
I applied via Naukri.com and was interviewed in Jul 2023. There were 2 interview rounds.
Spark internal working and optimization techniques
Spark uses Directed Acyclic Graph (DAG) for optimizing workflows
Lazy evaluation helps in optimizing transformations by combining them into a single stage
Caching and persistence of intermediate results can improve performance
Partitioning data can help in parallel processing and reducing shuffle operations
I applied via LinkedIn and was interviewed in Jun 2023. There were 4 interview rounds.
I applied via Referral and was interviewed in Dec 2024. There were 2 interview rounds.
30 Questions in 20 Minutes
The aptitude test lasts 30 minutes and focuses on topics relevant to data engineering, including Spark, SQL, Azure, and PySpark.
The coding test is a one-hour examination on PySpark.
I applied via AmbitionBox and was interviewed in Nov 2024. There were 4 interview rounds.
I utilize tools such as Excel, Python, SQL, and Tableau for data analysis.
Excel for basic data manipulation and visualization
Python for advanced data analysis and machine learning
SQL for querying databases
Tableau for creating interactive visualizations
Data analysis of code in the context of data analysis.
Coding logical question paper.
posted on 17 Jul 2024
I applied via Naukri.com and was interviewed in Aug 2024. There were 2 interview rounds.
I am a Senior Data Engineer with experience in developing data pipelines and optimizing data storage for various projects.
Developed data pipelines using Apache Spark for real-time data processing
Optimized data storage using technologies like Hadoop and AWS S3
Worked on a project to analyze customer behavior and improve marketing strategies
My day-to-day job in the project involved designing and implementing data pipelines, optimizing data workflows, and collaborating with cross-functional teams.
Designing and implementing data pipelines to extract, transform, and load data from various sources
Optimizing data workflows to improve efficiency and performance
Collaborating with cross-functional teams including data scientists, analysts, and business stakeholde...
DAGs handle fault tolerance by rerunning failed tasks and maintaining task dependencies.
DAGs rerun failed tasks automatically to ensure completion.
DAGs maintain task dependencies to ensure proper sequencing.
DAGs can be configured to retry failed tasks a certain number of times before marking them as failed.
Shuffling is the process of redistributing data across partitions in a distributed computing environment.
Shuffling is necessary when data needs to be grouped or aggregated across different partitions.
It can be handled efficiently by minimizing the amount of data being shuffled and optimizing the partitioning strategy.
Techniques like partitioning, combiners, and reducers can help reduce the amount of shuffling in MapRed
Repartition increases or decreases the number of partitions in a DataFrame, while Coalesce only decreases the number of partitions.
Repartition can increase or decrease the number of partitions in a DataFrame, leading to a shuffle of data across the cluster.
Coalesce only decreases the number of partitions in a DataFrame without performing a full shuffle, making it more efficient than repartition.
Repartition is typically...
Incremental data is handled by identifying new data since the last update and merging it with existing data.
Identify new data since last update
Merge new data with existing data
Update data warehouse or database with incremental changes
SCD stands for Slowly Changing Dimension, a concept in data warehousing to track changes in data over time.
SCD is used to maintain historical data in a data warehouse.
There are three types of SCD - Type 1, Type 2, and Type 3.
Type 1 SCD overwrites old data with new data.
Type 2 SCD creates a new record for each change, preserving history.
Type 3 SCD maintains both old and new values in the same record.
SCD is important for...
Reverse a string using SQL and Python codes.
In SQL, use the REVERSE function to reverse a string.
In Python, use slicing with a step of -1 to reverse a string.
Use Spark and SQL to find the top 5 countries with the highest population.
Use Spark to load the data and perform data processing.
Use SQL queries to group by country and sum the population.
Order the results in descending order and limit to top 5.
Example: SELECT country, SUM(population) AS total_population FROM table_name GROUP BY country ORDER BY total_population DESC LIMIT 5
To find different records for different joins using two tables
Use the SQL query to perform different joins like INNER JOIN, LEFT JOIN, RIGHT JOIN, and FULL JOIN
Identify the key columns in both tables to join on
Select the columns from both tables and use WHERE clause to filter out the different records
A catalyst optimizer is a query optimization tool used in Apache Spark to improve performance by generating an optimal query plan.
Catalyst optimizer is a rule-based query optimization framework in Apache Spark.
It leverages rules to transform the logical query plan into a more optimized physical plan.
The optimizer applies various optimization techniques like predicate pushdown, constant folding, and join reordering.
By o...
Used query optimization techniques to improve performance in database queries.
Utilized indexing to speed up search queries.
Implemented query caching to reduce redundant database calls.
Optimized SQL queries by restructuring joins and subqueries.
Utilized database partitioning to improve query performance.
Used query profiling tools to identify and optimize slow queries.
Use the len() function to check the length of the data frame.
Use len() function to get the number of rows in the data frame.
If the length is 0, then the data frame is empty.
Example: if len(df) == 0: print('Data frame is empty')
Cores and worker nodes are decided based on the workload requirements and scalability needs of the data processing system.
Consider the size and complexity of the data being processed
Evaluate the processing speed and memory requirements of the tasks
Take into account the parallelism and concurrency needed for efficient data processing
Monitor the system performance and adjust cores and worker nodes as needed
Enforcing schema ensures that data conforms to a predefined structure and rules.
Ensures data integrity by validating incoming data against predefined schema
Helps in maintaining consistency and accuracy of data
Prevents data corruption and errors in data processing
Can lead to rejection of data that does not adhere to the schema
I applied via Job Fair and was interviewed in Nov 2024. There were 2 interview rounds.
based on 5 reviews
Rating in categories
Software Developer
552
salaries
| ₹4.1 L/yr - ₹16.1 L/yr |
Software Development Engineer
511
salaries
| ₹3 L/yr - ₹13.2 L/yr |
Assistant Manager
452
salaries
| ₹3 L/yr - ₹10 L/yr |
Product Manager
417
salaries
| ₹10.2 L/yr - ₹40 L/yr |
Senior Manager
358
salaries
| ₹13.5 L/yr - ₹44.3 L/yr |
Jio
Reliance Industries
Bharti Airtel
Vodafone Idea