i
Egen
Filter interviews by
Top trending discussions
I applied via Campus Placement and was interviewed before Dec 2021. There were 4 interview rounds.
Simple questions on aptitude, verbal, and behavioral questions.
Simple coding questions which can be solved in 2 mins.
posted on 10 Jun 2024
I applied via Naukri.com and was interviewed before Jun 2023. There were 3 interview rounds.
It’s just reasoning type questions.
SSIS stands for SQL Server Integration Services, a tool provided by Microsoft for data integration and workflow applications.
SSIS is a platform for building high-performance data integration and workflow solutions.
It allows you to create packages that move data from various sources to destinations.
SSIS includes a visual design interface for creating, monitoring, and managing data integration processes.
You can use SSIS ...
SSIS packages are used for ETL processes in SQL Server. Union combines datasets vertically, while merge combines them horizontally.
SSIS packages are used for Extract, Transform, Load (ETL) processes in SQL Server.
Union in SSIS combines datasets vertically, stacking rows on top of each other.
Merge in SSIS combines datasets horizontally, matching rows based on specified columns.
Union All in SSIS combines datasets vertica...
reduceByKey is more efficient than groupByKey for aggregating data in Spark due to reduced shuffling.
reduceByKey combines values for each key in each partition before shuffling data
groupByKey shuffles all data to a single partition before combining values for each key
reduceByKey is preferred for large datasets to minimize data movement and improve performance
Scala provides a simple way to count words in a string using built-in functions.
Use the split function to split the string into an array of words
Use the length function to get the count of words in the array
Use SQL query with ORDER BY and LIMIT to find the second highest salary.
Use ORDER BY clause to sort salaries in descending order
Use LIMIT 1,1 to skip the first highest salary and get the second highest salary
posted on 17 Jul 2024
I applied via Naukri.com and was interviewed in Aug 2024. There were 2 interview rounds.
I am a Senior Data Engineer with experience in developing data pipelines and optimizing data storage for various projects.
Developed data pipelines using Apache Spark for real-time data processing
Optimized data storage using technologies like Hadoop and AWS S3
Worked on a project to analyze customer behavior and improve marketing strategies
My day-to-day job in the project involved designing and implementing data pipelines, optimizing data workflows, and collaborating with cross-functional teams.
Designing and implementing data pipelines to extract, transform, and load data from various sources
Optimizing data workflows to improve efficiency and performance
Collaborating with cross-functional teams including data scientists, analysts, and business stakeholde...
DAGs handle fault tolerance by rerunning failed tasks and maintaining task dependencies.
DAGs rerun failed tasks automatically to ensure completion.
DAGs maintain task dependencies to ensure proper sequencing.
DAGs can be configured to retry failed tasks a certain number of times before marking them as failed.
Shuffling is the process of redistributing data across partitions in a distributed computing environment.
Shuffling is necessary when data needs to be grouped or aggregated across different partitions.
It can be handled efficiently by minimizing the amount of data being shuffled and optimizing the partitioning strategy.
Techniques like partitioning, combiners, and reducers can help reduce the amount of shuffling in MapRed
Repartition increases or decreases the number of partitions in a DataFrame, while Coalesce only decreases the number of partitions.
Repartition can increase or decrease the number of partitions in a DataFrame, leading to a shuffle of data across the cluster.
Coalesce only decreases the number of partitions in a DataFrame without performing a full shuffle, making it more efficient than repartition.
Repartition is typically...
Incremental data is handled by identifying new data since the last update and merging it with existing data.
Identify new data since last update
Merge new data with existing data
Update data warehouse or database with incremental changes
SCD stands for Slowly Changing Dimension, a concept in data warehousing to track changes in data over time.
SCD is used to maintain historical data in a data warehouse.
There are three types of SCD - Type 1, Type 2, and Type 3.
Type 1 SCD overwrites old data with new data.
Type 2 SCD creates a new record for each change, preserving history.
Type 3 SCD maintains both old and new values in the same record.
SCD is important for...
Reverse a string using SQL and Python codes.
In SQL, use the REVERSE function to reverse a string.
In Python, use slicing with a step of -1 to reverse a string.
Use Spark and SQL to find the top 5 countries with the highest population.
Use Spark to load the data and perform data processing.
Use SQL queries to group by country and sum the population.
Order the results in descending order and limit to top 5.
Example: SELECT country, SUM(population) AS total_population FROM table_name GROUP BY country ORDER BY total_population DESC LIMIT 5
To find different records for different joins using two tables
Use the SQL query to perform different joins like INNER JOIN, LEFT JOIN, RIGHT JOIN, and FULL JOIN
Identify the key columns in both tables to join on
Select the columns from both tables and use WHERE clause to filter out the different records
A catalyst optimizer is a query optimization tool used in Apache Spark to improve performance by generating an optimal query plan.
Catalyst optimizer is a rule-based query optimization framework in Apache Spark.
It leverages rules to transform the logical query plan into a more optimized physical plan.
The optimizer applies various optimization techniques like predicate pushdown, constant folding, and join reordering.
By o...
Used query optimization techniques to improve performance in database queries.
Utilized indexing to speed up search queries.
Implemented query caching to reduce redundant database calls.
Optimized SQL queries by restructuring joins and subqueries.
Utilized database partitioning to improve query performance.
Used query profiling tools to identify and optimize slow queries.
Use the len() function to check the length of the data frame.
Use len() function to get the number of rows in the data frame.
If the length is 0, then the data frame is empty.
Example: if len(df) == 0: print('Data frame is empty')
Cores and worker nodes are decided based on the workload requirements and scalability needs of the data processing system.
Consider the size and complexity of the data being processed
Evaluate the processing speed and memory requirements of the tasks
Take into account the parallelism and concurrency needed for efficient data processing
Monitor the system performance and adjust cores and worker nodes as needed
Enforcing schema ensures that data conforms to a predefined structure and rules.
Ensures data integrity by validating incoming data against predefined schema
Helps in maintaining consistency and accuracy of data
Prevents data corruption and errors in data processing
Can lead to rejection of data that does not adhere to the schema
I applied via Recruitment Consulltant and was interviewed before Nov 2022. There were 3 interview rounds.
I am a Senior Data Engineer with expertise in data processing and analysis.
Experienced in designing and implementing data pipelines
Proficient in programming languages like Python and SQL
Skilled in working with big data technologies like Hadoop and Spark
Familiar with data warehousing and ETL processes
Strong problem-solving and analytical skills
I am a Senior Data Engineer with expertise in data processing and analysis.
Experienced in designing and implementing data pipelines
Proficient in programming languages like Python and SQL
Skilled in working with big data technologies such as Hadoop and Spark
Familiar with data warehousing concepts and ETL processes
Strong problem-solving and troubleshooting skills
Effective communication and collaboration with cross-functio
I was interviewed in Jul 2023.
Snowflake architecture is used in our project for cloud-based data warehousing.
Snowflake follows a multi-cluster shared data architecture.
It separates storage and compute resources, allowing for independent scaling.
Data is stored in virtual warehouses, which are compute clusters that can be scaled up or down based on workload.
Snowflake uses a unique architecture called a multi-cluster, shared data architecture, which s...
Database roles in Snowflake define permissions and access control for users and objects.
Database roles in Snowflake are used to manage permissions and access control for users and objects.
Roles can be assigned to users or other roles to grant specific privileges.
Examples of roles in Snowflake include ACCOUNTADMIN, SYSADMIN, SECURITYADMIN, and PUBLIC.
Session Policy in Snowflake defines the behavior of a session, including session timeout and idle timeout settings.
Session Policy can be set at the account, user, or role level in Snowflake.
Session Policy settings include session timeout, idle timeout, and other session-related configurations.
Example: Setting a session timeout of 30 minutes will automatically end the session if there is no activity for 30 minutes.
SSO process between Snowflake and Azure Active Directory involves configuring SAML-based authentication.
Configure Snowflake to use SAML authentication with Azure AD as the identity provider
Set up a trust relationship between Snowflake and Azure AD
Users authenticate through Azure AD and are granted access to Snowflake resources
SSO eliminates the need for separate logins and passwords for Snowflake and Azure AD
Network Policy in Snowflake controls access to Snowflake resources based on IP addresses or ranges.
Network Policies are used to restrict access to Snowflake resources based on IP addresses or ranges.
They can be applied at the account, user, or role level.
Network Policies can be used to whitelist specific IP addresses or ranges that are allowed to access Snowflake resources.
They can also be used to blacklist IP addresse...
Automate data loading from pipes into Snowflake for efficient data processing.
Use Snowpipe, a continuous data ingestion service provided by Snowflake, to automatically load data from pipes into Snowflake tables.
Snowpipe monitors a stage for new data files and loads them into the specified table in real-time.
Configure Snowpipe to trigger a data load whenever new data files are added to the stage, eliminating the need fo...
Query acceleration speeds up query processing by optimizing query execution and reducing the time taken to retrieve data.
Query acceleration uses techniques like indexing, partitioning, and caching to optimize query execution.
It reduces the time taken to retrieve data by minimizing disk I/O and utilizing in-memory processing.
Examples include using columnar storage formats like Parquet or optimizing join operations.
I applied via Naukri.com and was interviewed in Jul 2021. There were 4 interview rounds.
based on 2 interviews
Interview experience
based on 1 review
Rating in categories
Talent Acquisition Specialist
12
salaries
| ₹0 L/yr - ₹0 L/yr |
Associate Application Engineer
10
salaries
| ₹0 L/yr - ₹0 L/yr |
Software Engineer
7
salaries
| ₹0 L/yr - ₹0 L/yr |
Data Engineer
6
salaries
| ₹0 L/yr - ₹0 L/yr |
Senior Software Engineer
5
salaries
| ₹0 L/yr - ₹0 L/yr |
TCS
Infosys
Wipro
HCLTech