70+ Sismanage Integrated Solutions Interview Questions and Answers

To get values present in table A but not in B without using joining conditions, minus, intersect, union, you can use a subquery.

• Use a subquery to select values from table A that are not present in table B

• Example: SELECT * FROM A WHERE value NOT IN (SELECT value FROM B)

Generate permutations and combinations of a given string without stored procedures.

• Use recursion to generate all possible permutations and combinations.

• Swap characters in the string to generate permutations.

• Use a boolean array to keep track of used characters in combinations.

• Store the generated permutations and combinations in an array of strings.

Questions related to Java programming language and memory management.

• Overriding is a concept where a subclass provides its own implementation of a method that is already present in its superclass.

• JVM stores class information in the method area and objects in the heap area.

• Serialization is the process of converting an object into a stream of bytes to store it in a file or send it over a network.

• Thread class methods include start(), run(), sleep(), join(), etc.

• StackOverflow exc...read more

Explaining partitioning, bucketing, joins, optimization, broadcast variables, ORC vs Parquet, RDD vs Dataframe, project architecture and responsibilities for Big Data Engineer role.

• Partitioning is dividing data into smaller chunks for parallel processing, while bucketing is organizing data into buckets based on a hash function.

• Types of joins in Spark include inner join, outer join, left join, right join, and full outer join.

• Optimization techniques in Spark include caching, re...read more

Get 3 highest used words in a column from a table.

• Use GROUP BY and ORDER BY clauses in SQL query.

• Apply LIMIT 3 to get top 3 words.

• Use COUNT() function to count occurrences of each word.

Explains the working of hashmap and the difference between abstract class and interface.

• Hashmap is a data structure that stores key-value pairs and uses hashing to retrieve values quickly.

• Abstract classes are classes that cannot be instantiated and can have both abstract and non-abstract methods.

• Interfaces are contracts that define a set of methods that a class must implement.

• Using interfaces is better than abstract classes when multiple inheritance is needed or when a class ...read more

Reverse array in-place, find frequency of distinct elements, and implement Singleton class.

• To reverse an array in-place, swap the first and last elements, then move towards the center until the entire array is reversed.

• To find the frequency of distinct elements, use a hash table to count occurrences of each element.

• To implement a Singleton class, create a private constructor, a private static instance variable, and a public static method to return the instance.

Questions related to implementation and performance of Hadoop cluster.

• SSL implementation for Hadoop services involves generating SSL certificates and configuring SSL/TLS protocols for secure communication between Hadoop components.

• Kerberos implementation for Hadoop components involves setting up a Kerberos KDC, creating principals and keytabs, and configuring Hadoop services to use Kerberos authentication.

• Performance benchmarking of Hadoop cluster involves measuring various m...read more

Check if a list is sorted and sort the list without using Python methods.

• To check if a list is sorted, iterate through the list and compare each element with the next one. If any element is greater than the next one, the list is not sorted.

• To sort the list without using Python methods, implement a sorting algorithm like bubble sort, selection sort, or insertion sort.

• Example for checking if a list is sorted: ['a', 'b', 'c'] is sorted, ['c', 'b', 'a'] is not sorted.

• Example for ...read more

Spark can handle upserts using merge() function

• Use merge() function to handle upserts in Spark

• Specify the primary key column(s) to identify matching rows

• Specify the update column(s) to update existing rows

• Specify the insert column(s) to insert new rows

• Example: df1.merge(df2, on='id', whenMatched='update', whenNotMatched='insert')

Remove duplicate records and find 5th highest salary department wise using SQL.

• Use DISTINCT keyword to remove duplicate records.

• Use GROUP BY clause to group the records by department.

• Use ORDER BY clause to sort the salaries in descending order.

• Use LIMIT clause to get the 5th highest salary.

• Combine all the above clauses to get the desired result.

Handling out of memory issue in Spark involves optimizing memory usage, partitioning data, and increasing resources.

• Optimize memory usage by tuning Spark configurations like executor memory, driver memory, and shuffle partitions.

• Partition data to distribute workload evenly across nodes and avoid data skew.

• Increase resources by adding more nodes, increasing memory allocation, or using a larger cluster.

• Use persistence mechanisms like caching or checkpointing to reduce recomputa...read more

Multithreading allows concurrent execution of multiple threads. Future and CompletableFuture are used for asynchronous programming.

• Multithreading enables parallel processing of tasks

• Future is used to represent a result that may not be available yet

• CompletableFuture is a subclass of Future that provides more flexibility and functionality

• CompletableFuture can be used to chain multiple asynchronous tasks

• CompletableFuture can also be used to handle exceptions and timeouts

Data structures are fundamental in programming. Python and Java have built-in data structures like lists, tuples, and dictionaries.

• Python has built-in data structures like lists, tuples, and dictionaries

• Java has built-in data structures like arrays, lists, sets, and maps

• Data structures are used to store and organize data efficiently

• Choosing the right data structure is important for optimizing performance

• Examples of data structure implementation in Python: creating a list, app...read more

Spark wordcount and sort in descending order with their number occurrence in dataframe

• Read text file into dataframe using spark.read.text()

• Split each line into words using split() function

• Use groupBy() and count() functions to count the occurrence of each word

• Sort the resulting dataframe in descending order using orderBy() function

• Show the resulting dataframe using show() function

I have extensive experience in leading data migration projects from start to finish, ensuring seamless transition and minimal disruption.

• Led a team in migrating legacy data from on-premise servers to cloud storage

• Developed data mapping strategies to ensure accurate transfer of information

• Implemented data validation processes to identify and rectify any discrepancies

• Collaborated with stakeholders to define project scope and timelines

• Utilized ETL tools such as Informatica and T...read more

Partitioning and bucketing are techniques used in Hive to improve query performance and manage large datasets.

• Partitioning involves dividing data into smaller, more manageable parts based on a specific column or columns.

• Bucketing is a technique where data is divided into equal-sized buckets based on a hash function applied to a column.

• Partitioning and bucketing can be used together to further optimize query performance.

• Partitioning and bucketing help in reducing the amount of...read more

S3 bucket types are Standard, Intelligent-Tiering, Standard-Infrequent Access, One Zone-Infrequent Access, and Glacier. Life cycle policy automates data movement.

• S3 bucket types are designed to optimize storage costs and access patterns.

• Standard is for frequently accessed data, Intelligent-Tiering for variable access patterns, Standard-Infrequent Access for infrequent access, One Zone-Infrequent Access for infrequent access in a single availability zone, and Glacier for long-...read more

Overloading with different return type is not allowed in Java.

• Overloading is based on method signature, not return type

• Methods with same name and different return type will result in compile-time error

• Example: int add(int a, int b) and double add(int a, int b) is not allowed

Spark architecture involves master-slave setup with driver and executor nodes. Cluster configurations include memory allocation, cores allocation, and parallelism settings.

• Spark architecture consists of a master node (driver) and multiple worker nodes (executors)

• Cluster configurations involve setting memory allocation for driver and executors, specifying number of cores per executor, and adjusting parallelism settings

• Example: Setting executor memory to 4GB, number of cores pe...read more

Use a set to remove duplicates from a string array.

• Create a set to store unique strings.

• Iterate through the array and add each string to the set.

• Convert the set back to an array to get the unique strings.

Authentication in web API is achieved by using tokens or API keys to verify the identity of users.

• Use tokens (such as JWT) to authenticate users

• Implement OAuth for secure authentication

• Utilize API keys for authentication purposes

Find the intersection point of two arrays of strings.

• Convert arrays to sets to find common elements

• Iterate through one array and check if element exists in the other array

• Use built-in intersection method in Python

• Sort arrays and use two pointers to find common elements

AKS stands for Azure Kubernetes Service, a managed Kubernetes service provided by Microsoft Azure. Deployment strategy refers to the approach used to deploy applications or updates.

• AKS is a managed Kubernetes service offered by Microsoft Azure

• It simplifies the process of deploying, managing, and scaling containerized applications using Kubernetes

• Deployment strategy refers to the method used to deploy applications or updates, such as rolling updates, blue-green deployments, ca...read more

Broadcast Join is a type of join operation in distributed computing where one smaller dataset is broadcasted to all nodes for joining with a larger dataset.

• In Broadcast Join, one smaller dataset is broadcasted to all nodes in a distributed system.

• This smaller dataset is then joined with a larger dataset that is partitioned across the nodes.

• Broadcast Join is efficient when the smaller dataset can fit in memory across all nodes.

• It reduces the amount of data shuffling and networ...read more

Data lake is a storage repository that holds a vast amount of raw data in its native format, while Delta table is a versioned parquet table that stores data in a structured format.

• Data lake stores raw data in its original form, while Delta table stores structured data in a versioned parquet format.

• Data lake is suitable for storing large amounts of unstructured data, while Delta table is ideal for structured data with ACID transactions.

• Data lake allows for schema-on-read, mean...read more

Troubleshooting in Hadoop involves identifying and resolving issues related to data processing and storage in a Hadoop cluster.

• Identify and resolve issues with data ingestion, processing, and storage in Hadoop

• Check for errors in log files and analyze them to determine the root cause of the problem

• Monitor resource utilization and performance metrics to identify bottlenecks

• Optimize Hadoop configuration settings for better performance

• Ensure proper connectivity and communication ...read more

NameNode is a component in Hadoop that manages the file system metadata and keeps track of the location of data blocks.

• NameNode is the master node in Hadoop's HDFS (Hadoop Distributed File System).

• It stores the metadata of all the files and directories in the HDFS.

• NameNode maintains the mapping of data blocks to DataNodes where the actual data is stored.

• It handles client requests for file operations like read, write, and delete.

• NameNode is a single point of failure in Hadoop,...read more

Spark memory optimisation techniques

• Use broadcast variables to reduce memory usage

• Use persist() or cache() to store RDDs in memory

• Use partitioning to reduce shuffling and memory usage

• Use off-heap memory to avoid garbage collection overhead

• Tune memory settings such as spark.driver.memory and spark.executor.memory

The highest salary in SQL depends on the data and industry.

• The highest salary in SQL varies depending on the industry and location.

• Factors such as experience, education, and job role also impact salary.

• For example, a senior data engineer in Silicon Valley may earn a higher salary than a junior data engineer in a smaller city.

Hadoop serialisation techniques are used to convert data into a format that can be stored and processed in Hadoop.

• Hadoop uses Writable interface for serialisation and deserialisation of data

• Avro, Thrift, and Protocol Buffers are popular serialisation frameworks used in Hadoop

• Serialisation can be customised using custom Writable classes or external libraries

• Serialisation plays a crucial role in Hadoop performance and efficiency

Find and print the occurrence of each repeated character in sequence in a given string.

• Iterate through the string and keep track of the count of each character.

• Print the character and its count whenever a character is repeated in sequence.

Sealed virtual static new keywords are used in C# to control inheritance and method hiding.

• Sealed keyword prevents a class from being inherited.

• Virtual keyword allows a method to be overridden in derived classes.

• Static keyword makes a member of a class accessible without creating an instance of the class.

• New keyword hides a method from the base class in the derived class.

To create a stack from two queues, we need to use one queue for storing elements and the other for temporary storage.

• Push elements into the first queue

• When popping, move all elements except the last one to the second queue

• Pop the last element from the first queue and return it

• Swap the names of the two queues to make the second queue the first one

Performance optimization of Spark involves tuning various parameters and optimizing code.

• Tune memory allocation and garbage collection settings

• Optimize data serialization and compression

• Use efficient data structures and algorithms

• Partition data appropriately

• Use caching and persistence wisely

• Avoid shuffling data unnecessarily

• Monitor and analyze performance using Spark UI and other tools

Default size of block in HDFS is 128 MB.

• The default block size in HDFS is 128 MB.

• This block size can be changed by modifying the 'dfs.blocksize' property in the HDFS configuration.

• A larger block size can improve throughput for large files, while a smaller block size can reduce storage wastage for small files.

Interface defines a contract for classes to implement, while abstract class can have both abstract and concrete methods.

• Interfaces can only have abstract methods and constants, while abstract classes can have both abstract and concrete methods.

• A class can implement multiple interfaces but can only inherit from one abstract class.

• Interfaces are used to achieve multiple inheritance in Java, while abstract classes are used to provide a common base for subclasses.

Parallel execution in Selenium and Revert string

• Parallel execution in Selenium allows running multiple test cases simultaneously

• It helps in reducing the overall execution time

• Revert string means reversing the order of characters in a string

• In Java, we can use StringBuilder or StringBuffer class to reverse a string

Aks upgrade process involves upgrading the Kubernetes version of Azure Kubernetes Service clusters.

• AKS upgrade can be performed using Azure Portal, Azure CLI, or Azure PowerShell.

• Before upgrading, it is recommended to check for any compatibility issues with the new Kubernetes version.

• During the upgrade process, AKS nodes are drained and replaced with nodes running the new Kubernetes version.

• AKS supports both minor and major version upgrades of Kubernetes.

• It is important to ba...read more

SCD Types refer to Slowly Changing Dimensions Types used in data warehousing.

• SCD Type 1: Overwrite the old data with new data

• SCD Type 2: Create a new record for the new data and keep the old record

• SCD Type 3: Add new columns to the existing record to store new data

• SCD Type 4: Create a separate table to store historical data

• SCD Type 6: Combination of Type 1 and Type 2

Yes, it is possible to have a MapReduce job with 0 reducers.

• In some cases, the output of the map phase may not require any further processing or aggregation, so having 0 reducers is sufficient.

• For example, if the goal of the MapReduce job is simply to filter out certain data points based on a condition, there may be no need for a reducer.

• Having 0 reducers can also be useful for jobs where the output of the map phase is already in the desired format and does not need any addit...read more

Java collection is a single interface while collections is a utility class.

• Java collection is an interface that provides a unified architecture for manipulating and storing groups of objects.

• Collections is a utility class that provides static methods for working with collections.

• Java collection is a part of the Java Collections Framework while collections is not.

• Examples of Java collections include List, Set, and Map while examples of methods in collections include sort, reve...read more

A data node is a component in a distributed computing system that stores and manages data.

• Data nodes are part of a distributed file system, such as Hadoop's HDFS.

• They store and manage data blocks, and handle read and write operations.

• Data nodes communicate with the NameNode to maintain file system metadata.

• Examples of data nodes include the data nodes in a Hadoop cluster or the data storage nodes in a distributed database system.

I do not manage financial reports as a Scrum Master, but I ensure that the team follows the budget and financial guidelines set by the organization.

• As a Scrum Master, my focus is on facilitating the Scrum process and helping the team deliver value to the customer.

• I work closely with the Product Owner to prioritize the product backlog based on business value and budget constraints.

• I may help the team estimate the effort required for each user story or task, which can indirectl...read more

Calculate the frequency of a number in a list

• Iterate through the list and count occurrences of the number

• Use a dictionary to store the count of each number

• Return the count of the specified number

Hadoop queue policy determines how resources are allocated to different jobs in a Hadoop cluster.

• Queue policies can be configured at the cluster level or at the job level.

• Different queue policies include FIFO, Fair, and Capacity.

• FIFO policy allocates resources to jobs in the order they are submitted.

• Fair policy allocates resources fairly to all jobs based on their priority and resource requirements.

• Capacity policy allocates a fixed amount of resources to each queue and jobs w...read more

To sort an array in ascending order, use the sort() method.

• Use the sort() method to sort the array in ascending order.

• Pass a compare function to the sort() method to specify the sorting order.

• For an array of strings, the compare function should use the localeCompare() method.

• Example: array.sort((a, b) => a.localeCompare(b));

Water trapping problem involves calculating the amount of water that can be trapped between bars in an array.

• The problem can be solved using two pointers approach.

• Iterate through the array and keep track of the maximum height on the left and right side of each bar.

• Calculate the amount of water trapped at each bar by subtracting the bar's height from the minimum of the maximum heights on both sides.

• Sum up the trapped water at each bar to get the total amount of water trapped.

Keyvault is a secure storage service provided by cloud providers to manage and safeguard cryptographic keys, secrets, and certificates.

• Keyvault helps in securely storing and managing sensitive information such as passwords, API keys, and encryption keys.

• It provides centralized management of keys, secrets, and certificates, allowing for easier access control and auditing.

• Keyvault integrates with other services and applications to securely retrieve and use stored secrets and ke...read more

Diff string string builder compares two strings and returns the differences.

• Use a loop to iterate through each character of the strings

• Compare characters at the same index in both strings

• Build a new string with the differing characters

Arrays are fixed in size, while ArrayLists can dynamically resize.

• Arrays have a fixed size determined at initialization, while ArrayLists can dynamically resize.

• Arrays are faster for accessing elements, while ArrayLists are better for adding or removing elements.

• Arrays use [] syntax for accessing elements, while ArrayLists use get() and set() methods.

• Arrays can only store primitive types, while ArrayLists can store objects.

Network plugin in AKS is used to enable communication between pods in a Kubernetes cluster.

• Network plugin in AKS is responsible for setting up networking rules and policies within the cluster.

• Popular network plugins for AKS include Azure CNI, Kubenet, and Calico.

• Azure CNI is recommended for production workloads as it provides better performance and scalability.

• Network plugins help in enabling communication between pods running on different nodes in the cluster.

I have extensive experience working with various Azure services such as Azure DevOps, Azure Functions, Azure SQL Database, and Azure Virtual Machines.

• Experience with Azure DevOps for CI/CD pipelines

• Knowledge of Azure Functions for serverless computing

• Hands-on experience with Azure SQL Database for data storage

• Proficiency in managing Azure Virtual Machines for hosting applications

Optimization techniques in SQL query

• Use indexes to speed up query execution

• Avoid using SELECT * and only select necessary columns

• Use subqueries instead of joins for small tables

• Use UNION ALL instead of UNION if possible

• Avoid using functions in WHERE clause

• Use EXPLAIN to analyze query performance

Use the sort() method to arrange list items in Python.

• Use the sort() method to arrange list items in ascending order: list_name.sort()

• Use the sort() method with reverse=True parameter to arrange list items in descending order: list_name.sort(reverse=True)

• Use the sorted() function to return a new sorted list without modifying the original list: sorted_list = sorted(list_name)

Reverse each word in a sentence

• Split the sentence into words

• Reverse each word using built-in function or loop

• Join the reversed words back into a sentence

The question is asking about the average level of difficulty.

• Explain how you determine the difficulty level of a task

• Provide examples of tasks you have worked on and their difficulty level

• Discuss how you approach tasks of varying difficulty levels

Developed a data pipeline to ingest, process, and analyze customer feedback data

• Designed and implemented ETL processes to extract data from various sources

• Used tools like Apache Spark and Kafka for real-time data processing

• Built data models and visualizations to identify trends and insights

• Collaborated with cross-functional teams to improve data quality and accuracy

A linked list is a linear data structure where elements are stored in nodes and each node points to the next node in the sequence.

• Consists of nodes where each node contains data and a reference to the next node

• Can be singly linked (each node points to the next node) or doubly linked (each node points to the next and previous nodes)

• Allows for efficient insertion and deletion of elements compared to arrays

• Example: Singly linked list: 1 -> 2 -> 3 -> 4

• Example: Doubly linked list:...read more

Diff method compares values of two objects while equal method compares references of two objects.

• Diff method compares values of two objects, while equal method compares references.

• Diff method is used to check if two objects are different, while equal method is used to check if two objects are the same.

• Example: obj1.diff(obj2) vs obj1.equal(obj2)

Encapsulation, Inheritance, Polymorphism, Abstraction

• Encapsulation: Bundling data and methods that operate on the data into a single unit

• Inheritance: Ability of a class to inherit properties and behavior from another class

• Polymorphism: Ability to present the same interface for different data types

• Abstraction: Hiding the complex implementation details and showing only the necessary features

Lambda functions are anonymous functions that can be defined inline and do not require a name.

• Lambda functions are often used in functional programming languages.

• They are concise and can be used as arguments to higher-order functions.

• Lambda functions do not have a name and are defined using the 'lambda' keyword.

• Example: lambda x: x*2 defines a lambda function that doubles the input x.

Reconciliation is the process of ensuring two sets of records are in agreement.

• Reconciliation involves comparing financial transactions to ensure accuracy.

• It is commonly used in accounting to match bank statements with internal records.

• Reconciliation helps identify discrepancies and errors that need to be resolved.

• Automated tools like accounting software can streamline the reconciliation process.

My current CTC is confidential.

• I am unable to disclose my current CTC as it is confidential information.

• Discussing current CTC is not considered appropriate in interviews.

• I am more interested in discussing the potential salary for this role.

• I believe my skills and experience make me a strong candidate for this position.

The == operator checks for equality, while the === operator checks for strict equality.

• The == operator performs type coercion before comparing two values, while the === operator does not.

• Using === is generally considered safer and more predictable in JavaScript.

• Example: 1 == '1' will return true, but 1 === '1' will return false.

Join types in SQL are used to combine data from two or more tables based on a related column.

• Inner join returns only the matching rows from both tables

• Left join returns all rows from the left table and matching rows from the right table

• Right join returns all rows from the right table and matching rows from the left table

• Full outer join returns all rows from both tables, with NULL values in the columns where there is no match

Different types of performance testing include load testing, stress testing, and scalability testing.

• Load testing: Evaluates system performance under normal and peak load conditions.

• Stress testing: Tests system performance beyond its normal capacity to identify breaking points.

• Scalability testing: Measures system's ability to handle increased workload by adding resources.

• Endurance testing: Checks system performance over an extended period to ensure stability.

• Spike testing: Te...read more

Coupling refers to the degree of interdependence between software modules or components.

• Coupling measures how closely connected two modules are in a system.

• There are different types of coupling such as tight coupling and loose coupling.

• Tight coupling means high interdependence between modules, while loose coupling means low interdependence.

• Reducing coupling in a system can improve maintainability and flexibility.

• Example: A system with high coupling may require changes in mult...read more

Reverse a linked list by changing the pointers direction

• Create three pointers: prev, current, next

• Iterate through the list, updating pointers to reverse the direction

• Return the new head of the reversed list

React is a JavaScript library for building user interfaces. Redux is a predictable state container for managing application state.

• React is a front-end library developed by Facebook for building user interfaces.

• React uses a virtual DOM to efficiently update the actual DOM.

• Redux is a state management tool commonly used with React to manage application state.

• Redux stores the entire application state in a single immutable object.

Steam API is a set of tools and services provided by Steam for developers to integrate Steam features into their games or applications.

• Steam API allows developers to access features like user authentication, in-game purchases, and multiplayer functionality.

• Developers can use Steamworks SDK to integrate Steam API into their games.

• Examples of Steam API features include Steam Cloud for saving game data online and Steam Workshop for user-generated content.

A function is a block of code that performs a specific task when called.

• Functions can take input parameters and return output values.

• Functions can be reused multiple times in a program.

• Functions help in organizing code and making it more modular.

• Example: function add(a, b) { return a + b; }

Accessing the Document Object Model (DOM) allows developers to interact with and manipulate elements on a web page.

• Use document.getElementById() to access an element by its ID

• Use document.getElementsByClassName() to access elements by their class name

• Use document.getElementsByTagName() to access elements by their tag name

• Use document.querySelector() to access the first element that matches a specified CSS selector

• Use document.querySelectorAll() to access all elements that mat...read more

I'm sorry, I don't understand the question. Could you please provide more context?

• Please provide more information about what you mean by '3rd class experience'

• Is this related to a specific project or industry?

• I am happy to answer any questions you have, but I need more information to do so

Code in Spark for Big Data Engineer Lead interview

• Use SparkSession to create a Spark application

• Read data from a source like HDFS or S3

• Perform transformations and actions on the data using Spark RDDs or DataFrames

• Write the processed data back to a sink like HDFS or S3