Sigmoid Interview Questions, Process, and Tips

Q1. What is scd in dw?
Ans. 
SCD stands for Slowly Changing Dimension in Data Warehousing.
• SCD is a technique used in data warehousing to track changes to dimension data over time.

• There are different types of SCDs - Type 1, Type 2, and Type 3.

• Type 1 SCD overwrites old data with new data, Type 2 creates new records for changes, and Type 3 maintains both old and new values in separate columns.

• Example: In a customer dimension table, if a customer chan...read more
Answered by AI
Add your answer
Q2. Inferschema in pyspark when reading file
Ans. 
inferschema in pyspark is used to automatically infer the schema of a file when reading it.
• inferschema is a parameter in pyspark that can be set to true when reading a file to automatically infer the schema based on the data

• It is useful when the schema of the file is not known beforehand

• Example: df = spark.read.csv('file.csv', header=True, inferSchema=True)
Answered by AI
Add your answer
Q3. Triplet program
Add your answer
Q4. Rank vs dense rank
Ans. 
Rank assigns unique ranks to each distinct value, while dense rank assigns ranks without gaps.
• Rank function assigns unique ranks to each distinct value in a result set.

• Dense rank function assigns ranks to rows in a result set without any gaps between the ranks.

• Rank function may skip ranks if there are ties in values, while dense rank will not skip ranks.
Answered by AI
Add your answer
Q5. Some numbers gave how many rows for inner join,left,right,full join
Add your answer
Q6. Read parquet file and remove duplicates
Add your answer
Q7. Optimizing techniques in spark
Ans. 
Optimizing techniques in Spark involve partitioning, caching, and tuning resources for efficient data processing.
• Use partitioning to distribute data evenly across nodes for parallel processing

• Cache frequently accessed data in memory to avoid recomputation

• Tune resources such as memory allocation and parallelism settings for optimal performance
Answered by AI
Add your answer
Q8. Repartition vs coalesce
Ans. 
Repartition is used to increase the number of partitions in a DataFrame, while coalesce is used to decrease the number of partitions.
• Repartition involves shuffling data across the network, which can be expensive in terms of performance and resources.

• Coalesce is a more efficient operation as it minimizes data movement by only merging existing partitions.

• Repartition is typically used when there is a need for more paralle...read more
Answered by AI
Add your answer
Q9. Normalization in db and types
Ans. 
Normalization in databases is the process of organizing data in a database to reduce redundancy and improve data integrity.
• Normalization is used to eliminate redundant data and ensure data integrity.

• It involves breaking down a table into smaller tables and defining relationships between them.

• There are different normal forms such as 1NF, 2NF, 3NF, and BCNF.

• Normalization helps in reducing data redundancy and improving qu...read more
Answered by AI
Add your answer
Q10. Transformation vs action
Ans. 
Transformation involves changing the data structure, while action involves performing a computation on the data.
• Transformation changes the data structure without executing any computation

• Action performs a computation on the data and triggers the execution

• Examples of transformation include map, filter, and reduce in Spark or Pandas

• Examples of action include count, collect, and saveAsTextFile in Spark
Answered by AI
Add your answer

Q1. How does dropout help in neural networks?
Ans. 
Dropout helps prevent overfitting in neural networks by randomly setting a fraction of input units to zero during training.
• Dropout helps in preventing overfitting by reducing the interdependence between neurons

• It acts as a regularization technique by randomly setting a fraction of input units to zero during training

• Dropout forces the network to learn redundant representations, making it more robust and generalizable

• It ...read more
Answered by AI
Add your answer
Q2. How does xgboost deal with nan values?
Ans. 
XGBoost can handle missing values (NaN) by assigning them to a default direction during tree construction.
• XGBoost treats NaN values as missing values and learns the best direction to go at each node to handle them

• During tree construction, XGBoost assigns NaN values to the default direction based on the training data statistics

• XGBoost can handle missing values in both input features and target variables
Answered by AI
Add your answer
Q3. What would you do if you see your model not performing well on a time series prediction specifically on peaks and troughs?
Add your answer
Q4. How would you deal with datasets having lots of categories
Ans. 
Utilize feature engineering techniques like one-hot encoding or target encoding to handle datasets with many categories.
• Use feature engineering techniques like one-hot encoding to convert categorical variables into numerical values

• Consider using target encoding to encode categorical variables based on the target variable

• Apply dimensionality reduction techniques like PCA or LDA to reduce the number of features

• Use tree-b...read more
Answered by AI
Add your answer

Case study involved creating a churn model with an imbalanced dataset. It contained a lot of missing values in numerical features which were correlated, Also the scaling was highly skewed. Categorical data contained a lot of low frequency categories. They wanted a final model performance on a test dataset on chosen KPIs (I chose F1-score).

Q1. Questions on the case study - assumptions made, why did you choose a particular KPI? What was the loss function? How did you deal with class imbalance, nan values , high number of categories? Did you perfo...read more
Add your answer
Q2. Questions related to past experience - I told him about my last project which was based on computer vision. Interviewer asked a lot of clarifying questions and inquired about the process
Add your answer

Q1. Asked about previous projects and the business impact.
Add your answer
Q2. What challenges have you faced managing a team?
Add your answer
Q3. Asked about a hypothetical scenario, how would you help the customer with that
Add your answer
Q4. What would you do if someone in your team is not performing well
Add your answer

Q1. Salary Discussion
Add your answer
Q2. Previous employment history
Add your answer

Q1. Pyspark, sql, Python.
Add your answer

Q1. Python, dsa,PySpark
Add your answer

Q1. Simple questions
Add your answer

1 hour aptitude test

Q1. What is one hot encoding
Add your answer

Q1. What is your long term goal
Add your answer

Consist of 2 coding questions, multiple choice of around 30 question including, math, code, logic and core concepts.

1 hr of coding test mostly on leetcode platform. level : Hard

Q1. Db related question
Add your answer

Q1. Why this company?
Ans. 
I am impressed by the company's innovative products, strong company culture, and opportunities for growth.
• Innovative products: I am excited about the cutting-edge technology and solutions the company is developing.

• Strong company culture: I have heard great things about the supportive and collaborative work environment at the company.

• Opportunities for growth: The company's commitment to employee development and career a...read more
Answered by AI
Add your answer