LTIMindtree Junior Data Analyst Interview Questions and Answers

SCD (Slowly Changing Dimensions) manages historical data changes in data warehouses.

• SCD Type 1: Overwrite old data (e.g., updating a customer's address without keeping history).

• SCD Type 2: Create new records for changes (e.g., adding a new row for a customer's address change).

• SCD Type 3: Store current and previous values in the same record (e.g., adding a 'previous address' column).

• Implementation can be done using ETL ...read more

Handling multiple inputs in data sources requires effective integration, transformation, and validation strategies.

• Use ETL (Extract, Transform, Load) processes to consolidate data from various sources.

• Implement data validation checks to ensure data quality from each input source.

• Utilize data orchestration tools like Apache Airflow to manage workflows and dependencies.

• Consider using a message queue (e.g., Kafka) for rea...read more

Questions on big data, Hadoop, Spark, Scala, Git, project and Agile.

• Hadoop architecture and HDFS commands for copying and listing files in HDFS

• Spark architecture and Transformation and Action question

• What happens when we submit a Spark program

• Spark DataFrame coding question

• Scala basic program on List

• Git and Github

• Project-related question

• Agile-related

Working with nested JSON using PySpark involves using the StructType and StructField classes to define the schema and then using the select function to access nested fields.

• Define the schema using StructType and StructField classes

• Use the select function to access nested fields

• Use dot notation to access nested fields, for example df.select('nested_field.sub_field')

Implementing SCD2 involves tracking historical changes in data over time.

• Identify the business key that uniquely identifies each record

• Add effective start and end dates to track when the record was valid

• Insert new records with updated data and end date of '9999-12-31'

• Update end date of previous record when a change occurs

Use a SQL query to select data from table 2 where data exists in table 1

• Use a JOIN statement to link the two tables based on a common column

• Specify the columns you want to select from table 2

• Use a WHERE clause to check for existence of data in table 1

The number of records retrieved after performing joins depends on the type of join - inner, left, right, or outer.

• Inner join retrieves only the matching records from both tables

• Left join retrieves all records from the left table and matching records from the right table

• Right join retrieves all records from the right table and matching records from the left table

• Outer join retrieves all records from both tables, filling