State Street Corporation Production Engineer Interview Questions and Answers

Design a system similar to YouTube for video sharing and streaming

• Implement user authentication and authorization for uploading and viewing videos

• Create a database to store video metadata, user information, and comments

• Develop a recommendation algorithm based on user preferences and viewing history

• Utilize a content delivery network (CDN) for efficient video streaming

• Include features like video monetization, live stream...read more

Kafka is a distributed streaming platform used for building real-time data pipelines and streaming applications.

• Kafka uses a publish-subscribe model for messaging, allowing multiple producers and consumers.

• Topics in Kafka are partitioned for scalability; for example, a topic with 3 partitions can handle 3 consumers simultaneously.

• Kafka ensures message durability through replication; if one broker fails, others can stil...read more

Bigdata systems architecture involves distributed storage, processing, and analysis of large volumes of data.

• Utilize distributed file systems like HDFS for storage

• Use parallel processing frameworks like Apache Spark or Hadoop for data processing

• Implement data pipelines for ETL processes

• Leverage NoSQL databases like Cassandra or MongoDB for real-time data querying

• Consider data partitioning and replication for fault tole...read more

Big data system distribution for storage and compute involves partitioning data across multiple nodes for efficient processing.

• Data is partitioned across multiple nodes to distribute storage and processing load.

• Hadoop Distributed File System (HDFS) is commonly used for storage distribution.

• Apache Spark utilizes a cluster computing framework for distributed computing.

• Data locality is important to minimize data transfer ...read more