System Design Concerns

Introduction

In this post, I list and groups questions arise in System Design. The questions and answers are extracted from the book System Design Interview - An Insider guide by Alex Xu. The questions are drawn up by me, hinted by materials from the book. The answers are definitely provided in the book. Some concepts, terminology, or keywords from the books are bold. Source from the book are briefly cited for ease of look up.

Questions regarding Service Amount.

Those questions evolve around the concern how to provide increasing requests for service from the product.

1. How to support failover and redundancy?
   • Answer with concept of Horizontal Scale vs Vertical Scale. (Fig 1-20)
   • Implement Vertical Scale with improved hardware.
   • Implement Horizontal Scale with multiple servers/ database servers where each server different services, thus improving redundancy. (See Fig 1-5)
   • Implement Horizontal Scale with multiple backup servers to support failure scenarios. (See Fig 1-4)
   • Implement Load balancer to direct traffic.
   • Implement Cache/Proxy and appropriate caching policies. (Fig 1-11)
2. How to improve service availability over wide geographic area.
   • Answer with concept of Horizontal Scale: replicate database on multiple servers.
   • However, it may not be possible or maybe expensive to replicate data across all locations. Data for certain users may be stored only at certain locations. (See Figure 1-12)
   • Answer with concept of Stateful and Stateless architecture. (Fig 1-13)
3. How to reduce overload, handle multiple requests?
   • Implement Message Queue for asynchronous response.
   • Implement buffer to store not yet handled data.
   • Separate large database using Shard, harshmap data structure. Shard contains isolated database whose inputs are hashed. Consider adding separated shard for highly requested data. (Fig 1-21, Fig-22). See more in Chapter 5
4. How to manage system?
   • Use logs, metrics, counters. Fig(2-23)

Question regarding capacity estimation

1. How to give estimation of total capacity?
   • Estimate the amount of data needed per basic service such as basic message, basic packet, and estimate the number of users
   • Estimate can also be made by considering the target number of users per unit time such as per TTI, per ms, days, etc.

Question on making High Level Design

1. What limitation expected of the product? What scope of the design?
2. How to construct functional architecture.
   • Consider functional architecture that focuses on creating and justify modules based on functions or services. After that, adding interfaces or communication between those modules.
   • Make design choices to spawn all services at start up or initialize service only when needed
   • Identify possible bottlenecks in data flow.
3. How to make improvement and refinement.
   • Suppose all services requested from customers are already delivered in the design, there are still room for improvement.
   • Consider possibility of requirement changes such as added service in future.
   • Consider possibility of more or less demand in output data. How to handle next scale curve? (Chapter 3, Step 4)
   • Logs/Error handling/Metrics may not be included in the requirements, but should be added, for future testing and bug fix.

Question on database design and data handling

1. How to achieve horizontal scaling with database?
   • Consider using hasing, and store data in shards. (Chapter 5)
2. How to achieve fast searching, retrieving data?
3. How to achieve fast reading and high availability readings?
4. How to achieve fast writing and high availability write?
5. What are trade-offs in the design?
   • According to CAP theorem, e for a distributed system to simultaneously provide more than two of these three guarantees: consistency, availability, and partition tolerance. (Fig 6-1).
6. What are choices to handle temporary failures?
7. What are choices to handle permanent failures?
8. What are choices to handle data center outage?