Python - Asynchronous Client Request Processor

[aditya-xq]

Asynchronous Client Request Processor Design Interview Question

Company: Eightfold AI (Source)

Problem Statement:

Design a system that asynchronously processes client requests based on distinct time-based priorities. The system should be able to intake a massive number of requests and prioritize their processing based on the given criteria.

Requirements:

1. Request Intake and Prioritization:

   • The system should intake client requests and classify them into different priority levels.
   • Example Priority Levels:
     • P0: Requests should begin processing within 2 minutes of request initiation.
     • P1: Requests should begin processing within 10 minutes of request initiation.
     • P2: Requests should begin processing within 30 minutes of request initiation.
       (Note: You can define more priority levels if necessary.)

2. Asynchronous Processing:

   • The system should process requests asynchronously, ensuring that higher-priority requests are processed before lower-priority ones.
   • There should be mechanisms in place to manage system resources efficiently, preventing bottlenecks or resource starvation.

3. Monitoring & Alerting:

   • Implement a monitoring system to ensure that requests are processed within their designated priority windows.
   • If any request misses its window, an alert should be generated.

4. Scalability:

   • The system should be scalable to handle a high influx of simultaneous requests.
   • Consider designing components that can be horizontally scaled as the request load increases.

5. Failure Handling:

   • The system should gracefully handle failures, ensuring that in the event of a processing node going down, the requests are redistributed and processed by other available nodes.

Constraints:

• The system should be designed to be fault-tolerant and resilient.
• Focus on ensuring that high-priority requests always meet their processing time requirements.
• Consider the possibility of changing priority criteria in the future (e.g., changing the processing time for P0 requests).

Evaluation Criteria:

Candidates will be evaluated on:

• System design principles and clarity of design components.
• Handling of scalability and high availability.
• Strategies for ensuring priority-based request processing.
• Identification and handling of potential bottlenecks or failure points.
• Thought process behind resource allocation and asynchronous processing.

Hints/Tips:

Consider aspects like:

• Efficient data structures and algorithms to manage and process priority-based requests.
• Integration points with external monitoring and alerting tools.
• Potential use of distributed systems principles to ensure system scalability and reliability.