Python - Log Watcher

[aditya-xq]

Log Watcher System Design Interview Question

Company: BrowserStack Source

Problem Statement:

Design and implement a system inspired by the UNIX 'tail -f' command to monitor a log file on a remote server. Your solution will consist of a server-side component that watches and streams log updates and a web-based client to display these updates in real time.

Requirements:

1. Server-side component:

   • Monitors a given log file for updates. The server and the log file reside on the same machine.
   • Capable of streaming log updates to the client.
   • Supports multiple client connections simultaneously.
   • Efficiently retrieves and sends the last 10 lines of a log file that could be several GB in size.
   • Streams only new updates to the client, not the entire log file.
   • The server component can be implemented in a programming language of your choice.

2. Web-based client:

   • Accessible via a URL (e.g., http://localhost/log).
   • Displays the last 10 lines of the log file upon initial load.
   • Continuously updates in real-time without requiring page refreshes.
   • Should not be in a loading state after the initial page load.

Constraints:

• The server should push updates to the clients, emphasizing real-time data transfer.
• Optimize for efficiently retrieving the last 10 lines from a potentially large log file.
• Ensure the solution does not retransmit the entire log file, but only new updates.
• The system should handle simultaneous multiple client connections.
• Post the initial load, the web client should not stay in a loading state or require page refreshes.
• External libraries or tools that provide read or tail-like functionalities are not permitted.

Evaluation Criteria:

Candidates will be evaluated on:

• Code quality and readability.
• Modularity and design principles.
• Testability and the presence of relevant test cases.
• Handling of potential edge cases and system robustness.

Hints/Tips:

Consider challenges like:

• How to efficiently read from the end of a large file.
• Handling potential concurrency issues with multiple clients.
• Strategies for pushing updates from the server to the client.

Submission:

Please provide:

• Source code for both the server-side component and web-based client.
• Any necessary instructions for running and testing the solution.
• A brief design document explaining your choices and potential trade-offs.