O(1) scheduler: Complete implementation

[vicLin8712]

O(1) scheduler: Complete implementation

This PR provides the complete O(1) scheduler implementation and serves as the final part of the 3-part stacked PR series.
It integrates all components introduced in the earlier patches and replaces the legacy O(n) linear scheduler with the new ready-queue–based, RR-cursor-based, bitmap-assisted O(1) design.

Feature of O(1) scheduler

• Priority-indexed ready queues
  Each priority level maintains an independent ready queue.

• Bitmap + De Bruijn–based highest-priority lookup
  The scheduler locates the next runnable task in constant time using priority bitmaps and De Bruijn table lookup.

• RR cursor for fair round-robin scheduling
  Each priority queue maintains a cursor to provide O(1) fair scheduling among tasks of the same priority.

• Full integration into the scheduler execution path
  The legacy O(n) priority scanning algorithm is completely replaced by the new O(1) logic; the iteration limit IMAX=500 is removed.

• Idle task fully integrated into new design
  System execution starts in the idle task, which serves as the initial execution context.
  Whenever the idle task yields, control deterministically transitions to the highest-priority runnable task.

Unit tests

Commit: Add unit test suite for RR-cursor scheduler

make sched_cmp
make run

Approach

• A dedicated controller task is created with priority TASK_PRIO_CRIT to orchestrate the entire test process and enforce deterministic sequencing.
• After each state change of the test tasks, the unit test verifies both bitmap correctness and per-priority task-count consistency, ensuring alignment with the ready-queue and priority-bitmask invariants maintained by the O(1) scheduler.
  Task types
• Controller task: Coordinates the test flow, triggers all state transitions, and validates ready-queue invariants after each step.
• Delay task: A runnable task that transitions into TASK_BLOCKED through mo_task_delay().
  Used to verify dequeue behavior and correct clearing of priority bits when a task leaves the schedulable state set.
• Normal task: A simple infinite-loop runnable task that remains schedulable unless externally suspended or cancelled.
  Serves as the primary subject for testing state transitions and enqueue/dequeue correctness.

Verified state points
The following state transitions are validated by checking both ready-queue task counts and bitmap updates after each operation:

• Normal task state transitions

  • Creation (TASK_READY) – initial enqueue and priority bit set.
  • Priority change – priority migration updates to queue placement and the corresponding bitmap bit.
  • Suspension (TASK_READY → TASK_SUSPEND) – dequeued from the ready queue and priority bit cleared.
  • Resumption (TASK_SUSPEND → TASK_READY) – re-enqueued with correct priority placement.
  • Cancellation (TASK_READY → TASK_CANCELLED) – removed from ready queues and all bitmap bits fully cleared.

• Blocked task behavior (TASK_RUNNING → TASK_BLOCKED)

  • The delay task is created and its priority is promoted to match the controller task’s priority (TASK_READY).
  • After the controller yields, the delay task becomes the running task, invokes mo_task_delay(), and transitions to TASK_BLOCKED.
  • Control returns to the controller task, and the test verifies:
    • the delay task is completely removed from the ready queue
    • its priority bit is cleared from the bitmap
    • scheduler selection falls back to the highest remaining runnable task

Results

Linmo kernel is starting...
Heap initialized, 130005992 bytes available
idle id 1: entry=80001900 stack=80004488 size=4096
task 2: entry=80000788 stack=80005508 size=4096 prio_level=4 time_slice=5
Scheduler mode: Preemptive
Starting RR-cursor based scheduler test suits...

=== Testing Bitmap and Task Count Consistency ===
task 3: entry=80000168 stack=80006634 size=4096 prio_level=4 time_slice=5
PASS: Bitmap is consistent when TASK_READY
PASS: Task count is consistent when TASK_READY
PASS: Bitmap is consistent when priority migration
PASS: Task count is consistent when priority migration
PASS: Bitmap is consistent when TASK_SUSPENDED
PASS: Task count is consistent when TASK_SUSPENDED
PASS: Bitmap is consistent when TASK_READY from TASK_SUSPENDED
PASS: Task count is consistent when TASK_READY from TASK_SUSPENDED
PASS: Bitmap is consistent when task canceled
PASS: Task count is consistent when task canceled
task 4: entry=80000178 stack=80006634 size=4096 prio_level=4 time_slice=5
PASS: Task count is consistent when task canceled
PASS: Task count is consistent when task blocked

=== Test Results ===
Tests passed: 12
Tests failed: 0
Total tests: 12
All tests PASSED!
RR-cursor based scheduler tests completed successfully.

Note

1. The term TASK_CANCELLED in this document is used only for explanation. It is not an actual state in the task state machine, but represents the condition where a task has been removed from all scheduling structures and no longer exists in the system.
2. The task states shown in parentheses (e.g., (TASK_READY)) refer to the state of the test tasks being created or manipulated, not the state of the controller task.

Benchmark

Commit: Add benchmarking files

python3 bench.py

Approach

1. Spawn N=500 normal tasks to populate the scheduling domain.
   All tasks begin in the TASK_READY state, ensuring the ready queues and bitmap are fully populated.
2. Scenario configuration (active ratio)
   For each benchmark scenario, suspend a portion of tasks to reach the desired active-ratio load:

• 2% active
• 4% active
• 20% active
• 50% active
• 100% active

3. Benchmark execution
   To compare the legacy O(n) scheduler with the new O(1) scheduler, a compile-time flag OLD is passed to select which scheduling algorithm is active.
   The original linear-search scheduler is preserved in task.c for baseline measurement.
   For each benchmark scenario, the scheduler is executed 20 times to obtain stable timing data.
   The average and maximum scheduling latencies are collected, and the performance improvement is computed as the ratio between the old and new scheduler times (e.g., 1.5× faster).

4. Metrics collected
   The benchmark collects the following metrics for each scenario:

   • Mean improvement
     Average speedup factor computed as (old_latency / new_latency) across 20 runs.

   • Standard deviation of improvement
     Measures the variability of speedup across repeated runs.

   • Minimum / maximum improvement
     Best and worst observed speedup factors among the 20 runs.

   • 95% confidence interval (CI)
     Statistical confidence bounds for the mean improvement.

   • Mean scheduling latency (old / new)
     Average schedule-selection time for both the legacy O(n) scheduler and the new O(1) scheduler.

   • Maximum scheduling latency (old / new)
     Worst-case schedule-selection time observed for each scheduler.
     Results

Scenario 'Minimal Active':                                                                                                                                                                                                 
  mean improvement        = 2.68x faster                                                                                                                                                                                   
  std dev of improvement  = 0.34x                                                                                                                                                                                          
  min / max improvement   = 1.75x  /  3.35x                                                                                                                                                                                
  95% CI of improvement   = [2.54x, 2.83x]                                                                                                                                                                                 
  mean old sched time     = 5616.25 us                                                                                                                                                                                     
  mean new sched time     = 2119.0 us                                                                                                                                                                                      
  max  old sched time     = 47.0 us 
  max  new sched time     = 37.0 us 

Scenario 'Moderate Active':
  mean improvement        = 1.80x faster
  std dev of improvement  = 0.27x
  min / max improvement   = 1.27x  /  2.51x
  95% CI of improvement   = [1.68x, 1.92x]
  mean old sched time     = 3887.6 us 
  mean new sched time     = 2179.45 us 
  max  old sched time     = 40.0 us 
  max  new sched time     = 23.0 us 

Scenario 'Heavy Active':
  mean improvement        = 1.02x faster
  std dev of improvement  = 0.08x
  min / max improvement   = 0.84x  /  1.17x
  95% CI of improvement   = [0.98x, 1.06x]
  mean old sched time     = 2150.15 us 
  mean new sched time     = 2119.1 us 
  max  old sched time     = 73.0 us 
  max  new sched time     = 33.0 us 

Scenario 'Stress Test':
  mean improvement        = 0.93x (slower than OLD)
  std dev of improvement  = 0.11x
  min / max improvement   = 0.65x  /  1.20x
  95% CI of improvement   = [0.88x, 0.98x]
  mean old sched time     = 1874.35 us 
  mean new sched time     = 2032.55 us 
  max  old sched time     = 23.0 us 
  max  new sched time     = 20.0 us 

Scenario 'Full Load Test':
  mean improvement        = 0.89x (slower than OLD)
  std dev of improvement  = 0.11x
  min / max improvement   = 0.63x  /  1.07x
  95% CI of improvement   = [0.84x, 0.94x]
  mean old sched time     = 1798.8 us 
  mean new sched time     = 2048.55 us 
  max  old sched time     = 33.0 us 
  max  new sched time     = 52.0 us

Reference

#23 - Draft discussion
#36 - Infrustrue
#37 - Task state transitions APIs
ae35c84 - unit test test suite
11e9ee6 - benchmark

---

Summary by cubic

Complete O(1) scheduler with priority queues, bitmap selection, and RR cursors, replacing the legacy O(n) scan. Adds an idle task and updates task lifecycle to use ready queues; up to ~2.7x faster under light load.

• New Features

  • Priority-indexed ready queues with O(1) highest-priority selection via bitmap.
  • Per-priority round-robin cursors for fair rotation without list churn.
  • Scheduler state in kcb (ready_bitmap, ready_queues[], rr_cursors[]) and a dedicated idle task as the safe fallback.
  • Intrusive ready-queue design: TCB embeds rq_node; helpers list_pushback_node() and list_remove_node() manage nodes safely.
  • Unit tests validate bitmap/queue invariants; benchmarks show strong gains at low/moderate activity.

• Refactors

  • Tasks explicitly enqueue/dequeue on READY/RUNNING transitions (spawn, delay/block, suspend/resume, cancel, priority change).
  • Blocking paths use _sched_block_dequeue() and _sched_block_enqueue() for mutex/cond/semaphore to reinsert tasks correctly.
  • Priority changes migrate tasks between queues and yield if the running task changes its priority.
  • Startup launches into the idle task (idle_task_init) and removes the IMAX scan limit.

^{Written for commit 77bb7ae. Summary will update automatically on new commits.}

[jserv]

Do not include numbers in pull-request titles.