-
Notifications
You must be signed in to change notification settings - Fork 5
Description
asyncio has been introduced to reduce the complexity of the scheduler – and to avoid many deadlocks. To finish the transitions, the following would be needed:
- Job dependencies should be handled directly through asyncio futures; a job basically loops over all its dependencies and wait for them to be finished (instead of relying on asyncio-based updates)
- (most important) Token dependencies should also be handled this way, since this would prevent current (rare) deadlocks
Roadmap:
-[x] SchedulerCentral should have a unique instance with a event loop that manages all the jobs – and we should merge SchedulerCentral into Scheduler (since they will mostly do the same thing). This will make the distinction between experiment and scheduler clearer: the experiment uses a unique scheduler;
-[ ] Use asyncio for the locking mechanism of tokens
-[ ] Create a TokenResourceLocker that handles the task of locking all the tokens at once
-[ ] Starting jobs should be directly done in Job
Job handling
Job launching sequence:
- (to adapt) Checking that the task is not already registered in a scheduler (otherwise, return the scheduled job);
- (to adapt) Check that the job is not already done or running
- Wait that all job dependencies are completed (simple loop over futures) – instead of monitor the jobs.
- Lock all the token dependencies (if one lock fails, just put this token first and start again)
- (to adapt) Create the directory and other resources needed to actually run the script.
- Run and monitor
- Resolve the future
Sub-issues
Metadata
Metadata
Assignees
Labels
No labels
Type
Projects
Status
Done