Perf: Limit concurrency for parallel requests #26
Description
A file may contain a large number of chunks.
To prevent overwhelming system resources, utilize tokio::sync::Semaphores to limit concurrency. A semaphore maintains a set of permits, and a task must acquire a permit from the semaphore before proceeding.
// Declare number of permits
let semaphore = Arc::new(Semaphore::new(max_concurrent_tasks));
// Before task starts
let permit = semaphore.clone().acquire_owned().await.expect("Failed to acquire semaphore permit");
// Release the permit when task finishes
drop(permit);