Skip to content

Commit 1d99fab

Browse files
authored
Merge pull request #99 from crawler-commons/87
Fix #87, similar behaviour re-blocking whether specitying a key or not
2 parents bd4e209 + 6daf49e commit 1d99fab

File tree

1 file changed

+6
-0
lines changed

1 file changed

+6
-0
lines changed

service/src/main/java/crawlercommons/urlfrontier/service/AbstractFrontierService.java

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -595,6 +595,12 @@ public void getURLs(GetParams request, StreamObserver<URLInfo> responseObserver)
595595
return;
596596
}
597597

598+
// already has its fill of URLs in process
599+
if (queue.getInProcess(now) >= maxURLsPerQueue) {
600+
responseObserver.onCompleted();
601+
return;
602+
}
603+
598604
int totalSent =
599605
sendURLsForQueue(
600606
queue, qwc, maxURLsPerQueue, secsUntilRequestable, now, synchStreamObs);

0 commit comments

Comments
 (0)