-
Notifications
You must be signed in to change notification settings - Fork 3.6k
Improve Kafka event publishing reliability #24350
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: rel-10.0
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR enhances Kafka event publishing reliability by implementing stronger guarantees for message delivery and adding validation to detect failures. The changes focus on ensuring that published events are actually persisted to Kafka before considering the operation successful.
Key Changes:
- Configured Kafka producers with
Acks.AllandEnableIdempotence=trueto prevent message loss and duplicates - Added
PersistenceStatusvalidation after publishing events, throwing exceptions on failure instead of silently continuing - Changed from fire-and-forget
Produce()to awaitedProduceAsync()for proper async/await patterns and error detection
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| ProducerPool.cs | Sets default Kafka producer reliability configurations (Acks.All, EnableIdempotence=true) and removes trailing whitespace |
| KafkaDistributedEventBus.cs | Fixes async modifier ordering, validates persistence status after publishing, and changes from synchronous Produce() to asynchronous ProduceAsync() with proper awaiting |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
framework/src/Volo.Abp.EventBus.Kafka/Volo/Abp/EventBus/Kafka/KafkaDistributedEventBus.cs
Outdated
Show resolved
Hide resolved
framework/src/Volo.Abp.EventBus.Kafka/Volo/Abp/EventBus/Kafka/KafkaDistributedEventBus.cs
Outdated
Show resolved
Hide resolved
framework/src/Volo.Abp.EventBus.Kafka/Volo/Abp/EventBus/Kafka/KafkaDistributedEventBus.cs
Outdated
Show resolved
Hide resolved
…KafkaDistributedEventBus.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…KafkaDistributedEventBus.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…KafkaDistributedEventBus.cs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Resolves #24340