Merge pull request #266 from j3-signalroom/265-add-some-best-practice-kafka-producer-configuration-settings

Resolved #265.

Author: j3-signalroom

CHANGELOG.md

@@ -10,6 +10,7 @@ The format is base on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 - Issue [#241](https://github.com/j3-signalroom/kafka_cluster-topics-partition_count_recommender-tool/issues/241)
 - Issue [#246](https://github.com/j3-signalroom/kafka_cluster-topics-partition_count_recommender-tool/issues/246)
 - Issue [#248](https://github.com/j3-signalroom/kafka_cluster-topics-partition_count_recommender-tool/issues/248)
+- Issue [#265](https://github.com/j3-signalroom/kafka_cluster-topics-partition_count_recommender-tool/issues/265)
 
 ### Changed
 - Issue [#247](https://github.com/j3-signalroom/kafka_cluster-topics-partition_count_recommender-tool/issues/247)

CHANGELOG.pdf

@@ -204,10 +204,13 @@ def __init__(self,
                 "sasl.mechanism": "PLAIN",
                 "sasl.username": sasl_username,
                 "sasl.password": sasl_password,
-                "linger.ms": 100,           # Batch messages for efficiency
+                "linger.ms": 100,               # Batch messages for efficiency
                 "compression.type": "lz4",
-                "acks": "all",              # Wait for all replicas
-                "retries": 3
+                "acks": "all",                  # Wait for all replicas
+                "retries": 10,
+                'retry.backoff.ms': 100,
+                'delivery.timeout.ms': 300000,  # Delivery timeout 5 minutes
+                'enable.idempotence': True,     # Prevents duplicate messages on retry
             })
         except Exception as e:
             logging.error(f"Failed to create Kafka producer: {e}")