logspout-fluentd fails to reconnect if it loses connectivity with fluentd

[michaeltravisuk]

We're seeing issues with logspout-fluentd where connectivity is never re-established with fluentd if it is lost, what we see instead is continuous logging of broken pipe messages in stderr, which causes a large spike in CPU activity on that docker host:

2018/01/09 15:07:31 fluentd-adapter: write tcp 127.0.0.1:38830->127.0.0.1:24224: write: broken pipe

We're running logspout in a container on each of our Rancher hosts talking to a separate container running fluentd. This issue requires us to restart the logspout container to re-establish connection. I saw a similar issue raised for the syslog logspout adapter: gliderlabs/logspout#107 in which they raised gliderlabs/logspout#291 to resolve the issue by adding a configurable number of reconnection attempts.

We much prefer the richness of the logging using logspout-fluentd over syslog, so would it be possible to implement a similar fix to the fluentd adapter?

[michaeltravisuk]

Raised #2 to resolve the issue using code from https://github.com/gliderlabs/logspout/blob/master/adapters/syslog/syslog.go.

[pilgrim2go]

Any update on this ticket?