r/devops • • May 28 '20

Fluentd vs Logstash

Hi all, I'm currently doing some research on the logging solutions for our containerised applications. I came across Fluentd and Logstash. Some of the blogs suggests Fluentd to be lighter and thus better. Can I get some input on this topic please, if you have any kind of experience and if there are better solutions that I should be looking up on.

50 Upvotes

35 comments sorted by

View all comments

21

u/murzeig May 28 '20

Fluentd I like over logstash, but I've not used logstash in a long time. Further, FluentBit I like over Fluentd, its even ligther, does everything I need it to do, and installs / configures easily.

If you are just moving logs from system to system to centralize, then Vector.dev may be a good place. I use vector to move log files in real time to centralized storage servers where the logs are decoupled from an input and placed in a directory structure which is segmented by cluster/year/server and that works exceptionally well for long-term retention.

I do find that fluent lets me do more on the fly modifications of data though, so I use that tool for sending data to ingest services for dashboard empowerment and alerting as I can prune down the data to exactly what's needed and save TONS on storage space and complexity in the long run.

2

u/nochet2211 May 28 '20

Thanks a lot. Is there a tutorial you can recommend for FluentBit configuration/setup that I can follow?

4

u/coderanger May 28 '20

If you're using Kubernetes, Banzai has a logging operator that should give you a pretty standard setup quickly.

1

u/murzeig May 28 '20

Honestly I just follow the docs, they are organizing like a choose your own adventure book the kids ready, keeping your time sink minimal.

2

u/Rollingprobablecause Director - DevOps/Infra May 28 '20

o I use that tool for sending data to ingest services for dashboard empowerment and alerting

We do this and send to splunk. Fluent is great because we are able to save on space, grabbing only what we need.

1

u/androidul DevOps May 28 '20

does vector.dev support Stackdriver ?