February 14, 2021
Fluent Bit is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of Fluent Bit v1.7.
For people upgrading from previous versions you must read the Upgrading Notes section of our documentation:
https://docs.fluentbit.io/manual/installation/upgrade_notes
The Fluent ecosystem keeps growing and now we are bringing the best of Fluentd and Fluent Bit! More information click the banner below:
Fluent Bit v1.7 is the next major release!, here you get the exciting news:
Most of users are very pleased with the high performance of Fluent Bit at a low cost of CPU and Memory. Our event-driven architecture and optimized handling of I/O through co-routines and buffering techniques is more than sufficient for most of cases. But running at high scale a single-thread program might face some limits since we were using a single CPU, and after working hard with our Cloud Providers partners like Google and Amazon, now we are pleased to introduce our new multi-thread architecture support: 5x times faster throughput!.
Running in multiple threads allows us to scale to multiple CPU and scale data processing and delivery at higher levels. This new mechanism can be configured at start time and can be enabled optionally per output connector (plugin). Now output connectors can run in multi thread mode using the new option workers
. Example:
[OUTPUT]
name http
host 192.168.3.4
port 443
tls on
format json_lines
workers 4
The example above enable 4 workers
for the connector, so every data delivery procedure will run independently in a separate thread, further connections are balanced in a round-robin fashion.
The multithread implementation is lock-free at runtime. One tip: if you are curious about the improvement, just try setting workers 1
in your configuration, that option will force the engine to spawn a dedicated thread for data delivery, but note that this thread also can handle hundred of I/O requirements.
Since day zero Fluent Bit supported TLS/SSL through the mbedTLS library. The library is very lightweight and gets the job done. Doing more performance research we found that replacing mbedTLS with OpenSSL will bring even more performance improvements due to it optimized crypto routines. Starting from Fluent Bit v1.7 we provide an abstracted layer for TLS/SSL handling. Now OpenSSL is the preferred option for performance and ecosystem compatibility.
The are two major enhancements on this version, now we provide a new configuration property called offset_key
which optionally allows to append a key with the file offset to each record, and the known option ignore_older
when enabled, now allows to skip unmodified files at start time and also stop monitoring files that have not modified in the interval of time set. This last property behavior change helps to reduce significantly the number of open file descriptors.
We are including a new beta plugin that allows to receive data over HTTP protocol. This input plugin is fully functional and we expect extending it capabilities over the 1.7.x release cycle.
Enrich your records with geoip2 data!. If you have records that contains IP addressed and need a country reference, this is the filter for you.
We have introduced a new native Websocket output plugin.
There are tons of other imrpovements and fixes on this major release, here is a list of the most relevant ones:
Core
Time_Strict
(#2813)filter_rewrite_tag
Libraries
On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey!
We want to hear about you, our community is growing and you can be part of it!, you can contact us at:
Check out the Release Notes, read the Updated Documentation or jump directly to the Downloads Section.
We are part of a wide community, no vendor lock-in.