v1.3.0

We provides the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD and OSX. We are proud to announce the availability of Fluent Bit v1.3.0.

September 30, 2019

KNOWLEDGE BASE

Release Notes v1.3.0

Fluent Bit is a Fast and Lightweight Data Forwarder for Linux, BSD and OSX. We are proud to announce the availability of Fluent Bit v1.3.0.

For people upgrading from previous versions you must read the Upgrading Notes section of our documentation:

https://docs.fluentbit.io/manual/installation/upgrade_notes

Changes

Fluent Bit v1.3.0 is the next major release and include several improvements:

Stream Processor

Our built-in Stream Processor keeps improving, on this release we have integrated support for linear regression algorithms to provide timeseries functionalities, we expose two new functions that aim to expose Machine Learning capabilities:

  • TIMESERIES_FORECAST()
  • TIMESERIES_FORECAST_R()
New Plugins & Connectors

Fluent Bit community continues contributing improvements and extensions, the following is a summary of the new plugins that are part of v1.3 release:

Input

  • thermal: available on Linux, collect temperature samples from the registered thermal zones in the system.
  • collectd: new collectd UDP service mode that collect metrics from remote collectd services.

Output

  • datadog: new native connector that ship records to Datadog service.
Kubernetes Filter

A major improvement has landed to our Kubernetes Filter. The way to extract docker id and container id has been improved, annotation handling has been extended, now is possible to exclude records per stream or container name. For cases where the metadata cannot be obtained now the filter will add at least the metadata extracted from the local file name.

Prometheus Metrics

The exposed Prometheus metrics, now are sorted and have the proper TYPE labels for easy processing on the Prometheus server side.

Performance & Tools

Performance is usually a complex topic due to diverse scenarios of environments and data loads. When performing benchmarks is important to have proper tooling, so we have started a sub-project that provides simple tooling for testing:

List of general changes

Core

  • io: tls: support SNI routing for outputs (#1313)
  • utils: fix integer size to avoid overflow on 32 bits (#1328)
  • parser: decoder: json: do not unescape and skip empty spaces (#1278)
  • scheduler: fix backoff jitter calculation (#798 #670 #649)
  • conf: parsers: docker: comment out decoders, not longer required on k8s
  • regex: remove onigmo data types from regex context
  • input: new ‘routable’ option for input plugins
  • input: chunk: safe chunk creation validating file up/down
  • config: add new storage option ‘storage.max_chunks_up’
  • core: allow use of relative paths in configuration files on Windows
  • compat: add realpath(3) emulation for Windows
  • compat: introduce FLB_DIRCHAR
  • upstream: always do a reset of connection event structure
  • oauth2: fix to try ipv6 when getting upstream connection
  • unescape: utf-8: do not skip byte if character cannot be encoded (#1278)
  • lib: msgpack: upgrade from 3.1.1 to 3.2.0

Stream Processor

  • add @record.time() function for conditionals
  • add support for sub-key selection in ‘group by’ statement
  • parser: support keys that starts with an underscore
  • add record-specific functions (@record.contains) (#1369)

Storage

  • set default storage.max_chunks_up to 128
  • initialize using new API content loader

Plugins

  • Systemd (Input)
    • Workaround for systemd tail/skip issue 9934 (#1565)
    • Validate if flb_sqldb is set
  • Lib (Input)
    • Destroy packer state on exit
  • Winlog (Input)
    • Listen to ‘application’ channel by defAult
  • Tail (Input)
    • Prevent file duplication on macOS (#1596)
    • Fix conditionals in case ‘file’ becomes null on exception (cid 251480)
    • Prevent wrong memory accesses and memory leaks on exit
    • Improve performance (3x) when database is enabled
    • Optional compile-time dependency on SQLlite (#1472)
  • Tcp (Input)
    • New ‘format’ and ‘separator’ options to allow raw text
  • Cpu (Input)
    • New ‘pid’ option to monitor a specific process
  • Syslog (Input)
    • Fix protocol parser for concatenated messages
    • Add ‘unix_perm’ option to specify socket file permission (#1554)
  • Thermal (Input)
    • Add temperature input plugin for Linux (#1509)
  • MQTT (Input)
    • Fix ping response
    • Properly encode packet identifier
  • Collectd (Input)
    • Implement a new input plugin for collectd (#1506)
  • Docker (Input)
    • New Docker container metrics collector (original pr #790)
  • Serial (Input)
    • Fix leak on failure plugin initialization (cid 251471)
  • Grep (Filter)
    • Add error messages for troubleshooting
  • Kubernetes (Filter)
    • Strict validation of file descriptor (cid 184257)
    • Always add meta extracted from the record itself
    • Add per stream and per container exclude annotation support
    • Fix parser annotation handling
    • Fix container docker id and image id extraction
  • Nest (Filter)
    • Free unused temporary buffer(#1540) (#1541)
    • Use kv interface to query properties
    • Copy original event if condition is not mached(#1434)
    • Use temporal variable for warning message
  • Modify (Filter)
    • Use new regex_find() api and fix group matching (#1433)
    • Copy original event if condition is not matched (#1077 #1416)
  • Datadog (Output)
    • Add new official Datadog output plugin (#1502)
  • TCP (Output)
    • New tcp output plugin with optional TLS support
  • HTTP (Output)
    • Fix gelf multiline, group all records in one payload
    • New ‘compress’ option to compress payload with gzip
  • File (Output)
    • Add support for arbitrary templating (#1480)
  • Lib (Output)
    • Do not append empty space in json record after timestamp (#1396)
  • Influxdb (Output)
    • Add escape for backslash (#1123 #1455)
  • Kafka (Output)
    • Upgrade librdkafka v0.11.6 -> v1.2.0
    • Always call rd_kafka_poll() after sleep to reduce saturation (#923)
  • Bigquery (Output)
    • Use string length to calculate buffer size (cid 185609)
    • Enable ‘ipv6 mode’ support (#761)
  • Forward (Output)
    • Add support for user-based authentication
  • Stackdriver (Output)
    • Add functionality to send log level info to stackdriver (#1418)

Contributors

On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey!

Since the list grows every time, it’s hard to mention every single person involved. In a best effort possible, we are mentioning people who have signed contributions in our GIT project history through commits. We would like to thanks to the following people who have been involved doing coding, bug fixes, documentation and general contributions on this release:

Join us

We want to hear about you, our community is growing and you can be part of it!, you can contact us at:

Latest Version

New release on Mar 20, 2024

Fluent Bit v3.0.0 is out!

Check out the Release Notes, read the Updated Documentation or jump directly to the Downloads Section.

We are part of a wide community, no vendor lock-in.