Knowledge Base

Release Notes v1.5

Fluent Bit is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of Fluent Bit v1.5.

For people upgrading from previous versions you must read the Upgrading Notes section of our documentation:

https://docs.fluentbit.io/manual/installation/upgrade_notes


Changes

Fluent Bit v1.5 is the next major release and include several improvements:

Network I/O

In the last v1.4 release cycle we exposed an experimental KeepAlive capability for TCP connections, which aims to allow to re-use the TCP channel to help improve performance and reduce overall TLS handshakes on high demand environments. In this new v1.5 release, we have a unified mechanism with extended improvements, now Fluent Bit officially supports:

  • Connect Timeouts
  • Custom Source Address
  • KeepAlive
  • KeepAlive Idle Timeout

Starting from this release, all TCP connections are KeepAlive enabled by default. Please refer to the following Networking section to learn more about these features:

https://docs.fluentbit.io/manual/administration/networking

Monitoring

Understanding how the logging layer is operating is critical, as part of this release we have done two improvements that aims to help our users to troubleshoot and optimize properly:

Storage Layer

Our storage layer that handle all data buffering requirements for memory and file system now exposes an optional HTTP end-point to consume it metrics, so now is possible to measure all pipeline buffers and get a clear status of each component:

  • https://docs.fluentbit.io/manual/administration/monitoring#rest_api

Dashboards and Alerts

We have introduced tips and examples on how to integrate better Grafana and create alerts in your Dashboards:

https://docs.fluentbit.io/manual/administration/monitoring#dashboard-and-alerts

New and updated Input Plugins

Forward

The Forward input plugin speaks the Fluentd Forward protocol. In this version we added support for ACK feature to enable at-least-once. It also adds support to handle GZIP compressed payloads.

Docker Events

The Docker Events input plugin allows to retrieve any event generated by Docker engine on any management task of containers. All possible events are listed here.

New and updated enterprise services Output Plugins

On this release cycle, we did a joint work with companies who are contributing with improvements and new connectors for their own services. Primary, we have the official addition of the following output plugins:

In addition the following output plugins has been extended:

Special thanks to Amazon, LogDNA, New Relic, Google, Sumo Logic and 2nd Quadrant for their contributions!

Amazon Credentials and CloudWatch

In Fluent Bit 1.5, AWS adds full support for all standard credential sources:

  • Environment Variables

  • AWS Profile
  • EC2 Instance Role
  • ECS IAM Roles for Tasks
  • EKS IAM Roles for Service Accounts
  • STS Assume Role

These credential sources can be used to sign requests made to Amazon ElasticSearch Service by Fluent Bit’s Elasticsearch plugin. Finally, a CloudWatch Logs plugin was contributed for Amazon CloudWatch Logs which can replace the external Golang plugin launched last year. The new plugin was written in C in the core of Fluent Bit; it is much more performant and efficient that the Golang plugin.

LogDNA

In this version we are announcing the new certified connector for LogDNA service called LogDNA Output Plugin. This plugin is fully compatible with the main service and also with IBM Log Analysis.

The plugin comes with Auto Enrichment and Data discovery features.

New Relic

New Relic integration is not new to Fluent Bit, the company already ships their own Go connector. But this time, we did a joint work and we are bringing a new native/built-in connector for the service. The name of the built-in plugin is nrlogs.

New Relic customers can use any of both certified connectors for now, users can migrate at their own pace.

PostgreSQL

The updated PostgreSQL connector now support both asynchronous and synchronous connections. It also adds support for .pgpass that allows clients to store database credentials outside of the main Fluent Bit configuration.

The plugin also comes with granular log_level support.

Google Stackdriver

Now you can configure resource types for Kubernetes containers, pods and nodes, so you can search logs by the corresponding Kubernetes resource.

You can add an associated operation as a special field in a record and the Stackdriver output plugin will extract it. For example, if you have a long running machine learning model training operation, you can mark multiple logs with the same operation ID, so you can search logs by the operation ID as a top-level field.

Also you can add labels as a special field in a record and the Stackdriver output will extract them. If your application is deployed in several environments: test, staging and production, you can specify them as with the label “environment,” and you can search logs by the environments as a top-level field to troubleshoot issues for that environment specifically.

Windows Support

Maintainers and overall community have worked hard to get things working properly on Windows, this version comes with exciting enhancements!:

Windows Service Support

Windows Service is a standard interface to run programs in system background, equivalent of daemons on Linux and BSD. Starting from v1.5, now is possible to register Fluent Bit as a Windows Service.

Windows Event Log Input Plugin

The output format of the data collected by Winlog input plugin has been improved. It now explicit exports UTF-8 encoded fields, and also human-readable values for event types and log messages.

{
    "RecordNumber" : 1394,
    "TimeGenerated": "2020-07-06 05:59:43 +0900",
    "TimeWritten"  : "2020-07-06 05:59:43 +0900",
    "EventType"    : "Information",
    "EventCategory": 0,
    "Channel"      : "system",
    "SourceName"   : "Service Control Manager",
    "ComputerName" : "fluent-bit",
    "Data"         : "",
    "Sid"          : "",
    "Message"      : "The Windows Time service entered the running state."
}

Windows Kubernetes Support

Fluent Bit Kubernetes filter needed some extra features to workaround some networking issues on starting up, in this version we have added two new configuration properties to handle DNS issues on Windows environments: DNS_Retries and DNS_Wait_Time

In addition, the Tail input plugin now handle Tags properly when using the standard file layout in Windows Pod.

Contributors

On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey!

Join us!

We want to hear about you, our community is growing and you can be part of it!, you can contact us at: