KNOWLEDGE BASE
Release Notes v1.9.0
Fluent Bit is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of Fluent Bit v1.9.0.
For people upgrading from previous versions you must read the Upgrading Notes section of our documentation:
https://docs.fluentbit.io/manual/installation/upgrade_notes
Introduction
Fluent Bit v1.9 is the start of the new stable series of the project. During the last months our primary focus has been around scalability and performance, metrics and overall usability.
In the performance area, we have created a new priority queue mechanism to handle the processing order of internal events in the events loop, this lead to scale event more in high
demand scenarios, this is an internal change that will benefit majority of users, no user config changes are needed.
We have been hard working on extending metrics support in Fluent Bit, meaning the input and output metrics plugins, where now is possible to perform end-to-end metrics collection
and delivery. We fully support Prometheus & OpenMetrics and we are also shipping experimental OpenTelemetry metrics support (spoiler: traces will come shortly!). A new Nginx Metrics scraper plugin
is part of this release that supports Nginx OSS and Nginx+ enterprise edition.
In the Loggin side, we have a new experimental Kafka input plugin, enhanced performance for Tail, a new Windows winevtlog that supports non-classic channels, new OpenSearch output plugin
and a hot new Apache Arrow encoding support for our S3 output.
Below you will find a more complete summary of the changes on this release.
> note that not all commits or specific changes are listed, for full details please refer to the GIT history
News
Fluent Bit v1.9.0 is the stable release!, new changes on this version:
-
Core
- output_thread: fix shutdown message byte size for 32 bits
- output_thread: fixed multiple initialization of local_thread_instance in emulated TLS
- base64: fork mbedtls 2.25.0 base64 utility to avoid 2.26.0+ performance hit
- signv4: fix NULL dereference
- input: re-architect coroutines messaging
- input: initialize cmetrics to zero
- bucket_queue: implement priority bucket queue for priority event loop
- metrics_exporter: use flb_hs_push_health_metrics when health check is registered
- metrics_exporter: new cmetrics context exporter
- random: use getentropy if available.
- network : enable transport layer protocol selection for dns
- input chunk: fix linkage error when storage backlog is disabled
- input chunk: correctly calculate the difference when chunks grow (#4318)
- plugin_proxy: use new flb_event_chunk structure
- winevtlog: add new
winevtlog input plugin (#4179)
- output: allow to define a number of workers
- output: initialize cmetrics to zero
- output: initialize network defaults for output instances (#4050) (#4088)
- output: added multithreaded worker support for proxy plugins
- output: don’t return uninitialized variable
- hash: add new API to check if hash exists
- event: expose new FLB_EVENT_TYPE_ macro
- event: new ‘flb_event_’ interface
- network: properly react to being awoken due to a timeout
- network: preserve addrlinfo list release address
- network: added dns result priorization option
- network: fixed async usage when not in a coroutine error
- network: added dns context to hold active and drop lists
- network: added async DNS UDP timeout management
- network: refactored the async DNS client
- pack_gelf: make time precision explicit(#3727)
- pack_gelf: ensure proper cleanup in case of failure.
- lua: Add test module for lua util functions
- health: use input coroutine flag to correctly use async network I/O
- ra: allow slash character (#4370)
- ra: allow dash character as an IDENTIFIER(#4236)
- core: integrate priority event loop
- ml_parser_cri: add missing arg skip_empty
- aws_util: support auto_retry_requests for failed connections
- aws_util: added index recognition for flb_get_s3_key
- tls: improve TLS handshake error message (#4561)
- tls: Load system certificates on Windows
- tls: openssl: fix error handling for OpenSSL API #4098 (#4100)
- installation: add script to install in one liner (#4740)
- plugin: add missing char cast
- plugin: add support for config format
- config_map: support deprecated property
- filter: initialize cmetrics to zero
- filter: allow case-insensitive filter name
- codeowners: update to include packaging directory
- ci: update the yaml configuration file for the smoke test.
- regex: log allocation failures in Onigmo
- git: add clangd files to .gitignore (#4299)
- custom: add new ‘custom’ plugin type support
- upstream: avoid dns timeout null event injection (#4956)
- upstream: replaced timeout wakeup mechanism
- upstream: add new option ‘net.connect_timeout_log_error’ (#4473)
- upstream: added dns result priorization option
- upstream: added dns resolver configuration switch
- upstream: detect shutdown and reduce log noise
- sha512: use openssl implementation if available (#4303)
- in stdin: Initialize memory to 0
- engine: unify flush protocol for thread and engine
- engine: add flb_mp_count_remaining to flb_mp
- engine: fix linkage error when storage backlog is disabled
- engine: do not retry if the engine is shutting down
- engine: enforce and reduce shutdown time
- engine: dispatch: update task by using new API
- ra_key: reverse order of map lookup when extracting values
- ra_key: check subkeys length (#3896)
- event_loop: implement priority event loop
- sds: fix missing length set when using flb_sds_snprintf()
- sds: new flb_sds_trim() function
- oauth2: release upstream connection on exit (#4095)
- ml: support FLB_ML_TYPE_MAP to parse (#4034)
- mp: add new function to validate metric chunks
- str: use flb_strndup to implement flb_strdup (#4302)
- error: register custom invalid error
- input_chunk: warn the user about tag truncation due to limits (#4467)
- input_chunk: don’t pause ingestion if filesystem storage is enabled (#4221) (#4411)
- mem: fix alloc_size attribute and remove extra ifs (#4305)
- pack: added java_sql_timestamp format (#4811)
- pack: ensure minimum allocation is 256 bytes (unit test)
- pack: enhance memory allocation strategy for large JSON payloads
- pack: only count records when packing plain JSON
- file: add new utility module
- utils: expose missing flb_utils_get_os_name()
- utils: lowercase text errors
- utils: detect and replace ill-formed utf-8 bytes (#4346)
- utils: fix return value handling of uuid generator
- utils: new machine-id reader and OS detector
- parser: support
skip_empty_values (#1486) If enabled, the parser ignores empty value of the record.
- parsers: add rabbitmq neo4j and dns custom parsers (#4324)
- conf: set default flush intervals to ‘1’
- cmake: add support for jemalloc page size config (#5020)
- cmake: Enable CMAKE_EXPORT_COMPILE_COMMANDS
- storage: don’t ingest metrics when storage.metrics is off
- storage: added extra argument to flb_sched_timer_cb_create call
- config: set default flush interval to ‘1’
- config: support lowercase proxy environment variables
- config: added dns result priorization option
- config: increase coro stack size if it is less than pagesize (#3716) (#4434)
- config: rename coro stack size define
- config: added dns resolver configuration switch
- config: add new ‘is_shutting_down’ field
- config: add new ‘grace_count’ structure field
- io: check connection error only on net_io_read
- io: fixed a bug in net_io_write_async
- io: keep u_conn->coro as NULL unless awaiting event
- aws: create shared compression utility restructure aws cmake to make maintaining nested directories easier
- aws: Add imds tests and mock flb_aws_client via flb_aws_client_mock struct
- aws: Include IMDSv2 support for EC2 instances
- aws: util: set max response buffer size for the http client (#4047)
- aws: Rename struct process stdin,stdout,stderr to fix build
- http_client: fix wrong host header when using proxy (#4810)
- http_client: remove ability to see HTTP request for non-dev env
- task: process direct routes if they are set
- task: use new flb_event_chunk structure
- windows: build legacy td-agent-bit package (#4862)
- windows: cmake: Add version information metadata with version.rc (#4531)
-
Binary (fluent-bit)
- bin: on ‘-J’, print all JSON config schema
- bin: add support for ‘custom’ plugin help
- bin: report nightly build info if found
- bin: register config format env variables
- bin: do not overwrite context on static config
- bin: add support for static config
- bin: release config file optarg and always stop the context
- bin: use config_format to compose pipeline from the command line
- bin: read configuration by using new config_format API
- bin: exit with status code
- bin: load ‘custom’ plugins with ‘-C’
- bin: add SIGFPE handler
-
Libraries
- lib: cmetrics: upgrade to v0.3.0
- lib: libbacktrace: upgrade to 2446c66
- lib: monkey: fix priority initialization for kqueue backend
- lib: xxhash: upgrade to v0.8.1
- lib: chunkio: upgrade to v1.1.6
- lib: mbedtls: upgrade from v2.26 to v2.27
-
Plugins
Contributors
On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey!
Join us
We want to hear about you, our community is growing and you can be part of it!, you can contact us at: