Fluent Bit is a straightforward tool and to get started with it we need to understand a few concepts involved.
The nature of Fluent Bit is to gather data from an input interface, do buffering and then flush the records through an output. Every input and output interfaces are implemented through a plugin.
When a plugin is loaded, multiple instances of it can be started and in order to instruct the core how to route the records between input and output instances, a tag and a match are required:
- Input plugin instances are tagged
- Output plugin instances defines a match
When a record is buffered it contains a tag, then the router lookup the output instances that have a match for it, then it delivers the data as required.
In order to understand more how it works we will introduce some concepts:
Fluent Bit provides different input plugins to gather information from, some of them just collect operating system metrics while others interact with radio devices or listen from specific protocols data.
When an input plugin is loaded, an internal instance of it is created. An instance can have it own configuration, the settings available are called properties. Input plugins inherit some fixed properties and also the plugin may expose their own too.
Similar to the input plugins, Fluent Bit provides output plugins which are in charge to deliver the data generated by the inputs to some destination, which can be network service or the file system.
When an output plugin is enabled, an internal instance of it is created. This instance is also configurable through it supported properties.
The Engine is the core runtime who schedule to retrieve data from the input plugin or receive notifications from them to collect it. Once it buffers the data, it lookup the destination of the records using the router paths defined at start time, then the records are passed to the right output plugins to deliver the data.
For Fluent Bit, routing means the action to define paths for the data collected. So for each input plugin instance, exists a configuration property called tag. In the other side the output plugin instances are configured with a property called match, which is a rule to match incoming data under a specific tag pattern.