The Grep Filter plugin allows to match or exclude specific records based in regular expression patterns.
The plugin supports the following configuration parameters:
|Regex||FIELD REGEX||Keep records which field matches the regular expression.|
|Exclude||FIELD REGEX||Exclude records which field matches the regular expression.|
In order to start filtering records, you can run the filter from the command line or through the configuration file. The following example assumes that you have a file called lines.txt with the following content
aaa aab bbb ccc ddd eee fff ggg
Note: using the command line mode need special attention to quote the regular expressions properly. It's suggested to use a configuration file.
The following command will load the tail plugin and read the content of lines.txt file. Then the grep filter will apply a regular expression rule over the log field (created by tail plugin) and only pass the records which field value starts with aa:
$ bin/fluent-bit -i tail -p 'path=lines.txt' -F grep -p 'regex=log aa' -m '*' -o stdout
[INPUT] Name tail Path lines.txt [FILTER] Name grep Match * Regex log aa [OUTPUT] Name stdout Match *
The filter allows to use multiple rules which are applied in order, you can have many Regex and Exclude entries as required.