The key to make include_lines work is to understand that (1) Filebeat uses its own set of regular expressions and (2) you should match the whole line.

Regarding Filebeat’s own regular expressions you can go here . To test if your regular expressions work you can try them out here. Beware, you should match the whole line! (if you use regexr.com make sure that the whole line that you are testing is highlighted.

For example, in a big message queue, I am interested only in the log lines that contain the word “apache”. The Filebeat.yml for this requirement would be like this:

filebeat.inputs:
– type: log
paths:
– ‘/usr/share/filebeat/my_logs/rsyslog_log/messages’
include_lines: [‘.apache.‘]
output.logstash:
hosts: [‘localhost:5044’]
timeout: 30s

It is important to mention that, if you use include_lines: [‘apache’] it won’t work!