Skip to main content

file_tail

EXPERIMENTAL

This component is experimental and therefore subject to change or removal outside of major version releases.

Tails a file

Introduced in version 1.15.0.

# Common config fields, showing default values
input:
  label: ""
  file_tail:
    path: ./folder/file.log # No default (required)
    poll_interval: 1s
    start_position: start
    auto_replay_nacks: true

Reads lines from a file continuously with handling of log rotation.

Metadata​

This input adds the following metadata fields to each message:

- file_tail_path 
- file_tail_position

You can access these metadata fields using function interpolation.

Fields​

path​

Path to file to consume lines from.

Type: string

# Examples

path: ./folder/file.log

poll_interval​

The time between checks for new lines.

Type: string
Default: "1s"

# Examples

poll_interval: 1s

poll_interval: 200ms

line_buffer​

Number of lines to buffer internally before backpressure is applied.

Type: int
Default: 1000

start_position​

Where to begin reading the file from. start will consume all existing lines in the file, end will only consume new lines after input has started.

Type: string
Default: "start"
Options: start, end.

auto_replay_nacks​

Whether messages that are rejected (nacked) at the output level should be automatically replayed indefinitely, eventually resulting in back pressure if the cause of the rejections is persistent. If set to false these messages will instead be deleted. Disabling auto replays can greatly improve memory efficiency of high throughput streams as the original shape of the data can be discarded immediately upon consumption and mutation.

Type: bool
Default: true