Skip to main content

file

EXPERIMENTAL

This component is experimental and therefore subject to change or removal outside of major version releases.

Performs operations (read, write, delete, move, rename, stat) on files.

# Common config fields, showing default values
label: ""
file:
  operation: "" # No default (required)
  path: /tmp/data.txt # No default (required)
  destination_path: /tmp/backup/${! json("document.id") }.txt # No default (optional)

This processor allows you to perform various file operations based on message content. The operation is specified by the 'operation' field, and paths can be dynamically generated using interpolation functions.

Operations​

  • read: Read content at 'path' into the message
  • write: Write message content to 'path'
  • delete: Delete file at 'path'
  • move: Move a file at 'path' to 'destination_path'
  • rename: Rename/move a file at 'path' to 'destination_path' using os.Rename.
  • stat: Get file information from 'path' without reading content

move vs rename​

You may want to use the rename operation instead of using move and therefore avoid copying bytes. However you should be aware that there are OS-specific restrictions with this approach and it will not work across different filesystems.

Metadata​

When reading or getting file info (stat), this processor adds the following metadata fields:

- file_path: The path of the file
- file_size: The size of the file in bytes
- file_mod_time_unix: File modification time as Unix timestamp
- file_mod_time: File modification time in RFC3339 format
- file_name: The name of the file
- file_is_dir: Whether the file is a directory (true/false)
- file_mode: File permissions and mode

Fields​

operation​

The file operation to perform.

Type: string
Options: read, write, delete, move, rename, stat.

path​

The path used for reads, writes, deletes and stat. Source path for move and rename. This field supports interpolation functions.

Type: string

# Examples

path: /tmp/data.txt

path: /tmp/${! json("document.id") }.txt

destination_path​

The destination path for 'move' and 'rename' operations. This field supports interpolation functions.

Type: string

# Examples

destination_path: /tmp/backup/${! json("document.id") }.txt

scanner​

The scanner to use for reading files.

Type: scanner