Skip to main content

cache

Stores each message in a cache.

# Common config fields, showing default values
output:
  label: ""
  cache:
    target: "" # No default (required)
    key: ${!count("items")}-${!timestamp_unix_nano()}
    max_in_flight: 64

Caches are configured as resources, where there's a wide variety to choose from.

The target field must reference a configured cache resource label like follows:

output:
  cache:
    target: foo
    key: ${!json("document.id")}

cache_resources:
  - label: foo
    memcached:
      addresses:
        - localhost:11211
      default_ttl: 60s

In order to create a unique key value per item you should use function interpolations described here.

Performance​

This output benefits from sending multiple messages in flight in parallel for improved performance. You can tune the max number of in flight messages (or message batches) with the field max_in_flight.

Fields​

target​

The target cache to store messages in.

Type: string

key​

The key to store messages by, function interpolation should be used in order to derive a unique key for each message. This field supports interpolation functions.

Type: string
Default: "${!count(\"items\")}-${!timestamp_unix_nano()}"

# Examples

key: ${!count("items")}-${!timestamp_unix_nano()}

key: ${!json("doc.id")}

key: ${!meta("kafka_key")}

ttl​

The TTL of each individual item as a duration string. After this period an item will be eligible for removal during the next compaction. Not all caches support per-key TTLs, and those that do not will fall back to their generally configured TTL setting. This field supports interpolation functions.

Type: string
Requires version 1.0.0 or newer

# Examples

ttl: 60s

ttl: 5m

ttl: 36h

max_in_flight​

The maximum number of messages to have in flight at a given time. Increase this to improve throughput.

Type: int
Default: 64