Skip to main content

etcd

BETA

This component is mostly stable but breaking changes could still be made outside of major version releases if a fundamental problem with the component is found.

Configures an etcd event watcher on a key or prefix.

Introduced in version 1.2.0.

# Common config fields, showing default values
input:
  label: ""
  etcd:
    endpoints: [] # No default (required)
    auto_sync_interval: "" # No default (optional)
    key: "" # No default (required)
    options:
      with_prefix: false
      with_progress_notify: false
      with_put_filter: false
      with_delete_filter: false
      with_created_notify: false
      with_range: ""
    auto_replay_nacks: true

Watches an etcd key or prefix for new events.

From the etcd docs:

The Watch API provides an event-based interface for asynchronously monitoring changes to keys. An etcd watch waits for changes to keys by continuously watching from a given revision, either current or historical, and streams key updates back to the client.

Watches are long-running requests and use gRPC streams to stream event data. A watch stream is bi-directional; the client writes to the stream to establish watches and reads to receive watch events. A single watch stream can multiplex many distinct watches by tagging events with per-watch identifiers. This multiplexing helps reducing the memory footprint and connection overhead on the core etcd cluster.

Events​

Each event object is flattened and returned as an array, with each individual event in the form:

{
  "key": "<string or []byte>",
  "value": "<string or []byte>",
  "type": "<'PUT' or 'DELETE'>",
  "version": "<int64>",
  "mod_revision": "<int64>",
  "create_revision": "<int64>",
  "lease": "<int64>"
}

Where a key or value is only a string if it is valid UTF-8.

Fields​

endpoints​

A set of URLs (schemes, hosts and ports only) that can be used to communicate with a logical etcd cluster. If multiple endpoints are provided, the Client will attempt to use them all in the event that one or more of them are unusable.

Type: array

# Examples

endpoints:
  - etcd://:2379

endpoints:
  - etcd://localhost:2379

endpoints:
  - etcd://localhost:2379
  - etcd://localhost:22379
  - etcd://localhost:32379

auth​

Optional configuration of etcd authentication headers.

Type: object

auth.enabled​

Whether to use password authentication

Type: bool
Default: false

auth.username​

The username to authenticate as.

Type: string
Default: ""

auth.password​

The password to authenticate with.

Secret

This field contains sensitive information that usually shouldn't be added to a config directly, read our secrets page for more info.

Type: string
Default: ""

dial_timeout​

Timeout for failing to establish a connection.

Type: string
Default: "5s"

keep_alive_time​

Time after which client pings the server to see if transport is alive.

Type: string
Default: "5s"

keep_alive_timeout​

Time that the client waits for a response for the keep-alive probe. If the response is not received in this time, the connection is closed.

Type: string
Default: "1s"

request_timeout​

Timeout for a single request. This includes connection time, any redirects, and header wait time.

Type: string
Default: "1s"

tls​

Custom TLS settings can be used to override system defaults.

Type: object

tls.enabled​

Whether custom TLS settings are enabled.

Type: bool
Default: false

tls.skip_cert_verify​

Whether to skip server side certificate verification.

Type: bool
Default: false

tls.enable_renegotiation​

Whether to allow the remote server to repeatedly request renegotiation. Enable this option if you're seeing the error message local error: tls: no renegotiation.

Type: bool
Default: false
Requires version 1.0.0 or newer

tls.root_cas​

An optional root certificate authority to use. This is a string, representing a certificate chain from the parent trusted root certificate, to possible intermediate signing certificates, to the host certificate.

Secret

This field contains sensitive information that usually shouldn't be added to a config directly, read our secrets page for more info.

Type: string
Default: ""

# Examples

root_cas: |-
  -----BEGIN CERTIFICATE-----
  ...
  -----END CERTIFICATE-----

tls.root_cas_file​

An optional path of a root certificate authority file to use. This is a file, often with a .pem extension, containing a certificate chain from the parent trusted root certificate, to possible intermediate signing certificates, to the host certificate.

Type: string
Default: ""

# Examples

root_cas_file: ./root_cas.pem

tls.client_certs​

A list of client certificates to use. For each certificate either the fields cert and key, or cert_file and key_file should be specified, but not both.

Type: array
Default: []

# Examples

client_certs:
  - cert: foo
    key: bar

client_certs:
  - cert_file: ./example.pem
    key_file: ./example.key

tls.client_certs[].cert​

A plain text certificate to use.

Type: string
Default: ""

tls.client_certs[].key​

A plain text certificate key to use.

Secret

This field contains sensitive information that usually shouldn't be added to a config directly, read our secrets page for more info.

Type: string
Default: ""

tls.client_certs[].cert_file​

The path of a certificate to use.

Type: string
Default: ""

tls.client_certs[].key_file​

The path of a certificate key to use.

Type: string
Default: ""

tls.client_certs[].password​

A plain text password for when the private key is password encrypted in PKCS#1 or PKCS#8 format. The obsolete pbeWithMD5AndDES-CBC algorithm is not supported for the PKCS#8 format. Warning: Since it does not authenticate the ciphertext, it is vulnerable to padding oracle attacks that can let an attacker recover the plaintext.

Secret

This field contains sensitive information that usually shouldn't be added to a config directly, read our secrets page for more info.

Type: string
Default: ""

# Examples

password: foo

password: ${KEY_PASSWORD}

auto_sync_interval​

The interval to update endpoints with its latest members. 0 disables auto-sync. By default auto-sync is disabled.

Type: string

max_call_send_msg_size​

The client-side request send limit in bytes. If 0, it defaults to 2.0 MiB (2 * 1024 * 1024).

Type: int

max_call_recv_msg_size​

The client-side response receive limit. If 0, it defaults to math.MaxInt32.

Type: int

reject_old_cluster​

When set, will refuse to create a client against an outdated cluster.

Type: bool
Default: false

permit_without_stream​

When set, will allow client to send keepalive pings to server without any active streams (RPCs).

Type: bool
Default: false

max_unary_retries​

The maximum number of retries for unary RPCs.

Type: int

backoff_wait_between​

The wait time before retrying an RPC.

Type: string

backoff_jitter_fraction​

The jitter fraction to randomize backoff wait time.

Type: float

key​

The key or prefix being watched. For prefix watching, options.with_prefix should be true

Type: string

options​

Collection of options to configure an etcd watcher.

Type: object

options.with_prefix​

Whether to watch for events on a prefix.

Type: bool
Default: false

options.with_progress_notify​

Whether to send periodic progress updates every 10 minutes when there is no incoming events.

Type: bool
Default: false

options.with_put_filter​

Whether to discard PUT events from the watcher.

Type: bool
Default: false

options.with_delete_filter​

Whether to discard DELETE events from the watcher.

Type: bool
Default: false

options.with_created_notify​

Whether to send CREATED notify events to the watcher.

Type: bool
Default: false

options.with_range​

Will cause the watcher to return a range of lexicographically sorted keys to return in the form [key, end) where end is the passed parameter.

Type: string
Default: ""

auto_replay_nacks​

Whether messages that are rejected (nacked) at the output level should be automatically replayed indefinitely, eventually resulting in back pressure if the cause of the rejections is persistent. If set to false these messages will instead be deleted. Disabling auto replays can greatly improve memory efficiency of high throughput streams as the original shape of the data can be discarded immediately upon consumption and mutation.

Type: bool
Default: true