Skip to main content

aws_bedrock_chat

EXPERIMENTAL

This component is experimental and therefore subject to change or removal outside of major version releases.

Generates responses to messages in a chat conversation, using the AWS Bedrock Converse API.

# Common config fields, showing default values
label: ""
aws_bedrock_chat:
  model: amazon.titan-text-express-v1 # No default (required)
  prompt: "" # No default (optional)
  system_prompt: "" # No default (optional)
  max_tokens: 0 # No default (required)
  temperature: 0 # No default (optional)

This processor sends prompts to your chosen large language model (LLM) and generates text from the responses, using the AWS Bedrock Converse API.

Unlike the aws_bedrock_invoke processor which uses the model-specific InvokeModel API, this processor uses the model-agnostic Converse API, which provides a unified request/response format across all supported models.

Currently only text content is supported for both the input prompt and the model response; you cannot yet send images, documents, or other content blocks.

For more information, see the AWS Bedrock documentation.

Examples​

Send each message as a prompt to an Anthropic model and replace the message with the generated text using the Converse API.

pipeline:
  processors:
    - aws_bedrock_chat:
        model: anthropic.claude-3-5-sonnet-20241022-v2:0
        system_prompt: You are a helpful assistant.
        max_tokens: 1024
        region: us-east-1

Fields​

model​

The model ID to use. For a full list see the AWS Bedrock documentation.

Type: string

# Examples

model: amazon.titan-text-express-v1

model: anthropic.claude-3-5-sonnet-20241022-v2:0

model: cohere.command-text-v14

model: meta.llama3-1-70b-instruct-v1:0

model: mistral.mistral-large-2402-v1:0

prompt​

The prompt you want to generate a response for. By default, the processor submits the entire payload as a string. This field supports interpolation functions.

Type: string

system_prompt​

The system prompt to submit to the AWS Bedrock LLM. This field supports interpolation functions.

Type: string

max_tokens​

The maximum number of tokens to allow in the generated response. Set to -1 in order to not apply a limit.

Type: int

temperature​

The likelihood of the model selecting higher-probability options while generating a response. A lower value makes the model more likely to choose higher-probability options, while a higher value makes the model more likely to choose lower-probability options.

Type: float

stop​

A list of stop sequences. A stop sequence is a sequence of characters that causes the model to stop generating the response.

Type: array

top_p​

The percentage of most-likely candidates that the model considers for the next token. For example, if you choose a value of 0.8, the model selects from the top 80% of the probability distribution of tokens that could be next in the sequence.

Type: float

region​

The AWS region to target.

Type: string
Default: ""

endpoint​

Allows you to specify a custom endpoint for the AWS API.

Type: string
Default: ""

credentials​

Optional manual configuration of AWS credentials to use. More information can be found in this document.

Type: object

credentials.profile​

A profile from ~/.aws/credentials to use.

Type: string
Default: ""

credentials.id​

The ID of credentials to use.

Type: string
Default: ""

credentials.secret​

The secret for the credentials being used.

Secret

This field contains sensitive information that usually shouldn't be added to a config directly, read our secrets page for more info.

Type: string
Default: ""

credentials.token​

The token for the credentials being used, required when using short term credentials.

Type: string
Default: ""

credentials.from_ec2_role​

Use the credentials of a host EC2 machine configured to assume an IAM role associated with the instance.

Type: bool
Default: false

credentials.role​

A role ARN to assume.

Type: string
Default: ""

credentials.role_external_id​

An external ID to provide when assuming a role.

Type: string
Default: ""

credentials.expiry_window​

Allow the credentials to trigger refreshing prior to the credentials actually expiring. This is beneficial so race conditions with expiring credentials do not cause requests to fail. For example '10s' would refresh credentials ten seconds before expiration. Setting to a duration of 0 disables the expiry window.

Type: string
Default: ""