Skip to main content

aws_bedrock_invoke

EXPERIMENTAL

This component is experimental and therefore subject to change or removal outside of major version releases.

Invokes an Amazon Bedrock model for each message, replacing the message contents with the raw model response.

# Common config fields, showing default values
label: ""
aws_bedrock_invoke:
  model: amazon.titan-embed-text-v2:0 # No default (required)

Sends the contents of each message to the Amazon Bedrock InvokeModel API and replaces the message with the response body.

Message payloads are forwarded verbatim as the request body and are neither parsed nor validated by this processor, and responses are written back unmodified. The request and response formats differ between model providers, so it is up to you to construct a body the target model accepts and to extract whatever you need from the reply — typically with a mapping processor either side. Both the request and response content types are set to application/json.

Requests are signed with AWS Signature Version 4 by the AWS SDK, so no additional authentication configuration is required beyond standard AWS credentials.

The model may be a foundation model ID (e.g. amazon.titan-embed-text-v2:0) or an inference profile ID or ARN. Some models can only be invoked through an inference profile.

Each message in a batch is invoked separately and concurrently, and messages larger than one are invoked in parallel. A failed invocation is flagged on the individual message that caused it, leaving the rest of the batch untouched. Note that this means a batch of N messages results in N Bedrock requests; models that accept multiple inputs per request are not batched into a single call.

Streaming APIs (InvokeModelWithResponseStream, Converse, ConverseStream) are not supported.

Preserving the original message​

By default the message contents are replaced with the model response. To keep the original payload and store the response alongside it, use a branch processor.

Credentials​

By default Bento will use a shared credentials file when connecting to AWS services. It's also possible to set them explicitly at the component level. You can find out more in this document.

Examples​

Generate an embedding for each message using an Amazon Titan model, keeping the original document and adding the embedding under a vector field. The request_map builds the body Titan expects, and the result_map pulls the vector out of the response.

pipeline:
  processors:
    - branch:
        request_map: 'root.inputText = content().string()'
        processors:
          - aws_bedrock_invoke:
              model: amazon.titan-embed-text-v2:0
              region: us-east-1
        result_map: 'root.vector = this.embedding'

Fields​

model​

The ID of the Bedrock model to invoke, or an inference profile ID or ARN.

Type: string

# Examples

model: amazon.titan-embed-text-v2:0

model: cohere.embed-english-v3

model: anthropic.claude-3-5-sonnet-20241022-v2:0

model: us.anthropic.claude-3-5-sonnet-20241022-v2:0

rate_limit​

An optional rate_limit resource to throttle Bedrock InvokeModel requests by. The limit is applied per request, and each message in a batch counts as one request. Note that Bento rate limits are enforced per running instance, so this does not bound the aggregate rate across horizontally scaled deployments.

Type: string
Default: ""

region​

The AWS region to target.

Type: string
Default: ""

endpoint​

Allows you to specify a custom endpoint for the AWS API.

Type: string
Default: ""

credentials​

Optional manual configuration of AWS credentials to use. More information can be found in this document.

Type: object

credentials.profile​

A profile from ~/.aws/credentials to use.

Type: string
Default: ""

credentials.id​

The ID of credentials to use.

Type: string
Default: ""

credentials.secret​

The secret for the credentials being used.

Secret

This field contains sensitive information that usually shouldn't be added to a config directly, read our secrets page for more info.

Type: string
Default: ""

credentials.token​

The token for the credentials being used, required when using short term credentials.

Type: string
Default: ""

credentials.from_ec2_role​

Use the credentials of a host EC2 machine configured to assume an IAM role associated with the instance.

Type: bool
Default: false

credentials.role​

A role ARN to assume.

Type: string
Default: ""

credentials.role_external_id​

An external ID to provide when assuming a role.

Type: string
Default: ""

credentials.expiry_window​

Allow the credentials to trigger refreshing prior to the credentials actually expiring. This is beneficial so race conditions with expiring credentials do not cause requests to fail. For example '10s' would refresh credentials ten seconds before expiration. Setting to a duration of 0 disables the expiry window.

Type: string
Default: ""