What is Bento for?
Bento is a declarative data streaming service that solves a wide range of data engineering problems with simple, chained, stateless processing steps. It implements transaction based resiliency with back pressure, so when connecting to at-least-once sources and sinks it's able to guarantee at-least-once delivery without needing to persist messages during transit.
It's simple to deploy, comes with a wide range of connectors, and is totally data agnostic, making it easy to drop into your existing infrastructure. Bento has functionality that overlaps with integration frameworks, log aggregators and ETL workflow engines, and can therefore be used to complement these traditional data engineering tools or act as a simpler alternative.
Bento is ready to commit to this relationship, are you?
Get StartedComponents
Inputs
- Services
- Network
- AWS
- GCP
- Azure
- Social
- Local
- Utility
- Integration
Inputs that consume from storage or message streaming services.
amqp_0_9amqp_1aws_kinesisaws_s3aws_sqsazure_blob_storageazure_queue_storageazure_table_storagebeanstalkdcassandradiscordetcdgcp_bigquery_selectgcp_cloud_storagegcp_pubsubhdfskafkakafka_franzmongodbmqttnatsnats_jetstreamnats_kvnats_streamnsqpulsarredis_listredis_pubsubredis_scanredis_streamssql_rawsql_selecttwitter_searchInputs that consume directly from low level network protocols.
http_clienthttp_servernanomsgsftpsocketsocket_serverwebsocketzmq4Inputs that consume from Google Cloud Platform services.
gcp_bigquery_selectgcp_cloud_storagegcp_pubsubInputs that consume from Microsoft Azure services.
azure_blob_storageazure_cosmosdbazure_queue_storageazure_table_storageInputs that provide utility by generating data or combining/wrapping other inputs.
batchedbrokerdynamicgenerateinprocread_untilresourcesequencesubprocessProcessors
- Mapping
- Integration
- Parsing
- Composition
- Utility
- Services
- Aws
- Azure
Processors that specialize in restructuring messages.
awkbloblangjavascriptjmespathjqjson_schemamappingmutationProcessors that interact with external services.
aws_dynamodb_partiqlaws_lambdacachecommandcouchbasegcp_bigquery_selecthttpredisredis_scriptschema_registry_decodeschema_registry_encodesql_insertsql_rawsql_selectsubprocessProcessors that specialize in translating messages from one format to another.
archiveavrobloblangcompressdecompressgrokmappingmsgpackmutationparquet_decodeparquet_encodeparse_logprotobufschema_registry_decodeschema_registry_encodeunarchivexmlHigher level processors that compose other processors and modify their behavior.
branchcatchfor_eachgroup_bygroup_by_valueinsert_partparallelprocessorsretryswitchtrywhileworkflowProcessors that provide general utility or do not fit in another category.
archivebounds_checkcacheddedupelogmetricrate_limitresourceselect_partssleepsplitsync_responseunarchivewasmOutputs
- Services
- Network
- AWS
- GCP
- Azure
- Social
- Local
- Utility
Outputs that write to storage or message streaming services.
amqp_0_9amqp_1aws_dynamodbaws_kinesisaws_kinesis_firehoseaws_s3aws_snsaws_sqsazure_blob_storageazure_queue_storageazure_table_storagebeanstalkdcachecypherdiscordelasticsearchgcp_bigquerygcp_bigquery_write_apigcp_cloud_storagegcp_pubsubhdfskafkakafka_franzmongodbmqttnatsnats_jetstreamnats_kvnats_streamnsqopensearchpulsarpusherquestdbredis_hashredis_listredis_pubsubredis_streamssnowflake_putsplunk_hecsql_insertsql_rawOutputs that write directly to low level network protocols.
http_clienthttp_servernanomsgsftpsocketwebsocketzmq4Outputs that write to Amazon Web Services products.
aws_dynamodbaws_kinesisaws_kinesis_firehoseaws_s3aws_snsaws_sqsOutputs that write to Google Cloud Platform services.
gcp_bigquerygcp_bigquery_write_apigcp_cloud_storagegcp_pubsubOutputs that write to Microsoft Azure services.
azure_blob_storageazure_cosmosdbazure_queue_storageazure_table_storageOutputs that write to social applications and services.
discordOutputs that provide utility by combining/wrapping other outputs.
brokerdropdrop_ondynamicfallbackinprocrejectreject_erroredresourceretrysubprocessswitchsync_response