diff --git a/content/en/user-guide/aws/kinesis/index.md b/content/en/user-guide/aws/kinesis/index.md index 0105cc3657..d6af1028dd 100644 --- a/content/en/user-guide/aws/kinesis/index.md +++ b/content/en/user-guide/aws/kinesis/index.md @@ -1,24 +1,22 @@ --- -title: "Kinesis" -linkTitle: "Kinesis" -description: Get started with Kinesis on LocalStack +title: "Kinesis Data Streams" +linkTitle: "Kinesis Data Streams" +description: Get started with Kinesis Data Streams on LocalStack persistence: supported --- ## Introduction -Kinesis is a platform provided by Amazon Web Services (AWS) that enables your application to ingest, buffer, and process data in real-time. -Kinesis is suitable for applications that require processing and deriving insights from data streams such as logs, metrics, user interactions, and IoT sensor readings. -Kinesis offers three main services: Kinesis Data Streams, Kinesis Data Firehose, and Kinesis Data Analytics. -In this page, we take a look at Kinesis Data Streams which allows you to capture and store real-time data streams. +Kinesis Data Streams is an AWS service for ingesting, buffering, and processing data in high throughput data streams. +It is used for applications that require real-time processing and deriving insights from data streams such as logs, metrics, user interactions, and sensor readings. -LocalStack allows you to use the Kinesis APIs in your local environment from setting up data streams and configuring data processing to building real-time analytics applications. -The supported APIs are available on our [API coverage page](https://docs.localstack.cloud/references/coverage/coverage_kinesis/), which provides information on the extent of Kinesis's integration with LocalStack. +LocalStack allows you to use the Kinesis Data Streams APIs in your local environment from setting up data streams and configuring data processing to building real-time applications. +The supported APIs are available on our [API coverage page](https://docs.localstack.cloud/references/coverage/coverage_kinesis/). ## Getting started -This guide is designed for users new to Kinesis and assumes basic knowledge of the AWS CLI and our [`awslocal`](https://github.com/localstack/awscli-local) wrapper script. +This guide is designed for users new to Kinesis Data Streams and assumes basic knowledge of the AWS CLI and our [`awslocal`](https://github.com/localstack/awscli-local) wrapper script. Start your LocalStack container using your preferred method. We will demonstrate how to create a Lambda function to consume events from a Kinesis stream with the AWS CLI. @@ -171,7 +169,16 @@ $ awslocal kinesis put-record \ You can fetch the CloudWatch logs for your Lambda function reading records from the stream, using AWS CLI or LocalStack Resource Browser. -## Configuration +## Implementations + +There are two implementations of Kinesis Data Streams available in LocalStack. + +### Kinesis Mock + +This is the default provider for Kinesis Data Streams on LocalStack. +It is uses [Kinesis Mock](https://github.com/etspaceman/kinesis-mock) under the hood. + +This provider has the following configuration options: | Variable | Description | | -------- | ----------- | @@ -180,6 +187,11 @@ You can fetch the CloudWatch logs for your Lambda function reading records from | `KINESIS_LATENCY` | Integer value in milliseconds (default: `500`) or `0` (to disable). Particularly useful for testing latency-sensitive applications. Since local Kinesis service lacks latency simulation, you can introduce artificial latency into your AWS calls using this variable. To disable this behavior, set `KINESIS_LATENCY=0`. | | `KINESIS_INITIALIZE_STREAMS` | **Deprecated.** Comma-delimited string with stream names, corresponding shard counts, and an optional region for initialization during startup. If no region is provided, the default region is used. For example, `KINESIS_INITIALIZE_STREAMS=my-first-stream:1,my-other-stream:2:us-west-2,my-last-stream:1`. In multi-account setups, the specified streams will be created for all accounts. | +### Native + +This provider is implemented purely in Python. +It can be enabled by setting the configuration option `PROVIDER_OVERRIDE_KINESIS=native` + ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing Kinesis Streams & Kafka Clusters. @@ -204,7 +216,14 @@ The following code snippets and sample applications provide practical examples o - [Search application with Lambda, Kinesis, Firehose, ElasticSearch, S3](https://github.com/localstack/sample-fuzzy-movie-search-lambda-kinesis-elasticsearch) - [Streaming Data Pipeline with Kinesis, Tinybird, CloudWatch, Lambda](https://github.com/localstack/serverless-streaming-data-pipeline) -## Current Limitations +## Limitations + +### Kinesis Mock + +- In multi-account setups, each AWS account launches a separate instance of Kinesis Mock, which is very resource intensive when a large number of AWS accounts are used. + An [open Kinesis Mock issue](https://github.com/etspaceman/kinesis-mock/issues/377) is being used to keep track of this feature. + +### Native + +- TODO -In multi-account setups, each AWS account launches a separate instance of Kinesis Mock, which is very resource intensive when a large number of AWS accounts are used. -An [open Kinesis Mock issue](https://github.com/etspaceman/kinesis-mock/issues/377) is being used to keep track of this feature.