Localstack

Summary

This LocalStack release features several improvements and new features in AWS services, a new CLI command, Python 3.10 support, and 10 new open source contributors!

AWS Services

* Support for [Cloudwatch Alarms](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/AlarmThatSendsEmail.html)
* Improved streaming support for Lambdas which better testing of common real-time data pipelines. Read more [on our blog](https://localstack.cloud/blog/2022-04-04-develop-and-test-data-analytics-pipelines-on-localstack/).
* New standalone SQS provider with improved stability and performance
* Near complete implementation of the [SQS Query API](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-making-api-requests.html) and new ways to [create Queue URLs](https://docs.localstack.cloud/aws/sqs/)
* [Support for .NET 6 Lambdas](https://aws.amazon.com/blogs/compute/introducing-the-net-6-runtime-for-aws-lambda/)
* Improved [OpenAPI support for API Gateway](https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-import-api.html).

CLI command to update LocalStack

We have introduced a new CLI command that automatically updates LocalStack and its requirements to the latest version:

bash
% localstack update --help
Usage: localstack update [OPTIONS] COMMAND [ARGS]...

Update LocalStack components

Options:
--help Show this message and exit.

Commands:
all Update all LocalStack components
docker-images Update container images LocalStack depends on
localstack-cli Update LocalStack CLI tools


Other notable updates

* LocalStack now runs on Python 3.10!
* Many services have now been successfully migrated to our new framework that handles AWS service requests, this dramatically improves parity and stability of services
* Deprecation note: The **legacy** record&replay based **persistence** mechanism (currently still available via `LEGACY_PERSISTENCE`) be entirely **removed in the next release** (as already previously announced in the [0.13.1 release](https://github.com/localstack/localstack/releases/tag/v0.13.1)).

What's Changed

Features

* add metric-alarm evaluation to Cloudwatch by steffyP in https://github.com/localstack/localstack/pull/5897
* add on failure destination implementation and TRIM_HORIZON starting position for create-event-source-mapping by steffyP and SwatsonCodes in https://github.com/localstack/localstack/pull/5807
* Add localstack update command by dfangl in https://github.com/localstack/localstack/pull/5955
* Upgrade python version to 3.10 by dfangl in https://github.com/localstack/localstack/pull/5632
* Add dotnet 6 runtime by yongliu-mdsol in https://github.com/localstack/localstack/pull/5815
* add SQS Query API by thrau in https://github.com/localstack/localstack/pull/5928
* Rework ASF SQS provider for RegionBackend by viren-nadkarni in https://github.com/localstack/localstack/pull/6016
* Apigateway OpenAPI import support for authorizers by calvernaz in https://github.com/localstack/localstack/pull/6013

Enhancement

* LocationConstraint should be None for us-east-1 by pinzon in https://github.com/localstack/localstack/pull/5765
* Unify Lambda Runtime using Runtime API by dfangl in https://github.com/localstack/localstack/pull/5306
* Add newline after XML preamble in S3 responses for Serverless compatibility by whummer in https://github.com/localstack/localstack/pull/5795
* Update ASF APIs by localstack-bot in https://github.com/localstack/localstack/pull/5798
* Deletion of individual statments of a function policy by pinzon in https://github.com/localstack/localstack/pull/5785
* Lambda Feature: Throw error when unzipped code exceeds maximum size (~256mb) by RodrigoATorres in https://github.com/localstack/localstack/pull/5641
* Rework Lambda Integration tests to work against AWS as well as LocalStack by dfangl in https://github.com/localstack/localstack/pull/5819
* Custom header x localstack authorization by calvernaz in https://github.com/localstack/localstack/pull/5809
* Migrate DynamoDB to ASF by whummer in https://github.com/localstack/localstack/pull/5796
* add cloudformation support for s3 bucket versioning by dominikschubert in https://github.com/localstack/localstack/pull/5841
* adds authorizer object to lambda event by calvernaz in https://github.com/localstack/localstack/pull/5837
* Added DYNAMODB_SHARE_DB to conf by giograno in https://github.com/localstack/localstack/pull/5842
* Add !Ref support for KMS::Alias CloudFormation resources by whummer in https://github.com/localstack/localstack/pull/5836
* Add additional awslocal commands for CI smoke tests by whummer in https://github.com/localstack/localstack/pull/5845
* Update ASF APIs by localstack-bot in https://github.com/localstack/localstack/pull/5838
* Update python to 3.8.13 by dfangl in https://github.com/localstack/localstack/pull/5856
* add type hints to PaginatedList by alexrashed in https://github.com/localstack/localstack/pull/5865
* [ASF] Simple Email Service migration by viren-nadkarni in https://github.com/localstack/localstack/pull/5771
* Sns unsubscribe should send a confirmation by pinzon in https://github.com/localstack/localstack/pull/5874
* Return successfully when attempting to delete a non-existent secret by foxmulder900 in https://github.com/localstack/localstack/pull/5873
* Error message for PAY_PER_REQUEST billing mode by giograno in https://github.com/localstack/localstack/pull/5877
* Update ASF APIs by localstack-bot in https://github.com/localstack/localstack/pull/5881
* update Docker base image (python:3.8.13-slim-buster) digest to 36a1e0b by renovate in https://github.com/localstack/localstack/pull/5880
* Add DynamoDB patch to enable more than 20 GSIs on tables by whummer in https://github.com/localstack/localstack/pull/5883
* Upgrade moto-ext to version 3.1.6 by whummer in https://github.com/localstack/localstack/pull/5882
* Migrate StepFunctions to ASF by whummer in https://github.com/localstack/localstack/pull/5869
* update localstack wait command to use docker logs stream by thrau in https://github.com/localstack/localstack/pull/5885
* get_handler_file_from_name when path contains periods by tmbobbins in https://github.com/localstack/localstack/pull/5849
* Support nested EventBridge patterns, improve parity with AWS by whummer in https://github.com/localstack/localstack/pull/5879
* rework localstack.http.Request object to use a dummy WSGi environment by thrau in https://github.com/localstack/localstack/pull/5876
* add task scheduler utility by thrau in https://github.com/localstack/localstack/pull/5857
* Refactor util to create S3 buckets with location constraints by whummer in https://github.com/localstack/localstack/pull/5898
* Test MISSING Function with PartiQL by giograno in https://github.com/localstack/localstack/pull/5899
* Bump moto-ext to 3.1.6.1 by whummer in https://github.com/localstack/localstack/pull/5900
* queryStringParameters set to None when there are no GET parameters by jops-wtakase in https://github.com/localstack/localstack/pull/5893
* Add javaagent loader for DynamoDB local process by whummer in https://github.com/localstack/localstack/pull/5905
* add headers to dummy WSGI environment by thrau in https://github.com/localstack/localstack/pull/5902
* Minor: Bump version of awscli for compabilibity with poetry by whummer in https://github.com/localstack/localstack/pull/5912
* specify content type header with curl PUT request in docker-entrypoint.sh by nao23 in https://github.com/localstack/localstack/pull/5903
* move functions to helper module by calvernaz in https://github.com/localstack/localstack/pull/5925
* Add SES HTML body to match v2 API spec by NickGrantX in https://github.com/localstack/localstack/pull/5920
* Add small utility for base64 URL safe decoding by whummer in https://github.com/localstack/localstack/pull/5932
* Handling TableClass in DynamoDB by giograno in https://github.com/localstack/localstack/pull/5923
* add decorator for adding routes to a Router by thrau in https://github.com/localstack/localstack/pull/5927
* Remove explicit object inheritance by dominikschubert in https://github.com/localstack/localstack/pull/5933
* Update ASF APIs by localstack-bot in https://github.com/localstack/localstack/pull/5929
* Allow multiple bind addresses to be passed to hypercorn by dfangl in https://github.com/localstack/localstack/pull/5937
* extend s3 virtual host addressing rules exemptions by thrau in https://github.com/localstack/localstack/pull/5952
* Apigateway import test extension by pinzon in https://github.com/localstack/localstack/pull/5953
* Ensure eventID is contained in DDB stream records by whummer in https://github.com/localstack/localstack/pull/5960
* Feature log for fifo queue creating by pinzon in https://github.com/localstack/localstack/pull/5962
* migrate cloudformation to ASF by thrau in https://github.com/localstack/localstack/pull/5965
* simplify StsAwsApiListener JSON response modification by thrau in https://github.com/localstack/localstack/pull/5968
* allow http.Request object to retain dashes and key casing by thrau in https://github.com/localstack/localstack/pull/5969
* update Docker base image (python:3.8.13-slim-buster) digest to 8b65f13 by renovate in https://github.com/localstack/localstack/pull/5971
* chore: add user survey over README by HarshCasper in https://github.com/localstack/localstack/pull/5975
* DynamoDB Streams - code removal by giograno in https://github.com/localstack/localstack/pull/5981
* Update README.md by dijonkitchen in https://github.com/localstack/localstack/pull/5979
* Update ASF APIs by localstack-bot in https://github.com/localstack/localstack/pull/5973
* Bump version of moto-ext to 3.1.8 by whummer in https://github.com/localstack/localstack/pull/5980
* chore: add a stale bot to manage issues by HarshCasper in https://github.com/localstack/localstack/pull/5974
* disable strict slashes for ASF REST operation router by thrau in https://github.com/localstack/localstack/pull/5983
* add ASGI to WSGI bridge and hypercorn server abstraction by thrau in https://github.com/localstack/localstack/pull/5970
* CI: Add CLA bot by HarshCasper in https://github.com/localstack/localstack/pull/5993
* Utility to list loaded services by giograno in https://github.com/localstack/localstack/pull/5996
* Update ReadMe to show support for 3.10 by silv-io in https://github.com/localstack/localstack/pull/6002
* Enable signal handlers per default for LocalStack inside a docker container by dfangl in https://github.com/localstack/localstack/pull/6000
* Allow docker client to modify security opts, explicit exception chaining by dfangl in https://github.com/localstack/localstack/pull/6001
* remove unused 'reactive mode' event source listener code by SwatsonCodes in https://github.com/localstack/localstack/pull/6004
* Apigateway ASF migration by calvernaz in https://github.com/localstack/localstack/pull/5731
* Add support for return macros in VTL templates by whummer in https://github.com/localstack/localstack/pull/6010
* improve openapi import by calvernaz in https://github.com/localstack/localstack/pull/5939
* refactor initial set of s3 tests by thrau in https://github.com/localstack/localstack/pull/6006
* Add filter for cognito identity fields by dfangl in https://github.com/localstack/localstack/pull/6005
* Introduce snapshot tests by dominikschubert in https://github.com/localstack/localstack/pull/5825
* configure pytest to log DEBUG to cli by default by thrau in https://github.com/localstack/localstack/pull/6015
* Added utility to filter a dictionary based on a TypedDict keys by giograno in https://github.com/localstack/localstack/pull/6018
* Asf sns migration by baermat in https://github.com/localstack/localstack/pull/5654
* [ASF] iam Migration by MEPalma in https://github.com/localstack/localstack/pull/5818
* [ASF] events Migration by MEPalma in https://github.com/localstack/localstack/pull/5844
* chore: Set permissions for GitHub actions by naveensrinivasan in https://github.com/localstack/localstack/pull/6014
* Update ASF APIs by localstack-bot in https://github.com/localstack/localstack/pull/6030
* update Docker base image (python:3.10.4-slim-buster) digest to 1678c20 by renovate in https://github.com/localstack/localstack/pull/6029
* add workaround for s3 virtual host addressing by thrau in https://github.com/localstack/localstack/pull/6026
* ignore GET and HEAD request body for rest-xml protocol by thrau in https://github.com/localstack/localstack/pull/6025
* Revert "add workaround for s3 virtual host addressing (6026)" by dfangl in https://github.com/localstack/localstack/pull/6034
* Explicit persistence hook argument for a Service by giograno in https://github.com/localstack/localstack/pull/6020
* Minor cleanup after events migration by MEPalma in https://github.com/localstack/localstack/pull/6036
* Add map, quiet return functions for VTL template rendering by whummer in https://github.com/localstack/localstack/pull/6035
* don't suppress stage variables from lambda event by calvernaz in https://github.com/localstack/localstack/pull/6042
* simplify cloudformation ui resource by thrau in https://github.com/localstack/localstack/pull/6047
* Bump moto-ext to 3.1.9 by whummer in https://github.com/localstack/localstack/pull/6037
* Refactor container network connection to reuse it outside of lambda by dfangl in https://github.com/localstack/localstack/pull/6052
* Initial CFn support for UsePreviousValue to enable idempotent CDK bootstraps by whummer in https://github.com/localstack/localstack/pull/6053
* Update ASF APIs by localstack-bot in https://github.com/localstack/localstack/pull/6067
* move import_rest_api patch into ASF handler by calvernaz in https://github.com/localstack/localstack/pull/6048
* Unify SES logging format by viren-nadkarni in https://github.com/localstack/localstack/pull/6060
* ci: add labeler action by HarshCasper in https://github.com/localstack/localstack/pull/6068
* emit analytics event containing CLI command whenever CLI is invoked by SwatsonCodes in https://github.com/localstack/localstack/pull/6043
* Add DISABLE_CORS_HEADERS option to disable default CORS headers by whummer in https://github.com/localstack/localstack/pull/6073
* Handle Cloudformations empty stack updates by changing the changeset status by dfangl in https://github.com/localstack/localstack/pull/6080
* remove delete integration patch by calvernaz in https://github.com/localstack/localstack/pull/6074
* migrate apigateway unittest tests to pytest by calvernaz in https://github.com/localstack/localstack/pull/6092
* sqs tag map parsing by thrau in https://github.com/localstack/localstack/pull/6091
* add initial support for streaming in ASF scaffold and serializer by alexrashed in https://github.com/localstack/localstack/pull/6086
* add empty-string validation for sqs user queue attributes by thrau in https://github.com/localstack/localstack/pull/6090
* add ASF prototype for server-side parameter validation by thrau in https://github.com/localstack/localstack/pull/6088

Fixes

* Fix `KMS_PROVIDER=kms-local` back to `local-kms` by davecardwell in https://github.com/localstack/localstack/pull/5793
* Minor fixes for data types handling in CloudFormation deployer by whummer in https://github.com/localstack/localstack/pull/5794
* Fix zone-name filter in describe-availability-zones by viren-nadkarni in https://github.com/localstack/localstack/pull/5800
* Fix S3 download of XML documents with preamble by whummer in https://github.com/localstack/localstack/pull/5805
* fix dynamodb index query validation by foxmulder900 in https://github.com/localstack/localstack/pull/5799
* Fix flaky tests: test_dynamodb_stream_records_with_update_item, test_rotate_secret_with_lambda_2 by steffyP in https://github.com/localstack/localstack/pull/5826
* fix ApiGateway ASF request matching by alexrashed in https://github.com/localstack/localstack/pull/5827
* Fixed issue with spaces in Windows user names by us88 in https://github.com/localstack/localstack/pull/5834
* fix ASF operation routing (empty query args, deprecated ops) by alexrashed in https://github.com/localstack/localstack/pull/5840
* Fix ResourceNotFoundException for BatchGetItem operation by giograno in https://github.com/localstack/localstack/pull/5848
* fix queries for local secondary index on dynamodb by steffyP in https://github.com/localstack/localstack/pull/5847
* Fix 5821 - ElasticMQ always downloaded due to incorrect cache path in Dockerfile by wimax-grapl in https://github.com/localstack/localstack/pull/5822
* Target HttpParameters and Input support and basic auth fix for Events service by pinzon in https://github.com/localstack/localstack/pull/5738
* fix api detection to apply custom rules and normalize CORS behaviour by calvernaz in https://github.com/localstack/localstack/pull/5839
* fix extraction of oauth_http_parameters for api destinations by pinzon in https://github.com/localstack/localstack/pull/5861
* Increment kinesis-mock version to 0.2.4 for validation fix by afleishaker in https://github.com/localstack/localstack/pull/5884
* add optional .fifo suffix to sqs queue url matcher by thrau in https://github.com/localstack/localstack/pull/5892
* Fix backdoor API provided on health endpoint by nao23 in https://github.com/localstack/localstack/pull/5907
* Fix DynamoDB TransactGetItems action by giograno in https://github.com/localstack/localstack/pull/5909
* Fix ExecuteTransaction action in DynamoDB by giograno in https://github.com/localstack/localstack/pull/5918
* Minor fix to ensure that /static/ paths are properly served by the proxy by whummer in https://github.com/localstack/localstack/pull/5917
* Fix BatchExecuteStatement action in DynamoDB by giograno in https://github.com/localstack/localstack/pull/5919
* fix localstack.http.Request path encoding by thrau in https://github.com/localstack/localstack/pull/5924
* Fix lambda package type and code response by dominikschubert in https://github.com/localstack/localstack/pull/5930
* Fix ApiGW: accept yaml format for openapi(swagger) import by pinzon in https://github.com/localstack/localstack/pull/5871
* fix invalid characters in asf rest-protocol operation router by alexrashed in https://github.com/localstack/localstack/pull/5948
* [minor] Fix text/html Content-Type for local CloudFormation UI by whummer in https://github.com/localstack/localstack/pull/5950
* remove Werkzeug pin after bugfix release by alexrashed in https://github.com/localstack/localstack/pull/5954
* fix get_raw_path to unify wsgi/asgi specs by thrau in https://github.com/localstack/localstack/pull/5956
* Fix Firehose.PutRecord to extended s3 by pinzon in https://github.com/localstack/localstack/pull/5967
* fix op router when matching keys with slashes by thrau in https://github.com/localstack/localstack/pull/5972
* Fix ValidationError for CloudFormation.DescribeStacks by dominikschubert in https://github.com/localstack/localstack/pull/5978
* fix secretsmanager to use ASF instead of moto exceptions by thrau in https://github.com/localstack/localstack/pull/5976
* fix HTTP HEAD method operation detection in ASF by alexrashed in https://github.com/localstack/localstack/pull/5991
* fix test_cloudwatch.py mock patches by thrau in https://github.com/localstack/localstack/pull/5984
* fix: fix cla bot by HarshCasper in https://github.com/localstack/localstack/pull/5999
* fixed metric alarm evaluation for data points with value 0 by steffyP in https://github.com/localstack/localstack/pull/5998
* fix http path encoding in asgi/wsgi adapter by thrau in https://github.com/localstack/localstack/pull/6012
* fix concurrency issues of cfn and sns tests by thrau in https://github.com/localstack/localstack/pull/6023
* Provide hotfix for CFn parameter updates and further CFn tests by dominikschubert in https://github.com/localstack/localstack/pull/6024
* fix CLA assistant allowlist by alexrashed in https://github.com/localstack/localstack/pull/6032
* Fix bytes size calculation with binary items by giograno in https://github.com/localstack/localstack/pull/6027
* Fix shutdown error in pytest by waiting for LS to finish correctly by dfangl in https://github.com/localstack/localstack/pull/6039
* fix ASF docs by sanitizing pandoc results by alexrashed in https://github.com/localstack/localstack/pull/6045
* fix multiple authorizer creation by calvernaz in https://github.com/localstack/localstack/pull/6049
* Restructure test fixtures/utilities by dominikschubert in https://github.com/localstack/localstack/pull/6050
* Fix DynamoDB error injection and re-enable tests by whummer in https://github.com/localstack/localstack/pull/6063
* Minor fixes in edge forwarder and zip utils for dev/host mode by whummer in https://github.com/localstack/localstack/pull/6065
* fix request encoding in generic proxy listener chain and forwarding by alexrashed in https://github.com/localstack/localstack/pull/6070
* Minor fix for CFn deletion of IAM::Role resources by whummer in https://github.com/localstack/localstack/pull/6072
* fix pycharm unit test debug execution by renaming http package by alexrashed in https://github.com/localstack/localstack/pull/6071
* fix ARN rewriting for DynamoDB ASF provider by alexrashed in https://github.com/localstack/localstack/pull/6075
* fix ARN rewriting for incoming requests, fix lambda arn creation by alexrashed in https://github.com/localstack/localstack/pull/6078
* fix test_get_queue_url_contains_request_host for pro integration by thrau in https://github.com/localstack/localstack/pull/6079
* ci: fix the stale bot by removing bug from exempted labels by HarshCasper in https://github.com/localstack/localstack/pull/6082
* Fix CLI usage with python 3.7 by dfangl in https://github.com/localstack/localstack/pull/6087
* fix cleanup of event source mapping test by alexrashed in https://github.com/localstack/localstack/pull/6094
* fix message attribute filter for SQS ReceiveMessage by thrau in https://github.com/localstack/localstack/pull/6097

New Contributors
* davecardwell made their first contribution in https://github.com/localstack/localstack/pull/5793
* RodrigoATorres made their first contribution in https://github.com/localstack/localstack/pull/5641
* foxmulder900 made their first contribution in https://github.com/localstack/localstack/pull/5799
* us88 made their first contribution in https://github.com/localstack/localstack/pull/5834
* afleishaker made their first contribution in https://github.com/localstack/localstack/pull/5884
* tmbobbins made their first contribution in https://github.com/localstack/localstack/pull/5849
* jops-wtakase made their first contribution in https://github.com/localstack/localstack/pull/5893
* nao23 made their first contribution in https://github.com/localstack/localstack/pull/5907
* NickGrantX made their first contribution in https://github.com/localstack/localstack/pull/5920
* naveensrinivasan made their first contribution in https://github.com/localstack/localstack/pull/6014

**Full Changelog**: https://github.com/localstack/localstack/compare/v0.14.2...v0.14.3

Change log

1. New Features

* add ability to import keys into KMS
* enable and refactor rotation logic and version stages in SecretsManager
* support "Accept" header to switch response format
* add provider for S3Control service

2. Enhancements

* migrate services to ASF: EC2, KMS, Route53, STS
* skip installing -ext dependencies in CI tests
* lazily load subclasses for instance manager to enable plugin loading at runtime
* add missing UpdateDomainConfig for OpenSearch and Elasticsearch
* pin werkzeug version to fix Transfer-Encoding "chunked" on empty responses
* apply filter pattern for log subscriptions
* skip "online-only" tests, fix some tests
* use valid references for EC2 resources to fix CloudFormation tests
* enhance handling of KMS key pairs for SIGN_VERIFY keys
* add CloudFormation support for IAM::ServiceLinkedRole
* reintroduce Lambda log output str casting
* source API Gateway domain name and prefix from invocation context
* pin airspeed version and remove patch code
* add CloudFormation support for AWS::CDK::Metadata
* remove SERVICES from default docker-compose.yaml
* round DynamoDB ApproximateCreationDateTime to seconds
* add GetAtt support for AWS::EC2::VPC DefaultSecurityGroup and DefaultNetworkAcl
* minor update in patch decorator to allow multiple/composite patches of methods

3. Bug Fixes

* fix PaginatedList nextToken generation
* fix MessageGroupId for publish batch to SQS fifo queue
* fix elliptic curve key specs for KMS keys
* fix broken transitive imports of quart from jinja
* fix Terraform creation of EC2 Security Groups with default egress
* fix eager deserialization for 404 responses returned from Lambda
* fix CloudFormation Route53 deployment without ResourceRecords
* fix ACM DomainValidationOptions to support waiters
* fix Flask app routes for Lambda API
* fix broken patch for moto S3 is_delete_keys(..)
* fix CFn name autogeneration for SQS fifo queues
* fix CFn Route53 deployment
* fix CFn default name for Kinesis streams
* fix logs subscription filter
* fix query request parser for operations without input shape
* fix nested stack outputs in CloudFormation
* fix handling encoded `$` for default stage in API Gateway path regex
* fix redirect to index file in S3 website

Change log

1. New Features

* push Docker images to public ECR registry on CI build
* add internal endpoint to output diagnostics data
* add ability to build, tag, and push images using Docker client
* add API Gateway SNS integration
* add OpenSearch destination to Firehose
* add support for CloudWatch set_alarm_state and trigger actions

2. Enhancements

* implement in-memory request dispatching to moto
* implement generic HTTP route dispatching
* migrate several services to ASF (ACM, CloudWatch, CloudWatch Logs, ConfigService, DynamoDB Streams, Redshift, ResourceGroups, SecretsManager, SSM, Support, SWF)
* migrate from requirements.txt/setup.py to setup.cfg
* CircleCI pipeline parallelization
* add GitHub action to automatically update ASF APIs
* migrate resourcegroupstagging API to ASF
* split up Docker client, remove obsolete `docker` client binary from image
* refactor handling of API Gateway request/response templates
* introduce CODEOWNERS file for code ownership
* add timeout to status services cli command
* add DynamoDB Streams integration for PartiQL query updates
* add ability to extract cookies from API Gateway invocation context
* update generated ASF APIs to latest version
* add ability to specify timeout and max content length for HTTP server
* improve cli startup performance
* refactor utils imports in utils and services packages
* patch moto Lambda backend to access LocalStack Lambdas directly
* add ability to extract tgz archives via download_and_extract utility
* allow specifying working directory when running Docker exec
* implement regex rules for the http Router
* migrate from localstack-plugin-loader to plux
* add support for define function in API Gateway velocity templates
* add a GHA workflow to track first-time pull request and issue creators
* rework docker capabilities, remove intermediate containers after building images in docker-sdk client
* add optimizing and sorting of imports to ASF scaffold
* add support for response templates in API Gateway MOCK responses
* add mock ECR CFN resource and integration test for CDK bootstrap template v10
* add flag to call_moto to exclude the ResponseMetadata
* upgrade airspeed dependency and remove custom patches
* add helpers to unpause Docker containers
* reorganize utility package
* add deep copy of handler results to avoid unwanted mutations in parser
* add error check for fifo attribute when creating SQS fifo queues
* update renovatebot config to ignore Dockerfile.rh
* add ApiGateway S3 integration using action name
* add method to allow calling moto with a new request in request dispatcher
* add resolve_hostname(..) util function, refactor some utils into sub-modules
* improve performance of localstack status command
* remove `six` library from dependencies
* add failsafe with try/except for creating log streams to guard against potential duplicates
* add the ability to snapshot running Docker containers
* make autocreating log groups optional
* allow service restart in ServiceContainer from STOPPED state
* add support for DefinitionS3Location in StepFunctions::StateMachine model
* remove superfluous CloudWatch response processing
* add PaginatedList for paginating and filtering response lists
* add default entries in CloudFormation stack template dict to avoid KeyError
* docs: refactor third party software tools docs and remove announcements in README
* refactor some tests to pytest (e.g., Lambda, EC2)
* configure test coverage exclusions
* make Lambda output truncation configurable

3. Bug Fixes

* fix ASF parser and serializer issues
* fix custom error responses from lambda when handler throws an exception
* fix moto dispatching for trailing slashes and response headers
* fix OpenSearch data dir issue that would block startup in pro
* fix encoding problem for AlarmDescription in put-metric-alarm
* fix making typing_extensions import dependent on python version
* fix ApiGateway v2 path-base style requests without stages
* fix several ASF parser, serializer, and scaffold issues
* fix replacement of AWS::NoValue, add default props for Elasticsearch::Domain resources
* fix service provider configuration override
* fix edge route mapping for STS AssumeRole
* fix API Gateway integration with Kinesis
* fix linting issues by pinning flake8-black
* fix python 3.7 compatibility with TypedDict for cli
* fix ASF xml-rest parser location- and payload-trait handling
* fix CloudFormation stack class to return latest raw template if change sets are present
* fix OpenSearch cluster shutdown in tests
* fix parameter for StateMachine type in CFN model
* fix uncapped recursion bug in parser

Announcements

- **Breaking changes:**
- This version introduces the external service port range in LocalStack. This port range has been used in LocalStack Pro before, but has now been moved to the community edition. While doing so, the port range has been broadened (it now ranges from 4510 to 4559). Please make sure to adjust your Docker configurations to expose this port range (such that external services started within the container can be reached) or use the latest version of the LocalStack CLI to manage your LocalStack container.
- When using ElasticSearch with the `ES_ENDPOINT_STRATEGY=port` (formerly `off`) and `ES_MULTI_CLUSTER=0`, a port from the external service port range is used (instead of `PORT_ELASTICSEARCH` or `4571` by default).
- **Upcoming breaking change:** This version deprecates the `ES_ENDPOINT_STRATEGY`, `ES_MULTI_CLUSTER`, and `ES_CUSTOM_BACKEND`. Instead, the `OPENSEARCH_*` variables should be used. The `ES_ENDPOINT_STRATEGY=off` is being renamed to `port` (i.e. please use `OPENSEARCH_ENDPOINT_STRATEGY=port` from now on).

Change log

1. New Features

* add initial support for SNS PublishBatch API
* add support for email-json protocol in SNS subscriptions
* add support for AWS OpenSearch
* support multi arch for local-kms provider
* support invocation of local API URLs via stepfunctions

2. Enhancements

* bump moto-ext to latest version
* minor refactoring in SNS get_message_attributes() method
* add utilities for migrating services to ASF
* minor refactoring in selecting handler in edge.py for better extensibility
* add check for supported protocols on SNS subscription
* adjust routing to enable using anonymous boto3 client to fetch S3 object
* refactor Lambda executors to allow modification of container settings using hooks
* minor adjustments in patch decorator for bound methods of class instances
* hardcode default tmp and data paths
* Add handling for DynamoDB stream iterator expiration, bump kinesis-mock to 0.2.2
* remove obsolete moto imports in CloudFormation resource classes
* remove obsolete patch for CloudWatch metrics filtering
* cleanup CloudFormation logic and remove old/obsolete code
* add utility to manage / reserve ports for external services
* invalidate builder stage in dockerfile on package version changes
* use base_url instead of url for determining SQS queue URL
* add cachetools to CLI dependencies
* set MessageGroupId and MessageDeduplicationId on SQS message from SNS subscription
* Introduce lazily-created localstack-internal-awssdk utility lambda function
* capture X-AMAZON-APIGATEWAY-ANY-METHOD for cases where route key is for integrations
* provide better log info when CloudFormation resources cannot be found
* clean up old/unused CloudFormation logic
* refactor SQS queue-url generation
* extend SNS publish batch support for FIFO topics
* remove monkey patching and circular dependency
* refactor sqs queue resolving and url rendering
* minor refactoring of CFN logging code to dump resources as JSON string
* clean up aws_stack utils, remove obsolete global cached boto3 resources
* add missing CloudFormation parameters for Firehose delivery streams
* migrate firehose to ASF
* stop setting environment variable if cognito_identity if empty
* Kinesis refactor server abstraction
* Allow container client to connect/disconnect containers to/from a network
* handle SSE for sqs
* refactor config and rip out global port/url variables
* add connection parameters and credentials to EventBridge API destinations
* replace function calls with literal syntax and remove unnecessary comprehensions
* broaden the external service port range, adjust docker-compose
* update Docker base image (python:3.8.12-slim-buster) digest to 3544d06
* add util function for deletion of None-values from dicts
* combine OpenSearch and Elasticsearch backends
* update exposed ports and docker-compose


3. Bug Fixes

* fix CloudFormation deployment of SNS::TopicPolicy
* fix handling of empty prefix for Firehose S3 uploads to properly trigger event notification
* fix external host setting for sqs
* fix acronym handling in camel_to_snake_case
* fix API Gateway to match the most specific path in the presence of "{proxy+}" placeholders
* fix asf scaffold to generate TypedDict declarations correctly
* fix sqs port extraction from headers and other related issues
* fix ASF SQS provider to allow direct calls to queue URLs
* fix DynamoDB local/localhost region for NoSQL Workbench
* fix ASF SQS xml response encoding
* fix request parser to enable X-Amz-Target strings with multiple dots
* fix expectations around Kinesis record encoding, always re-encode records for shard subscribers
* fix CloudFormation deployer to recursively resolve string placeholders
* fix ASF REST operation lookup with conflicting path params
* fix DomainValidationOptions property in ACM Certificate CFN deployment
* fix access to listener in logic for restarting Kinesis
* fix issue rendering VTL templates for API Gateway by flattening the input
* fix accessing external SQS port for intra-service communication
* fix positional args in CloudFormation find_change_set(..) method
* fix SNS PublishBatch logic for optional Subject atttribute
* fix Kinesis installation via LPM, add Docker check for StepFunctions installation
* fix docker client ctx manager and json decode issues on windows
* fix passing Conditions on stack updates, refactor/simplify CFN function arguments
* fix EC2 route tables and service backend initializations for recent upstream moto changes
* add config for host path to fix mounting of libs_dir
* minor fix in SNS tests to resolve stale PR merge
* minor fix in printing result for local node.js Lambda executor
* ASF rest-json parser fixes
* minor fix in request parsing to match path parameters with slashes
* replace get_data() with response.content to fix logic in SNS HTTP subscriptions
* fix gzip handling in SSL proxy

Change Log

1. New Features

* add initial ASF (AWS Service Framework) SQS provider implementation
* add automatic Docker network detection for Lambda containers
* add support for dynamic SSM/SecretsManager references in CloudFormation templates
* add version info to internal health HTTP endpoint

2. Enhancements

* add support for gzipped responses in SSL proxy via Accept-Encoding headers
* lowercase 'accept' headers in API Gateway for parity with AWS
* add negative test for SNS FIFO topic validation, migrate SNS tests from unittest to pytest
* add ability to customize default IAM/STS user via TEST_IAM_USER_ID/TEST_IAM_USER_NAME
* add integration test that passes binary data via APIGateway to Lambda
* refactor StepFunctions multi-region support to use upstream changes
* add proper validations for SNS/SQS integration with fifo queues
* allow provider overrides to be passed to the started LocalStack container
* ensure SequenceNumber is present in Lambda events from DynamoDB Streams
* support updating of API Gateway resources via PATCH operations
* refactor logging code to replace string interpolation with passing arguments to log methods
* refactor code to remove unnecessary list comprehensions
* refactor code to remove mutable default arguments
* pull out subtypes instance manager into separate util class for reusability
* update and clean up outdated documentation
* replace dict and list function calls to literal syntax
* remove `uname` command in system check for windows compatibility

3. Bug Fixes

* fix API gateway proxy resources
* fix passing of request parameters to API Gateway HTTP integrations
* fix Firehose-ElasticSearch integration, allow S3Backup AllDocuments with ElasticSearchDestination
* fix multiple service container creation
* fix extraction of filter values for describing EC2 prefix lists
* fix single-space env default in docker-compose
* fix RenovateBot config to only enable patch updates

Announcements

* **Security fixes**: This release upgrades **log4j** dependencies to version `2.17.0` to fix the critical security vulnerabilities [CVE-2021-44228](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44228), [CVE-2021-45046](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-45046)

Change Log

1. New Features

* add initial support for region-based in-place partition rewriting
* add "--offline" pytest flag, skip "online-only" tests, fix some tests
* add SKIP_SSL_CERT_DOWNLOAD option to allow skipping download of SSL cert

2. Enhancements

* bump version of Java utils to 0.2.18 to fix log4j vulnerability CVE-2021-44228, CVE-2021-45046
* refactor README.md to make it crisp and readable
* refactor Lambda logic to remove local executor Callable from model entities
* automatically pull non-cached Docker images on image inspection
* replace dict calls with comprehensions
* small fixes to allow running LocalStack with podman
* reduce log level of edge port configuration hook
* correctly wait for stay-open port to be available, fix fallback to exec mode
* add compatibility checks for S3 copy object with metadata
* add proper error response message to reject empty SNS messages
* minor refactoring to use negative index -1 to get the last element of sequence
* minor refactoring of Lambda API for better extensibility

3. Bug Fixes

* fix updating of HTTP method in message handler chain to avoid None methods
* fix misc. tests failing for non-default region
* fix Terraform test issue related to SQS changes in the AWS provider
* fix association of VPCs in Route53 HostedZone responses
* fix CloudFormation updates for EC2::Instance with empty SecurityGroups property
* fix listing of KMS signing keys for asymmetric key pairs