This is our second major release with many new features, enhancements, and improved parity for more extensive integration across the AWS tooling ecosystem. With LocalStack 2.0, we have significantly optimized the internals of the platform and moved to new service implementations, images, and internal toolings to make it easy for developers to build & test their cloud applications locally!
Several of these changes require a migration, and we have done our best to make the migration as smooth as possible. Please find more in the _How to migrate_ section.
Summary
Community
- LocalStack community image (`localstack/localstack`) [no longer contains LocalStack Pro](https://discuss.localstack.cloud/t/separating-localstack-community-and-pro-containers/236) (`localstack/localstack-pro`) code, effectively cutting the community image size in half
- Removal of `localstack-full` and `localstack-light` since they have become obsolete thanks to our new dependency packaging system (simply use `localstack` or `localstack-pro`)
- Completely new AWS Lambda provider with improved parity and performance
- Completely new S3 provider with improved parity and features
- Improved supports for community cloud pods
- Simplified host configuration and docker networking
- Internal endpoints have moved into `/_localstack` and `/_aws`
Pro
- Cross-service IAM enforcement
- Completely new snapshot persistence (`PERSISTENCE=1`) mechanism with more flexible load and save strategies
- New simplified container setup for [AWS Big Data Technologies](https://aws.amazon.com/big-data/datalakes-and-analytics/) (Athena, Glue, EMR, etc)
How to migrate
General
* ⚠️ If you are using LocalStack Pro with a `LOCALSTACK_API_KEY`, please change your container configuration to use `localstack/localstack-pro`. More details can be found here: https://github.com/localstack/localstack/issues/7854
* LocalStack Pro will fail to start if the LocalStack Key activation fails. You can disable this behavior by setting `ACTIVATE_PRO=0`.
Networking
- We are unifying the variables `EDGE_BIND_HOST`, `EDGE_PORT` and `EDGE_PORT_HTTP` into `GATEWAY_LISTEN`, which will allow configuration of the addresses and ports the LocalStack process listens on. It takes the form `<ip address>[:port][,<ip address>:<port>...]` where multiple values can be given, separated by commas. LocalStack will listen on all interfaces, but will ask for superuser permission for privileged ports. We will still accept `EDGE_BIND_HOST`, `EDGE_PORT`, and `EDGE_PORT_HTTP` during a deprecation period following the release of v2, but they will not be used by LocalStack for configuration at some point in the future. Please migrate your use of `EDGE_BIND_HOST`, `EDGE_PORT` or `EDGE_PORT_HTTP` to `GATEWAY_LISTEN`. If you experience issues, please try the deprecated configuration variables.
- For example, if you previously ran LocalStack with the command `EDGE_BIND_HOST=0.0.0.0 EDGE_PORT=5000 EDGE_PORT_HTTP=5001 localstack start` or used these configuration variables in your `docker-compose.yml`, please replace it with `GATEWAY_LISTEN=0.0.0:5000,0.0.0.0:5001 localstack start`.
- We are unifying the variables `HOSTNAME_EXTERNAL` and `LOCALSTACK_HOSTNAME` into `LOCALSTACK_HOST`, which will allow configuration of hostnames returned by LocalStack in a more consistent way. If provided, this variable will be used in preference to `HOSTNAME_EXTERNAL` and `LOCALSTACK_HOSTNAME`, and will be used as the hostname returned in URLs for created resources such as OpenSearch clusters, SQS queues, or RDS databases. We will still accept `HOSTNAME_EXTERNAL` or `LOCALSTACK_HOSTAME` during a deprecation period following the release of v2, but they will not be used by LocalStack for configuration at some point in the future. Please migrate your use of `HOSTNAME_EXTERNAL` or `LOCALSTACK_HOSTNAME` to `LOCALSTACK_HOST`. If you experience issues, please try the deprecated configuration variables.
- For example, if you previously ran LocalStack with the command `HOSTNAME_EXTERNAL=<hostname> localstack start` or used `HOSTNAME_EXTERNAL` in your `docker-compose.yml`, please replace it with `LOCALSTACK_HOST=<hostname>[:port]`.
- Starting LocalStack Pro using the CLI will no longer publish port `53` to the host if it is already bound by some other service like `systemd-resolved` on Linux, or `mDNSResponder` on macOS.
Persistence & cloud pods
Using LocalStack Pro with `PERSISTENCE=1` (which we term *snapshot-based persistence*) now has two behavioral changes with respect to when data is restored and saved. More information can be found [in our docs](https://docs.localstack.cloud/references/persistence-mechanism/).
- Restoring state/loading a snapshot (load): Previously, snapshots could only be loaded from disk per-service when services were first initialized, lazy-loading the state the first time a service was used. With the new persistence mechanism introduced in v2, persistent data can be loaded on LocalStack startup. You can configure this behavior by setting `SNAPSHOT_LOAD_STRATEGY` to `on_request` or `on_startup`. The default strategy currently is still `on_request` until we iron out some quirks.
- Saving state/creating a snapshot (save): Previously, a snapshot was created for a particular service on each request. This was mainly to protect against potential data loss if LocalStack would suddenly terminate. We found that this approach leads to several problems, specifically related to concurrency and performance. An alternative approach we have introduced is to store snapshots on LocalStack shutdown, which produces no performance overhead during runtime, but will not protect you against data loss if LocalStack does not terminate correctly. The default strategy is on a scheduled basis, specifically, we take snapshots of services that have changed every 15 seconds. You can configure this behavior by setting `SNAPSHOT_SAVE_STRATEGY` to `on_request`, `on_shutdown` or `scheduled`, respectively.
The layout of **Cloud Pods** has been changed and cloud pods created with v1.x.x may be incompatible with v2.0. Please re-create your cloud pods with the latest LocalStack version.
Lambda
**Lambda** has been completely re-written and the current documentation can be found [here](https://docs.localstack.cloud/references/lambda-asf-provider/). There are several behavioral changes to lambda that mostly affect users of the `LOCAL` executor mode. If you run into issues, you can, for now, use the legacy provider using `PROVIDER_OVERRIDE_LAMBDA=legacy`.
- Mounting the Docker socket `"/var/run/docker.sock:/var/run/docker.sock"` into the LocalStack container is now required to run Lambdas.
- The default behavior is now equivalent to the old `docker-reuse` executor, there are no longer multiple options.
- Local executor: with `LAMBDA_EXECUTOR=local` , lambdas were executed within the LocalStack container. This was mostly used as a fallback if the docker socket could not be mounted into the LocalStack container. If you cannot mount the docker socket or don’t have an external `DOCKER_HOST` available, we provide a new way to run Lambda functions via static worker containers that can be configured manually. The only requirement here is connectivity between the static worker and the LocalStack instance. Predefined workers will be available via images hosted by us: e.g. `localstack/lambda-worker:python3.9`.
- There are several new ways to configure the lambda provider variables that can be found [in our docs](https://docs.localstack.cloud/references/configuration/), which we will update in the upcoming weeks.
- We have migrated our Lambda Docker images from lambci to use the official AWS images, which will now be pulled by default from `public.ecr.aws/lambda/`
- Functions were previously created synchronously, i.e., `CreateFunction` calls would block until the [function state](https://docs.aws.amazon.com/lambda/latest/dg/functions-states.html) was “Active”. Functions are now created asynchronously, and their state will move from “Pending” to “Active”, which you can check with a `GetFunction` call. This behavior can be disabled with `LAMBDA_SYNCHRONOUS_CREATE=1` (not recommended).
- Stricter input validation: previously, when creating functions, the `Role` attribute could be any value, and many of our examples included something like `awslocal create-function --role r1`. This will no longer work, as roles now have to have an ARN format. We do not t validate whether the role exists, so you can use any ARN.
- Hot reloading: previously, the magic S3 bucket name for lambda code hot reloading was `__local__`, which was changed to `hot-reload`. Please change your deployment configs accordingly, or use the `BUCKET_MARKER_LOCAL` configuration to customize the value.
- Lambda in LocalStack Pro supports “transparent endpoint injection”, which allows Lambdas to resolve domains like `s3.amazonaws.com` to the LocalStack container. Previously, this was based on patching SDKs, but is now completely DNS-based, and will be disabled if `DNS_ADDRESS=0` is set.
S3
S3 has been completely re-written and its behavior is aligned with AWS. Users should be mostly unaffected, but may experience some breakage depending on previous behavior that was not aligned with AWS. Should you run into problems, you can activate the old provider with `PROVIDER_OVERRIDE_S3=legacy`.
Big Data container
- The mono container mode (`BIGDATA_MONO_CONTAINER=1`) for BigData services (Glue, EMR, Athena, etc) is now the default, the previous implementation with a separate sidecar `localstack_bigdata` container is deprecated and will be removed in a future release.
- Please note that some of the required dependencies (Spark, Hive, etc) are lazily downloaded and installed at runtime, which increases the processing time on first load. The libraries are cached locally in the `var_libs` directory - please make sure to properly mount the LocalStack volume into your container. We also provide a separate `localstack/localstack-pro:latest-bigdata` BigData mono container image which has the default dependencies pre-installed.
Other notable changes
* supervisord is no longer used to start LocalStack in the Docker container, instead we use our own init program.
* LocalStack no longer automatically restarts on failure
* /var/lib/localstack/logs/localstack_infra.log was removed
* `REQUIRE_PRO` has been replaced with `ACTIVATE_PRO`, which sets whether or not LocalStack pro should be activated when using the pro image.
New deprecations
- With all new provider implementations, some configuration variables have been deprecated with this release and will be removed in future releases:
- `HOSTNAME_EXTERNAL` - This configuration will be migrated to `LOCALSTACK_HOST`
- `LOCALSTACK_HOSTNAME` - This configuration will be migrated to `LOCALSTACK_HOST`
- `EDGE_BIND_HOST` - This configuration will be migrated to `GATEWAY_LISTEN`
- `EDGE_PORT` - This configuration will be migrated to `GATEWAY_LISTEN`
- `EDGE_PORT_HTTP` - This configuration will be migrated to `GATEWAY_LISTEN`
- `LAMBDA_EXECUTOR` - This configuration is obsolete with the new lambda provider
- `LAMBDA_STAY_OPEN_MODE` - Stay open mode is the default behavior in the new lambda provider
- `LAMBDA_REMOTE_DOCKER` - The new lambda provider copies zip files by default and automatically configures hot reloading
- `LAMBDA_CODE_EXTRACT_TIME` - Function creation now happens asynchronously in the new lambda provider
- `LAMBDA_CONTAINER_REGISTRY` - The new lambda provider uses `LAMBDA_RUNTIME_IMAGE_MAPPING` instead
- `LAMBDA_XRAY_INIT` - The X-Ray daemon is always initialized in the new lambda provider
- `LAMBDA_DOCKER_DNS` - This feature is currently not supported in the new lambda provider
- `HOSTNAME_FROM_LAMBDA` - This feature is currently not supported in the new lambda provider
- `LAMBDA_FALLBACK_URL` - This feature is not supported in the new lambda provider
- `LAMBDA_FORWARD_URL` - This feature is not supported in the new lambda provider
Deprecation removals
- Deprecation removals:
- `LEGACY_EDGE_PROXY` has been removed
- `LEGACY_DIRECTORIES` has been removed, please migrate to the current [Filesystem Layout](https://docs.localstack.cloud/references/filesystem/)
- `LEGACY_IAM_ENFORCEMENT` has been removed
- `/docker-entrypoint-initaws.d` for initializing LocalStack has been removed, please migrate to the modern [Initialization Hooks](https://docs.localstack.cloud/references/init-hooks/)
- please find other minor deprecation notices in previous release notes.
What's Changed
* Update ASF APIs by localstack-bot in https://github.com/localstack/localstack/pull/7669
* [SFN] basic version of stepfunctions v2 by MEPalma in https://github.com/localstack/localstack/pull/7464
* Avoid installing amazon_kclpy>=2.1.0 by dominikschubert in https://github.com/localstack/localstack/pull/7676
* Fix CFn Route53 Record Set TTL by pinzon in https://github.com/localstack/localstack/pull/7677
* use lifecycle hook for registering lambda function URL routes by thrau in https://github.com/localstack/localstack/pull/7668
* add proper shutdown for the hypercorn server that serves the ASF gateway by thrau in https://github.com/localstack/localstack/pull/7609
* fix encoding issues in router, proxy, partition rewriting by alexrashed in https://github.com/localstack/localstack/pull/7682