localstack / localstack

Change Log:

LocalStack release 0.12.16

1. New Features

• add integration for Thundra tracing via THUNDRA_APIKEY
• add support for local execution of Node.js lambdas
• add support for DeleteAlias and TracingConfig in Lambda API
• support sequential Lambda execution for Kinesis streams, using ParallelizationFactor
• support resolving SSM parameter values in CloudFormation templates
• add possibility for services to subscribe to runtime config changes
• add TEST_TARGET=AWS_CLOUD option to run tests against real AWS

2. Enhancements

• update README with instructions to build the Docker image
• add integration test for EC2 route table association
• adjust updated download URL for patched DynamoDBLocal.zip
• update kinesis-mock version to fix missing Logger classes
• store SES templated email messages to DATA_DIR or tmp folder for easier debugging
• refactor and extend AWS response parsing utils
• remove Web dashboard process from supervisord.conf
• extend test utils to pass in an auth_creator_func(..) for creating API GW authorizers
• minor: remove deprecated web UI documentation
• add option to disable boto3 retries/timeouts
• remove EventBus proxy listener patches for Tags operations handled upstream
• integrate and use the util function to remove Docker env files
• print version string if Terraform version check fails in integration tests
• replace unparse with custom XML serializer for AWS responses
• support CloudFormation string placeholders with triples like "${resource.attr.subattr}"
• improve kinesis startup routine and add KINESIS_MOCK_FORCE_JAVA config option
• replace botostubs with boto3-stubs in test fixtures
• add retry logic for downloading Elasticsearch plugins to fix temporary network issues
• extend ACM ARN integration test when importing certificates
• improve infra graph code, extend tests, add new fixtures
• add tests for Lambda invocation destinations
• add unit tests for common and config
• add JSON safe parsing when processing API GW event in local Javascript Lambda
• patch moto to include region header for GET and HEAD requests
• update CI workflow to aggregate coverage report
• use JAVA_TOOL_OPTIONS to apply LAMBDA_JAVA_OPTS in Java Lambdas
• cleanup and refactor CI configurations
• add default account_id for API GW Lambda invocation events
• add ability to define UDP Docker ports in PortMappings class
• strip off warning messages when extracting Docker container ID from stdout
• add exception response when calling DeletePolicy on non-existent IAM policy
• refactor Lambda executor to use unified container API and break apart container/Lambda logic
• pin version of "requests" library to fix import errors in Lambda tests
• add feature that saves LocalStack version info into data dir
• add pre-commit hook that runs linting and formatting
• add support for CloudFormation Fn::GetAtt WebsiteURL for S3 buckets
• refactor JSONPatch implementation for API GW to handle edge cases more reliably
• add make targets for linting and formatting only modified files
• add proper headers when triggering API destination target
• add CORS headers required for AWS SDK v3
• refactor Firehose and properly return "ExtendedS3DestinationDescription" with extended attributes
• add support for API Gateway documentation parts (tf-compat)

3. Bug Fixes

• fix regex to extract API ID in hostname from API GW endpoint URL
• fix logic in inject_test_credentials_into_env(..) to set custom env parameters for Lambda functions
• fix DynamoDB to Kinesis stream event format - add "tableName", remove "StreamViewType"
• fix Docker detection on hosts with cgroup v2 (unified hierachy)
• fix running multiple API Gateway JSON Patch operations in sequence (tf-compat)
• fix timestamp format of Lambda event source mappings for TF compatibility
• fix returning error response when requesting ACL for non-existing S3 bucket
• fix edge routing to match rules if request host contains a dot "."
• fix EXTRA_CORS_ALLOWED_ORIGINS environment variable to match docs
• fix ARN structure for EventBridge resources
• fix extraction of EventBridge API destination name using new ARN format
• fix S3 host-based routing detection
• fix accessing missing "body" from Lambda response in API GW
• fix JSON Patch for API GW method settings
• fix some data types for API GW patch operations
• fix "application/xml" content-type header for S3 responses
• minor: fix file path in "config validate" CLI command

Change Log:

LocalStack release 0.12.15

1. New Features

• add support for Kinesis CBOR error messages
• add CF support for CertificateManager::Certificate

2. Enhancements

• security fixes: bind edge service to 127.0.0.1 by default (instead of 0.0.0.0)
• format the codebase with black and isort
• make for-loop in Makefile /bin/sh compatible
• add flake8 rule F401 (unused imports)
• update code style badge in README.md
• run pre-release smoke tests for Pro features in CI builds
• remove quiet parameter from service starting
• refactor common.py and add tests
• remove dead/unused CloudFormation code
• update bug-report issue template to assign correct label
• add workaround for kinesis-mock persistence startup failure
• reduce log verbosity for CORS/origin checks
• update .dockerignore to exclude files generated in CI build
• restructure Elasticsearch process management
• add unit test for moto parsing of complex params from querystring
• refactor tests to fix/invert order of arguments in test assertions
• refactor test_integration.py to use randomly generated resource names
• minor formatting in bug-report issue template

3. Bug Fixes

• add additional allowed CORS origins to fix Cognito issues
• fix processing of empty values in EventBridge event filters
• fix JSON Patch operations for API Gateway usage plans
• fix detection of resource status to allow re-deployment of CF stack after deletion
• fix installation of debugpy via pip module instead of calling pip CLI
• fix minor formatting issues in http/proxy server

Change Log:

LocalStack release 0.12.14

1. New Features

• add support for EventBridge API destinations
• add KINESIS_INITIALIZE_STREAMS configuration to initialize Kinesis streams on startup
• add support for API GW MOCK integration and response parameters
• add CloudFormation support for CloudWatch::Alarm resources
• add support for V4 S3 presigned URLs with custom headers
• add support for isBase64Encoded flag with Lambda integrations in API Gateway

2. Enhancements

• security fixes: fix CORS setup to prevent CSRF, add DISABLE_*_CORS_* configs
• migrate from nose to pytests
• refactor test_error_injection.py into a unittest.TestCase class
• refactor tests and invert/fix order of assertion arguments
• update coveralls config for new test setup with pytests
• improve version check in check_terraform_version
• parallelize builds and improve CircleCI workflow
• pin serverless version in TestServerless to 2.48.0 to fix test issues
• add threading event to indicate infrastructure has started
• add coverage install to Dockerfile.base to enable coverage C extensions
• remove superfluous coveralls config, now managed via CI env
• fall back to using JAR for kinesis-mock installation for ARM systems (e.g., Macbook M1)
• add config option TF_COMPAT_MODE to enable TF compatibility mode (for TF integration tests)
• minor: update badges in README
• minor refactoring: add dynamodb_table_exists() to common utils
• minor: fix KeyError for missing EventSourceArn in Lambda API lookup
• minor: make "enabled" flag to add_job(..) optional
• minor: add parse_json_or_yaml() util function

3. Bug Fixes

• fix prefix filtering of events in EventBridge API in case of exact match
• fix deprecation warnings surfaced when introducing pytest
• fix extracting build report step in docker-build CI phase
• fix passing environment variables to container creation for LAMBDA_EXECUTOR=docker-reuse
• fix linter exclusions in install.py
• fix request params for CF deployment of ApiGateway::Stage
• fix time duration unit for profiler log_duration output
• minor fix in file regex for plugin lookup to accommodate Windows path separators
• minor fix to avoid exception on missing CloudFormation exports
• minor fixes in S3 API

Change Log:

LocalStack release 0.12.13

1. New Features

• introduce kinesis-mock as default provider for Kinesis (replacing kinesalite)
• add support for API Gateway client certificates
• add support for API Gateway - S3 integration for GET requests
• add BUCKET_MARKER_LOCAL option to customize S3 bucket name (__local__) for local Lambda code mounting
• add support for API Gateway VPC Links
• add support for requestParameters in API Gateway method integrations

2. Enhancements

• add CF support for ResourceGroups::Group
• add support for x-id query param for S3 delete object requests
• add tagging support for IAM policies and instance profiles
• add support for Kinesis EnableEnhancedMonitoring
• add support for EC2 create_vpc_endpoint_service_configuration
• use the new MOTO_S3_DEFAULT_KEY_BUFFER_SIZE variable for S3 in-memory buffer sizes
• refactor shutdown procedure
• use --env-file to pass env vars to Lambda containers to prevent cmd line arguments from becoming too long
• ignore trailing slashes for some routes in Lambda API
• update PYTHONPATH in image to accommodate both python v3.8 and v3.7
• refactor API GW impl. for better compatibility with Terraform
• parallelize SNS message delivery for improved performance
• make parsing of IAM::Role properties more resilient
• minor refactoring around Kinesis handler and x-amzn-errortype response header
• minor: update Helm repo URL in README from HTTP to HTTPS
• minor refactoring of Dockerfile and API Gateway tests

3. Bug Fixes

• fix minor formatting issue in profiler log
• fix S3 bucket notification FilterRule name validation
• fix deprecation warnings in unit tests
• fix timezone issues in timestamp calculation and datetime comparisons
• fix response for Lambda configuration update API
• fix passing payload via stdin for large Lambda events using docker-reuse
• fix duplicate Content-Type header in Lambda invocation responses
• fix API GW JSONPatch operations and error responses
• fix start_edge to gracefully fail when stdin for sudo is not available
• fix continuation sequence number in event returned by SubscribeToShard Kinesis API
• fix symlink to aws CLI executable in Dockerfile
• fix datetime format for Lambda put-function-event-invoke-config response
• fix skipping S3 signature validation if S3_SKIP_SIGNATURE_VALIDATION is enabled
• fix stripping off qualifier suffix when constructing ARN for Lambda lookup
• fix returning error response for attempting to delete non-existing S3 bucket
• minor fix in extraction of Lambda results from log output lines
• minor fix for Kinesis max stream retention period
• minor fix when extracting URI for API GW integrations
• minor CF fix to extract PhysicalResourceId via get_physical_resource_id()

Change Log:

LocalStack release 0.12.12

1. New Features

• add initial support for Resource Groups API
• add support for managing request payers on S3 buckets
• save SES email contents to disk in machine-readable format
• add support for LAMBDA_DOCKER_FLAGS to support custom Lambda Docker run flags
• add S3_SKIP_SIGNATURE_VALIDATION config to skip S3 presign URL signature validation
• add DEVELOP configuration for debugpy integration
• add support for servicediscovery endpoints in API GW invocations
• add Lambda Ruby 2.7 image support and update tests
• add support for DynamoDB table replicas
• add support for Route53 reusable-delegation-set APIs

2. Enhancements

• enhance validation of presigned URLs to support different port numbers in S3 URLs
• add XRay trace headers in notifications from S3 to SQS
• support ElasticsearchClusterConfig parameter for creation of ES clusters
• add support for KinesisStreamSpecification parameter for AWS::DynamoDB::Table CF resource
• add patch for stepfunctions-local to enable cross-region Lambda access
• add test for creating bucket with existing bucket name
• add support for session_token in S3 presign URLs
• remove obsolete DynamoDBLocal fix for Alpine
• add EventsBackend to manage state of Events API resources
• add UnsubscribeURL in payload sent to HTTP SQS endpoints
• add docs for Kafka and Elasticsearch external services integration
• temporarily set S3_SKIP_SIGNATURE_VALIDATION=1 by default (until all issues with S3 presign URLs are resolved)
• add CF support for ApiGateway::RequestValidator, Logs::SubscriptionFilter, Redshift::Cluster
• add region backend for DynamoDB state
• add support for arn:aws-us-gov:iam:... managed IAM policies
• update version in sample docker-compose.yml
• add test case for generating S3 presign URL from Node.js Lambda
• add simple TCP proxy server util
• add "running" property to FuncThread
• refactor logic for sending SQS messages to DLQ
• cache target API in edge requests for improved performance
• skip gzip encoding for S3 responses to fix content-length issue with Go SDK
• rename currentThread to current_thread for deprecation in Python 3.10
• add missing endpoints for StepFunctions startup command
• minor: add utility function to clean up Docker containers
• minor refactoring in API Gateway implementation
• minor enhancements in base image to optimize disk space
• minor refactoring for EC2::VPCGatewayAttachment CF model
• minor: add simple items_equivalent() util function
• minor refactoring of docker-reuse Lambda executor for better extensibility
• minor: allow passing keyword args to async_utils.run_sync(..)
• minor: initialize Properties for CF resource instances

3. Bug Fixes

• fix missing SQS message attributes for notifications via SNS
• fix using configured edge port in S3 presign URL calculation
• fix EC2::Route CF model to properly lookup deployment state if DestinationIpv6CidrBlock not set
• fix fetching CF resource state for EC2::Route and Logs::LogGroup
• fix broken SNS test related to UnsubscribeURL returned on subscription attributes
• fix edge route mapping for SES requests using /email route
• fix syntax in aws_stack.py for compatibility with python 2.x
• fix incorrect suffixes in SecretManager ARNs for secret lookups
• fix Python sys path for module resolution when using local Lambda executor
• fix integration test assertions to accommodate recent CloudWatch metrics changes
• fix idempotence for SNS CreatePlatformEndpoint API calls
• fix correct error code for getting attributes of non-existing SNS subscription
• fix correct Authn header for internal call when replaying non HMAC request
• fix filtering of CloudWatch metrics and dimensions
• fix URL escaping when calculating presigned URLs to accommodate recent Quart server update
• fix double response headers in CORS preflight API Gateway requests
• fix resetting ETags when copying S3 object from multipart upload
• fix accessing raw response when returning gzip encoded content from Lambdas via API GW
• fix copying missing events file into container for Java Lambdas with docker-reuse
• fix Kinesis response payload encoding for use with Java SDK
• minor fix in ACM to avoid returning duplicate DomainValidationOptions

Change Log:

LocalStack release 0.12.11

1. New Features

• add support for ResourceGroupsTaggingAPI
• add support for Kinesis targets in EventBridge
• add PERSISTENCE_SINGLE_FILE configuration
• add HOSTNAME_FROM_LAMBDA configuration to enable connectivity in compose stacks with dynamic container names
• add filtering in EventBus using event patterns: prefix,numeric,anything-but
• add throttling support for DynamoDB BatchWriteItem requests

2. Enhancements

• add dynamic DNS check to fix issues with resolving s3.localhost.localstack.cloud
• refactor CF model and fix deletion method for Lambda::Function
• add missing $context variable for VLT templates
• add requestTemplates attr for ApiGateway::Method
• add domainName and apiId to Lambda events for API Gateway invocations
• remove default cache-control header in S3 responses
• handle x-id query parameter for S3 DeleteObjects requests
• remove obsolete USE_HTTP2_SERVER and HOSTNAME configuration variables
• add missing resourcePath to API GW Lambda events to fix Chalice invocations
• strip quotes off StreamARNs when creating Kinesis stream consumers
• enhance CloudFormation support for API GW Model resources
• minor: refactor Lambda impl. for better extensibility
• minor refactoring in tests and AWS response utils

3. Bug Fixes

• fix missing PATCH method in API GW CORS configuration
• fix Event::Rule CF model to properly deploy rule with custom event bus
• fix fetching Lambda details using ARN as well as partial ARN
• fix bug in handler for DynamoDB BatchWriteItem API
• fix S3 presigned URLs with response header overrides
• fix object tagging for S3 multipart uploads
• fix API Gateway invocation paths in Lambda events
• fix returning Arn attribute for Logs::LogGroup CF resource
• fix idempotency for creation of SNS topics with tags
• fix DynamoDB to return empty response on no unprocessed items
• minor fix for accessing non-existing attribute in ACM models

Change Log:

LocalStack release 0.12.10

1. New Features

• add initial support for CloudFormation stack sets and stack instances
• initial support for AWS Support API
• add support for iterator types on Kinesis SubscribeToShard requests
• support Kinesis streaming destinations for DynamoDB
• initial support for put_metric_filter for CloudWatch log events
• add support for HttpEndpoint destinations in Firehose API
• add health check status for init scripts and persistence initialization

2. Enhancements

• replace AWS CLI commands with boto3 calls for /graph endpoint
• remove obsolete code for S3 bucket encryptions
• add CloudFormation support for EC2::NatGateway, KMS::Key, KMS::Alias, SecretsManager resources
• return PhysicalResourceId for S3::BucketPolicy CloudFormation model
• add edge mapping for EC2 requests with missing Auth headers
• remove deprecated USE_MOTO_CF config switch
• resolve refs in ExportName of CF stacks
• add missing attributes in Kinesis SubscribeToShard response
• minor refactoring and adding missing path in Dockerfile
• cache cfn-response module in local infra folder instead of re-downloading each time
• support JSON message structure with raw delivery in SNS
• add CloudFormation support for additional EC2 resources
• refactor logic for CF stack set instances
• add support for getting SSM parameters by path and filtering by labels
• remove "content-encoding: identity" header in S3 responses to fix compatibility with PHP/Guzzle 6
• add SNS XRay propagation to SQS using ElasticMQ
• add Circle-CI project setup and build badge
• move CI build commands into Makefile for easier migration
• refactor utils for resource tagging
• refactor logic for proxy listener, fix edge forward headers
• update S3 regex to support HOSTNAME_EXTERNAL in bucket hostnames
• add missing aws_request_id and context attributes for Python Lambdas with local executor
• add support for SES GetIdentityVerificationAttributes
• add LS_LOG=trace option to print detailed request/response messages
• add CloudFormation support for EC2::VPC, EC2::RouteTable, EC2::Route
• add Location response header on S3 bucket creation
• minor refactorings for "config validate" command

3. Bug Fixes

• fix API Gateway integration to allow binary payload data
• fix reprocessing of SQS messages in case multiple Lambda event source mappings are defined
• fix API GW integrations with no uri/integrationUri attribute (for mock integrations)
• fix StreamViewType in DynamoDB Streams event records
• fix stack status filter in CloudFormation API
• fix CF dependency resolution to avoid unresolved/nil resource refs
• fix Events API logic for recent upstream changes
• fix API Gateway when applying JSON patch with empty patch value
• fix passing region when constructing resource graph
• fix CloudFormation GetAtt intrinsic function for attr string refs
• fix proper stream view type of DynamoDB stream records
• fix error logs on subscribe_to_shard for deleted Kinesis streams
• fix build script to detect git:// repo URLs
• fix compatibility of S3 API with Cyberduck
• fix local URLs for nested CFN stacks
• fix Kinesis DescribeStreamConsumer to return consumer correctly
• fix content-type for requests forwarded to Lambda fallback URL
• fix returning None value on TransactWrite DynamoDB operation
• fix incorrect URL modification when sending S3 requests to backend
• fix encoding and timestamp format for Kinesis SubscribeToShard from Java SDK
• fix returning code 200 for S3 HEAD range requests
• fix S3 notifications for URLs with "x-id" query parameters
• minor fix for missing Lambda attributes on legacy persistence states
• minor fix in util function for parsing URL-encoded request payloads
• minor fix for is_port_open(..) util function to parse port parameters
• minor fix for extracting region from auth headers

Change Log:

LocalStack release 0.12.9

1. New Features

• major: add support for virtual host based addressing in S3 requests
• add support for signature authentication for S3 host addressing based presigned URLs
• add support for IAM service linked roles
• add ability to import REST APIs for APIGateway

2. Enhancements

• enable and fix asynchronous CF stack deployments
• add small util function to list local Docker images
• configure KMS to use $DATA_DIR as persistence folder
• support PutRecord Kinesis endpoint in API Gateway integrations
• minor refactoring in download util functions
• add small util function to create Lambda API Gateway integration
• remove duplicated CFN integration test
• add test to validate export names for CFN deployment
• add LastUpdatedTimestamp for CF resources
• minor refactoring in configs and S3 presigned URL tests
• add missing get_physical_resource_id for IAM::InstanceProfile CF model

3. Bug Fixes

• fix missing parameters in download utils function
• fixed slash prefixes for path in S3 static website hosting
• fix addressing style by setting "auto" in default S3 clients
• fix incorrect date format for policy expiration date on S3 signing
• fix S3 Location Constraint for access with Node.js AWS SDK
• fix key name with special char and multipart upload in S3 presigned URL
• fix ARN format for IAM service roles
• fix encoding and strip off quotes for kinesis bytes input
• fix S3 bucket location constraint value for us-east-1 region
• fix error response for S3 get-bucket-cors in case no CORS config exists
• fix EC2 responses for compatibility with Terraform VPC module
• fix using regexes to match and extract RSA private keys
• fix determining ID for deletion of ApiGateway::RestApi resources
• fix failing tests for S3 presign URLs using incorrect bucket host
• fix lookup of CF resource ID for IAM::InstanceProfile model
• fix determining state for EC2::SecurityGroup CF resources
• fix Serverless tests and run CF deployments asynchronously
• fix presigned GET requests with querystring auth parameters
• minor fix for API GW patch operations on non-existing attributes
• minor fix for accessing IAM role attributes
• minor fix: make Lambda handler file executable in test utils
• minor fix for SNS topic deletion in CloudFormation

Change Log:

LocalStack release 0.12.8

1. New Features

• add support for v4 signature calculations in S3 presigned URLs
• add support for nodejs14.x Lambda runtimes
• add CF support for EC2::Subnet, EC2::VPC, Events::EventBus, ApiGateway::Model

2. Enhancements

• add auto_restart option to ShellCommandThread to restart DynamoDB process on error
• run CF deployments asynchronously in the background
• extract auth details from query parameters in edge service
• add test for Java Lambda archive built from Gradle
• check existence of KinesisConsumer and add missing Header to binary Kinesis messages
• add small util function to get or create S3 bucket
• automatically confirm SNS subscriptions to fix TF deployments
• extend TF tests with EC2 resources
• add logs with build info when process starts up
• add support for deletion of SES templates
• refactor tests and logic for EventBridge targets

3. Bug Fixes

• fix extraction of SQS QueueArn in CloudFormation models
• fix mechanism to determine whether LocalStack is running inside Docker for Github Actions compatibility
• fix extracting list of changes in CloudFormation change sets
• fix result processing of external Lambda invocations via LAMBDA_FORWARD_URL
• fix edge route mapping logic to identify EventBridge API calls
• fix incorrect region parameter in EventBridge tests
• fix SQS redrive policy tests failing after recent upgrade
• fix tests for new SNS subscription attributes
• fix missing 'detail' field when putting EventBridge events with empty details
• fix resource path in event sent to Lambda from API Gateway
• fix log output for "localstack config" CLI commands
• minor fix in API GW request routing
• minor fix for SES response parsing