- Feature: AWS Glue Data Catalog now enhances managed table optimizations of Apache Iceberg tables that can be accessed only from a specific Amazon Virtual Private Cloud (VPC) environment.
- Dependency Update: Update to smithy-go v1.22.1.
- Dependency Update: Updated to the latest SDK module versions
- No change notes available for this release.
- Bug Fix: Adds case-insensitive handling of error message fields in service responses
- Dependency Update: Updated to the latest SDK module versions
- Feature: Add schedule support for AWS Glue column statistics
- Dependency Update: Updated to the latest SDK module versions
- Dependency Update: Updated to the latest SDK module versions
- Dependency Update: Updated to the latest SDK module versions
- Feature: Add support for HTTP client metrics.
- Dependency Update: Updated to the latest SDK module versions
- No change notes available for this release.
- No change notes available for this release.
- No change notes available for this release.
- Feature: Added AthenaProperties parameter to Glue Connections, allowing Athena to store service specific properties on Glue Connections.
- Feature: Add tracing and metrics support to service clients.
- Dependency Update: Updated to the latest SDK module versions
- Feature: This change is for releasing TestConnection api SDK model
- Bug Fix: BREAKFIX: Only generate AccountIDEndpointMode config for services that use it. This is a compiler break, but removes no actual functionality, as no services currently use the account ID in endpoint resolution.
- Feature: AWS Glue is introducing two new optimizers for Apache Iceberg tables: snapshot retention and orphan file deletion. Customers can enable these optimizers and customize their configurations to perform daily maintenance tasks on their Iceberg tables based on their specific requirements.
- No change notes available for this release.
- Dependency Update: Updated to the latest SDK module versions
- Feature: Add optional field JobRunQueuingEnabled to CreateJob and UpdateJob APIs.
- Dependency Update: Bump minimum Go version to 1.21.
- Dependency Update: Updated to the latest SDK module versions
- Feature: Add AttributesToGet parameter support for Glue GetTables
- Feature: This release adds support to retrieve the validation status when creating or updating Glue Data Catalog Views. Also added is support for BasicCatalogTarget partition keys.
- Feature: Introducing AWS Glue Data Quality anomaly detection, a new functionality that uses ML-based solutions to detect data anomalies users have not explicitly defined rules for.
- Feature: Add recipe step support for recipe node
- Dependency Update: Updated to the latest SDK module versions
- Feature: Add recipe step support for recipe node
- Dependency Update: Updated to the latest SDK module versions
- Feature: Added AttributesToGet parameter to Glue GetDatabases, allowing caller to limit output to include only the database name.
- Dependency Update: Updated to the latest SDK module versions
- Feature: Support list-of-string endpoint parameter.
- Dependency Update: Updated to the latest SDK module versions
- Feature: Track usage of various AWS SDK features in user-agent string.
- Dependency Update: Updated to the latest SDK module versions
- Feature: This release introduces a new feature, Usage profiles. Usage profiles allow the AWS Glue admin to create different profiles for various classes of users within the account, enforcing limits and defaults for jobs and sessions.
- Dependency Update: Updated to the latest SDK module versions
- Feature: This release adds support for configuration of evaluation method for composite rules in Glue Data Quality rulesets.
- Bug Fix: Add clock skew correction on all service clients
- Dependency Update: Updated to the latest SDK module versions
- Feature: This release adds support for creating and updating Glue Data Catalog Views.
- Feature: AWS Glue now supports native SaaS connectivity: Salesforce connector available now
- Dependency Update: Updated to the latest SDK module versions
- Feature: Add optional field JobMode to CreateJob and UpdateJob APIs.
- No change notes available for this release.
- Feature: Add Maintenance window to CreateJob and UpdateJob APIs and JobRun response. Add a new Job Run State for EXPIRED.
- Dependency Update: Updated to the latest SDK module versions
- Dependency Update: Updated to the latest SDK module versions
- Bug Fix: GoDoc improvement
- Feature: Adding RowFilter in the response for GetUnfilteredTableMetadata API
- Feature: Modifying request for GetUnfilteredTableMetadata for view-related fields.
- Feature: Adding View related fields to responses of read-only Table APIs.
- Dependency Update: Updated to the latest SDK module versions
- Dependency Update: Updated to the latest SDK module versions
- Bug Fix: Remove dependency on go-cmp.
- Dependency Update: Updated to the latest SDK module versions
- No change notes available for this release.
- Bug Fix: Move all common, SDK-side middleware stack ops into the service client module to prevent cross-module compatibility issues in the future.
- Dependency Update: Updated to the latest SDK module versions
- Feature: Add middleware stack snapshot tests.
- Dependency Update: Updated to the latest SDK module versions
- Bug Fix: When sourcing values for a service's
EndpointParameters
, the lack of a configured region (i.e.options.Region == ""
) will now translate to anil
value forEndpointParameters.Region
instead of a pointer to the empty string""
. This will result in a much more explicit error when calling an operation instead of an obscure hostname lookup failure.
- Bug Fix: Correct failure to determine the error type in awsJson services that could occur when errors were modeled with a non-string
code
field.
- Feature: Bump minimum Go version to 1.20 per our language support policy.
- Dependency Update: Updated to the latest SDK module versions
- Feature: Introduce Catalog Encryption Role within Glue Data Catalog Settings. Introduce SASL/PLAIN as an authentication method for Glue Kafka connections
- Feature: Update page size limits for GetJobRuns and GetTriggers APIs.
- Dependency Update: Updated to the latest SDK module versions
- Feature: This release adds additional configurations for Query Session Context on the following APIs: GetUnfilteredTableMetadata, GetUnfilteredPartitionMetadata, GetUnfilteredPartitionsMetadata.
- Bug Fix: Reinstate presence of default Retryer in functional options, but still respect max attempts set therein.
- Dependency Update: Updated to the latest SDK module versions
- Bug Fix: Restore pre-refactor auth behavior where all operations could technically be performed anonymously.
- Bug Fix: Correct wrapping of errors in authentication workflow.
- Bug Fix: Correctly recognize cache-wrapped instances of AnonymousCredentials at client construction.
- Dependency Update: Updated to the latest SDK module versions
- Feature: Adds observation and analyzer support to the GetDataQualityResult and BatchGetDataQualityResult APIs.
- Dependency Update: Updated to the latest SDK module versions
- Feature: Expose Options() accessor on service clients.
- Dependency Update: Updated to the latest SDK module versions
- Dependency Update: Updated to the latest SDK module versions
- Bug Fix: Respect setting RetryMaxAttempts in functional options at client construction.
- Feature: add observations support to DQ CodeGen config model + update document for connectiontypes supported by ConnectorData entities
- Dependency Update: Updated to the latest SDK module versions
- Feature: Introduces new column statistics APIs to support statistics generation for tables within the Glue Data Catalog.
- Dependency Update: Updated to the latest SDK module versions
- Feature: Introduces new storage optimization APIs to support automatic compaction of Apache Iceberg tables.
- Dependency Update: Updated to the latest SDK module versions
- Feature: This release introduces Google BigQuery Source and Target in AWS Glue CodeGenConfigurationNode.
- Feature: Adds support for configured endpoints via environment variables and the AWS shared configuration file.
- Dependency Update: Updated to the latest SDK module versions
- Feature: BREAKING CHANGE: Bump minimum go version to 1.19 per the revised go version support policy.
- Dependency Update: Updated to the latest SDK module versions
- Feature: BREAKFIX: Correct nullability and default value representation of various input fields across a large number of services. Calling code that references one or more of the affected fields will need to update usage accordingly. See 2162.
- Feature: Extending version control support to GitLab and Bitbucket from AWSGlue
- Dependency Update: Updated to the latest SDK module versions
- Dependency Update: Updated to the latest SDK module versions
- Feature: Added API attributes that help in the monitoring of sessions.
- Dependency Update: Updated to the latest SDK module versions
- Dependency Update: Updated to the latest SDK module versions
- Dependency Update: Updated to the latest SDK module versions
- Feature: AWS Glue Crawlers can now accept SerDe overrides from a custom csv classifier. The two SerDe options are LazySimpleSerDe and OpenCSVSerDe. In case, the user wants crawler to do the selection, "None" can be selected for this purpose.
- Dependency Update: Updated to the latest SDK module versions
- Feature: This release includes additional Glue Streaming KAKFA SASL property types.
- No change notes available for this release.
- Feature: Adds support for smithy-modeled endpoint resolution. A new rules-based endpoint resolution will be added to the SDK which will supercede and deprecate existing endpoint resolution. Specifically, EndpointResolver will be deprecated while BaseEndpoint and EndpointResolverV2 will take its place. For more information, please see the Endpoints section in our Developer Guide.
- Dependency Update: Updated to the latest SDK module versions
- No change notes available for this release.
- Dependency Update: Updated to the latest SDK module versions
- Feature: Release Glue Studio Snowflake Connector Node for SDK/CLI
- Feature: Added support for Data Preparation Recipe node in Glue Studio jobs
- Feature: This release adds support for AWS Glue Crawler with Apache Hudi Tables, allowing Crawlers to discover Hudi Tables in S3 and register them in Glue Data Catalog for query engines to query against.
- Feature: Adding new supported permission type flags to get-unfiltered endpoints that callers may pass to indicate support for enforcing Lake Formation fine-grained access control on nested column attributes.
- Dependency Update: Updated to the latest SDK module versions
- Feature: This release enables customers to create new Apache Iceberg tables and associated metadata in Amazon S3 by using native AWS Glue CreateTable operation.
- Feature: This release adds support for AWS Glue Crawler with Iceberg Tables, allowing Crawlers to discover Iceberg Tables in S3 and register them in Glue Data Catalog for query engines to query against.
- Feature: Timestamp Starting Position For Kinesis and Kafka Data Sources in a Glue Streaming Job
- Feature: This release adds support for creating cross region table/database resource links
- No change notes available for this release.
- Dependency Update: Updated to the latest SDK module versions
- Feature: Added Runtime parameter to allow selection of Ray Runtime
- Feature: Added ability to create data quality rulesets for shared, cross-account Glue Data Catalog tables. Added support for dataset comparison rules through a new parameter called AdditionalDataSources. Enhanced the data quality results with a map containing profiled metric values.
- Feature: Add Support for Tags for Custom Entity Types
- Feature: This release adds AmazonRedshift Source and Target nodes in addition to DynamicTransform OutputSchemas
- Feature: Support large worker types G.4x and G.8x for Glue Spark
- No change notes available for this release.
- Dependency Update: Updated to the latest SDK module versions
- No change notes available for this release.
- Dependency Update: Updated to the latest SDK module versions
- No change notes available for this release.
- Feature: Add support for database-level federation
- Feature: This release adds support for AWS Glue Data Quality, which helps you evaluate and monitor the quality of your data and includes the API for creating, deleting, or updating data quality rulesets, runs and evaluations.
- Dependency Update: Updated to the latest SDK module versions
- Dependency Update: Updated to the latest SDK module versions
- Bug Fix: Prevent nil pointer dereference when retrieving error codes.
- Dependency Update: Updated to the latest SDK module versions
- Feature: Release of Delta Lake Data Lake Format for Glue Studio Service
- Announcement: When receiving an error response in restJson-based services, an incorrect error type may have been returned based on the content of the response. This has been fixed via PR #2012 tracked in issue #1910.
- Feature: Fix DirectJDBCSource not showing up in CLI code gen
- Bug Fix: Correct error type parsing for restJson services.
- Feature: DirectJDBCSource + Glue 4.0 streaming options
- Dependency Update: Updated to the latest SDK module versions
- No change notes available for this release.
- Feature: Release Glue Studio Hudi Data Lake Format for SDK/CLI
- Feature: Add
ErrorCodeOverride
field to all error structs (aws/smithy-go#401).
- No change notes available for this release.
- Feature: This release adds support for AWS Glue Crawler with native DeltaLake tables, allowing Crawlers to classify Delta Lake format tables and catalog them for query engines to query against.
- Dependency Update: Updated to the latest SDK module versions
- Dependency Update: Updated to the latest SDK module versions
- Feature: This release adds support for AWS Glue Data Quality, which helps you evaluate and monitor the quality of your data and includes the API for creating, deleting, or updating data quality rulesets, runs and evaluations.
- Feature: This release allows the creation of Custom Visual Transforms (Dynamic Transforms) to be created via AWS Glue CLI/SDK.
- Feature: AWSGlue Crawler - Adding support for Table and Column level Comments with database level datatypes for JDBC based crawler.
- Documentation: Added links related to enabling job bookmarks.
- Feature: Added support for custom datatypes when using custom csv classifier.
- Dependency Update: Updated to the latest SDK module versions
- Dependency Update: Updated to the latest SDK module versions
- Feature: This SDK release adds support to sync glue jobs with source control provider. Additionally, a new parameter called SourceControlDetails will be added to Job model.
- Feature: Added support for S3 Event Notifications for Catalog Target Crawlers.
- Dependency Update: Updated to the latest SDK module versions
- Feature: Fixed a bug in the API client generation which caused some operation parameters to be incorrectly generated as value types instead of pointer types. The service API always required these affected parameters to be nilable. This fixes the SDK client to match the expectations of the the service API.
- Dependency Update: Updated to the latest SDK module versions
- Dependency Update: Updated to the latest SDK module versions
- Dependency Update: Updated to the latest SDK module versions
- Dependency Update: Updated to the latest SDK module versions
- No change notes available for this release.
- Feature: Add support for Python 3.9 AWS Glue Python Shell jobs
- Dependency Update: Updated to the latest SDK module versions
- Dependency Update: Updated to the latest SDK module versions
- Feature: Add an option to run non-urgent or non-time sensitive Glue Jobs on spare capacity
- Dependency Update: Updated to the latest SDK module versions
- Dependency Update: Updated to the latest SDK module versions
- Documentation: Documentation updates for AWS Glue Job Timeout and Autoscaling
- Feature: This release adds an additional worker type for Glue Streaming jobs.
- Dependency Update: Updated to the latest SDK module versions
- Feature: This release adds tag as an input of CreateDatabase
- Dependency Update: Updated to the latest SDK module versions
- Dependency Update: Updated to the latest SDK module versions
- Feature: This release adds a new optional parameter called codeGenNodeConfiguration to CRUD job APIs that allows users to manage visual jobs via APIs. The updated CreateJob and UpdateJob will create jobs that can be viewed in Glue Studio as a visual graph. GetJob can be used to get codeGenNodeConfiguration.
- Dependency Update: Updated to the latest SDK module versions
- Documentation: This release adds documentation for the APIs to create, read, delete, list, and batch read of AWS Glue custom patterns, and for Lake Formation configuration settings in the AWS Glue crawler.
- Dependency Update: Updated to the latest SDK module versions
- Feature: This release adds APIs to create, read, delete, list, and batch read of Glue custom entity types
- Feature: Auto Scaling for Glue version 3.0 and later jobs to dynamically scale compute resources. This SDK change provides customers with the auto-scaled DPU usage
- Dependency Update: Updated to the latest SDK module versions
- Dependency Update: Updated to the latest SDK module versions
- Dependency Update: Updated to the latest SDK module versions
- Feature: Added 9 new APIs for AWS Glue Interactive Sessions: ListSessions, StopSession, CreateSession, GetSession, DeleteSession, RunStatement, GetStatement, ListStatements, CancelStatement
- Feature: Updated
github.com/aws/smithy-go
to latest version - Dependency Update: Updated to the latest SDK module versions
- Feature: API client updated
- Feature: Adds RetryMaxAttempts and RetryMod to API client Options. This allows the API clients' default Retryer to be configured from the shared configuration files or environment variables. Adding a new Retry mode of
Adaptive
.Adaptive
retry mode is an experimental mode, adding client rate limiting when throttles reponses are received from an API. See retry.AdaptiveMode for more details, and configuration options. - Feature: Updated
github.com/aws/smithy-go
to latest version - Dependency Update: Updated to the latest SDK module versions
- Feature: Updated API models
- Feature: Updated
github.com/aws/smithy-go
to latest version - Dependency Update: Updated to the latest SDK module versions
- Feature: API client updated
- Feature: Updated
github.com/aws/smithy-go
to latest version - Dependency Update: Updated to the latest SDK module versions
- Feature: API Paginators now support specifying the initial starting token, and support stopping on empty string tokens.
- Feature: API client updated
- Bug Fix: Fixes a bug that prevented aws.EndpointResolverWithOptions from being used by the service client. (#1514)
- Dependency Update: Updated to the latest SDK module versions
- Dependency Update: Updated to the latest SDK module versions
- Feature: Service clients now support custom endpoints that have an initial URI path defined.
- Feature: The SDK now supports configuration of FIPS and DualStack endpoints using environment variables, shared configuration, or programmatically.
- Feature: Updated
github.com/aws/smithy-go
to latest version - Dependency Update: Updated to the latest SDK module versions
- Feature: API client updated
- Feature: Updated to latest version
- Dependency Update: Updated to the latest SDK module versions
- Feature: API client updated
- Dependency Update: Updated to the latest SDK module versions
- Dependency Update: Updated to the latest SDK module versions
- Feature: Updated API model to latest revision.
- Feature: Updated
github.com/aws/smithy-go
to latest version - Dependency Update: Updated to the latest SDK module versions
- Dependency Update: Updated to the latest SDK module versions
- Feature: Updated to latest API model.
- Dependency Update: Updated
github.com/aws/smithy-go
to latest version. - Dependency Update: Updated to the latest SDK module versions
- Feature: Updated service model to latest version.
- Dependency Update: Updated
github.com/aws/smithy-go
to latest version - Dependency Update: Updated to the latest SDK module versions
- Feature: API client updated
- Feature: Updated
github.com/aws/smithy-go
to latest version - Dependency Update: Updated to the latest SDK module versions
- Feature: Updated to latest API model.
- Dependency Update: Updated to the latest SDK module versions
- Feature: Constant has been added to modules to enable runtime version inspection for reporting.
- Dependency Update: Updated to the latest SDK module versions