Status of testing Providers that were prepared on December 22, 2024

[eladkal]

Body

I have a kind request for all the contributors to the latest provider packages release.
Could you please help us to test the RC versions of the providers?

The guidelines on how to test providers can be found in

Verify providers by contributors

Let us know in the comment, whether the issue is addressed.

Those are providers that require testing as there were some substantial changes introduced:

Provider airbyte: 5.0.0rc2

• Remove deprecated code from Airbyte provider (#44577): @ajitg25

Provider alibaba: 3.0.0rc2

• Removed deprecated code from Alibaba provider package (#44576): @Tushar4432

Provider amazon: 9.2.0rc2

• Add DMS Serverless Operators (#43988): @ellisms
• Add fail_on_file_not_exist option to SFTPToS3Operator (#44320): @Guaqamole
• Add wait_policy option to EmrCreateJobFlowOperator. (#44055): @adrian-adikteev
• Use S3CopyObjectOperator in example_comprehend_document_classifier (#44160): @vincbeck
• Remove unnecessary compatibility code in S3 asset import (#44714): @potiuk
• Remove AIP-44 from taskinstance (#44540): @potiuk
• Add do_xcom_push documentation in EcsRunTaskOperator (#44440): @leonidasefrem
• Move Asset user facing components to task_sdk (#43773): @Lee-W
• Set up JWT token authentication in Fast APIs (#42634): @vincbeck
• Bump to mypy-boto3-appflow and pass without # type: ignore[arg-type] (#44115): @jx2lee
  Linked issues:
  • Linked Issue #44111: @eladkal

Provider apache.beam: 6.0.0rc2

• Removed deprecated code from provider Apach.beam (#44700): @ajitg25
  Linked issues:
  • Linked Issue #44559: @eladkal
• Fix deferrable mode for BeamRunPythonPipelineOperator (#44386): @MaksYermak

Provider apache.drill: 3.0.0rc2

• Remove provider deprecations in Apache Drill (#44575): @kunaljubce
  Linked issues:
  • Linked Issue #44559: @eladkal

Provider apache.druid: 4.0.0rc2

• Remove provider deprecations in Apache Druid (#44765): @kunaljubce
  Linked issues:
  • Linked Issue #44559: @eladkal
• Rename execution_date to logical_date across codebase (#43902): @sunank200

Provider apache.hdfs: 4.7.0rc2

• Add mTLS support to WebHDFSHook (#44561): @markhatch

Provider apache.hive: 9.0.0rc2

• Remove deprecations from Apache hive Provider (#44715): @vatsrahul1001
• Rename execution_date to logical_date across codebase (#43902): @sunank200

Provider apache.kafka: 1.7.0rc2

• FIX add error_cb to confluent.Consumer config in ConsumerFromTopic (#44307): @SuccessMoses

Provider apache.livy: 4.0.0rc2

• Remove Provider Deprecations in Apache Livy (#44631): @jason810496
  Linked issues:
  • Linked Issue #44559: @eladkal

Provider apache.spark: 5.0.0rc2

• Remove deprecated code from apache spark provider (#44567): @kunaljubce
  Linked issues:
  • Linked Issue #44559: @eladkal
• Fix failing mypy check on main (#44191): @kaxil
• spark-submit: replace principle by principal (#44150): @brouberol
  Linked issues:
  • Linked Issue #43679: @brouberol

Provider apprise: 2.0.0rc2

• Remove Provider Deprecations in Apprise (#44764): @jscheffl

Provider arangodb: 2.7.0rc2

• Added the ArangoDBCollectionOperator that executes collection operations in a ArangoDB database (#44676): @harjeevanmaan

Provider atlassian.jira: 3.0.0rc2

• Remove Provider Deprecations in Atlassian Jira (#44644): @jason810496
  Linked issues:
  • Linked Issue #44559: @eladkal

Provider celery: 3.9.0rc2

• AIP-81 Move CLI Commands to directories according to Hybrid, Local and Remote (#44538): @bugraoz93
  Linked issues:
  • Linked Issue #44204: @bugraoz93
• Remove AIP-44 configuration from the code (#44454): @potiuk
  Linked issues:
  • Linked Issue #44441: @potiuk
  • Linked Issue #44436: @potiuk

Provider cncf.kubernetes: 10.1.0rc2

• Add logging support for init containers in KubernetesPodOperator (#42498) (#43853): @mrk-andreev
• New Kubernetes Kueue Operators (#44568): @moiseenkov
• support grouping of log lines for KubernetesPodOperator (#44428): @karunpoudel
• Compare k8s executor against alias, not full ExecutorName repr (#44967): @o-nikolas
  Linked issues:
  • Linked Issue #44931: @amoghrajesh
• Fix failing KubeExecutor tests due to #44710 (#44931): @amoghrajesh

Provider common.compat: 1.3.0rc2

• fix(providers/common/compat): add back add_input_dataset and add_output_dataset to NoOpCollector (#44681): @Lee-W
• Fix name of private function in compat provider (#44680): @jedcunningham
• Remove unnecessary compatibility code in S3 asset import (#44714): @potiuk
• Move Asset user facing components to task_sdk (#43773): @Lee-W
• Make AssetAliasEvent a class context.py (#44709): @uranusjr
• Move triggers to standard provider (#43608): @gopidesupavan
  Linked issues:
  • Linked Issue #43641: @gopidesupavan

Provider common.io: 1.5.0rc2

• feat: add OpenLineage support for transfer operators between gcs and local (#44417): @kacpermuda
  Linked issues:
  • Linked Issue #44410: @kacpermuda
• Move Asset user facing components to task_sdk (#43773): @Lee-W
• Migrate pickled data & change XCom value type to JSON (#44166): @kaxil

Provider common.sql: 1.21.0rc2

• [Added output_processor parameter to SQLQueryOperator and fixed bug with return_single_query_results handler when None is passed as split_statements (Added output_processor parameter to SQLQueryOperator and fixed bug with return_single_query_results handler when None is passed as split_statements #44781)](https://github.com/apache/airflow/
• Moved common SQL handler methods of common-sql-provider into dedicated module (#43747): @dabla
• Fix static checks in common SQL hooks (#44930): @shahar1
  Linked issues:
  • Linked Issue #43747: @dabla
• Remove Provider Deprecations in Common SQL (#44645): @jason810496
  Linked issues:
  • Linked Issue #44559: @eladkal

Provider databricks: 7.0.0rc2

• Remove deprecations from Databricks Provider (#44566): @vatsrahul1001
• [FIX] Fixed databricks repair run deferrable (#44213): @raghvendra-singh1
• fix(providers/databricks): remove additional argument passed to repair_run (#44140): @Lee-W
• Added job_clusters as a templated parameter to CreateDatabricksWorkflowOperator (#45022): @kunaljubce
  Linked issues:
  • Linked Issue #42438: @sushil-louisa
• Upgrade databricks provider dependency (#43272): @dcmshi
• Rename execution_date to logical_date across codebase (#43902): @sunank200

Provider dbt.cloud: 4.0.0rc2

• Remove Provider Deprecations in DBT (#44638): @jason810496
  Linked issues:
  • Linked Issue #44559: @eladkal
• utilize map_index for deterministic generation of OpenLineage's run_id (#43936): @mobuchowski
• Remove commented breakpoint in dbt provider (#44163): @dstandish
• Rename execution_date to logical_date across codebase (#43902): @sunank200

Provider docker: 4.0.0rc2

• Remove Provider Deprecations in Docker (#44583): @jscheffl
• Fix docker documentation auth url (#44112): @gopidesupavan

Provider elasticsearch: 6.0.0rc2

• Remove Provider Deprecations in Elasticsearch (#44629): @jason810496
  Linked issues:
  • Linked Issue #44559: @eladkal
• Rename execution_date to logical_date across codebase (#43902): @sunank200

Provider google: 12.0.0rc2

• Google provider delete deprecated reaching removal date (December 2024) (#45084): @olegkachur-e
• Add Google Vertex AI Feature Store - Feature View Sync Operators, Sensor (#44891): @CYarros10
• Introduce GCP translation (V3), translate document providers (#44971): @olegkachur-e
• Introduce gcp advance (V3) API translate native models operators (#44627): @olegkachur-e
• Support multiple SQL queries in Dataproc SQL job (#44890): @amirmor1
• feat: add OpenLineage support for BigQuery Create Table operators (#44783): @kacpermuda
• feat: add OpenLineage support for S3ToGCSOperator (#44426): @kacpermuda
• feat: automatically inject OL info into spark job in DataprocSubmitJobOperator (#44477): @kacpermuda
• Implement AlloyDB operators: create/update/delete clusters (#45027): @moiseenkov
• Fix MetastoreHivePartitionSensor failing due to duplicate aliases (#45001): @CYarros10
• Fix failing OpenLineage emition for InsertBigQueryOperator (#44650): @spapi17
• BigQueryInsertJobOperator: log transient error and check job state before marking task as success (#44279): @pankajastro
• FIX make CloudBatchSubmitJobOperator fail when job fails (#44425): @SuccessMoses
  Linked issues:
  • Linked Issue #43744: @adamszustak
• feat: add OpenLineage support for transfer operators between gcs and local (#44417): @kacpermuda
  Linked issues:
  • Linked Issue #44410: @kacpermuda
• chore: remove deprecated BigQuery facets from OpenLineage utils (#44838): @kacpermuda
• New Kubernetes Kueue Operators (#44568): @moiseenkov
• Fix system test for Dataform operators (#44729): @MaksYermak
• Deprecate VertexAI PaLM text generative model (#44719): @MaksYermak
• Show prominent warning for deprecations in docs (#44479): @omkar-foss
• chore: unify handling of gcs paths in OpenLineage processes (#44410): @kacpermuda

Provider hashicorp: 4.0.0rc2

• Removed deprecated code from hashicorp provider (#44598): @Prab-27
  Linked issues:
  • Linked Issue #44559: @eladkal

Provider http: 5.0.0rc2

• Remove Provider Deprecations in HTTP (#44542): @jscheffl
• [Providers/HTTP] Add adapter parameter to HttpHook to allow custom requests adapters (#44302): @jieyao-MilestoneHub
  Linked issues:
  • Linked Issue #44285: @kiaradlf
• Rename execution_date to logical_date across codebase (#43902): @sunank200
• Bump aiohttp to 3.11.1 (#44036): @kaxil

Provider jdbc: 5.0.0rc2

• Remove Provider Deprecations in JDBC (#44662): @jscheffl
• Made get_conn in JdbcHook threadsafe to avoid OSError: JVM is already started (#44718): @dabla
• Suppress JException when get_autocommit and set_autocommit methods aren't supported on JDBC driver (#43786): @dabla

Provider jenkins: 4.0.0rc2

• Remove Provider Deprecations in Jenkins (#44630): @jason810496
  Linked issues:
  • Linked Issue #44559: @eladkal

Provider microsoft.azure: 12.0.0rc2

• Remove Provider Deprecations in Microsoft-AZURE (#44763): @vatsrahul1001
• Added test-case for callable values in path and query parameters of MSGraphAsyncOperator (#43799): @dabla
• PowerBIDatasetRefreshOperator should fail when refresh fails (#44696): @Ohashiro
• Move triggers to standard provider (#43608): @gopidesupavan
  Linked issues:
  • Linked Issue #43641: @gopidesupavan
• Remove XCom pickling (#43905): @kaxil
• Update path of example dags in docs (#45069): @eladkal
• Avoid 1.1.8 version of msgraph-core (#45044): @potiuk
• Added MS Graph connection type (#45006): @dabla

Provider microsoft.mssql: 4.0.0rc2

• Remove Provider Deprecations in Microsoft-MSSQL (#44762): @jscheffl
• Support connection extra parameters in MsSqlHook (#44310): @jx2lee
  Linked issues:
  • Linked Issue #43798: @ilarionkuleshov

Provider microsoft.psrp: 3.0.0rc2

• Remove Provider Deprecations in Microsoft-PSRP (#44761): @jscheffl

Provider microsoft.winrm: 3.7.0rc2

• Remove XCom pickling (#43905): @kaxil

Provider mongo: 5.0.0rc2

• Remove Provider Deprecations in Mongo (#44632): @jason810496
  Linked issues:
  • Linked Issue #44559: @eladkal

Provider mysql: 6.0.0rc2

• Remove Provider Deprecations in MySQL (#44665): @jscheffl
• Fix error file not found. tmp file is deleted before inserting rows to DB in VerticaToMySQLOperator bulk (#44028): @bareketamir
• Add basic asyncio support (#43944): @dstandish

Provider openlineage: 2.0.0rc2

• Remove Provider Deprecations in OpenLineage (#44636): @jason810496
  Linked issues:
  • Linked Issue #44559: @eladkal
• add clear_number to OpenLineage's dagrun-level event run id generation (#44617): @mobuchowski
• utilize map_index for deterministic generation of OpenLineage's run_id (#43936): @mobuchowski
• feat: automatically inject OL info into spark job in DataprocSubmitJobOperator (#44477): @kacpermuda
• add basic system tests for OpenLineage (#43643): @mobuchowski
• Move Asset user facing components to task_sdk (#43773): @Lee-W
• Rename execution_date to logical_date across codebase (#43902): @sunank200

Provider opensearch: 1.6.0rc2

• Rename execution_date to logical_date across codebase (#43902): @sunank200

Provider oracle: 4.0.0rc2

• Remove Provider Deprecations in Oracle (#44704): @jscheffl

Provider pagerduty: 4.0.0rc2

• Remove deprecated code from Pagerduty provider (#44653): @Prab-27
  Linked issues:
  • Linked Issue #44559: @eladkal

Provider postgres: 6.0.0rc2

• Remove Provider Deprecations in Postgres (#44705): @jscheffl
• Add basic asyncio support (#43944): @dstandish

Provider presto: 5.8.0rc2

• Rename execution_date to logical_date across codebase (#43902): @sunank200

Provider redis: 4.0.0rc2

• Remove Provider Deprecations in Redis (#44633): @jason810496
  Linked issues:
  • Linked Issue #44559: @eladkal

Provider sendgrid: 4.0.0rc2

• Remove Provider Deprecations in SendGrid (#44637): @jason810496
  Linked issues:
  • Linked Issue #44559: @eladkal

Provider sftp: 5.0.0rc2

• Remove deprecations from SFTP Provider (#44740): @vatsrahul1001
• feat: retrieve sftp file attrs onces instead multiple time (#44625): @dondaum
• Add host_proxy_cmd parameter to SSHHook and SFTPHook (#44565): @ajitg25

Provider slack: 9.0.0rc2

• Remove deprecations from Slack Provider (#44693): @vatsrahul1001

Provider snowflake: 6.0.0rc2

• Remove deprecations from Snowflake Provider (#44756): @vatsrahul1001
• [Snowflake] enable client_store_temporary_credential for snowflake provider (#44431): @yzliao
• Allow json_result_force_utf8_encoding specification in providers.snowflake.hooks.SnowflakeHook extra dict (#44264): @ttzhou
• Make host/port configurable for Snowflake connections (#44079): @whummer

Provider sqlite: 4.0.0rc2

• Remove Provider Deprecations in Sqlite (#44707): @jscheffl
• Add basic asyncio support (#43944): @dstandish

Provider ssh: 4.0.0rc2

• Remove Provider Deprecations in SSH (#44544): @jscheffl
• Add host_proxy_cmd parameter to SSHHook and SFTPHook (#44565): @ajitg25
• Remove XCom pickling (#43905): @kaxil

Provider standard: 0.0.3rc2

• Remove Provider Deprecations in Standard (#44541): @jscheffl
• Add backward compatibility check for StartTriggerArgs import in filesystem sensor (#44458): @kunaljubce
  Linked issues:
  • Linked Issue #44324: @eladkal
• Remove Pydanitc models introduced for AIP-44 (#44552): @potiuk
  Linked issues:
  • Linked Issue #44436: @potiuk
• Deferrable sensors can implement sensor timeout (#33718): @dstandish
• Remove AIP-44 code from renderedtifields.py (#44546): @potiuk
  Linked issues:
  • Linked Issue #44436: @potiuk
• Remove AIP-44 from taskinstance (#44540): @potiuk
• Move LatestOnlyOperator operator to standard provider. (#44309): @hardeybisey
  Linked issues:
  • Linked Issue #43641: @gopidesupavan
• Remove AIP-44 configuration from the code (#44454): @potiuk
  Linked issues:
  • Linked Issue #44441: @potiuk
  • Linked Issue #44436: @potiuk
• Move external task sensor to standard provider (#44288): @kunaljubce
  Linked issues:
  • Linked Issue #43641: @gopidesupavan
• Move triggers to standard provider (#43608): @gopidesupavan
  Linked issues:
  • Linked Issue #43641: @gopidesupavan

Provider tableau: 5.0.0rc2

• Remove deprecations from Tableau Provider (#44757): @vatsrahul1001

Provider telegram: 4.7.0rc2

• Feat: telegram send file Operator and Hook (#44040): @HaKkaz

Provider teradata: 3.0.0rc2

• Remove deprecations from Teradata Provider (#44746): @vatsrahul1001

Provider trino: 6.0.0rc2

• Remove Provider Deprecations in Trino (#44717): @Prab-27
  Linked issues:
  • Linked Issue #44559: @eladkal
• Rename execution_date to logical_date across codebase (#43902): @sunank200

Provider vertica: 4.0.0rc2

• Remove Provider Deprecations in Vertica (#44748): @jscheffl

Provider weaviate: 3.0.0rc2

• Remove deprecations from Weaviate Provider (#44745): @vatsrahul1001

Provider yandex: 4.0.0rc2

• Remove Provider Deprecations in Yandex provider (#44754): @rawwar

All users involved in the PRs:
@kaxil @raghvendra-singh1 @uranusjr @Lee-W @mobuchowski @leonidasefrem @adrian-adikteev @dstandish @omkar-foss @HaKkaz @jedcunningham @shahar1 @potiuk @mrk-andreev @hardeybisey @o-nikolas @Tushar4432 @sunank200

Committer

• I acknowledge that I am a maintainer/committer of the Apache Airflow project.

[jscheffl]

Checked the deprecations which were my main contributions and all seem to be OK.

[potiuk]

All my removals checked. All looks good !

[markhatch]

• apache.hdfs: 4.7.0rc2

Tested WebHDFS hook with mTLS - looks good.

[amoghrajesh]

#44931 works as expected, niko reworked it and even that fix works as expected.

[moiseenkov]

Hi,
#44568 and #45027 work as expected

[Ohashiro]

Hello
I think #44696 works well

[hardeybisey]

Hi, #44309 works as expected.

[kacpermuda]

All my changes look good and produce expected results with some sample DAGs.

[dondaum]

Tested #44625. All good.

One thing. It seems that some contributors, including myself, are somehow not tagged correctly. That's why I didn't get a notification.

[jx2lee]

#44115
#44310
works fine!

[potiuk]

One thing. It seems that some contributors, including myself, are somehow not tagged correctly. That's why I didn't get a notification.

Interesting seems it's only @dondaum @yzliao @HaKkaz @hardeybisey - interesting why, as it's not something that we have control over.

[potiuk]

Likely some intermittent GitHub Issue - now all the users are tagged properly when mentioned....

[gopidesupavan]

Verified my changes all looks good.

[yzliao]

Verified Snowflake provider and all LGTM. My change #44431 is good to go.

[pankajastro]

#44279 looks good.

[olegkachur-e]

The #44971 <#44971>, #44627 <#44627>, #45084 <#45084> looks good. Thank you!

[HaKkaz]

#44040 looks good.