Skip to content

Beam 2.61.0 release

Latest
Latest
Compare
Choose a tag to compare
@damccorm damccorm released this 14 Nov 14:46
· 248 commits to master since this release
v2.61.0
acf49f9

We are happy to present the new 2.61.0 release of Beam.
This release includes both improvements and new functionality.

For more information on changes in 2.61.0, check out the detailed release notes.

Highlights

  • [Python] Introduce Managed Transforms API (#31495)
  • Flink 1.19 support added (#32648)

I/Os

  • [Managed Iceberg] Support creating tables if needed (#32686)
  • [Managed Iceberg] Now available in Python SDK (#31495)
  • [Managed Iceberg] Add support for TIMESTAMP, TIME, and DATE types (#32688)
  • BigQuery CDC writes are now available in Python SDK, only supported when using StorageWrite API at least once mode (#32527)
  • [Managed Iceberg] Allow updating table partition specs during pipeline runtime (#32879)
  • Added BigQueryIO as a Managed IO (#31486)
  • Support for writing to Solace messages queues (SolaceIO.Write) added (Java) (#31905).

New Features / Improvements

  • Added support for read with metadata in MqttIO (Java) (#32195)
  • Added support for processing events which use a global sequence to "ordered" extension (Java) #32540
  • Add new meta-transform FlattenWith and Tee that allow one to introduce branching
    without breaking the linear/chaining style of pipeline construction.
  • Use Prism as a fallback to the Python Portable runner when running a pipeline with the Python Direct runner (#32876)

Deprecations

  • Removed support for Flink 1.15 and 1.16
  • Removed support for Python 3.8

Bugfixes

  • (Java) Fixed tearDown not invoked when DoFn throws on Portable Runners (#18592, #31381).
  • (Java) Fixed protobuf error with MapState.remove() in Dataflow Streaming Java Legacy Runner without Streaming Engine (#32892).
  • Adding flag to support conditionally disabling auto-commit in JdbcIO ReadFn (#31111)

Known Issues

N/A

For the most up to date list of known issues, see https://github.com/apache/beam/blob/master/CHANGES.md

List of Contributors

According to git shortlog, the following people contributed to the 2.60.0 release. Thank you to all contributors!

Ahmed Abualsaud, Ahmet Altay, Arun Pandian, Ayush Pandey, Chamikara Jayalath, Chris Ashcraft, Christoph Grotz, DKPHUONG, Damon, Danny Mccormick, Dmitry Ulyumdzhiev, Ferran Fernández Garrido, Hai Joey Tran, Hyeonho Kim, Idan Attias, Israel Herraiz, Jack McCluskey, Jan Lukavský, Jeff Kinard, Jeremy Edwards, Joey Tran, Kenneth Knowles, Maciej Szwaja, Manit Gupta, Mattie Fu, Michel Davit, Minbo Bae, Mohamed Awnallah, Naireen Hussain, Rebecca Szper, Reeba Qureshi, Reuven Lax, Robert Bradshaw, Robert Burke, S. Veyrié, Sam Whittle, Sergei Lilichenko, Shunping Huang, Steven van Rossum, Tan Le, Thiago Nunes, Vitaly Terentyev, Vlado Djerek, Yi Hu, claudevdm, fozzie15, johnjcasey, kushmiD, liferoad, martin trieu, pablo rodriguez defino, razvanculea, s21lee, tvalentyn, twosom

Assets 20
Loading