v2.11.1

Broker

• AbstractBatchedMetadataStore - use AlreadyClosedException instead of IllegalStateException #19284
• Add ref count for sticky hash to optimize the performance of Key_Shared subscription #19167
• Pass subName for subscription operations in ServerCnx #19184
• Enable custom metadata stores #19208
• Topics stats shows msgBacklog but there reality no backlog #19275
• Fix open cursor with null-initialPosition result with earliest position #18416
• Added isActive in ManagedCursorImpl #19341
• Replaced checkBackloggedCursors with checkBackloggedCursor(single subscription check) upon subscription #19343
• Close TransactionBuffer when MessageDeduplication#checkStatus failed #19289
• Fix the incorrect total size if use ML interceptor #19404
• Support deleting partitioned topics with the keyword -partition- #19230
• Correct the prompt message #17009
• Expect msgs after server initiated CloseProducer #19446
• Fix Npe thrown by splitBundle #17370
• Fix executeWithRetry result is null #17694
• ManagedLedger: move to FENCED state in case of BadVersionException #17736
• Fix can not revoke permission after update topic partition #17393
• Allow deletion of a namespace that was left in deleted status #17592
• Add UncaughtExceptionHandler for every thread pool #18211
• Support setting ForceDeleteNamespaceAllowed dynamically #18181
• Support setting forceDeleteTenantAllowed dynamically #18192
• Remove locallyAcquiredLock when removeOwnership #18197
• Update the log print content of createSubscriptions #18024
• Unify time unit at dropping the backlog on a topic #17957
• Fix mutex never released when trimming #17911
• Fix getChildren in MemoryMetadataStore and EtcdMetadataStore #18172
• Fix the order of resource close in the InMemoryDelayedDeliveryTracker #18000
• Allow to configure and disable the size of lookahead for detecting fixed delays in messages #17907
• Make BookieId work with PulsarRegistrationDriver #17922
• In the trimDeletedEntries method, release the removed entry #18305
• Make Consumer#equals more effective #18662
• PulsarRegistrationClient - implement getAllBookies and follow BookieServiceInfo updates #18133
• Fix PulsarRegistrationClient and ZkRegistrationClient not aware rack info problem #18672
• Simplify extract entryMetadata code in filterEntriesForConsumer #18729
• Fix uncompleted future when getting the topic policies of a deleted topic #18824
• Close transactionBuffer after MessageDeduplication#checkStatus failed #19157
• GetLastMessageId returns a wrong batch index of last message if enabled read compacted #18877
• Fix multi invocation for ledger createComplete #18975
• Fix txn coordinator recover handle committing and aborting txn race condition #19201
• Reset individualDeletedMessagesSerializedSize after acked all messages #19428
• Fix potential NPE cause future never complete #19415
• Fix the return value of canConsumeAsync #19412
• Close TransactionBuffer when creating persistent topic timeout #19384
• Handle changeToReadyState failure correctly in TC client #19308
• The atomicity of multiple fields of ml is broken #19346
• MessagesConsumedCounter of NonDurableCursor was initialized incorrectly #19355
• Improve PersistentMessageExpiryMonitor expire speed when ledger not existed #17842
• Always send correct transaction id in end txn response #19137
• Allow superusers to abort transactions #19467
• Make ServerCnx#originalAuthData volatile #19507
• Validate originalPrincipal earlier in ServerCnx #19270
• Require authRole is proxyRole to set originalPrincipal #19455
• ServerCnx broken after recent cherry-picks #19521
• ServerCnx: go to Failed state when auth fails #19312
• Add test to verify authRole cannot change #19430
• Make authentication refresh threadsafe #19506
• OneStageAuth State: move authn out of constructor #19295
• Allow proxy to pass same role for authRole and originalRole #19557
• Copy command fields and fix potential thread-safety in ServerCnx #19517
• Topic load fail by ledger lost #19444
• Incorrect service name selection logic #19505
• Make the service name resolver cache of PulsarWebResource expire after access #19532
• Simplify enabling Broker, WS Proxy hostname verification #19674
• Improve tls config on replication client and cluster cli #19588
• Fix geo-replication admin #19686
• Filter system topic when getting topic list by binary proto #19667
• Fixed history load not releasing #19560
• Fix loadbalance score calculation problem #19420
• Use shrink map for trackerCache #19534
• Fix individual ack batch message with transaction abort redevlier duplicate messages #14327
• Fix receive duplicated messages due to pendingAcks in PendingAckHandle #19581
• Fail to update partition meta of topic due to ConflictException: subscription already exists for topic #17251
• Fix catching ConflictException when update topic partition #17374
• Only create extended partitions when updating partition number #17349
• Create replicated subscriptions for new partitions when needed #18659
• Fix race condition while updating partition number #19199
• Topic close failure leaves subscription in a permanent fence state #19692
• Fix potential exception cause the policy service init fail #19746
• Allow version to start positive and grow by more than one #19503
• Fix deadlock causes session notification not to work #19754
• Fix issue where msgRateExpired may not refresh forever #19759
• Delete topic failed if disabled system topic #19735
• Fix index generator is not rollback after entries are failed added #19727
• Store the original authentication data #19519
• Copy subscription properties during updating the topic partition number #19800
• Fix close borrowed executor #19761
• Fixed race condition between ResourceLock update and invalidation #19817
• Counter of pending send messages in Replicator incorre...

v2.10.4

Broker

• [improve][broker] Add ref count for sticky hash to optimize the performance of Key_Shared subscription (#19167)
• [fix][broker] Pass subName for subscription operations in ServerCnx (#19184)
• [fix][broker] AbstractBatchedMetadataStore - use AlreadyClosedException instead of IllegalStateException (#19284)
• [fix][broker] Fix open cursor with null-initialPosition result with earliest position (#18416)
• [fix] [ml] Topics stats shows msgBacklog but there reality no backlog (#19275)
• [improve][broker] Added isActive in ManagedCursorImpl (#19341)
• [improve][broker] Replaced checkBackloggedCursors with checkBackloggedCursor(single subscription check) upon subscription (#19343)
• [fix][broker] fixed the build error for pattern matching variable in lower JVM versions (#19362)
• [fix][broker]optimize the shutdown sequence of broker service when it close (#16756)
• [fix][broker] Pass subscriptionName to auth service (#17123) (#19423)
• [fix][broker]fix multi invocation for ledger createComplete (#18975)
• [fix] [broker] getLastMessageId returns a wrong batch index of last message if enabled read compacted (#18877)
• [fix][broker] Support deleting partitioned topics with the keyword -partition- (#19230)
• [fix] [ml] Fix the incorrect total size if use ML interceptor (#19404)
• [improve][broker] Added isActive in ManagedCursorImpl (#19341)
• [improve][broker] Added isActive in ManagedCursorImpl (#19341)
• [improve][broker] Copy subscription properties during updating the topic partition number. (#19223)
• [fix][broker] Expect msgs after server initiated CloseProducer (#19446)
• [fix][broker] Fix PulsarRegistrationClient and ZkRegistrationClient not aware rack info problem. (#18672)
• [fix][ml] Fix potential NPE cause future never complete. (#19415)
• [fix] [ml] The atomicity of multiple fields of ml is broken (#19346)
• [fix][broker] Fix race condition while updating partition number (#19199)
• [Improve][broker] Support clear old bookie data for BKCluster (#16744)
• [fix][ml] Reset individualDeletedMessagesSerializedSize after acked all messages. (#19428)
• [fix][broker] Make ServerCnx#originalAuthData volatile (#19507)
• [fix][broker] ServerCnx broken after recent cherry-picks (#19521)
• [fix][broker] Fix loadbalance score caculation problem (#19420)
• [fix] [ml] messagesConsumedCounter of NonDurableCursor was initialized incorrectly (#19355)
• [fix][broker][branch-2.10] Replace sync method call in async call chain to prevent ZK event thread deadlock (#19539)
• [improve][broker] Require authRole is proxyRole to set originalPrincipal (#19455)
• [improve][broker] ServerCnx: go to Failed state when auth fails (#19312)
• [feat][broker] Cherry-pick tests from (#19409)
• [improve][broker] Add test to verify authRole cannot change (#19430)
• [feat][broker] OneStageAuth State: move authn out of constructor (#19295)
• [fix][broker] Correct MockAlwaysExpiredAuthenticationState test impl
• [cleanup][broker] Validate originalPrincipal earlier in ServerCnx (#19270)
• [fix][broker] ServerCnx broken after recent cherry-picks (#19521)
• [fix][broker] Fix loadbalance score caculation problem (#19420)
• [fix] [ml] messagesConsumedCounter of NonDurableCursor was initialized incorrectly (#19355)
• [fix][broker][branch-2.10] Replace sync method call in async call chain to prevent ZK event thread deadlock (#19539)
• [improve][broker] Require authRole is proxyRole to set originalPrincipal (#19455)
• [improve][broker] ServerCnx: go to Failed state when auth fails (#19312）
• [feat][broker] Cherry-pick tests from (#19409)
• [improve][broker] Add test to verify authRole cannot change (#19430)
• [fix][broker] Call originalAuthState.authenticate in ServerCnx
• [fix][broker] Correct MockAlwaysExpiredAuthenticationState test impl
• [fix][broker] Make authentication refresh threadsafe (#19506)
• [fix][broker] Allow proxy to pass same role for authRole and originalRole (#19557)
• [fix][broker] Copy command fields and fix potential thread-safety in ServerCnx (#19517)
• [fix][broker][branch-2.10] Fix geo-replication admin (#19608）
• [fix][broker] PulsarRegistrationClient - implement getAllBookies and follow BookieServiceInfo updates (#18133)
• [Improve][Broker]Reduce GetReplicatedSubscriptionStatus local REST call (#16946)
• [fix] [broker] Incorrect service name selection logic (#19505)
• [improve][broker] Use shrink map for trackerCache (#19534)
• [improve][broker] Add UncaughtExceptionHandler for every thread pool (#18211)
• [cleanup][broker] Simplify extract entryMetadata code in filterEntriesForConsumer (#18729)
• [improve][broker] Follow up #19230 to tighten the validation scope (#19234)
• [fix] [ml] topic load fail by ledger lost (#19444)
• [fix] [broker] Topic close failure leaves subscription in a permanent fence state (#19692)
• [cherry-pick][branch-2.10] Fix deadlock causes session notification not to work (#19754) (#19768)
• [fix][broker] Fix potential exception cause the policy service init fail. (#19746)
• [improve][broker] Authorize originalPrincipal when provided (#19830)
• [fix][broker] Only validate superuser access if authz enabled (#19989)
• [fix][broker] Ignore and remove the replicator cursor when the remote cluster is absent (#19972)
• [fix][broker] Fix index generator is not rollback after entries are failed added (#19980)
• [fix] [proxy] Used in proxyConf file when configuration is missing in the command line (#15938)
• [fix][broker] Remove timestamp from broker metrics (#17419)
• [fix] [broker] Counter of pending send messages in Replicator incorrect if schema future not complete (#19242)

Transaction

• [fix][txn] Correct the prompt message (#17009)
• [fix][txn] Catch and log runtime exceptions in async operations (#19258)
• [fix][txn] fix txn coordinator recover handle committing and aborting txn race condition. (#19201)
• [improve][txn] Handle changeToReadyState failure correctly in TC client (#19308)
• [cherry-pick][branch-2.10] Allow superusers to abort transactions (#1…9467) (#19473)
• [feature][txn] Fix individual ack batch message with transaction abor…
• Close TransactionBuffer when MessageDeduplication#checkStatus failed (#19288)
• [improve][txn] Cleanup how superusers abort txns (#19976)
• [feature][txn] Fix individual ack batch message with transaction abort redevlier duplicate messages (#14327)

Pulsar IO and Pulsar Functions

• [fix][io] Update Elasticsearch sink idle cnx timeout to 30s (#19377)
• [fix][fn] Fix k8s merge runtime opts bug (#19481)
• [refactor][fn] Use AuthorizationServer more in Function Worker API (#19975)

CLI

• [fix] [cli] Fix Broker crashed by too much memory usage of pulsar tools (#20031)

Admin

• [fix][admin] Fix validatePersistencePolicies that Namespace/Topic persistent policies cannot set to < 0 (#18999)
• [improve][admin][branch-2.10] Unset namespace policy to improve deleting namespace (#17033) (#19865)
• [fix] [admin] fix incorrect state replication.connected on API partitioned-topic stat (#19942)
• [fix] [admin] Make response code to 400 instead of 500 when delete topic fails due to enabled geo-replication (#19879)

Security

• [fix][sec] Upgrade jettison to 1.5.3 (#19038)
• [fix][sec] Upgrade woodstox to 5.4.0 (#19041)
• [improve][sec] Suppress false positive OWASP reports (#19105)
• [fix][misc] do not require encryption on system topics (#18898)
• [fix][authorization] Fix the return value of canConsumeAsync (#19412)
• [fix][security] Fix secure problem CVE-2017-1000487 (#19479)
• [improve] Upgrade wildfly-eytron (used by debezium) to fix CVE-2022-3143 (#19333)
• [improve][misc] Upgrade Netty to 4.1.87.Final (#19417)
• [fix][broker] Ignore and remove the replicator cursor when the remote cluster is absent (#19972)
• [Authenticate] fix Invalid signature error when use Kerberos Authentication (#15121)
• [fix][sec] Fix transitive critical CVEs in file-system tiered storage (#19957)
• [fix][sec] Fix transitive critical CVEs in file-system tiered storage (#19957)

CI & Test

• [fix][test] ProxyWithAuthorizationTest remove SAN from test certs (#19594)
• [branch-2.10][test]Run and fix tests (#19636)
• [test] Fix ServerCnxTest failing after merge of #19830

Others

• [fix][build] Resolve OWASP Dependency Check false positives (#19120)
• Debezium sources: Support loading config from secrets (#19163)
• [fix][proxy] Only go to connecting state once (#19331)
• [fix][build] Upgrade dependency-check-maven plugin to fix broken OWASP check (#19170)
• [branch-2.10][fix][proxy] Fix using wrong client version in pulsar proxy (#19576)
• [branch-2.10][fix][proxy] Fix using wrong client version in pulsar proxy (#19576)
• [improve] Upgrade lombok to 1.8.26 (#19426)
• [improve] upgrade the bookkeeper version to 4.14.7 (#19179)
• [branch-2.10][broker] Support zookeeper read-only config. (#19156) (#19637)
• [improve] Simplify enabling Broker, WS Proxy hostname verification (#19674)
• [cherry-pick][branch-2.10] KCA: picking fixes from master (#19788)
• [Build] Make the test JVM exit if OOME occurs (#14509)

v2.11.0

Important notice

• Rename Pulsar txn metrics to specify OpenMetrics #16581
• Rename Pulsar schema metrics to specify OpenMetrics #16610
• Rename Pulsar lb metrics to specify OpenMetrics #16611
• Remove jul-to-slf4j #16320 #16320
• Fix get non-persistent topics issue in Namespaces #16170
• Fix producer/consume permission can’t get schema #15956
• Add verification for configured default backlog quota and retention #15441
• Fix ineffective broker LoadBalance #15314
• Ensure the deletion consistency of topic and schema #14608
• Transaction cumulative ack redeliver change #14371
• Bump prometheus client version from 0.5.0 to 0.15.0 #13785
• [PIP-158] Split client TLS transport encryption from authentication #15634
• [PIP-155] Remove Python 2 support #15376
• [PIP-156] Build Pulsar Server on Java 17 #15264
• [PIP-162] Enable system topic by default #15619

PIPs

• [PIP 37] Support chunking with Shared subscription #16202
• [PIP-74] Pulsar client memory limits #14400
• [PIP-81] Split createNewMetadataLedger into multiple methods for reuse #15425
• [PIP 97] Update Authentication Interfaces to Include Async Authentication Methods #12104
• [PIP-100] Support pluggable topic factory #12235
• [PIP-105] Store Subscription properties #15757
• [PIP-117] Change Pulsar standalone defaults #15478
• [PIP-132] Include message header size when checking maxMessageSize for non-batch messages on the client side #14007
• [PIP-136] Sync Pulsar metadata across multiple clouds #16425
• [PIP-143] Support split bundle by specified boundaries #13796
• [PIP-145] Improve performance of regex subscriptions #14505
• [PIP-146] ManagedCursorInfo compression #14542
• [PIP 149] Make the REST Admin API fully async #14365
• [PIP-152] Support subscription level dispatch rate limiter setting #15295
• [PIP-153] Optimize metadataPositions in MLPendingAckStore #15137
• [PIP-160] Make transactions work more efficiently by aggregation operation for transaction log and pending ack store #15370
• [PIP-161] Exclusive Producer: ability to fence out an existing Producer #15488
• [PIP-163] Add lowWaterMark check before appending entry to TB #15424
• [PIP-166] Add MANUAL delivery semantics for Functions #16279
• [PIP-179] Support the admin API to check unknown request parameters #16135
• [PIP-181] Pulsar Shell #16250
• [PIP-184] Topic specific consumer priorityLevel #16715
• [PIP-187] Add API to analyze a subscription backlog and provide an accurate value #16545
• [PIP-189] No batching if only one message in batch #18548
• [PIP-201] Extensions mechanism for Pulsar Admin CLI tools #17158

Broker

• [PIP 81] Split the individual acknowledgments into multiple entries

  • Add range set wrapper to support record dirty ledgers #15607
  • Notifications for faster topic discovery #16062
  • Split createNewMetadataLedger into multiple methods for reuse #15425

• [PIP 105] Store Subscription properties

  • Store Subscription properties #15757
  • Pulsar Admin: create subscription with Properties #15503
  • Fix error on recycled SubscriptionPropertiesList #15335

• [PIP 160] Make transactions work more efficiently by aggregation operation for transaction log and pending ack store

  • Pending ack log store enables the batch feature #16707
  • Metrics stats of Transaction buffered writer #16758
  • Transaction buffered writer supports Timer #16727
  • Transaction log store enables the batch feature #16685
  • Protocol changes and configuration changes for transaction batch log #16617
  • Txn buffered writer for transaction log batch #16428

• [PIP 145] Improve performance of regex subscriptions

  • Enable evaluating subscription pattern on broker side #14804
  • Notifications for faster topic discovery #16062

• [PIP-156] Build Pulsar Server on Java 17

  • Build changes by dropping some required checks #15496
  • Build Pulsar Server on Java 17 #15264

• Add logs for why namespace bundle been split #19003

• Omit making a copy of CommandAck when there are no broker interceptors #18997

• Fix deadlock in PendingAckHandleImpl #18989

• Copy proto command fields into final variables in ServerCnx #18987

• Transaction pending ack store future not completely problem #18943

• Update interceptor handler exception #18940

• Do not require encryption on system topics #18898

• Fix incorrect Nic usage collected by pulsar #18882

• Fix PendingAckHandleImpl when pendingAckStoreProvider.checkInitializedBefore failed #18859

• Ignore the exception of creating namespace #18837

• Fixing can not delete namespace by force #18826

• Avoid OOM not trigger PulsarByteBufAllocator outOfMemoryListener when use ByteBufAllocator.DEFAULT.heapBuffer #18747

• System topic writer/reader connection not counted #18603

• DnsResolverUtil.TTL should be greater than zero #18565

• Persist correct markDeletePosition to prevent message loss #18237

• Fix delete_when_subscriptions_caught_up doesn't work while have active consumers #18320

• Read local cookie when start pulsar standalone #18260

• Remove the redundant judgment logic of ManagedCursorImpl #18205

• Fix unexpected behaviour by invoke PulsarConfigurationLoader#convertFrom #18816

• Fix the wrong behaviour when set overrideBrokerNicSpeedGbps #18818

• Fix the breaking change of standalone metadata initialization #18909

• Transaction buffer recover blocked by readNext #18969

• Fix duplicated schemas creation #18701

• Read local cookie when start pulsar standalone #18260

• The Pulsar standalone bookie is not getting passed the config from standalone.conf #18126

• Reduce unnecessary calling span() when filtering read entries #18106

• Change update schema auth from tenant to produce #18074

• Fixed delayed delivery after read operation error #18098

• Fix memory leak while Offloading ledgers #18500

• Cache invalidation due to concurrent access #18076

• Fix unab...

v2.10.3

Broker

• [improve][broker] Omit making a copy of CommandAck when there are no broker interceptors #18997
• [fix][broker] Copy proto command fields into final variables in ServerCnx #18987
• [improve][broker]Add a cache eviction policy：Evicting cache data by the slowest markDeletedPosition #14985
• [fix][broker] Fix broker cache eviction of entries read by active cursors #17273
• [enh][broker] Add metrics for entry cache insertion, eviction #17248
• [cleanup][broker] Extracted interface for EntryCacheManager #15933
• [improve][broker] Support dynamic update cache config #13679
• [improve][broker] refactor ManagedLedger cacheEvictionTask implement #14488
• [improve][broker] Add logs for why namespace bundle been split (#19003)
• [fix][broker] Fix deadlock in PendingAckHandleImpl (#18989)
• [fix][broker] Branch-2.10 Avoid endless blocking call. (#18914)
• [fix][broker][functions-worker] Ensure prometheus metrics are grouped by type (#8407, #13865) #15558
• [fix][broker]Update interceptor handler exception (#18940)
• [fix][broker] Fix incorrect bundle split count metric (#17970)
• [fix][broker] Avoid OOM not trigger PulsarByteBufAllocator outOfMemoryListener when use ByteBufAllocator.DEFAULT.heapBuffer in PrometheusMetricsGeneratorUtils (#18747)
• [improve][broker] Make Consumer#equals more effective (#18662)
• [fix][broker] In the trimDeletedEntries method, release the removed entry (#18305)
• [improve][broker] Remove locallyAcquiredLock when removeOwnership (#18197)
• [improve][broker] Avoid unnecessary creation of BitSetRecyclable objects #17998
• [fix][broker] add return for PersistentMessageExpiryMonitor#findEntryFailed #17816
• [fix][ml] Persist correct markDeletePosition to prevent message loss (#18237)
• [improve][broker] Support setting forceDeleteTenantAllowed dynamically (#18192)
• Make BookieId work with PulsarRegistrationDriver (second take) #17922
• [fix][broker] Extract additional servlets to the default directory by… (#17477)
• [improve][broker]Improve PersistentMessageExpiryMonitor expire speed when ledger not existed (#17842)
• [fix][broker]Fix mutex never released when trimming (#17911)
• [fix][broker] Fix if dynamicConfig item in ZK do not exist in broker cause NPE. #17705
• [fix][broker] Fix system service namespace create internal event topic. (#17867)
• [bugfix] Prevent Automatic Topic Creation during namespace deletion #17609
• [fix][broker]Cache invalidation due to concurrent access (#18076)
• [fix][broker]add test case for deleting namespace fail when has partitioned system topic (#17338)
• Skip creating a subscription replication snapshot if no messages have been published after the topic gets activated on a broker #16618
• [fix][broker] Fix uncompleted future when get the topic policies of a deleted topic (#18824)
• [fix][broker] Fix delete system topic clean topic policy (#18823)
• [improve][broker][PIP-149]Make getList async #16221
• [fix][flaky-test]ManagedCursorMetricsTest.testManagedCursorMetrics #16878
• [improve][broker] System topic writer/reader connection not counted (#18603)
• [improve][broker] System topic writer/reader connection not counted. (#18369)
• [fix][broker]unify time unit at dropping the backlog on a topic #17957[fix][broker]unify time unit at dropping the backlog on a topic #17957
• [improve][broker] Support setting ForceDeleteNamespaceAllowed dynamically #18181
• [fix][broker] Fix getPositionAfterN infinite loop. (#17971)
• [fix][broker] Update the log print content of createSubscriptions (#18024)
• [Broker] Make PersistentTopicsBase#internalGetPartitionedMetadata async #14153
• [fix][broker] Fix duplicated schemas creation #18701
• [improve][broker] Using handle instead of handleAsync to avoid using common pool thread (#17403)
• [fix][broker] Fix dispatch duplicated messages with Exclusive mode. (#17237)
• [fix][broker] Fix the order of resource close in the InMemoryDelayedDeliveryTracker (#18000)
• [improve][broker] reduce code duplication to avoid endless wait CompletableFuture (#14853)
• [improve][ML] Print log when delete empty ledger. (#17859)
• [improve] clean the empty topicAuthenticationMap in zk when revoke permission (#16815)
• [fix][broker] fix can not revoke permission after update topic partition (#17393)
• [fix][broker] Fix Npe thrown by splitBundle (#17370)
• [fix][broker] Fix executeWithRetry result is null (#17694)
• [fix][broker] Fix SystemTopicBasedTopicPoliciesService NPE issue (#17602)
• [fix][broker] Fix broker cache eviction of entries read by active cursors #17273
• [fix][broker] Fix namespace can not be deleted by force (#18686)
• [fix][broker] Create replicated subscriptions for new partitions when needed (#18659)
• [fix][broker] DnsResolverUtil.TTL should be greater than zero (#18565)
• [fix][broker] Correctly set byte and message out totals per subscription (#18451)
• [fix][broker] fix delete_when_subscriptions_caught_up doesn't work while have active consumers (#18320)
• [improve][broker]consumer backlog eviction policy should not reset read position for consumer (#18037)
• [fix][broker] Fix NPE when ResourceGroupService execute scheduled task. #17840
• [fix][storage] Autorecovery default reppDnsResolverClass to ZkBookieRackAffinityMapping (#15640)
• [broker] Fixed delayed delivery after read operation error (#18098)
• [fix][broker] Fix issue where leader broker information isn't available after 10 minutes (#17401)
• [fix][broker] Make full async call in PulsarAuthorizationProvider (#18050)

Transaction

• [fix][txn] Transaction buffer recover blocked by readNext #18833
• [fix][txn] transaction pending ack store future not completely problem (#18943)
• [fix][txn] Fix PendingAckHandleImpl when pendingAckStoreProvider.checkInitializedBefore failed (#18859)
• [improve][txn] Add getState in transaction for client API (#17423)
• [improve][txn] Implementation of Delayed Transaction Messages (#17548)

Pulsar IO and Pulsar Functions

• [fix][io] Only bundle kafka schema registry client (#18931)
• [fix][fn] Typo in method name (#18844)
• [fix][function] Fix invalid metric type gauge (#18129)
• [fix][fn] fix function failed to start if no typeClassName provided in FunctionDetails (#18111)

Tiered Storage

• [fix][offload] Fix numerical overflow bug while reading data from tiered storage (#18595)
• [fix][offload] Fix memory leak while Offloading ledgers (#18500)

Pulsar SQL

• [fix][sql] Fix message without schema issue. (#18745)

CLI

• [fix][cli] Fix CLI client produce don't able to use multiple -m send multiple messages (#18238)
• [fix][cli] Check numMessages after incrementing counter #17826

Admin

• [improve][admin] Fix NPE in admin-CLI topic stats command (#18326)
• [improve][admin] add topic name and sub name for NotFound error message (#15606)
• PIP-105: new API to get subscription properties #16095
• [fix][admin] returns 4xx error when pulsar-worker-service is disabled and trying to access it (#17901)
• Pulsar Admin: grab contextual stacktrace for sync methods (#14620)
• [fix][admin] Fix NPE when get OffloadThreshold on namespace (#18061)

Security

• [fix][sec] Upgrade scala-library to get rid of CVE-2022-36944 (#18021)
• [Improve][Auth]Update authentication failed metrics report (#17787)
• [fix][sec] Upgrade jackson-databind to 2.13.4.2 to get rid of CVE-2022-42003 (#18394)
• [fix][sec] Upgrade protobuf to 3.19.6 to get rid of CVE-2022-3171 (#18086)
• [fix][sec] File tiered storage: upgrade jettison to get rid of CVE-2022-40149 (#18022)
• [fix][sec] Upgrade JacksonXML to 2.13.4 (#18020)

Dependency & Library updates

• Bump jackson version from 2.13.2 to 2.13.3 #16508
• Remove invalid Netty system property which never was valid for Netty 4.1.x (#13563)
• [improve][misc] Upgrade Netty to 4.1.86.Final and Netty Tcnative to 2.0.54.Final (#18599)
• [improve][build] Remove versions that are handled by netty-bom (#18629)

CI & Test

• [improve][test] force initialize field to avoid polluted by mocks (#17022)
• [fix][flaky-test]ManagedCursorMetricsTest.testManagedCursorMetrics (#16878)
• [improve][test] remove WhiteBox on MockZooKeeper #17579
• [improve][test] try remove whitebox on MockZooKeeper (#17579)
• [improve][test] Add test case for system topic schema not compatible bug. (#17992)
• [fix][flaky-test]BatchMessageWithBatchIndexLevelTest.testBatchMessageAck (#17436)
• [fix][test]The pulsarMessageOverhead in 2.10 is different (#18704)
• [improve][test] Add subscribing regex topic test for delete_when_subscriptions_caught_up. (#18368)

Java Client

• [fix][client] For exclusive subscriptions, if two consumers are created repeatedly, the second consumer will block #18633
• [fix][client] Fixes batch_size not checked in MessageId#fromByteArrayWithTopic (#18405)
• [fix][client] Fix possible npe (#18406)
• [fix][client] Fix exception when calling loadConf on a ConsumerBuilder that has a KeySharedPolicy (#18345)
• [fix][client] Support LocalDateTime Conversion (#18334)
• [fix] [pulsar-client] Fix pendingLookupRequestSemaphore leak when Ser… (#18219)
• [fix][client] Fix failover/exclusive consumer with batch cumulate ack issue. (#18454)
• [improve][java-client]Add init capacity for messages in BatchMessageContainerImpl (#17822)
• [fix] [pulsar-client] Fix pendingLookupRequestSemaphore leak when channel inactive (#17856)
• [fix][client] Unwrap completion exception for Lookup Services (#17717)
• [fix][client] Avoid redelivering duplicated messages when batching is enabled #18486
• [fix][client] Fix multi-topic consumer stuck after redeliver messages (#18491)
• [fix][client] Fix IllegalThreadStateException when using newThread in ExecutorProvider.ExtendedThreadFactory
• [fix][client] Fix NPE of MultiTopicsConsumerImpl due to race condi...

v2.9.4

What's Changed

• [branch-2.9]add message ack rate by @tjiuming in #16146
• [branch-2.9] Fix cherry-pick issues. by @mattisonchao in #16346
• [Branch-2.9][Cherry-pick] fix bug: fail to expose managed ledger client stats to prometheus if bookkeeperClientExposeStatsToPrometheus is true by @HQebupt in #16343
• [Branch 2.9] Fix compaction subscription acknowledge Marker msg issue. by @mattisonchao in #16348
• [Branch-2.9] Fix passing incorrect authentication data by @mattisonchao in #16347
• [Branch-2.9][transaction] Cmd-Subscribe and Cmd-Producer will not succeed even after 100 retries. by @mattisonchao in #16349
• [Branch-2.9][Cherry-pick] Fix get non-persistent topics issue in Namespaces. by @Technoboy- in #16517
• [branch-2.9] Fix MockStatic method caused the exception by @mattisonchao in #16628
• [branch-2.9] Fix compatible issue. by @mattisonchao in #16779
• [branch-2.9]Add back FutureUtil#waitForAll and FutureUtil#waitForAny methods with List parameter by @mattisonchao in #16794
• [broker][monitoring][fix] fix pulsar_subscription_msg_ack_rate by @tjiuming in #16866
• [Branch-2.9][fix][client] Fix client memory limit currentUsage leak and semaphore release duplicated in ProducerImpl. by @Nicklee007 in #16971
• [improve][broker] Optimise msgOutCounter and bytesOutCounter (#16214) (#16286) by @nahguam in #16771
• [Branch-2.9][fix][client] Fix MaxQueueSize semaphore release leak in createOpSendMsg. by @Nicklee007 in #16958
• [Branch 2.9][fix][broker] Upgrade log4j2 version to 2.18.0 (#16884) by @liudezhi2098 in #16995
• [improve][authentication][branch-2.9] Support for get token from HTTP params by @nodece in #16987
• [Branch-2.9]Upgrade rocksdb version to 6.16.4 to keep sync with bookkeeper by @hangc0276 in #17246
• [branch-2.9] Support mock final class with PowerMockito by @mattisonchao in #17327
• [branch-2.9][fix][load-balancer] skip mis-configured resource usage(>100%) in loa… by @heesung-sn in #17285
• [fix][broker][branch-2.9] Fix calculate avg message per entry by @coderzc in #17281
• [branch-2.9] Rename test file name from *Test2.java to *Test.java to run all tests correctly by @equanz in #17048
• [fix][c++][branch-2.9]Fix wrong consumers size: execute callback before executing readerCreatedCallback_ by @coderzc in #17629
• [branch-2.9][fix][broker]Persist cursor info error when cursor close by @poorbarcode in #17604
• [branch-2.9][cherry-pick] Fix producer/consume permission can’t get schema. by @Technoboy- in #17730
• [fix][broker][functions-worker] Ensure prometheus metrics are grouped by type (#8407, #13865) by @marksilcox in #17618
• [branch-2.9] Group prometheus metrics. by @tjiuming in #17852
• [branch-2.9] Fix cherry-pick issue by @codelipenghui in #17894
• [fix][broker][branch-2.9] Fix system topic schema not compatible bug by @dragonls in #17986
• [improve][authentication][branch-2.9] Improve get the basic authentication config by @nodece in #16944
• [fix][client][txn] Use PulsarClient HashWheelTimer to schedule producer batch trigger task by @lifepuzzlefun in #18058
• [fix][sec] File tiered storage: upgrade jettison to get rid of CVE-2022-40149 by @mattisonchao in #18105
• [branch-2.9] Fixed key-shared delivery of messages with interleaved delays. by @mattisonchao in #18108
• [branch-2.9] Fix license check issue by @Technoboy- in #18117
• [fix][cpp][branch-2.9] Fix zlib dowload path. by @shibd in #18127
• [fix][sec][branch-2.9] Updating dependencies to get rid of CVEs brought in with kafka and lo… by @alpreu in #18304
• [cherry-pick][branch-2.9] Fix NPE when get OffloadThreshold on namespace. by @congbobo184 in #18638
• [cherry-pick][branch-2.9] cherry-pick #17736 problem and DnsResolverTest license header by @congbobo184 in #18640
• [cherry-pick][branch-2.9] Fix NPE when ResourceGroupService execute scheduled task. by @congbobo184 in #18685
• [cherry-pick][branch-2.9] fix cherry-pick #17592 missing code and flaky test testFailoverSingleAckedPartitionedTopic by @congbobo184 in #18702
• [branch-2.9] Fix no durable cursor leak problem by @gaozhangmin in #18710
• [branch-2.9][fix][broker] Fix duplicated schemas creation (#18701) by @codelipenghui in #18762
• [cherry-pick][branch-2.9] fix cherry-pick #17609 #17957 import by @congbobo184 in #18793
• [cherry-pick][branch-2.9] make getList async #16221 by @congbobo184 in #18811
• [cherry-pick][branch-2.9] Cherry-pick #18307 (Fix can not delete namespace by force) by @liangyepianzhou in #18803
• [cherry-pick][branch-2.9] change proto license version by @congbobo184 in #18821
• [cherry-pick][branch-2.9] Fix delete system topic clean topic policy (#18823) by @congbobo184 in #18831

Full Changelog: v2.9.3...v2.9.4

v2.10.2

Important notice

• [fix][admin] Fix get non-persistent topics issue in Namespaces. #16170

Broker

• [fix][broker] ManagedLedger: move to FENCED state in case of BadVersionException #17736
• [fix][broker] Issue 17588: Allow deletion of a namespace that was left in deleted status #17592
• [fix][broker] Fix BookKeeper packages npe #17291
• [fix][broker] Fix rack awareness cache expiration data race #16825
• [fix][broker] Fix stats-internal with option -m cause active ledger recover then close #16662
• [fix][broker] Fix deadlock in key-shared dispatcher #16660
• [fix][broker] PulsarLedgerManager: add missed return statement #16607
• [fix][broker] Ensure the ack-timeout task gets re-scheduled when there is an exception in the final stage #16337
• [fix][broker] Fix select broker is none #16316
• [fix][broker] Fixed deadlock when checking topic ownership #16310
• [fix][broker] Fix thread safety issues in accessing ManagedCursorContainer.heap ArrayList #16049
• [fix][broker] Avoid IllegalStateException while client_version is not set #16788
• [fix][broker] BadVersionException when splitting bundles, delay 100ms and try again. #16612
• [fix][broker] Expose timestamp field for SchemaData&SchemaInfo #16380
• [fix][broker] Fix NPE when drop backlog for time limit. #16235
• [fix][broker] Fix NPE when get /admin/v2/namespaces/public/default/maxTopicsPerNamespace #16076
• [fix][broker] Fix RawReader out of order #16390
• [fix][broker] Fix calculate avg message per entry #17046
• [fix][broker] Fix compaction subscription acknowledge Marker msg issue. #16205
• [fix][broker] Fix consumer does not abide by the max unacks limitation for Key_Shared subscription #16718
• [fix][broker] Fix consumer does not abide by the max unacks limitation for Shared subscription #16670
• [fix][broker] Fix create client with TLS config #16014
• [fix][broker] Fix namespace backlog quota check with retention. #17706
• [fix][broker] Fix out of order data replication #17154
• [fix][broker] Fix passing incorrect authentication data #16201
• [fix][broker] Fix potential exception causing the reader to be unable to continue reading #17556
• [fix][broker] Fix pulsarLedgerIdGenerator can't delete index path when zk metadata store config rootPath. #17192
• [fix][broker] Fix schema does not replicate successfully #17049
• [fix][broker] Fix the replicator unnecessary get schema request for BYTES schema #17523
• [fix][broker] Fixed error when delayed messages trackers state grows to >1.5GB #16490
• [fix][broker] Increment topic stats outbound message counters and update rate after messages have been written to the TCP/IP connection #17043
• [fix][broker] ManagedCursor: mark delete no callback when create meta-ledger fail #16841
• [fix][broker] Multiple consumer dispatcher stuck when unackedMessages greater than maxUnackedMessages #17483
• [fix][broker] PulsarLedgerManager to pass correct error code to BK client #16857
• [fix][broker] Release the entry in getEarliestMessagePublishTimeOfPos. #16386
• [fix][broker] Retry to delete the namespace if new topics are created during the namespace deletion #16676
• [fix][broker] Retry when DistributedIdGenerator has BadVersion error #16491
• [fix][broker] Skip reading more entries for a pending read with no more entries #16400
• [fix][broker] Support loadBalancerSheddingIntervalMinutes dynamic configuration #16408
• [fix][broker] The configuration loadBalancerNamespaceMaximumBundles is invalid #16552
• [fix][broker] Topic policy reader can't recover when getting any exception. #17562
• [fix][broker] Unregister topic policy listener if managed ledger close failed #17652
• [fix][broker] Upgrade log4j2 version to 2.18.0 #16884
• [fix][broker] fix No such ledger exception #16420
• [fix][broker] fix broker unackmessages become a negative number #17003
• [fix][broker] remove exception log when access status.html #17025
• [fix][broker] Consumer can't consume messages because there are two same topics in one broker #17526
• [fix][broker] Fix getInternalStats occasional lack of LeaderInfo again #16238
• [fix][broker] Fix subscribe dispatcher limiter not be initialized #16175
• [fix][broker] Fix topic policies update not check message expiry #15941
• [fix][broker] Fix topic-level replicator rate limiter not init #15825
• [fix][broker] ManagedLedger metrics fail cause of zero period #17257
• [fix][broker] Prevent StackOverFlowException in SHARED subscription #16968
• [fix][broker] Fix message ack rate #16108
• [fix][broker] Fix bug: fail to expose managed ledger client stats to prometheus if bookkeeperClientExposeStatsToPrometheus is true #16219
• [fix][broker] Fix npe when invoking replaceBookie. #16239
• [fix][broker] Fail to update partition meta of topic due to ConflictException: subscription already exists for topic #17488
• [fix][broker] Fix NPE when ledger id not found in OpReadEntry #15837
• [Fix][broker] Fix topic dispatch rate limiter not init on broker-level #16084
• [opt][broker] Optimize topic policy with HierarchyTopicPolicies about replicatorDispatchRate #14161
• [fix][broker] Fix the broker close hanged issue. #17689
• [fix][broker] Fix topic dispatch rate limiter not init on broker-level #16084 #17000
• [fix][broker] Prevent StackOverFlowException in SHARED subscription #17053
• [fix][broker] skip mis-configured resource usage(>100%) in load balancer #16937
• [fix][broker] Fix wrong unit of NIC speed on Linux #15304
• [fix][broker] Fix RawReader hasMessageAvailable returns true when no messages #16443
• [improve][broker] Expose topic level storage write and read rate metrics #16855
• [improve][broker] Broker/EntryFilter: make the delay for RESCHEDULED messages configurable (dispatcherFilterRescheduledMessageDelay) #16602
• [improve][broker] Use OrderedExecutor instead of OrderedScheduler for consumer dispatch #16115
• [improve][broker] Avoid going through all the consumers to get the message ack owner #16245
• [improve][broker] Avoid reconnection when a partitioned topic was created concurrently #16043
• [improve][broker] Cancel the loadShedding task when closing pulsar service #17632
• [improve][broker] Consolidate Netty channel flushes to mitigate syscall overhead #16361
• [improve][broker] Improve cursor.getNumberOfEntries if isUnackedRangesOpenCacheSetEnabled=true #17465
• [improve][broker] Make MessageRedeliveryController work more efficiently [#17804](https...

v2.8.4

Broker

• [improve][broker] Do not print the log that the topic does not exist #13535
• [fix][broker] Fix NPE when nextValidLedger is null #13975
• [fix][broker] Fix NPE when ReplicatedSubscriptionsController sends marker message with deduplication enabled #14017
• [fix][broker] Fix avg-messagePerEntry metrics for consumer #14330
• [improve][broker] Validate rack name when setting the bookie rack info #14336
• [improve][broker] Optimize find nics process. #14340
• [fix][broker] Fix NPE when subscription is already removed #14363
• [fix][broker] Fix Broker HealthCheck Endpoint Exposes Race Conditions #14367
• [improve][broker] Optimize RawReader#create when using Compactor #14447
• [fix][broker] Fix the latest message of the compacted topic cannot be read #14449
• [fix][broker] Fix producerFuture not completed in ServerCnx#handleProducer #14467
• [fix][broker] Fix NPW in ManagedLedgerImpl #14481
• [improve][broker] Support shrink in ConcurrentLongHashMap #14497
• [fix][broker] Fixed wrong behaviour caused by not cleaning up topic policy service state. #14503
• [improve][broker] Optimize memory usage: support to shrink for pendingAcks map #14515
• [fix][broker] Cancel offload tasks when managed ledger closed #14744
• [improve][broker] Reduce unnecessary expansions for ConcurrentLong map and set #14562
• [fix][broker] Fix metadata store deadlock when checking BacklogQuota #14634
• [improve][broker] Support shrink for map or set #14663
• [fix][broker] Fix lost message issue due to ledger rollover #14664
• [fix][broker] Set splitNamespaceBundle with readonly=false #14680
• [fix][broker] Fixed duplicated delayed messages when all consumers disconnect #14740
• [fix][broker] Fix wrong state for non-durable cursor #14869
• [fix][broker] Improve error logging for topic not found #14892
• [fix][broker] Fix topic policy reader close bug #14897
• [fix][broker] Return if reset in progress #14978
• [fix][broker] Fix potential NPE in Replicator #15003
• [fix][broker] Fix race condition in updating lastMarkDeleteEntry field #15031
• [fix][broker] Fix potential to add duplicated consumer #16826
• [fix][broker] Follow up on race condition fixes in ManagedCursorImpl #15031 #15067
• [cleanup] [broker] Remove useless code to avoid confusion in OpReadEntry#checkReadCompletion #15104
• [fix][broker] Cancel fencedTopicMonitoringTask when topic closed normally. #15202
• [fix][broker] Fix MessageDeduplication#inactiveProducers may not be persistence correctly #15206
• [fix][broker] Fix race condition between timeout and completion in OpAddEntry #15233
• [fix][broker] Fix problem at RateLimiter#tryAcquire #15306
• [improve][broker] Use shrink map for message redelivery. #15342
• [improve][broker] Support shrink for ConcurrentSortedLongPairSet #15354
• [improve][broker] Close connection if a ping or pong message cannot be sent #15382
• [cleanup][broker] Fix typo in enum name and handle closing of the channel properly since writeAndFlush is asynchronous #15384
• [fix][broker] Avoid heartbeat topic to offload. #15393
• [fix][broker] Fix deadlock in broker after race condition in topic creation failure #15570
• [fix][broker] fix calculation in getNumberOfEntriesInStorage #15627
• [fix][broker] Fix NPE when set AutoTopicCreationOverride #15653
• [fix][broker] Fix creating producer failure when set backlog quota #15663
• [fix][broker] Fix NPE when put value to RangeCache #15707
• [fix][broker] Fix can not enable system topic if AutoUpdateSchemaEnabled=false #15759
• [fix][broker] Fix wrong unit of NIC speed on Linux #15770
• [fix][broker] Fix NPE in MessageDeduplication #15820
• [fix][broker] Fix NPE when ledger id not found in OpReadEntry #15837
• [fix][broker] When skipping updating mark delete position, execute callback with executor to prevent deadlock #15971
• [fix][broker] Fix NPE when drop backlog for time limit. #16235
• [fix][broker]Fix getInternalStats occasional lack of LeaderInfo again #16238
• [fix][broker] Fix RawReader out of order #16390
• [fix][broker] Skip reading more entries for a pending read with no more entries #16400
• [fix][broker] Fix No such ledger exception #16420
• [fix][broker] Fix RawReader hasMessageAvailable returns true when no messages #16443
• [fix][broker] Fix stats-internal with option -m cause active ledger recover then close #16662
• [fix][broker] Fix passing incorrect authentication data #16840
• [fix][broker] ManagedCursor: mark delete no callback when create meta-ledger fail #16841
• [improve][broker] Avoid reconnection when a partitioned topic was created concurrently #16867
• [improve][broker] Recycle OpReadEntry in some corner cases #16869
• [fix][broker] Fix compaction subscription acknowledge Marker msg issue. #16918

Security

• [fix][sec] Use provider's canLookupAsync method for AuthorizationService #11777
• [improve][sec] Optimize the logic of allowing namespace operation #13090
• [improve][sec] Full-support set SSL provider, ciphers and protocols #13740
• [fix][sec] Role with namespace produce authz can also get topics #13773
• [improve][sec] Full-support SSL provider, ciphers, and protocols for broker service and proxy service #15034
• [improve][sec] Add load multiple certificates support #14798
• [fix][sec] Fix handling single role and non-jwt-token in MultiRolesTokenAuthorizationProvider #14857
• [fix][sec] Use tlsCertRefreshCheckDurationSec instead of 0 for refresh value #15075
• [improve][sec] Skip unnecessary DNS resolution when creating AuthenticationDataHttp instance #15221
• [improve][sec] Improve skipping of DNS resolution when creating AuthenticationDataHttp instance #15228
• [fix][sec] Fix MultiRolesTokenAuthorizationProvider authorize issue #15454
• [fix][sec] Fix grant all permissions but can't list topic #15501
• [improve][sec] Switch to rely on Netty for Hostname Verification #15824
• [fix][sec] Fix create client with TLS config #16014
• [fix][sec] Avoid AuthenticationDataSource mutation for subscription name #16065
• [improve][sec] Improve get the basic authentication config #16526
• [fix][sec] Fix multiple roles authorization #16645
• [fix][sec] Add timeout of sync methods and avoid call sync method for AuthoriationServi...

v2.7.5

Broker

• [fix][broker] Fix NPE when ledger id not found in OpReadEntry#16966
• [fix][broker] Validate rack name #16850
• [fix][broker] Fix invalid rack name cause bookie join rack failed #16845
• [fix][broker] Avoid potentially blocking calls to metadata in PersistentTopic#getMessageTTL #16899
• [fix][broker] Fix create client with TLS config #16326
• [fix][broker] Fixed deadlock on metadata cache missing while doing checkReplication #16889
• [fix][broker] Fix the wrong unit of NIC speed on Linux #15770
• [fix][broker] Close connection if a ping or pong message cannot be sent #15382
• [fix][broker] Fix MessageDeduplication#inactiveProducers may not be persistence correctly #15206
• [fix][broker] Fix messageDedup delete inactive producer name #12493
• [fix][broker] Fix the reader skips compacted data which original ledger been removed. #16407
• [fix][broker] Fix getting the last message-id from an empty compact ledger. #16402
• [fix][broker] Return message ID from compacted ledger while the compaction cursor reaches the end of the topic. #16395
• [fix][broker] Fix skips compacted data for reader/consumer #16301
• [fix][broker] Fix reader skipped remaining compacted data during the topic unloading #16300
• [fix][broker] Fix passing incorrect authentication data #16201
• [fix][broker] Fix incorrect returned last message ID while the lastConfirmedEntry with negative entry ID #16299
• [fix][broker] Fix can not enable system topic if AutoUpdateSchemaEnabled=false. #15754
• [fix][broker] Fix lost message issue due to ledger rollover. #14703
• [fix][broker] Fix lost message issues 12221 #12223
• [fix][broker] Fix TopicPoliciesCacheNotInitException issue. #14293
• [fix][broker] Fix deadlock when use key_shared mode #11965
• [fix][broker] Fix log content error when OverloadShedder is not used. #13540
• [improve][broker] Skip unnecessary DNS resolution when creating AuthenticationDataHttp instance #15221
• [improve][broker] Improve error logging for topic not found [#13950) #14893
• [improve][broker] Trim configuration value string which contains blank prefix or suffix string #13984
• [improve][broker] Configure DLog Bookie, Pulsar, and Admin clients via pass-through config #15818
• [cleanup][broker] Improve skipping of DNS resolution when creating AuthenticationDataHttp instance #15228
• [cleanup][broker] Remove useless code to avoid confusion in OpReadEntry#checkReadCompletion. #15104
• [cleanup][broker] Clean up individually deleted messages before the mark-delete position #14261

Security

• [improve][sec] Improve get the basic authentication config #16947
• [improve][sec] Support for get token from HTTP params #16892
• [fix][sec] AuthorizationService should use provider's canLookupAsync method #11777
• [fix][sec] Avoid AuthenticationDataSource mutation for subscription name #16065
• [fix][sec] Return if namespace policies are read only #12514
• [refactor][sec] Switch to rely on Netty for Hostname Verification #15824
• [cleanup][sec] Ignore case when obfuscating passwords in configuration scripts #15077

Storage

• [fix][storage] ManagedCursor: mark delete no callback when create meta-ledger fail #16841
• [fix][storage] Cancel offload tasks when managed ledger closed #16808
• [fix][storage] Fix lost compaction data due to compaction properties missed during reset-cursor. #16404
• [fix][storage] Do not move the non-durable cursor position when trimming ledgers while topic with compaction. #16403
• [fix][storage] Fix issues in advanceNonDurableCursors #10667
• [fix][storage] Follow up on race condition fixes in ManagedCursorImpl #15031 #15067
• [fix][storage] Fix race condition in updating lastMarkDeleteEntry field #15031
• [fix][storage] Fix NPE when removing cursor #12297
• [improve][storage] If mark-delete operation fails, mark the cursor as "dirty" #14256

Proxy

• [fix][proxy] Fix client service url #16894
• [fix][proxy] Prevent leak of unreleased lookupRequestSemaphore permits #13812
• [fix][proxy] Remove unnecessary blocking DNS lookup in LookupProxyHandler #15415
• [fix][proxy] Fix proxy connection leak when inbound connection closes while connecting is in progress #15366
• [fix][proxy] Fix port exhaustion and connection issues in Pulsar Proxy #14078
• [fix][proxy] Fix DNS server denial-of-service issue when DNS entry expires #15403
• [fix][proxy] Configure Netty DNS resolver to match JDK DNS caching setting, share DNS resolver instance in Proxy #15219
• [refactor][proxy] Refactor Proxy code and fix connection stalling by switching to auto read mode #14713
• [improve][proxy] Fail proxy startup if brokerServiceURL is missing scheme #14682
• [improve][proxy] Remove unnecessary Pulsar Client usage from Pulsar Proxy #13836

Admin

• [fix][admin] Fix validateGlobalNamespaceOwnership wrap exception issue. #14269
• [cleanup][admin] Update/fix Swagger Annotation for param: authoritative #16222

Function

• [fix][fn] Fix python instance not process zip file correctly #16697

Java Client

• [fix][java] Fix semaphore release duplicated in ProducerImpl #16972
• [fix][java] Fix MaxQueueSize semaphore release leak in createOpSendMsg #16915
• [fix][java] Fix PatternTopicsChangedListener blocked when topic removed #16842
• [fix][java] Fix ReconsumeLater will hang up if retryLetterProducer exception #16655
• [fix][java] Fix DNS server denial-of-service issue when DNS entry expires #15403
• [fix][java] Configure Netty DNS resolver to match JDK DNS caching setting, share DNS resolver instance in Proxy #15219
• [refactor][java] Switch to rely on Netty for Hostname Verification #15824
• [improve][java] Remove unnecessary Pulsar Client usage from Pulsar Proxy #13836

C++ Client

• [cleanup][cpp] Clean up C++ client curl configuration #16064
• [cleanup][cpp] Add braces around initialization of subobject #14735

Python Client

• [fix][python] Fix generated Python protobuf code not compatible with the latest protobuf package #15846

Test & Others

• [improve][test] Verify the authentication data in the authorization provider #16945
• [improve][test] Exclude inner classes in surefire #9875
• [fix][test] Enable ignored tests #16435
• [fix][test] Fix setting Bookie dbStorage_*CacheMaxSizeMb in pulsar-test-latest-version docker image #9623
• [improve][doc] Add more configuration methods for basic authentication #16941
• [fix][build] Use grpc-...

v2.9.3

Important notice

• [PIP-146] ManagedCursorInfo compression #14542
• [PIP-153] Optimize metadataPositions in MLPendingAckStore #15137
• [PIP-163] Add lowWaterMark check before appending entry to TB #15424

Broker

• [cleanup][broker] Cleanup already deleted namespace topics #12597
• [cleanup][broker] Override close method to avoid caching exception #15529
• [cleanup][broker] Remove useless code to avoid confusion in OpReadEntry#checkReadCompletion #15104
• [fix][broker] Avoid heartbeat topic to offload #15008
• [fix][broker] Cancel fencedTopicMonitoringTask when topic closed normally #15202
• [fix][broker] Check for blank advertised listener name #14306
• [fix][broker] Close publishLimiter when disable it #15520
• [fix][broker] Fast return if ack cumulative illegal #15695
• [fix][broker] Fix MessageDeduplication#inactiveProducers may not be persistence correctly #15206
• [fix][broker] Fix MultiRolesTokenAuthorizationProvider authorize issue #15454
• [fix][broker] Fix NPE in MessageDeduplication #15820
• [fix][broker] Fix NPE when ledger id not found in OpReadEntry #15837
• [fix][broker] Fix NPE when put value to RangeCache #15707
• [fix][broker] Fix NPE when set AutoTopicCreationOverride #15653
• [fix][broker] Fix NPE when subscription is already removed #14363
• [fix][broker] Fix REST produce msg redirect issue #15551
• [fix][broker] Fix call sync method in onPoliciesUpdate method #13885
• [fix][broker] Fix call sync method in onPoliciesUpdate method #15227
• [fix][broker] Fix cannot delete namespace with system topic #14730
• [fix][broker] Fix creating producer failure when set backlog quota #15663
• [fix][broker] Fix creating system namespace topic failure #14949
• [fix][broker] Fix deadlock in broker after race condition in topic creation failure #15570
• [fix][broker] Fix getPendingAckInternalStats redirect issue #14876
• [fix][broker] Fix inconsistent prompt message when schema version is empty using AVRO #14626
• [fix][broker] Fix incorrect entryId in warning logs when reading an entry from tiered storage #14685
• [fix][broker] Fix metadata store deadlock when checking BacklogQuota #14634
• [fix][broker] Fix no value present #14891
• [fix][broker] Fix normal topic named ends with healthcheck becomes system topic issue #14671
• [fix][broker] Fix parameter saslJaasBrokerSectionName in broker.conf #15110
• [fix][broker] Fix potential to add duplicated consumer #15051
• [fix][broker] Fix precision issue and initial value for Consumer#avgMessagesPerEntry #14666
• [fix][broker] Fix problem at RateLimiter#tryAcquire #15306
• [fix][broker] Fix producerFuture not completed in ServerCnx#handleProducer #14467
• [fix][broker] Fix race condition between timeout and completion in OpAddEntry #15233
• [fix][broker] Fix race condition in updating lastMarkDeleteEntry field #15031
• [fix][broker] Fix rewind failed when redeliverUnacknowledgedMessages #15046
• [fix][broker] Fix topic policy reader close bug #14897
• [fix][broker] Fix typo in enum name and handle closing of the channel properly since writeAndFlush is asynchronous #15384
• [fix][broker] Fix when nextValidLedger is null caused NPE #13975
• [fix][broker] Fix wrong prompt exception when getting the non-persistent topic list without GET_BUDNLE permission #14638
• [fix][broker] Fix wrong state for non-durable cursor #14869
• [fix][broker] Fix wrong unit of NIC speed on Linux #15770
• [fix][broker] Fix 404 error msg not being returned correctly using http lookup #14677
• [fix][broker] Follow up on race condition fixes in ManagedCursorImpl #15067
• [fix][broker] Remove the loadbalance/bundle-data node #13164
• [fix][broker] Return if reset in progress #14978
• [fix][broker] Use dispatchRateLimiterLock to update dispatchRateLimiter #15601
• [fix][broker] When skipping updating mark delete position, execute callback with executor to prevent deadlock #15971
• [fix][broker] Expose configurationMetadataStore and localMetadataStore #15661
• [fix][broker] Filter the virtual NIC with relative path #14829
• [fix][broker] Fix MetadataStoreException$NotFoundException while doing topic lookup #15633
• [fix][broker] Fix calculation in getNumberOfEntriesInStorage #15627
• [fix][broker] Fix error log miss stack trace when create tenant fail #14366
• [fix][broker] Fix resource group does not report usage #15292
• [fix][broker] Fix duplicated delayed messages when all consumers disconnect #14740
• [fix][broker] Fix the pid occupied check when using pulsar-daemon start or stop process #14701
• [fix][broker] Fix potential NPE in Replicator #15003
• [improve][broker] Add log when updating namespace policies with error #14850
• [improve][broker] Add publishRateLimitedTimes to topic metrics #15739
• [improve][broker] Avoid using blocking calls for the async method checkTopicOwnership #15023
• [improve][broker] Cancel offload tasks when managed ledger closed #14545
• [improve][broker] Close connection if a ping or pong message cannot be sent #15382
• [improve][broker] Configure DLog Bookie, Pulsar, and Admin clients via pass-through config #15818
• [improve][broker] Full-support SSL provider, ciphers, and protocols for broker service and proxy service #14569
• [improve][broker] Ignore the print the log that the topic does not exist #13535
• [improve][broker] Optimize RawReader#create when using Compactor #14447
• [improve][broker] Optimize find nics process #14340
• [improve][broker] Optimize memory usage: support to shrink for pendingAcks map #14515
• [improve][broker] Provide an accurate error message when set autoTopicCreation #14684
• [improve][broker] Reduce unnecessary expansions for ConcurrentLong map and set #14562
• [improve][broker] Set splitNamespaceBundle with readonly=false #14680
• [improve][broker] Skip unnecessary DNS resolution when creating AuthenticationDataHttp instance #15221
• [improve][broker] Support advertised listeners for HTTP and HTTPS services #14839
• [improve][broker] Support shrink for ConcurrentSortedLongPairSet #15354
• [improve][broker] Support shrink for map or set #14663
• [improve][broker] Support shrink in ConcurrentLongHashMap [#14497](https://github...

v2.10.1

Important notice

• [fix][broker] Fix broker LoadBalance uneffective 15314
• [fix][admin] Fix producer/consume permission can’t get schema 15956

Broker

• [fix][broker] Fix race condition in getManagedLedgerInternalStats when includeLedgerMetadata=true 15918
• [improve][broker] Avoid contended synchronized block on topic load 15883
• [fix][broker] Fix NPE in MessageDeduplication 15820
• [improve][broker] Add timeout to closing CoordinationServiceImpl 15777
• [improve][broker] Store Subscription properties 15757
• [improve][broker] Support for updating the Subscription properties 15751
• [improve][broker] Disable memory limit controller for broker client and replication clients 15723
• [fix][broker] Fix NPE when put value to RangeCache. 15707
• [fix][broker] Fast return if ack cumulative illegal 15695
• [fix][broker] Fix creating producer failure when set backlog quota. 15663
• [fix][broker] Expose configurationMetadataStore and localMetadataStore 15661
• [fix][broker] Fix NPE when set AutoTopicCreationOverride 15653
• [fix][broker] Fix MetadataStoreException$NotFoundException while doing topic lookup 15633
• [fix][broker] Fix calculation in getNumberOfEntriesInStorage 15627
• [fix][broker] Use dispatchRateLimiterLock to update dispatchRateLimiter 15601
• [fix][broker] Sync topicPublishRateLimiter update 15599
• [fix][broker] Fix deadlock in broker after race condition in topic creation failure 15570
• [cleanup][broker] Override close method to avoid caching exception. 15529
• [fix][broker] Close publishLimiter when disable it 15520
• [fix][broker] Fix to avoid TopicStatsImpl NPE even if producerName is null 15502
• [fix][broker] Fix key-shared delivery of messages with interleaved delays 15409
• [fix][Broker] Fix bug in contructor of RocksdbMetadataStore 15405
• [feature][broker] EntryFilter (PIP-105) - support per-Consumer filtering 15391
• [fix][broker/client] Close connection if a ping or pong message cannot be sent 15382
• [improve][broker] Support shrink for ConcurrentSortedLongPairSet 15354
• [improve][broker] Support properties on NonDurable subscriptions (PIP-105) 15345
• [improve][broker] Use shrink map for message redelivery. 15342
• [fix][Broker] Fix error on recycled SubscriptionPropertiesList 15335
• [fix][broker] Fix problem at RateLimiter#tryAcquire 15306
• [fix][broker] Fix totalEntries calculation problem in AbstractBaseDispatcher#filterEntriesForConsumere 15298
• [fix][broker] Fix resource group does not report usage 15292
• [fix][Broker] Fix race condition between timeout and completion in OpAddEntry 15233
• [fix][broker] Fix MessageDeduplication#inactiveProducers may not be persistence correctly 15206
• [fix][broker] Cancel fencedTopicMonitoringTask when topic closed normally 15202
• [fix][broker] Fix parameter saslJaasBrokerSectionName in broker.conf 15110
• [cleanup][broker] Remove useless code to avoid confusion in OpReadEntry#checkReadCompletion 15104
• [fix][broker] Ensure NamespaceEphemeralData has equals() operator 15092
• [fix][broker] Fix potential to add duplicated consumer 15051
• [fix][broker] Fix rewind failed when redeliverUnacknowledgedMessages 15046
• [fix][broker]Fix race condition in updating lastMarkDeleteEntry field 15031
• [improve][broker] Avoid using blocking calls for the async method checkTopicOwnership 15023
• [fix][broker] Avoid heartbeat topic to offload. 15008
• [fix][broker] Return if reset in progress 14978
• [fix][broker] Fix topic policy reader close bug 14897
• [fix][broker] Fix getPendingAckInternalStats redirect issue 14876
• [fix][broker] Fix wrong state for non-durable cursor 14869
• [improve][broker] Add log when update namespace policies with error 14850
• [feature][broker] Support advertised listeners for HTTP and HTTPS services 14839
• [fix][broker] Filter the virtual NIC with relative path 14829
• [fix][broker] Fixed duplicated delayed messages when all consumers disconnect 14740
• [fix][broker] Fix cannot delete namespace with system topic 14730
• [fix][broker] Fixed 404 error msg not being returned correctly using http lookup 14677
• [fix][broker] Fix normal topic named ends with healthcheck becomes system topic issue 14671
• [improve][broker] Support shrink for map or set 14663
• [improve][broker] Changing the topic creation flow and optimize heartbeat topic not trigger compaction 14643
• [fix][broker] Fix wrong prompt exception when getting the non-persistent topic list without GET_BUDNLE permission 14638
• [fix][broker] Fix inconsistent prompt message when schema version is empty using AVRO. 14626
• [fix][broker] Fix update replication cluster but not update replicator 14570
• [improve][broker] Reduce unnecessary expansions for ConcurrentLong map and set 14562
• [improve][broker] Support ManagedCursorInfo compression 14542
• [improve][broker] Optimize memory usage: support to shrink for pendingAcks map 14515
• [improve][broker] Support shrink in ConcurrentLongHashMap 14497
• [improve][broker] Optimize RawReader#create when using Compactor 14447
• [fix][broker] Fix NPE when subscription is already removed 14363
• [improve][broker] Optimize load manager find nics process 14340
• [improve][broker] Make revokePermissionsOnTopic method async 14149
• [fix][broker] Fix when nextValidLedger is null caused NPE 13975

Clients

• [fix][Java] Fixes NPE when TableView handles null value message 15951
• [fix][Java] Fix conversion of TimestampMillisConversion has no effect when Jsr310Conversion enabled 15863
• [fix][Java] Fix messages sent by producers without schema cannot be decoded 15622
• [improve][Java] improve logic when ACK grouping tracker checks duplicated message id 15465
• [improve][Java] Add pending messages information while printing the producer stats 15440
• [fix][Java] Fix negative ack not redelivery 15312
• [improve][admin/client] AsyncHttpConnector doesn't use the system properties configured 15307
• [fix][Java] TableView should cache created readers 15178
• [fix][Java] Fix performance regression with message listener 15162
• [fix][Java] Fix internal receive used wrong timeout type 15014
• [fix][Java] ConsumerBuilderImpl can not set null to deadLetterPolicy. [14980](https:...