Retries ignored and client stuck in retry loop when SASL authentication fails

[purplefox]

Description

Hi,

My producer is stuck in an infinite retry loop when SASL authentication with the server fails.
In this case, I do not want it to retry, I want an error returned from the delivery channel.
The librdkafka docs state that temporary errors are retried however I can set the properties retries or message.send.max.retries on the config map. I have tried both, and tried with values of 0 and 1.
Setting the properties doesn't seem to have any affect and the client retries indefinitely and does not return an error on the delivery channel.

How to reproduce

Here's my config:

cfg := kafkago.ConfigMap{
		"partitioner":       "murmur2_random", // This matches the default hash algorithm we use, and same as Java client
		"bootstrap.servers": ...,
		"security.protocol": "sasl_ssl",
		"sasl.mechanisms":   "SCRAM-SHA-256",
		"sasl.username":     "some_username",
		"sasl.password":     "some_password",
		"ssl.ca.location":   ...,
		//"debug":             "all",
		"message.send.max.retries": 1,
		"retries":                  1,
	}

producer, err := kafkago.NewProducer(&cfg)

require.NoError(t, err)

deliveryChan := make(chan kafkago.Event, 1)
err := producer.Produce(&kafkago.Message{
    TopicPartition: kafkago.TopicPartition{Topic: &topicName, Partition: kafkago.PartitionAny},   
    Key:            key,
    Value:          value},
    deliveryChan,
  )
require.NoError(t, err) // Never get an error here
e := <-deliveryChan
m := e.(*kafkago.Message)
if m.TopicPartition.Error != nil  {
       // Never get an error here!
       return nil, m.TopicPartition.Error
}
	

Logs look like:

%3|1723629669.142|FAIL|rdkafka#producer-1| [thrd:sasl_ssl://127.0.0.1:52151/bootstrap]: sasl_ssl://127.0.0.1:52151/bootstrap: SASL authentication error:  (after 0ms in state AUTH_REQ)
%3|1723629669.395|FAIL|rdkafka#producer-1| [thrd:sasl_ssl://127.0.0.1:52151/bootstrap]: sasl_ssl://127.0.0.1:52151/bootstrap: SASL authentication error:  (after 0ms in state AUTH_REQ, 1 identical error(s) suppressed)
%3|1723629701.429|FAIL|rdkafka#producer-1| [thrd:sasl_ssl://127.0.0.1:52151/bootstrap]: sasl_ssl://127.0.0.1:52151/bootstrap: SASL authentication error:  (after 0ms in state AUTH_REQ, 5 identical error(s) suppressed)
%3|1723629731.470|FAIL|rdkafka#producer-1| [thrd:sasl_ssl://127.0.0.1:52151/bootstrap]: sasl_ssl://127.0.0.1:52151/bootstrap: SASL authentication error:  (after 0ms in state AUTH_REQ, 3 identical error(s) suppressed)
%3|1723629766.753|FAIL|rdkafka#producer-1| [thrd:sasl_ssl://127.0.0.1:52151/bootstrap]: sasl_ssl://127.0.0.1:52151/bootstrap: SASL authentication error:  (after 0ms in state AUTH_REQ, 4 identical error(s) suppressed)
%3|1723629804.321|FAIL|rdkafka#producer-1| [thrd:sasl_ssl://127.0.0.1:52151/bootstrap]: sasl_ssl://127.0.0.1:52151/bootstrap: SASL authentication error:  (after 0ms in state AUTH_REQ, 4 identical error(s) suppressed)
%3|1723629841.875|FAIL|rdkafka#producer-1| [thrd:sasl_ssl://127.0.0.1:52151/bootstrap]: sasl_ssl://127.0.0.1:52151/bootstrap: SASL authentication error:  (after 0ms in state AUTH_REQ, 4 identical error(s) suppressed)
%3|1723629880.905|FAIL|rdkafka#producer-1| [thrd:sasl_ssl://127.0.0.1:52151/bootstrap]: sasl_ssl://127.0.0.1:52151/bootstrap: SASL authentication error:  (after 0ms in state AUTH_REQ, 4 identical error(s) suppressed)

Checklist

Please provide the following information:

• confluent-kafka-go and librdkafka version (LibraryVersion()):
• Apache Kafka broker version:
• Client configuration: ConfigMap{...}
• Operating system:
• Provide client logs (with "debug": ".." as necessary)
• Provide broker log excerpts
• Critical issue