SearchIndexAsync invalid nextToken

[Maverick1983]

Describe the bug

I want to share my problem.

I have simple request C# for SearchIndexAsync aws SDK. First time receive response with 'nextToken' field for request other page. I use nextToken value, but receive 'Invalid nextToken'

I perform same request by aws cli and it's working fine.

My code:

SearchIndexResponse response = await _iotClient.SearchIndexAsync(new SearchIndexRequest
{
    QueryString = queryString,
    MaxResults = maxResults,
    NextToken = nextToken
});

Example:

queryString = attributes.domain=*
maxResults = 100

_iotClient is IAmazonIoT

Exception

at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionStream(IRequestContext requestContext, IWebResponseData httpErrorResponse, HttpErrorResponseException exception, Stream responseStream)
   at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionAsync(IExecutionContext executionContext, HttpErrorResponseException exception)
   at Amazon.Runtime.Internal.ExceptionHandler`1.HandleAsync(IExecutionContext executionContext, Exception exception)
   at Amazon.Runtime.Internal.ErrorHandler.ProcessExceptionAsync(IExecutionContext executionContext, Exception exception)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Signer.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.MetricsHandler.InvokeAsync[T](IExecutionContext executionContext)

• nuget .net 8.0
• AWSSDK.IoT 3.7.404.2
• AWSSDK.Core 3.7.400.58

AmazonIoTClient 58|2024-12-02T15:36:19.097Z|ERROR|InvalidRequestException making request SearchIndexRequest to https://iot.eu-central-1.amazonaws.com/. Attempt 2. --> Amazon.IoT.Model.InvalidRequestException: Invalid nextToken

API response with this nextToken, when I perform first request:

"nextToken": "AQJnif/ptPp6otTULTCYZu3eV0Be458dLpq/pnT+pHC4bZjcDbBIiRNCLJ9PH7x9xCCIqQDMU0k4gbr+a3CHoMxs+yFnId9gdG2f+FnYFQc3xgSyj7jAyxNdXbN9izAC7KjfAgjs5Xw4smIK50bz3SBbGhrOc6u9eDK3A7qWXpUnsBzr4yh76ds5mfr/WeouzanV+aDz+RR/Gg=="

Update:
Response Log. I add 'xxxx' for privacy response

AmazonIoTClient 67|2024-12-02T15:38:31.771Z|DEBUG|Received response (truncated to 1024 bytes): [{"nextToken":"AQJnif/ptPp6otTULTCYZu3eV0Be458dLpq/pnT+pHC4bZjcDbBIiRNCLJ9PH7x9xCCIqQDMU0k4gbr+a3CHoMxs+yFnId9gdG2f+FnYFQc3xgSyj7jAyxNdXbN9izAC7KjfAgjs5Xw4smIK50bz3SBbGhrOc6u9eDK3A7qWXpUnsBzr4yh76ds5mfr/WeouzanV+aDz+RR/Gg==","thingGroups":null,"things":[{"attributes":{"xxxxx":"xxxxx","domain":"xx","id":"xxxxxx","ring":"xxx","type":"xxxx","vin":"xxxxxx"},"connectivity":{"connected":false,"disconnectReason":null,"timestamp":0},"deviceDefender":null,"shadow":"","thingGroupNames":["xxxxx","xxxxx","xxxxxx"],"thingId":"00001f4c-25f8-4d59-9906-6b0c26508c60","thingNam]

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

Response like this

{
	"nextToken": "string",
	"thingGroups": [
		{
			"attributes": {
				"string": "string"
			},
			"parentGroupNames": [
				"string"
			],
			"thingGroupDescription": "string",
			"thingGroupId": "string",
			"thingGroupName": "string"
		}
	],
	"things": [
		{
			"attributes": {
				"string": "string"
			},
			"connectivity": {
				"connected": "boolean",
				"disconnectReason": "string",
				"timestamp": "number"
			},
			"deviceDefender": "string",
			"shadow": "string",
			"thingGroupNames": [
				"string"
			],
			"thingId": "string",
			"thingName": "string",
			"thingTypeName": "string"
		}
	]
}

Current Behavior

AmazonIoTClient 58|2024-12-02T15:36:19.097Z|ERROR|InvalidRequestException making request SearchIndexRequest to https://iot.eu-central-1.amazonaws.com/. Attempt 2. --> Amazon.IoT.Model.InvalidRequestException: Invalid nextToken

Reproduction Steps

SearchIndexResponse response = await _iotClient.SearchIndexAsync(new SearchIndexRequest
{
    QueryString = queryString,
    MaxResults = maxResults,
    NextToken = nextToken
});

Example:

queryString = attributes.domain=*
maxResults = 100

_iotClient is IAmazonIoT

Possible Solution

No response

Additional Information/Context

No response

AWS .NET SDK and/or Package version used

• AWSSDK.IoT 3.7.404.2
• AWSSDK.Core 3.7.400.58

Targeted .NET Platform

.NET 8.0

Operating System and version

AWS Lambda

[dscpinheiro]

I asked in the discussion, but could you share how you're handling the pagination? If you follow this pattern I believe you should not be running into this error:

var iot = new AmazonIoTClient();
var request = new SearchIndexRequest 
{ 
    QueryString = "attributes.domain=*",
    MaxResults = 100
};

do
{
    var response = await iot.SearchIndexAsync(request);
    foreach (var item in response.Things) { ... }
    request.NextToken = response.NextToken;
} while (!string.IsNullOrEmpty(request.NextToken));

[Maverick1983]

I asked in the discussion, but could you share how you're handling the pagination? If you follow this pattern I believe you should not be running into this error:

var iot = new AmazonIoTClient();
var request = new SearchIndexRequest 
{ 
    QueryString = "attributes.domain=*",
    MaxResults = 100
};

do
{
    var response = await iot.SearchIndexAsync(request);
    foreach (var item in response.Things) { ... }
    request.NextToken = response.NextToken;
} while (!string.IsNullOrEmpty(request.NextToken));

I don't need to have a while, because API it's used from third portal. So, I need response and if nextToken exits, from web application ask pagination.

[dscpinheiro]

Is it possible whoever is invoking your API is passing a next token for a different request? The SDK does not modify the token at all, and that error message you're seeing is coming from the service.

As of now, I'm afraid we don't have enough information to help you.

[github-actions]

This issue has not received a response in 5 days. If you want to keep this issue open, please just leave a comment below and auto-close will be canceled.

[Maverick1983]

Any news?

[dscpinheiro]

I see you opened a support case (for reference, the number is 173289833900338/ V1602625419) where we asked for more details.

As mentioned in the other comments, there's not enough information on this issue at the moment, so please follow the support case so that we can help you.

[github-actions]

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.