Custom key deserialiser registered for String.class is ignored when Map key type not defined

[devdanylo]

Search before asking

• I searched in the issues and found nothing similar.

Describe the bug

I'm working on a piece of software which accepts arbitrary JSON, flattens it, and then writes it to CSV. The main issue here is that we don't know the schema of the JSON—it can be anything. This, in turn, results in untyped deserialization.

Before starting processing of the parsed object, the field names must be sanitised (yes, it's crucial to sanitise the field names during the parsing phase). Unfortunately UntypedObjectDeserializerNR ignores the custom key deserialiser I'm trying to provide:

package libraries;

import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.KeyDeserializer;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.module.SimpleModule;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;

import java.util.Map;

public class ObjectMapperTest {

    @Test
    void customKeyDeserializerShouldBeUsedWhenTypeNotDefined() throws Exception {
        // given
        var json = """
                {
                    "name*": "Erik",
                    "address*": {
                        "city*": {
                            "id*": 1,
                            "name*": "Berlin"
                        },
                        "street*": "Elvirastr"
                    }
                }
                                
                """;

        var objectMapper = new ObjectMapper();
        var keySanitizationModule = new SimpleModule("key-sanitization");
        keySanitizationModule.addKeyDeserializer(String.class, new KeyDeserializer() {
            @Override
            public Object deserializeKey(String key, DeserializationContext ctxt) {
                return key.replace("*", "_");
            }
        });
        objectMapper.registerModule(keySanitizationModule);

        // when
        var result = objectMapper.readValue(json, Object.class);

        // then
        Assertions.assertEquals("Erik", ((Map<String, Object>) result).get("name_"));

        var addressMap = (Map<String, Object>) ((Map<String, Object>) result).get("address_");
        Assertions.assertEquals("Elvirastr", addressMap.get("street_"));

        var cityMap = (Map<String, Object>) addressMap.get("city_");
        Assertions.assertEquals(1, cityMap.get("id_"));
        Assertions.assertEquals("Berlin", cityMap.get("name_"));
    }
}

I have 2 questions in regards of this behaviour:

1. Is that something expected?
2. Is there a workaround for the problem in question?

Version Information

2.14.3

Reproduction

No response

Expected behavior

No response

Additional context

No response

[JooHyukKim]

Have you tried mapper.readValue(json, new TypeReference<Map<String, Object>>(){})?

[devdanylo]

@JooHyukKim in such case for the root level object the custom key deserialiser is going to work, but for the nested objects it's still going to be ignored.

[cowtowncoder]

I think the problem is that unless you can provide tight type definition for nested Maps, key deserializer is looked up for type Object and not String. Whether this is a bug or not in Jackson is debatable (it can be definitely be unexpected); not 100% sure if changing it would be safe (esp. for backwards-compatibility).

But I think you can register same KeyDeserializer for Object.class too. Either by adding another registration call for SimpleModule, or implementing KeyDeserializers callback which could return same instance for different JavaTypes.
Second approach could be helpful for debugging as well.

If registering of key deserializer for Object.class doesn't work, that's a bug.

[JooHyukKim]

@JooHyukKim in such case for the root level object the custom key deserialiser is going to work, but for the nested objects it's still going to be ignored.

I see, so the input could be nested. Then what @cowtowncoder said above would suffice.

[devdanylo]

If registering of key deserializer for Object.class doesn't work, that's a bug.

@cowtowncoder I've tried keySanitizationModule.addKeyDeserializer(Object.class, ...); and the result is the same.

FYI: if you look at the implementation of UntypedObjectDeserializerNR, you will find out that it doesn't use any external key deserializers.

[JooHyukKim]

@devdanylo is right. on depth 2 it no longer works.

    @Test
    void customKeyDeserializerShouldBeUsedWhenTypeNotDefined() throws Exception {
        // GIVEN
        var json = """
                {
                    "name*": "Erik",
                    "address*": {
                        "city*": {
                            "id*": 1,
                            "name*": "Berlin"
                        },
                        "street*": "Elvirastr"
                    }
                }
                                
                """;

        var keySanitizationModule = new SimpleModule("key-sanitization");
        keySanitizationModule.addKeyDeserializer(String.class, new KeyDeserializer() {
            @Override
            public String deserializeKey(String key, DeserializationContext ctxt) {
                return key.replace("*", "_");
            }
        });

        keySanitizationModule.addKeyDeserializer(Object.class, new KeyDeserializer() {
            @Override
            public Object deserializeKey(String key, DeserializationContext ctxt) {
                throw new RuntimeException("Should be called, but never called");
            }
        });

        var mapper = JsonMapper.builder().addModule(keySanitizationModule).build();

        // WHEN
        var result = mapper.readValue(json, new TypeReference<Map<String, Object>>() { });

        // THEN
        // depth 1 works as expected
        Assertions.assertEquals("Erik", result.get("name_"));

        // depth 2 does NOT work as expected
        var addressMap = (Map<String, Object>) result.get("address_");
        // null?? Fails here
        Assertions.assertEquals("Elvirastr", addressMap.get("street_"));
        var cityMap = (Map<String, Object>) addressMap.get("city_");
        Assertions.assertEquals(1, cityMap.get("id_"));
        Assertions.assertEquals("Berlin", cityMap.get("name_"));
    }

[cowtowncoder]

Ok. So it sounds like we have a bug here. I suspect it might be because nested Maps will probably be deserialized using "UntypedObjectDeserializer" (being java.lang.Object). And maybe that does not try to fetch deserializer correctly wrt custom configuration.

[JooHyukKim]

hmmmm, I tried to come up with a fix, but failing.
The deserialization is being directed to UntypedObjectDeserializerNR and _deserializeNR() is invoked, but I don't think I am correctly understanding the mechanism of deserializing nested (therefore recursive) JSON objects as Object.class.

Any pointers u might want to share here?

[cowtowncoder]

Ah .NR meaning "Non-recursive". There should be reference to (optional) custom Map deserializer, which would be thing created with custom key deserializer. But I don't immediately recall how locating that works.

[JooHyukKim]

I just realized this issue may no longer be about String.class but Object.class for key deseriailzation.