Some doubts about the index

[imperugo]

Hi,
I'm writing this issue not in order to open a bug, but to share some possible scenarios related the index creation into MongoDb.

Looking the code it seems you are creating the indexes using the specific method that comes out of the box with the c# driver.
That's pretty good in commons scenarios but, if you have a sharded cluster probably the best way to manage the indexes into the MongoDb Ops Manager and not via code.

Let's think the scenario where you are running your application but you have to scale up your db, create a new sharded instance, new replicaset, disaster recovery, backup or whatever you need.

MongoDB backups automatically include any indexes you create using Ops Manager.

The ops manager is able to create all the indexes in a new instance easily, doing this via c# is more complicated because you have to stop & run again the application.
Moreover you are using unique index in some case and, in a sharded environment, the unique index works only if you include the shard key into the index.

MongoDB does not support unique indexes across shards, except when the unique index contains the full shard key as a prefix of the index. In these situations MongoDB will enforce uniqueness across the full key, not a single field.

Of course I'm not sure about everything I wrote in this post :)

[timbussmann]

Hi @imperugo

Thanks for the inputs, we haven't looked into the MongoDB Ops Manager so far. What would be the proper way to let the Ops Manager create the indexes? Do those have to be created manually in the Ops Manager?

Moreover you are using unique index in some case and, in a sharded environment, the unique index works only if you include the shard key into the index.

the unique index is required to avoid duplicate data created due to concurrent writes. But based on the documentation that means that this will affect the possible sharding strategies available if you want to shard the saga data (as you couldn't use hashed key partitioning if I understand correctly). It seems we'd have to disable the unique constraint on the object id because sharding wouldn't work with two separate unique indexes?

Are those issues you're currently being affected by?

[imperugo]

Hi @timbussmann
thanks for the quick reply. Right now I've no issues because I'm using it in a no-sharded collection, but the idea is to shard it as soon as possible.

Unfortunately I've no experience with Ops Manager (we have people that manage it, not myself). From my point of I think would be nice to have a flag that disable the index creation.
If I set it as true, it mean I want the responsibility to handle the indexes manually.

Does it make sense to you?

[timbussmann]

@imperugo good to hear that there is no urgency in this. We'd be happy to learn more about how we can improve the mongo persistence support for sharded environments 👍

Would disabling index creation help in that scenario? What happens if index creation is not disabled, would that result in duplicate indexes? If I understood you correctly the issue would be that indexes on shards added at runtime wouldn't be automatically created, so disabling indexes wouldn't really help?

[tekmaven]

@imperugo @timbussmann In MongoDB, if the index already exists, then the call to create the index will be a no-op. If you pre-create the index as a part of deployment (or ahead of deployment), then you should be good to go!