Update internal db storage

[kremnik]

What has been done

With this PR, the performance of the in and update operations has been improved by replacing list with set in the recognition module.

How to test

make lint && make test

[serengil]

suppose that refresh_database = False, new_images and old_images are not initialized anymore in line 163 and 164. Line 195 and 196 will throw expception in that case.

IMO initializing new_images and old_images as empty sets in line 163, 164 will sort the problem

Additionally, we should have an unit test for refresh_database = False

[serengil]

Secondly, do we have any stats about the improvement? The question is that will it be worth?

TLDR:

Why use a set instead of a list?

Time Complexity:

• list: Membership checks in a list have a time complexity of O(n), where n is the length of the list. This is because the list must potentially scan through all of its elements to determine if the item is present.

• set: Membership checks in a set have a time complexity of O(1) on average. This is because sets are implemented as hash tables, which allow for much faster lookups.

[kremnik]

@serengil There are three reasons:

1. Converting set to list is time consumption, but not neccessary operation;
2. There are a few places when membership checking occurs: here and here. If old_images includes a huge amount of images (n), as well as representations includes another huge amount of images (m), total time complexity will be O(nm) if old_images is a list. Instead, if old_images is a set, total time complexity will be O(m);
3. It is guaranteed that there is no re-processing of the same images.

[serengil]

LGTM