2.10.18

📦 Feature

[Experimental] Ability to finetune single term queries exact maching

When an exact match query is performed with multiple search terms, the targeted attributes/fields are :

1. the searchable attributes/fields in their 'standard' analyzed version with their own search weight

2. the searchable attributes/fields in their 'shingle'/phrase matching version with their own search weight multiplied by the phrase match boost value

3. the searchable AND sortable attributes/fields in their 'sortable' version with their own search weight multiplied by twice the phrase match boost value

The logic behind that query construction is to

1. collect all indexed documents (products, categories) containing the individual search terms in a strictly exact or "stemmed" version (ie, not differentiating between the singular and plural form of a word - grill vs grills for instance - or between a noun/verb and its participle present form - hammer vs hammering for instance)
2. boost (possibly heavily) the documents that contain the search terms in a strictly exact or "stemmed" version close to one another
3. boost (possibly quite heavily) the documents that have an attribute/field containing strictly exactly the search terms and only the search terms in their indexed value (for instance, when a user searches for a product name without any word missing or any typo/approximation)

Now, when an exact match query is performed with a single term, there is a slight change with regards to rule 2. : the searchable attributes/fields are targeted in their 'whitespace' version, but still with their own search weight multiplied by the phrase match boost value.
Considering that the 'whitespace' version of attributes/fields does not have any stemming component, that means that exact matching for single term searches puts a huge boost on product or categories that contain exactly the word the user typed, compared to product or categories that contain only the "stemmed" version of that word.

This is not an issue in most cases, but you could find yourself in a situation where you would like that this emphasis on strictly exact matching be a bit more relaxed, particularly in the context of matching categories in the autocomplete.
For instance, let's imagine you're selling grills, grill tools, grill accessories. You might want that when a user searches either for "grill" or "grills" in the autocomplete, the same list of categories appear in the same exact order: "Grills", "Grill Tools", "Grill Accessories".

It is now possible to achieve that behavior with new experimental settings located in "Elasticsuite > Search Relevance > Relevance Configuration > Exact Matching Configuration" :

• "[Experimental] Enable single term custom boost values" : Yes/No, enables customizing the boost value for rules 2. and 3. when a single term is searched. Defaults to No.
• "[Experimental] Single term phrase match boost value" : the boost value that will replace the phrase match boost value applied to 'whitespace' version of attributes/fields. Defaults to 10. It's possible to set it to 0 to totally disable the rule 2.
• "[Experimental] Single term sortable matches boost value" : the boost value that will replace "twice the phrase match boost value" applied to 'sortable' version of attributes/fields. Defaults to 20. It's possible to set it to 0 to totally disable the rule 3.

Those three new settings can be changed only for a specific scope, ie a specific search query type on a given store view.
You are thus allowed to change the native behavior for categories results in the autocomplete box without affecting the products search.

What's Changed

• Experimental single term exact matching relevance by @rbayet in #3050
• Only Add Frontend Tracker Blocks If Config Enabled by @pykettk in #3051

Full Changelog: 2.10.17.3...2.10.18

2.11.3.3

What's Changed

• Fixes #3036 Span queries must not bypass exact query restrictions by @rbayet in #3037
• Feat indices list detail minor enhancements by @rbayet in #3041
• Fix #3030, avoid deprecated access to an undefined property by @vahonc in #3032
• Fixes #3030 deprecated access to an undefined property (2.10.x) by @rbayet in #3042
• Fixes #3026 Cannot use default value of facet_sort_order in category … by @Nuranto in #3043

Full Changelog: 2.11.3.2...2.11.3.3

2.10.17.3

What's Changed

• Fixes #3036 Span queries must not bypass exact query restrictions by @rbayet in #3037
• Feat indices list detail minor enhancements by @rbayet in #3041
• Fixes #3030 deprecated access to an undefined property (2.10.x) by @rbayet in #3042

Full Changelog: 2.10.17.2...2.10.17.3

2.11.3.2

What's Changed

• Update return statement for ExportProductAttributeCsv controller by @vahonc in #3004
• Fix #3005, remove unnecessary parameter of column from db_schema.xml by @vahonc in #3007
• Fix #1621 indexing performances by @romainruaud in #2982
• Backport synonym fix from Gally and better check on self-included syn… by @romainruaud in #3010

Full Changelog: 2.11.3.1...2.11.3.2

2.10.17.2

What's Changed

• Update return statement for ExportProductAttributeCsv controller by @vahonc in #3004
• Fix #1621 indexing performances by @romainruaud in #2982
• Backport synonym fix from Gally and better check on self-included syn… by @romainruaud in #3010

Full Changelog: 2.10.17.1...2.10.17.2

2.11.3.1

What's Changed

• Prevent edge_ngram analyzer being used at query time by @rbayet in #3000
• Disable magento outOfStockSorting plugin globally by @PierreGauthier in #2999

Full Changelog: 2.11.3...2.11.3.1

2.10.17.1

What's Changed

• Prevent edge_ngram analyzer being used at query time by @rbayet in #3000

Full Changelog: 2.10.17...2.10.17.1

2.11.3

📦 Features

Edge-ngram analyzer

In this release we introduced the ability to defined edge_ngram analyzer to particular fields.

This analyzer can be defined per each attribute in Stores > Configuration > Attributes > Storefront properties > Advanced Elasticsuite Configuration and then switching the default analyzer to standard_edge_ngram.

This will require a full reindex (should be triggered automatically as soon as you save the attribute).

The idea is to be able to generate "ngrams" for this attribute to be able to search on them.

Eg : if an attribute has "Organic Cotton" as a value, the generated ngrams will be "org, orga, organ, organi, organic, cot, cott, cotto, cotton".

Therefore, searching for "org" or "cot" in Front-office will allow the products having the value "Organic cotton" to be shown.

By default, this is only used when executing a fuzzy search, if you want to use this also for exact matching, you can enable the new parameter that is in Elasticsuite > Search Relevance > Exact matching configuration > Use default analyzer in exact matching filter query.

Please keep in mind that this feature is not meant to be enabled on all attributes, and that it's not recommended to enable it on long text fields (description, etc...) because of the noise and the performance issues it can generates with such fields.

Min score

In this release we also introduced the ability to add the support of min_score. By going into Elasticsuite > Search Relevance > Relevance Configuration > Min score configuration you'll be able to set a minimum score that products must have to be displayed in search result page.

That's useful if you have a very long tail of not-that-much relevant products, because they might appear in first positions when the user is sorting by price or by name in the result page.

Experimental settings for SKU/reference matching

All searchable fields using the reference analyzer are now copied into a new reference collector field in the index (this will require a full re-index to take effect).

This can be combined with three new experimental settings (disabled by default) that should help provide, depending on your references structure, better matching and recall of product references.

The new section Elasticsuite > Search Relevance > Spellchecking Configuration > Term Vectors Configuration provides two new settings

• Use all tokens from term vectors, which, when enabled, makes sure that when searching a product reference like 'AN328CZ127' Elasticsuite no longer only search for the homogeneous alphanumeric components 'AN', '328', 'CZ' and '127' but also the whole original string 'AN328CZ127'. This is particularly useful if the actual product reference is for instance 'AN-328-CZ/127': this will allow a fuzzy match with better accuracy ;
• Use reference analyzer in term vectors, which, when enabled, makes sure that the specific way the fields using the reference analyzer are indexed is taken into account when Elasticsuite determines if what the user searches exists exactly, almost exactly, or in a fuzzy matching way in the index. Combined with the previous setting, it should blur the lines between searching for 'AN-328-CZ/127' and 'AN328CZ127' for instance, in both case making Elasticsuite perform an exact match query with pretty much the same results set.

The new section Elasticsuite > Search Relevance > Relevance Configuration > Exact Match Configuration provides the new setting

• Use reference analyzer in exact matching filter query, which, when enabled, makes sure that all searchable fields using the reference analyzer (and not just the sku field) are looked into when performing an exact match query. This should solve some strange issues where searching exactly for a product reference stored in a custom field item_number or manufacturer_part_number leads to 0 results.

Additionally any change to the relevance settings that should not require a full reindex to take effect (this includes the three settings above) is now immediately taken into account by Elasticsuite pre-analysis mechanism, helping you finetune the search engine faster (we still recommend to do that on a staging environment, of course 😉).

What's Changed

• Fix #2965, default value for the property 'display_pattern' of EAV attributes by @vahonc in #2970
• Fix is spellchecked graphql by @romainruaud in #2981
• Fixes #2987 Check if category is virtual when evaluating useVirtualCa… by @rbayet in #2988
• Feature #2917, add min_score support for matching documents by @vahonc in #2986
• Inject all fields with their default analyzer in fuzzy queries. by @romainruaud in #2991
• [Spellcheck] Experimental settings for better sku/reference token ext… by @rbayet in #2992
• Fix #2978, update db_schema.xml by @vahonc in #2993
• Target non-default analyzer fields in exact search. by @romainruaud in #2995
• Adding min_score i18n by @romainruaud in #2996
• [Spellchecker] Cleaning spelling type cache when relevance config is … by @rbayet in #2997

Full Changelog: 2.11.2...2.11.3

2.10.17

📦 Features

Edge-ngram analyzer

In this release we introduced the ability to defined edge_ngram analyzer to particular fields.

This analyzer can be defined per each attribute in Stores > Configuration > Attributes > Storefront properties > Advanced Elasticsuite Configuration and then switching the default analyzer to standard_edge_ngram.

This will require a full reindex (should be triggered automatically as soon as you save the attribute).

The idea is to be able to generate "ngrams" for this attribute to be able to search on them.

Eg : if an attribute has "Organic Cotton" as a value, the generated ngrams will be "org, orga, organ, organi, organic, cot, cott, cotto, cotton".

Therefore, searching for "org" or "cot" in Front-office will allow the products having the value "Organic cotton" to be shown.

By default, this is only used when executing a fuzzy search, if you want to use this also for exact matching, you can enable the new parameter that is in Elasticsuite > Search Relevance > Exact matching configuration > Use default analyzer in exact matching filter query.

Please keep in mind that this feature is not meant to be enabled on all attributes, and that it's not recommended to enable it on long text fields (description, etc...) because of the noise and the performance issues it can generates with such fields.

Min score

In this release we also introduced the ability to add the support of min_score. By going into Elasticsuite > Search Relevance > Relevance Configuration > Min score configuration you'll be able to set a minimum score that products must have to be displayed in search result page.

That's useful if you have a very long tail of not-that-much relevant products, because they might appear in first positions when the user is sorting by price or by name in the result page.

Experimental settings for SKU/reference matching

All searchable fields using the reference analyzer are now copied into a new reference collector field in the index (this will require a full re-index to take effect).

This can be combined with three new experimental settings (disabled by default) that should help provide, depending on your references structure, better matching and recall of product references.

The new section Elasticsuite > Search Relevance > Spellchecking Configuration > Term Vectors Configuration provides two new settings

• Use all tokens from term vectors, which, when enabled, makes sure that when searching a product reference like 'AN328CZ127' Elasticsuite no longer only search for the homogeneous alphanumeric components 'AN', '328', 'CZ' and '127' but also the whole original string 'AN328CZ127'. This is particularly useful if the actual product reference is for instance 'AN-328-CZ/127': this will allow a fuzzy match with better accuracy ;
• Use reference analyzer in term vectors, which, when enabled, makes sure that the specific way the fields using the reference analyzer are indexed is taken into account when Elasticsuite determines if what the user searches exists exactly, almost exactly, or in a fuzzy matching way in the index. Combined with the previous setting, it should blur the lines between searching for 'AN-328-CZ/127' and 'AN328CZ127' for instance, in both case making Elasticsuite perform an exact match query with pretty much the same results set.

The new section Elasticsuite > Search Relevance > Relevance Configuration > Exact Match Configuration provides the new setting

• Use reference analyzer in exact matching filter query, which, when enabled, makes sure that all searchable fields using the reference analyzer (and not just the sku field) are looked into when performing an exact match query. This should solve some strange issues where searching exactly for a product reference stored in a custom field item_number or manufacturer_part_number leads to 0 results.

Additionally any change to the relevance settings that should not require a full reindex to take effect (this includes the three settings above) is now immediately taken into account by Elasticsuite pre-analysis mechanism, helping you finetune the search engine faster (we still recommend to do that on a staging environment, of course 😉).

Fixes

• Fix #2965, default value for the property 'display_pattern' of EAV attributes by @vahonc in #2970
• Fix is spellchecked graphql by @romainruaud in #2981
• Fixes #2987 Check if category is virtual when evaluating useVirtualCa… by @rbayet in #2988
• Feature #2917, add min_score support for matching documents by @vahonc in #2986
• Inject all fields with their default analyzer in fuzzy queries. by @romainruaud in #2991
• [Spellcheck] Experimental settings for better sku/reference token ext… by @rbayet in #2992
• Target non-default analyzer fields in exact search. by @romainruaud in #2995
• Adding min_score i18n by @romainruaud in #2996
• [Spellchecker] Cleaning spelling type cache when relevance config is … by @rbayet in #2997

Full Changelog: 2.10.16...2.10.17

2.11.2

📦 Features

In this release we introduced the ability to export and import attributes configurations (by @vahonc in #2937)

You will now be able to proceed with exporting the attributes configuration fields related to elasticsuite : "is searchable", "is filterable", "search weight", etc... from the Stores > Attributes > Products grid.

Then you can proceed to the modifications you need, and you can re-import your file through System > Import > choose 'Elasticsuite Product Attributes' and then re-import your file.

This is intended to help the merchants by saving a lot of time at the start of a project, to configure everything in the exported file "offline", and then reimport it once it's ready. This is way more easy than going through all the attributes one by one.

What's Changed

• Fix #1985 Ensure proper loading of virtual categories rules. by @romainruaud in #2931
• Fix #1985 Ensure proper loading of virtual categories rules. by @romainruaud in #2933
• Prevent failures when using empty values in multiselect. by @romainruaud in #2938
• Fix #2934 by renaming test file by @romainruaud in #2939
• Feature #2911, allow to export/import attributes configuration from admin by @vahonc in #2937
• Fix filter rendering by @Nuranto in #2942

Full Changelog: 2.11.1.1...2.11.2