Handle HighEntropyValues as evidence in on-premise solution

[vpenyazkov]

I'm working on on-premise solution and I see that your code example does not handle HighEntropyValues similar to what cloud solution provides (e.g. 51D_GetHighEntropyValues in request body).
Am I missing something? Or, if this should be handle manually, what evidences should I set from the value of getHighEntropyValues method?

[kirstin51D]

Hi! We haven't implemented HighEntropyValues in Node yet. Node would require an additional flow element to process the 51D_GHEV value at the start of the pipeline. Here's a link to the .NET version: https://github.com/51Degrees/device-detection-dotnet/tree/main/FiftyOne.DeviceDetection/FiftyOne.DeviceDetection/Uach

So at the moment, this is a feature request (I'll change the label on this issue to reflect this). Alternatively, anyone can make PRs to add this feature to Node should they wish. We will update this issue with more information in due time, but we aren't able to look at adding this feature just yet. Are you able to successfully use the cloud solution in the meantime?

[vpenyazkov]

Hi,
thanks for the response. Cloud solution works, indeed, but for our needs we need a self-hosted solution, that's why I'm asking.
Can you tell me which on-premise SDKs are supported the most? Is .NET SDK is the primary one and the rest are updated a bit later? or perhaps python SDK is also up-to-date?

[kirstin51D]

Hi! Thanks for clarification. To answer your questions:
1 - For which on-prem SDK's are most supported, it's not really applicable as we generally don't have a preference.
2 - In this instance, we have updated HighEntropyValues in .NET first, but for other features, other languages may get priority. Since our code is open source, we do welcome community contributions. So if you wish to add a feature to a specific language SDK, we can review the pull requests. If you wanted to add HighEntropyValues to Node, you can use the .NET code as a base to convert to Node.

Are you able to use .NET for your self-hosted solution?

[vpenyazkov]

Thanks for the clarifications.

We don't have .NET expertise in our company, so other SDKs are more preferred (e.g. node.js/python).
We will consider our chances to implement HighEntropyValues support in Node.js SDK and will submit a PR in case we succeed there.

[kirstin51D]

Thanks for the quick response. Do let us know how you get on!

If you are unable to submit a PR, let us know. Unfortunately, this feature is not part of our immediate scope, so it could take some time before we are able to look at it internally. If you require the feature sooner, we can put you in touch with an external organization who can help out for a fee.

[vpenyazkov]

Hi,
I'm currently looking into the implementation of javascriptgethighentropyvalues and I face an issue -- javascriptgethighentropyvalues is not returned by the engine. I thought what's missing is converting of HighEntropyValues data (returned by getHighEntryValues method) into evidences. But I see that engine ignores javascriptgethighentropyvalues property completely.
Even when I pass it to restrictedProperties param -- the javascriptgethighentropyvalues is never returned in the flowData result.

[Sara51D]

Hi @vpenyazkov
Thank you for your question. A flow element needs to be created in Node to handle the GetHighEntropyValues data. An example of how to create a flow element can be found here.

The GetHighEntropyValues data needs to be converted before device detection takes place. We've already done this in .Net using a flow element called UACH Converter; this needs to be ported for Node: please see the link here

I hope this helps!

[vpenyazkov]

hi @Sara51D ,

as I can see, the Flow Element handles the conversion from GetHighEntropyValues data into evidences. That's understandable.
But how to retrieve a javascriptgethighentropyvalues property from engine, which should provide a script to be run on client side to gather that data?

The example of the response from the cloud solution showing the properties which I mean:

{
  "device": {
    ...
    "javascriptgethighentropyvalues": "if(navigator.userAgentData){navigator.userAgentData.getHighEntropyValues([\"model\",\"platform\",\"platformVersion\",\"fullVersionList\"]).then(t=>{document.cookie=`51D_GetHighEntropyValues=${btoa(JSON.stringify(t))}`\r\n// 51D replace this comment with callback function.\r\n})} else { // 51D replace this comment with callback function.\r\n}"
  },
  "javascriptProperties": [
    ...
    "device.javascriptgethighentropyvalues"
  ]
}

[kirstin51D]

Hi @vpenyazkov. Thanks for providing this extra information! Can I confirm which on-premise data file are you using? If you are able to provide both the file name and the date it was published that would be helpful.

Another thing to try might be to passing an empty restrictedProperties parameter and see what is returned. If everything is returned, then it could be an issue with the API. If nothing is returned, then it could be a data problem. Knowing the result from the empty restrictedProperties param would help us investigate the root cause.
Thanks,
Kirstin

[vpenyazkov]

Hi @kirstin51D,

I'm currently using Enterprise-HashV41.hash as of date Mon, 24 Jul 2023 11:27:32 GMT.

When I provide empty restrictedProperties e.g. restrictedProperties: [] then the empty result is returned:

{
  "device": {},
  "javascriptProperties": [],
}

[Sara51D]

Hi @vpenyazkov

Apologies for not being clearer. Could you please try passing a null restricted Properties parameter and see what is returned?

Thanks,
Sarandha

[vpenyazkov]

hi @Sara51D,

In the case restrictedProperties: null all data is returned. The data file is the same as mentioned above
Here is a list of js properties returned:

 {
 "device": { ...all properties... },
 "javascriptProperties": [
    "device.javascriptbandwidth",
    "device.javascriptimageoptimiser",
    "device.screenpixelsheightjavascript",
    "device.screenpixelswidthjavascript",
    "device.pixelratiojavascript",
  ],
}

[Sara51D]

Hi @vpenyazkov
We looked into this further and have identified what is causing this issue. Our data team are yet to release the property to the Enterprise-HashV41.hash file. They will be making the change today to include these in the data files to be released next week.
Our apologies for not highlighting this earlier, and thank you for providing this detailed information that helped us identify the issue.

[kirstin51D]

Hi @vpenyazkov! I just wanted to let you know that GetHighEntropyValues has now been added to the V4 Enterprise file and the data file is now available. Hopefully this resolves the issue you were facing regarding the property not being returned by the engine.

Do let us know if you encounter any other issues! Thanks.

[vpenyazkov]

hi @kirstin51D,

I can confirm that it's returned now, thanks

[kirstin51D]

Hi @vpenyazkov! Glad to hear that GetHighEntropyValues is returned. How have you been able to progress on raising a PR as mentioned above? Have you encountered any other issues or are we able to close this GitHub issue? Thanks!

[vpenyazkov]

Hi @kirstin51D. we implemented that on our side, so we are not going to submit a PR into this repo

[justadreamer]

this feature will be implemented as part of the C library, thus moved the issue here.