[QUESTION] Use Cache when starting Kodi

[leverageinventions]

Every time I open Kodi it says the Elementum daemon is starting and the catalog begins loading and takes some time to finish.

Is there a way to use the cached data and not update the catalog until I do it manually?

I wasn't sure if it was Elementum or TheMovieDBHelper which updates everythin at the start of Kodi but I tried the option in Elementum to delay the start, and when I did then no Catalog is loaded whatsoever so I think it has to do with it.

Checking the log when starting Kodi these lines appear:

`
2024-05-11 17:28:14.099 T:27275 DEBUG : CDirectoryProvider[plugin://plugin.video.elementum/movies/popular]: refreshing..

2024-05-11 17:28:14.099 T:27275 DEBUG : CDirectoryProvider[plugin://plugin.video.elementum/movies/top]: refreshing..`

Thanks.

[antonsoroko]

@leverageinventions elementum does use cache :-) otherwise it would be super slow to get all your data every time, like content of all your lists, all watched movies/episodes and etc.
but every launch and by timer it also reads data from trakt (if enabled) to see if anything changed (like you added something to a list or watched something) and updates internal elementum's database and (if enabled) it also writes .strm files and then launch update of kodi's library.

so before making any changes to code, can you please do some tests?
please do them 1 by 1, not simultaneously.

1. in trakt tab, disable the setting

   plugin.video.elementum/resources/language/messages.pot

   Line 1845 in d06801d

   msgid "Enable Trakt synchronization"

   
   relaunch kodi, see if elementum launched faster in your opinion. enable the setting.
2. in library tab, disable 2 settings
   

   plugin.video.elementum/resources/language/messages.pot

   Line 1789 in d06801d

   msgid "Enable library integration with Kodi"

   
   

   plugin.video.elementum/resources/language/messages.pot

   Line 1841 in d06801d

   msgid "Enable Kodi library synchronization"

   
   relaunch kodi, see if elementum launched faster in your opinion. enable the settings.
3. disable all 3 settings, relaunch kodi, see if elementum launched faster in your opinion. enable the settings.

and then write results here.

[leverageinventions]

@antonsoroko thanks for the insights. Here are my findings:

1. For the trakt synchronization setting it didn't seem to have improved the loading times. It was on but I don't use trakt for my own lists, I don't have it even configured or have an account, I just use it to get the top/popular movies/shows lists, so maybe that's why this setting didn't do anything.
2. For the library settings I already had the library integration disabled. The library synchronization setting was enabled because I thought that disabling library integration would disable the other library synchronization processes. Disabling library synchronization seems to have helped a little bit with the start up times. However, it could be just placebo since I don't have anything in my kodi library, I just use kodi for streaming with Elementum, I don't have local media library.
3. I left all 3 setting disabled because I don't even use a trakt account or kodi library, is there another setting I should consider?

Now my question is the following, is there a way that Elementum uses the caches BEFORE it looks for changes in the lists? So that when I start Kodi the movies and shows are already populated instead of being an empty canvas?

And another question, does Elementum does the triggering for the lists updates or the triggering of the updates is done by kodi or the skin or skinshortcuts?

Thanks for your help.

[antonsoroko]

@leverageinventions

I don't have it even configured or have an account, I

then it will not do any sync in background or during startup! i really mean ANY.

from your initial message i thought that you heavily use trakt, like many of us: huge watchlist, many own lists, collection.
only those 3 things are synced!

looks like you thought that we get info about public list during kodi start:

I just use it to get the top/popular movies/shows lists|

but these lists are not synced/cached in background/startup. their content is gathered only when you open them.

---

Now my question is the following, is there a way that Elementum uses the caches BEFORE it looks for changes in the lists?
So that when I start Kodi the movies and shows are already populated instead of being an empty canvas?

both these questions are interconnected so the answer is one:

as i said, public list content is gathered only when you open them.
then we cache it for 1 week. but we use that cache only if list was not changed.
so if we already have cache then we always check if list was changed (elementum does small api request to trakt that says if list was changed or not) and then elementum either gets new content or use old content from cache
https://github.com/elgatito/elementum/blob/2c644cdb6d2908b4d2dfbe0e17c1f2fdea5ef7d5/trakt/movies.go#L420
but again, it is done in real time. not during start.

(about "always check if list was changed" - we will not change that b/c it makes perfect sense. we use quick api call to see if list was changed, this call is quick so it does not slow down anything but gives a lot of benefits, like we always give you updated list content, but if list was not changed - we also give you valid list content from cache).

And another question, does Elementum does the triggering for the lists updates or the triggering of the updates is done by kodi or the skin or skinshortcuts?

actually YOU do when you open list. read above for details.
but if you added some list from elementum to your kodi's main page - then maybe your skin will open that list, consequently elementum will open it (and maybe read data from scratch if list was changed).

---

so in the end: cache or trakt is not your issue.

maybe you just think that elementum's start time is big, but maybe for your device it is actually normal start time.
e.g. you use some not powerful single board arm device. please try on more powerful device for comparison purpose.

you should not compare elementum with tmdb helper. elementum not only show you data from trakt/tmdb sites, but also is a bittorrent engine. so it takes time to start it.

provide proper debug log file (see issue template for guide) so maybe we can see if there is something that slows down your elementum.

[antonsoroko]

JFYI: on my quite old home pc ( Intel NUC Kit BOXNUC8i5BEH2 ) elementum is readly immediately if w/o tratk+library and it is still immediately ready with trakt+lib (i mean i do not see difference).

btw

I tried the option in Elementum to delay the start,

it simply means "wait X seconds and only then start elementum". thus obviously you can't open elementum immediately.

to give you background info:
elementum consists of 2 parts (so called server-client architecture):

1. python part. it is a kodi plugin. used for UI and for communication with kodi. client in terms of server-client architecture.
2. golang part. a binary. used for communication with trakt/tmdb/other web sites and for download torrents. server.

so kodi launch python plugin, and then plugin launch binary.
when binary is ready it shows you notification. you can disable that notification if you want.
if device is slow - then you can see notification that elementum is still starting.

[leverageinventions]

Thanks for the explanation. I have been seeing a little of the code since I have been debugging this issue, mainly the navigation.py file which seems to be the one that gets and processes all the calls.

You got it exactly right when you said the following:

but if you added some list from elementum to your kodi's main page - then maybe your skin will open that list, consequently elementum will open it (and maybe read data from scratch if list was changed).

Basically I have 3 pages with 2 to 3 lists of top/popular movies/shows on Kodi's main page which are part of the interface. So, when Kodi starts the skin loads the lists as you said (automatically since they are in the main page) and shows all the posters when the data is ready, then I can just enjoy and browse and use the torrenting part.

The problem is that it takes some time to show everything, from the kodi launch to elementum checking/updating the lists. In the meanwhile I only see an empty canvas with loading indicators until Elementum gets the lists to be displayed. This process happens every time I open Kodi since it is an android tv box and not a dedicated kodi system.

I have been debugging and reducing the overhead by disabling/uninstallng things I don't use in Kodi but still there is some wait for the canvas to be populated with the posters. That's why I wanted a way to show the cached data instantly and later (when elementum was ready or manually) to update that data.

[antonsoroko]

@leverageinventions
i see. but i do not see a good way to do what you want.
if we will not update list when user opens list - then how/when we should update list content?

1. by open settings and click new button called "invalidate lists cache"? this is counter-intuitive and and extremely inconvenient.
2. or by adding menu item for list like "update list content"? this is again counter-intuitive.

i mean, waiting 1 week (or any other amount of time) before update cache is not ok. this is not what i as a user want to have, i want to have accurate info, not outdated info.

it is possible to implement an option like "never update list content cache until cache is expired or user asked for update", but personally I do not know how valuable this would be. and again, to update cache you need to implement either button in settings or menu item.

anyway, let's wait for @elgatito opinion about this.

(btw, if you know some coding - feel free to add such option :-D )

[leverageinventions]

Well, for a regular user I think the way it works right now should be the default. I was not thinking on changing that behavior but in adding it as an option, like the ones you gave me at the beginning (don't synchronize kodi library, don't synchronize trakt).

I was thinking it could work like this: an option in settings to use caches on the start of kodi and update while on playback. That's it.

About the coding stuff, I know basic coding, I am also looking to see if I could just add that but I am missing some stuff, for example I don't know how to extract the lists from the caches. Currently what I was thinking was in adding a condition in the navigation.py code so that if the url is for example "plugin://plugin.video.elementum/movies/popular" (which is one of the lists I am using) and kodi or elementum have just started then immediately send back the list in the cache.

[antonsoroko]

@leverageinventions

I was thinking it could work like this: an option in settings to use caches on the start of kodi and update while on playback. That's it.

please elaborate. i do not see a good way to do that, apart from ugly ways i described above.
what do you mean by playback? we open list, not play it.
and there is not difference between if skin opens list and if you open list. same action.

cache is in golang part, so unfortunately to make any real fix you need to change golang part and compile it.

[leverageinventions]

Well I just meant to add an option in settings to delay (or turn off) the update of the lists like there are options already to turn off tract or library synchronization. So the default behavior would continue to be as is but this option will allow other users to delay it.

By playback I just meant what this setting does:

plugin.video.elementum/resources/language/messages.pot

Line 1849 in d06801d

msgid "Enable synchronization while playback"

Got it, I don't know golang just some basic python so I cant go there, unfortunately.

Thanks for the insights, now at least I know what does what and what can and cannot be done.

[antonsoroko]

"Enable synchronization while playback"

It means that if you do not disable it then we do background synchronization even if you play video which means that on very slow device video can stutter.

like there are options already to turn off tract or library synchronization

Those options are about the background synchronization, but for public lists we don't do background synchronization, as a I already mentioned.

allow other users to delay it.

I do not see a way to delay update (because there's is nothing to delay, we do not do background update for public list), only to disable update at all and then you have to manually update every list by clicking some button/menu (like I described above).

Probably it is possible to create some hack, like if Elementum started less than 1 minute ago and user opened list then do not update this list even if list was changed on trakt side. But this dirty hack for a very specific situation.

[elgatito]

I don't see any way to speed-up loading.
With latest version Elementum is starting almost immediately and is ready to accept rendering requests.
But each request to list movies/shows/episodes is cached for at least 6 hours (probably) now. Show/Movie is cached for longer time, since there is nothing to change frequently.

Trakt sync or library sync is not affecting list rendering, unless you have a massive Kodi library and a single core on the device.

@leverageinventions You can enable debugging in Kodi and share a log after you start Kodi, so that I could see what is done and how long - https://elementum.surge.sh/contribute/log/