Retext consumes a lot of memory

[embeddedden]

I use retext for simple markdown note taking. But the issue is that it consumes around 1 gigabyte of RAM when there are around 30 tabs. Even when the preview is disabled. The same amount of tabs in Pluma consumes only around 70 MB. Is it possible to make retext consume less memory?
I use retext 7.2.2 in Ubuntu Mate 20.04.

[mitya57]

Hi! Do you use WebKit or WebEngine renderer, or none of them?

[embeddedden]

I tried with both "Use WebKit renderer" being on and off (in Edit menu).

I can see that for 18 tabs there are 18 python3 processes. Each of them consumes around 83 MB of RAM.

[mitya57]

Thank you for the extra information. Indeed we have 1 process per tab, and these processes all load Qt libraries, so they consume a lot of memory in total.

So we should either limit the number of processes (which will need a rewrite of many things) or try to reduce the memory usage by these processes.

I will try to look at this when I have more time. Cc @Griffon26 — you wrote the multiprocessing code, so any ideas are welcome.

[Griffon26]

and these processes all load Qt libraries, so they consume a lot of memory in total.

The same shared libraries loaded in multiple processes do not use up more memory for code than a single process would.
If on the other hand those libraries contain or allocate writable memory, then they do use more with each additional process.

So @embeddedden, it would be interesting to know which measurement shows the memory usage. If you look at it per process it might suggest you have to multiply it while that is not the case.

[embeddedden]

@Griffon26 I saw it in htop.

I believe all python3 processes on the screenshot are launched by retext. They consume around 600 MB of RAM according to htop.

[Griffon26]

Where do you get the 600? From SHR? The actual amount of RAM being used by a process is under RES, but parts of that memory may be shared.

To really check how much physical memory is being used by additional tabs, you should check the output of the free command before and after opening a whole bunch of tabs. In the output of free you should look for what is listed as free and add what is listed as buffers/cache (it may provide this sum for you).

I can't try this myself at the moment, @mitya57 perhaps you can also take a look.

[embeddedden]

In htop there is a line in the "header" about how much memory is consumed. So, when I start retext, it is going, for example, from 4.16 to 4.76 GB of consumed RAM. Here is the output of free -h:

Before opening:

               total        used        free      shared  buff/cache   available`

 Mem:          7,7Gi       3,7Gi       1,2Gi       107Mi       2,8Gi       3,6Gi`

Being opened:

               total        used        free      shared  buff/cache   available
 Mem:          7,7Gi       4,3Gi       609Mi       110Mi       2,8Gi       3,0Gi

[mitya57]

I just checked on my machine based on the output of free:

• When I start ReText, it consumes around 150 MB.
• Each new tab reduces the free memory by around 25-30 MB.
• I tried measuring one by one or by opening multiple tabs at a time. It doesn't matter: if I open 10 tabs at once, free memory decreases by around 300MB.
• So based on these calculations, ReText with 30 tabs consumes around 150 + 30×30 = 1050 (MB), so yes, a gigabyte, like you say.

Of these 30 MB for each process, 10 MB is probably Python interpreter itself (when I start a bare interpreter, memory decreases by that amount). The remaining 20 MB are field for optimization.