Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

memory leak using SVR #1869

Open
montagne5641 opened this issue Jun 15, 2024 · 6 comments
Open

memory leak using SVR #1869

montagne5641 opened this issue Jun 15, 2024 · 6 comments
Labels
bug Something isn't working

Comments

@montagne5641
Copy link

When using SVR with patch, a memory leak occurs every time an assignment is made repeatedly with for.
This does not occur if patch is not used.
Similar issues have been reported previously with SVC, and daal4py was suggested as an alternative at the time. However, SVR does not support daal4py.
Is it possible to address the memory leak?

#-----------------------------------------------------
use_intelex =1

if use_intelex==1:
from sklearnex import patch_sklearn
patch_sklearn()

from sklearn.svm import SVR
import numpy as np

X = np.random.randint(10, size=(1000,500))
y = np.random.randint(2, size=(1000))

for i in range(10000):
results = SVR().fit(X,y)
#-----------------------------------------------------

@montagne5641 montagne5641 added the bug Something isn't working label Jun 15, 2024
@montagne5641
Copy link
Author

This issue was not resolved by using del and go.collect()....
I think this problem becomes even more pronounced when used in combination with gridsearchcv etc.


import gc

use_intelex =1

if use_intelex==1:
from sklearnex import patch_sklearn
patch_sklearn()

import os
from sklearn.svm import SVR
import numpy as np

X = np.random.randint(10, size=(1000,500))
y = np.random.randint(2, size=(1000))

for i in range(10000):
results = SVR().fit(X,y)
del results
gc.collect()

@samir-nasibli
Copy link
Contributor

Thank you @montagne5641 for the report!
In order to reproduce the issues, please share the version of daal4py and scikit-learn-intelex and more details about your env

@montagne5641
Copy link
Author

montagne5641 commented Jun 20, 2024

dear samir-nasibli
Thank you. This is my environment. If you need any other information, I'll be happy to help.

daal4py==2024.4.0
scikit-learn-intelex==2024.4.0
scikit-learn==1.5.0
Windows 10 Pro

Note: The same issue occurred even when I lowered the version as shown below.
daal4py-2023.1.1
scikit-learn-intelex-2023.1.1

@montagne5641
Copy link
Author

Any update on this?

@jamescporter
Copy link

Same here, any updates?

@jamescporter
Copy link

This issue was not resolved by using del and go.collect().... I think this problem becomes even more pronounced when used in combination with gridsearchcv etc.

import gc

use_intelex =1

if use_intelex==1: from sklearnex import patch_sklearn patch_sklearn()

import os from sklearn.svm import SVR import numpy as np

X = np.random.randint(10, size=(1000,500)) y = np.random.randint(2, size=(1000))

for i in range(10000):

results = SVR().fit(X,y)
del results
gc.collect()

did you find that you would always run out of memory doing a grid search using this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants