-
Notifications
You must be signed in to change notification settings - Fork 40
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: latest comfyui; fix: better GPU utilization for SD15 #270
Commits on Sep 23, 2024
-
Configuration menu - View commit details
-
Copy full SHA for daad6b5 - Browse repository at this point
Copy the full SHA daad6b5View commit details -
Despite the name, `--novram` still allows the GPU to be used. However, comfyui uses this flag to much more aggressively avoid leaving tensors in VRAM. I am hoping that this will reduce VRAM OOMs and/or shared memory usage (in windows).
Configuration menu - View commit details
-
Copy full SHA for 5dfc26b - Browse repository at this point
Copy the full SHA 5dfc26bView commit details -
refactor/fix:
unload_models_from_vram
more oftenWith some recent comfyui changes it appears that the logic prior to this commit was not aggressive enough to avoid OOMs with relying on comfyui's internal decision making alone. This commit causes the worker to unload models from VRAM immediately after an inference result (if it is not about to be used) and right before post processing. Post processing as implemented today almost always overestimates the amount of free VRAM, and tends to cause OOMs or shared memory usage (on window) so more proactively unloading the model should help minimize that problem.
Configuration menu - View commit details
-
Copy full SHA for aac8c6e - Browse repository at this point
Copy the full SHA aac8c6eView commit details -
Configuration menu - View commit details
-
Copy full SHA for c344985 - Browse repository at this point
Copy the full SHA c344985View commit details -
Configuration menu - View commit details
-
Copy full SHA for 364c7b1 - Browse repository at this point
Copy the full SHA 364c7b1View commit details -
fix: more aggressively unload from system ram
The worker seems to be holding onto too much system RAM on average. I previously relied on comfyui internals to handle this implicitly but recent changes seem to have broken some assumptions I was making. This is an purposely over-zealous attempt to keep system RAM usage down.
Configuration menu - View commit details
-
Copy full SHA for 4234412 - Browse repository at this point
Copy the full SHA 4234412View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5debcf2 - Browse repository at this point
Copy the full SHA 5debcf2View commit details -
Configuration menu - View commit details
-
Copy full SHA for c90a019 - Browse repository at this point
Copy the full SHA c90a019View commit details -
Configuration menu - View commit details
-
Copy full SHA for 73418c1 - Browse repository at this point
Copy the full SHA 73418c1View commit details -
feat: more configurable memory management
Redefines the broken existing `high_memory_mode` to leverage the recent memory management extension
Configuration menu - View commit details
-
Copy full SHA for 602a958 - Browse repository at this point
Copy the full SHA 602a958View commit details -
Configuration menu - View commit details
-
Copy full SHA for 47f08c7 - Browse repository at this point
Copy the full SHA 47f08c7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1c46af6 - Browse repository at this point
Copy the full SHA 1c46af6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9e60095 - Browse repository at this point
Copy the full SHA 9e60095View commit details -
Configuration menu - View commit details
-
Copy full SHA for 67ba52b - Browse repository at this point
Copy the full SHA 67ba52bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 55ddc03 - Browse repository at this point
Copy the full SHA 55ddc03View commit details -
Configuration menu - View commit details
-
Copy full SHA for 22202b3 - Browse repository at this point
Copy the full SHA 22202b3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6dd52aa - Browse repository at this point
Copy the full SHA 6dd52aaView commit details -
fix: print to console
PROCESS_ENDED
message's infoThis will clarify when the situations such as the shared model manager failing to load or no models being found occur (e.g., when download_models.py isn't)
Configuration menu - View commit details
-
Copy full SHA for 00b6033 - Browse repository at this point
Copy the full SHA 00b6033View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3d41fdd - Browse repository at this point
Copy the full SHA 3d41fddView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3f7a47c - Browse repository at this point
Copy the full SHA 3f7a47cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 70f7a0b - Browse repository at this point
Copy the full SHA 70f7a0bView commit details -
Configuration menu - View commit details
-
Copy full SHA for aa0ca39 - Browse repository at this point
Copy the full SHA aa0ca39View commit details -
Configuration menu - View commit details
-
Copy full SHA for e21f192 - Browse repository at this point
Copy the full SHA e21f192View commit details -
Configuration menu - View commit details
-
Copy full SHA for b0bf64b - Browse repository at this point
Copy the full SHA b0bf64bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 1ab8b07 - Browse repository at this point
Copy the full SHA 1ab8b07View commit details -
Configuration menu - View commit details
-
Copy full SHA for 48206d8 - Browse repository at this point
Copy the full SHA 48206d8View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6d74831 - Browse repository at this point
Copy the full SHA 6d74831View commit details -
Configuration menu - View commit details
-
Copy full SHA for b1f5013 - Browse repository at this point
Copy the full SHA b1f5013View commit details -
Configuration menu - View commit details
-
Copy full SHA for f1bbd5e - Browse repository at this point
Copy the full SHA f1bbd5eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 0934de5 - Browse repository at this point
Copy the full SHA 0934de5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6dfcf93 - Browse repository at this point
Copy the full SHA 6dfcf93View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0c1de03 - Browse repository at this point
Copy the full SHA 0c1de03View commit details -
Configuration menu - View commit details
-
Copy full SHA for e142601 - Browse repository at this point
Copy the full SHA e142601View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5ef71b4 - Browse repository at this point
Copy the full SHA 5ef71b4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 519f734 - Browse repository at this point
Copy the full SHA 519f734View commit details -
Configuration menu - View commit details
-
Copy full SHA for 91d6f35 - Browse repository at this point
Copy the full SHA 91d6f35View commit details -
Configuration menu - View commit details
-
Copy full SHA for ddc7bfe - Browse repository at this point
Copy the full SHA ddc7bfeView commit details -
Configuration menu - View commit details
-
Copy full SHA for a52d5a3 - Browse repository at this point
Copy the full SHA a52d5a3View commit details -
Configuration menu - View commit details
-
Copy full SHA for f5ba73c - Browse repository at this point
Copy the full SHA f5ba73cView commit details -
fix: detect more deadlocks; less crashes w/ unsresponsive logic
- More fallback logic if there are jobs popped, processes available, but nothing happening. - Resolves certain problems with the unresponsive logic - The case of it ending all jobs after a long period of "No Job" messages from the server followed by successful pops. - Now no longer shuts down in error while processes are restarting
Configuration menu - View commit details
-
Copy full SHA for 5e01bfe - Browse repository at this point
Copy the full SHA 5e01bfeView commit details -
Configuration menu - View commit details
-
Copy full SHA for d817884 - Browse repository at this point
Copy the full SHA d817884View commit details -
Configuration menu - View commit details
-
Copy full SHA for f972979 - Browse repository at this point
Copy the full SHA f972979View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2f88886 - Browse repository at this point
Copy the full SHA 2f88886View commit details -
feat: time spent w/o jobs logging
Tracks the time spent without any available jobs. This will help worker operators identify potential issues with their configuration. A warning will be logged if the worker spends more than 5 minutes without any jobs, suggesting possible actions to increase job demand.
Configuration menu - View commit details
-
Copy full SHA for 9aa79b7 - Browse repository at this point
Copy the full SHA 9aa79b7View commit details -
Configuration menu - View commit details
-
Copy full SHA for c5f1bd9 - Browse repository at this point
Copy the full SHA c5f1bd9View commit details -
Configuration menu - View commit details
-
Copy full SHA for b54376f - Browse repository at this point
Copy the full SHA b54376fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 8de3e4d - Browse repository at this point
Copy the full SHA 8de3e4dView commit details -
Configuration menu - View commit details
-
Copy full SHA for e8de582 - Browse repository at this point
Copy the full SHA e8de582View commit details -
Configuration menu - View commit details
-
Copy full SHA for 79e71b6 - Browse repository at this point
Copy the full SHA 79e71b6View commit details -
Configuration menu - View commit details
-
Copy full SHA for f89812c - Browse repository at this point
Copy the full SHA f89812cView commit details