Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No module named 'diffusers.pipelines.controlnet_xs' #2110

Open
3 tasks done
Winnie202 opened this issue Dec 28, 2023 · 9 comments
Open
3 tasks done

No module named 'diffusers.pipelines.controlnet_xs' #2110

Winnie202 opened this issue Dec 28, 2023 · 9 comments
Assignees
Labels
kind/bug something isn't working

Comments

@Winnie202
Copy link

Prerequisite

Task

I'm using the official example scripts/configs for the officially supported tasks/models/datasets.

Branch

main branch https://github.com/open-mmlab/mmagic

Environment

Python: 3.8.18 (default, Sep 11 2023, 13:40:15) [GCC 11.2.0]
CUDA available: True
numpy_random_seed: 2147483648
GPU 0: NVIDIA TITAN X (Pascal)
CUDA_HOME: /usr/local/cuda-10.2
NVCC: Cuda compilation tools, release 10.2, V10.2.8
GCC: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
PyTorch: 1.12.1+cu102
PyTorch compiling details: PyTorch built with:

  • GCC 7.3
  • C++ Version: 201402
  • Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
  • Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
  • OpenMP 201511 (a.k.a. OpenMP 4.5)
  • LAPACK is enabled (usually provided by MKL)
  • NNPACK is enabled
  • CPU capability usage: AVX2
  • CUDA Runtime 10.2
  • NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70
  • CuDNN 7.6.5
  • Magma 2.5.2
  • Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=10.2, CUDNN_VERSION=7.6.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.12.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF,

TorchVision: 0.13.1+cu102
OpenCV: 4.8.1
MMEngine: 0.8.4
MMCV: 2.0.1
MMCV Compiler: GCC 7.3
MMCV CUDA Compiler: 10.2
MMagic: 1.2.0dev0+1bdd952

Reproduces the problem - code sample

from mmagic.apis import MMagicInferencer

Reproduces the problem - command or script

python demo/mmagic_inference_demo.py

Reproduces the problem - error message

ie_xiong/SR2/mmagic/yfolder/mmagic_inference_demo.py
/media/veily/work/envs/openmmlab/lib/python3.8/site-packages/controlnet_aux/segment_anything/modeling/tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_5m_224 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_5m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
/media/veily/work/envs/openmmlab/lib/python3.8/site-packages/controlnet_aux/segment_anything/modeling/tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_11m_224 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_11m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
/media/veily/work/envs/openmmlab/lib/python3.8/site-packages/controlnet_aux/segment_anything/modeling/tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_21m_224 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_21m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
/media/veily/work/envs/openmmlab/lib/python3.8/site-packages/controlnet_aux/segment_anything/modeling/tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_21m_384 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_21m_384. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
/media/veily/work/envs/openmmlab/lib/python3.8/site-packages/controlnet_aux/segment_anything/modeling/tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_21m_512 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_21m_512. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
Traceback (most recent call last):
File "/media/veily/work/envs/openmmlab/lib/python3.8/site-packages/diffusers/utils/import_utils.py", line 710, in _get_module
return importlib.import_module("." + module_name, self.name)
File "/media/veily/work/envs/openmmlab/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'diffusers.pipelines.controlnet_xs'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/media/veily/other/winnie_xiong/SR2/mmagic/yfolder/mmagic_inference_demo.py", line 5, in
from mmagic.apis import MMagicInferencer
File "/media/veily/other/winnie_xiong/SR2/mmagic/mmagic/apis/init.py", line 2, in
from .inferencers.inference_functions import init_model
File "/media/veily/other/winnie_xiong/SR2/mmagic/mmagic/apis/inferencers/init.py", line 16, in
from .translation_inferencer import TranslationInferencer
File "/media/veily/other/winnie_xiong/SR2/mmagic/mmagic/apis/inferencers/translation_inferencer.py", line 12, in
from mmagic.models.base_models import BaseTranslationModel
File "/media/veily/other/winnie_xiong/SR2/mmagic/mmagic/models/init.py", line 6, in
from .editors import * # noqa: F401, F403
File "/media/veily/other/winnie_xiong/SR2/mmagic/mmagic/models/editors/init.py", line 2, in
from .animatediff import AnimateDiff, UNet3DConditionMotionModel
File "/media/veily/other/winnie_xiong/SR2/mmagic/mmagic/models/editors/animatediff/init.py", line 2, in
from .animatediff import AnimateDiff
File "/media/veily/other/winnie_xiong/SR2/mmagic/mmagic/models/editors/animatediff/animatediff.py", line 15, in
from mmagic.models.archs import TokenizerWrapper, set_lora
File "/media/veily/other/winnie_xiong/SR2/mmagic/mmagic/models/archs/init.py", line 80, in
register_diffusers_models()
File "/media/veily/other/winnie_xiong/SR2/mmagic/mmagic/models/archs/init.py", line 69, in register_diffusers_models
pipeline = getattr(diffusers.pipelines, pipeline_name)
File "/media/veily/work/envs/openmmlab/lib/python3.8/site-packages/diffusers/utils/import_utils.py", line 698, in getattr
value = self._get_module(name)
File "/media/veily/work/envs/openmmlab/lib/python3.8/site-packages/diffusers/utils/import_utils.py", line 712, in _get_module
raise RuntimeError(
RuntimeError: Failed to import diffusers.pipelines.controlnet_xs because of the following error (look up to see its traceback):
No module named 'diffusers.pipelines.controlnet_xs'

Additional information

No response

@Winnie202 Winnie202 added the kind/bug something isn't working label Dec 28, 2023
@ghost
Copy link

ghost commented Dec 29, 2023

Hello, I also encountered this problem just now, I reduced the diffusers version to 0.24.0 and this error did not occur.

@ghost
Copy link

ghost commented Dec 29, 2023

pip install diffusers==0.24.0

@carlomigliorini
Copy link

carlomigliorini commented Jan 8, 2024

I had the same problem, I installed diffusers==0.24.0 and the problem has been solved, but now I get a new error:
ModuleNotFoundError: No module named 'mmcv._ext'
The mmcv version is 2.0.1.
Any idea please ?

@leozjr
Copy link

leozjr commented Feb 24, 2024

followed by this offical guide.
I had the same problem, installed diffusers==0.24.0 not work.

/home/leo/.conda/envs/leo_inpainting/lib/python3.10/site-packages/controlnet_aux/segment_anything/modeling/tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_5m_224 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_5m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
  return register_model(fn_wrapper)
/home/leo/.conda/envs/leo_inpainting/lib/python3.10/site-packages/controlnet_aux/segment_anything/modeling/tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_11m_224 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_11m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
  return register_model(fn_wrapper)
/home/leo/.conda/envs/leo_inpainting/lib/python3.10/site-packages/controlnet_aux/segment_anything/modeling/tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_21m_224 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_21m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
  return register_model(fn_wrapper)
/home/leo/.conda/envs/leo_inpainting/lib/python3.10/site-packages/controlnet_aux/segment_anything/modeling/tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_21m_384 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_21m_384. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
  return register_model(fn_wrapper)
/home/leo/.conda/envs/leo_inpainting/lib/python3.10/site-packages/controlnet_aux/segment_anything/modeling/tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_21m_512 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_21m_512. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
  return register_model(fn_wrapper)
Traceback (most recent call last):
  File "/home/leo/MentorWorks/ImageInpainting/MMLab/test.py", line 2, in <module>
    sd_inferencer = MMagicInferencer(model_name='stable_diffusion')
  File "/home/leo/MentorWorks/ImageInpainting/MMLab/mmagic/mmagic/apis/mmagic_inferencer.py", line 147, in __init__
    self.inferencer = Inferencers(
  File "/home/leo/MentorWorks/ImageInpainting/MMLab/mmagic/mmagic/apis/inferencers/__init__.py", line 81, in __init__
    self.inferencer = Text2ImageInferencer(
  File "/home/leo/MentorWorks/ImageInpainting/MMLab/mmagic/mmagic/apis/inferencers/base_mmagic_inferencer.py", line 59, in __init__
    register_all_modules()
  File "/home/leo/MentorWorks/ImageInpainting/MMLab/mmagic/mmagic/utils/setup_env.py", line 25, in register_all_modules
    import mmagic.evaluation  # noqa: F401,F403
  File "/home/leo/MentorWorks/ImageInpainting/MMLab/mmagic/mmagic/evaluation/__init__.py", line 2, in <module>
    from .evaluator import Evaluator
  File "/home/leo/MentorWorks/ImageInpainting/MMLab/mmagic/mmagic/evaluation/evaluator.py", line 12, in <module>
    from .metrics.base_gen_metric import GenMetric
  File "/home/leo/MentorWorks/ImageInpainting/MMLab/mmagic/mmagic/evaluation/metrics/__init__.py", line 12, in <module>
    from .niqe import NIQE, niqe
  File "/home/leo/MentorWorks/ImageInpainting/MMLab/mmagic/mmagic/evaluation/metrics/niqe.py", line 12, in <module>
    from mmagic.datasets.transforms import MATLABLikeResize
  File "/home/leo/MentorWorks/ImageInpainting/MMLab/mmagic/mmagic/datasets/transforms/__init__.py", line 2, in <module>
    from .albu_function import AlbuCorruptFunction, PairedAlbuTransForms
  File "/home/leo/MentorWorks/ImageInpainting/MMLab/mmagic/mmagic/datasets/transforms/albu_function.py", line 4, in <module>
    import albumentations as albu
ModuleNotFoundError: No module named 'albumentations'

after pip install albumentations:

raise EnvironmentError(
OSError: runwayml/stable-diffusion-v1-5 does not appear to have a file named config.json.

@umaximu
Copy link

umaximu commented Mar 17, 2024

i'm having the same issue! i'm on a macbook air (m1), have followed several different official guides, and am about to try changing my python model with pyenv. which version of python have y'all been running?

@lewishaas
Copy link

I think this is an issue with the diffusers library. I commented out the line referring to controlnet_xs in site-packages/diffusers/pipelines/__init__.py and it worked afterward. I didn't change the diffusers version

@hullout
Copy link

hullout commented Apr 9, 2024

pip install diffusers==0.24.0

it works, thanks

@mcsmart76
Copy link

mcsmart76 commented Apr 25, 2024

I have not been able to resolve the issue with diffusers=0.24.0 nor by commenting out the controlnet_xs line.

I think this is an issue with the diffusers library. I commented out the line referring to controlnet_xs in site-packages/diffusers/pipelines/__init__.py and it worked afterward. I didn't change the diffusers version

This got me to a new error message: ModuleNotFoundError: No module named 'mmcv._ext'

My full repro from head using virtual Python environments:

$ cat ~/mmagic.sh 
#!/bin/bash

VENV=~/.mmagic

# To cleanup, run `deactivate` and `rm -r $VENV`.

# Create my virtual environment at $VENV and activate it.
python3 -m venv $VENV
source $VENV/bin/activate

# Dependency installs, trying to be as up to date as possible.
pip3 install 'torch>=2.3.0'
pip3 install torchvision
pip3 install -U openmim
# mmcv was bumped to 2.2.0 on 2024-04-16, which is incompatible
# with mmengine's version check as of 2024-04-25.
# https://github.com/open-mmlab/mmcv/pull/3087/commits
mim install 'mmcv>=2.0.0,<=2.1.0'
mim install mmengine
mim install mmagic

# Validate imports.
echo "torch working?"
python -c "import torch; print(torch.rand(5))"
echo
echo "mmagic working?"
python -c "import mmagic; print(mmagic.__version__)"
python -c "from mmagic.apis import MMagicInferencer"

@mcsmart76
Copy link

I was able to get things working with the addition of:

# Make sure we aren't caching wheels so mmcv builds the .so.
pip3 cache purge
pip3 install wheel

and changing mim install 'mmcv>=2.0.0,<=2.1.0' to:

# Use pip3 instead of mim to cause .so to be built.
#
# mmcv was bumped to 2.2.0 on 2024-04-16, which is incompatible
# with mmengine's version check as of 2024-04-25.
# https://github.com/open-mmlab/mmcv/pull/3087/commits
pip3 install 'mmcv>=2.0.0,<=2.1.0'

I also ran into #2110 for which I added:

pip3 install 'diffusers==0.24.0'

The full working script with a Python virtual environment is:

$ cat mmagic.sh
#!/bin/bash

VENV=~/.mmagic

# To cleanup, run `deactivate` and `rm -r $VENV`.

# Create my virtual environment at $VENV and activate it.
python3 -m venv $VENV
source $VENV/bin/activate

# Make sure we aren't caching wheels so mmcv builds the .so.
pip3 cache purge
pip3 install wheel

pip3 install torch
pip3 install torchvision
pip3 install -U openmim

# Use pip3 instead of mim to cause .so to be built.
#
# mmcv was bumped to 2.2.0 on 2024-04-16, which is incompatible
# with mmengine's version check as of 2024-04-25.
# https://github.com/open-mmlab/mmcv/pull/3087/commits
pip3 install 'mmcv>=2.0.0,<=2.1.0'

mim install mmengine
mim install mmagic

# https://github.com/open-mmlab/mmagic/issues/2110#issuecomment-1871812155
#
# Fixed "'No module named 'diffusers.pipelines.controlnet_xs'" issue by
# downgrading to 0.24.0.
pip3 install 'diffusers==0.24.0'

# Needed for the example at
# https://github.com/open-mmlab/mmagic/blob/main/docs/en/get_started/quick_run.md
pip3 install albumentations
pip3 install accelerate              

# Validate imports.
echo "torch working?"
python -c "import torch; print(torch.rand(5))"
echo
echo "mmagic working?"
python -c "import mmagic; print(mmagic.__version__)"
python -c "from mmagic.apis import MMagicInferencer"

echo
echo "source $VENV/bin/activate"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug something isn't working
Projects
None yet
Development

No branches or pull requests

8 participants