Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] fail to export torch to onnx ,mmpretrain #2812

Open
3 tasks done
Kenneth-X opened this issue Aug 14, 2024 · 0 comments
Open
3 tasks done

[Bug] fail to export torch to onnx ,mmpretrain #2812

Kenneth-X opened this issue Aug 14, 2024 · 0 comments

Comments

@Kenneth-X
Copy link

Checklist

  • I have searched related issues but cannot get the expected help.
  • 2. I have read the FAQ documentation but cannot get the expected help.
  • 3. The bug has not been fixed in the latest version.

Describe the bug

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (2.0.7) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
/usr/local/lib/python3.10/dist-packages/mmcv/cnn/bricks/transformer.py:33: UserWarning: Fail to import MultiScaleDeformableAttention from mmcv.ops.multi_scale_deform_attn, You should install mmcv rather than mmcv-lite if you need this module.
warnings.warn('Fail to import MultiScaleDeformableAttention from '
08/14 20:54:40 - mmengine - WARNING - Failed to search registry with scope "mmpretrain" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpretrain" is a correct scope, or whether the registry is initialized.
08/14 20:54:40 - mmengine - WARNING - Failed to search registry with scope "mmpretrain" in the "mmpretrain_tasks" registry tree. As a workaround, the current "mmpretrain_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpretrain" is a correct scope, or whether the registry is initialized.
08/14 20:54:43 - mmengine - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (2.0.7) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
/usr/local/lib/python3.10/dist-packages/mmcv/cnn/bricks/transformer.py:33: UserWarning: Fail to import MultiScaleDeformableAttention from mmcv.ops.multi_scale_deform_attn, You should install mmcv rather than mmcv-lite if you need this module.
warnings.warn('Fail to import MultiScaleDeformableAttention from '
08/14 20:54:44 - mmengine - WARNING - Failed to search registry with scope "mmpretrain" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpretrain" is a correct scope, or whether the registry is initialized.
08/14 20:54:44 - mmengine - WARNING - Failed to search registry with scope "mmpretrain" in the "mmpretrain_tasks" registry tree. As a workaround, the current "mmpretrain_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpretrain" is a correct scope, or whether the registry is initialized.
Loads checkpoint by local backend from path: work_dirs/config-ResNet50/best_accuracy_top1_epoch_275.pth
Process Process-2:
Traceback (most recent call last):
File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.10/dist-packages/mmdeploy/apis/core/pipeline_manager.py", line 107, in call
ret = func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/mmdeploy/apis/pytorch2onnx.py", line 63, in torch2onnx
torch_model = task_processor.build_pytorch_model(model_checkpoint)
File "/usr/local/lib/python3.10/dist-packages/mmdeploy/codebase/base/task.py", line 125, in build_pytorch_model
model = revert_sync_batchnorm(model)
File "/usr/local/lib/python3.10/dist-packages/mmengine/model/utils.py", line 174, in revert_sync_batchnorm
from mmcv.ops import SyncBatchNorm
File "/usr/local/lib/python3.10/dist-packages/mmcv/ops/init.py", line 3, in
from .active_rotated_filter import active_rotated_filter
File "/usr/local/lib/python3.10/dist-packages/mmcv/ops/active_rotated_filter.py", line 10, in
ext_module = ext_loader.load_ext(
File "/usr/local/lib/python3.10/dist-packages/mmcv/utils/ext_loader.py", line 13, in load_ext
ext = importlib.import_module('mmcv.' + name)
File "/usr/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ImportError: /usr/local/lib/python3.10/dist-packages/mmcv/_ext.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN2at4_ops10zeros_like4callERKNS_6TensorEN3c108optionalINS5_10ScalarTypeEEENS6_INS5_6LayoutEEENS6_INS5_6DeviceEEENS6_IbEENS6_INS5_12MemoryFormatEEE
08/14 20:54:47 - mmengine - ERROR - /usr/local/lib/python3.10/dist-packages/mmdeploy/apis/core/pipeline_manager.py - pop_mp_output - 80 - mmdeploy.apis.pytorch2onnx.torch2onnx with Call id: 0 failed. exit.
root@yq01-sys-hic-k8s-v100-box-a225-0426.yq01.baidu.com bmat $ bash export_classification.sh
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (2.0.7) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
/usr/local/lib/python3.10/dist-packages/mmcv/cnn/bricks/transformer.py:33: UserWarning: Fail to import MultiScaleDeformableAttention from mmcv.ops.multi_scale_deform_attn, You should install mmcv rather than mmcv-lite if you need this module.
warnings.warn('Fail to import MultiScaleDeformableAttention from '
08/14 20:57:10 - mmengine - WARNING - Failed to search registry with scope "mmpretrain" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpretrain" is a correct scope, or whether the registry is initialized.
08/14 20:57:10 - mmengine - WARNING - Failed to search registry with scope "mmpretrain" in the "mmpretrain_tasks" registry tree. As a workaround, the current "mmpretrain_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpretrain" is a correct scope, or whether the registry is initialized.
08/14 20:57:13 - mmengine - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (2.0.7) or chardet (3.0.4) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
/usr/local/lib/python3.10/dist-packages/mmcv/cnn/bricks/transformer.py:33: UserWarning: Fail to import MultiScaleDeformableAttention from mmcv.ops.multi_scale_deform_attn, You should install mmcv rather than mmcv-lite if you need this module.
warnings.warn('Fail to import MultiScaleDeformableAttention from '
08/14 20:57:14 - mmengine - WARNING - Failed to search registry with scope "mmpretrain" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpretrain" is a correct scope, or whether the registry is initialized.
08/14 20:57:14 - mmengine - WARNING - Failed to search registry with scope "mmpretrain" in the "mmpretrain_tasks" registry tree. As a workaround, the current "mmpretrain_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpretrain" is a correct scope, or whether the registry is initialized.
Loads checkpoint by local backend from path: work_dirs/config-ResNet50/best_accuracy_top1_epoch_275.pth
Process Process-2:
Traceback (most recent call last):
File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.10/dist-packages/mmdeploy/apis/core/pipeline_manager.py", line 107, in call
ret = func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/mmdeploy/apis/pytorch2onnx.py", line 63, in torch2onnx
torch_model = task_processor.build_pytorch_model(model_checkpoint)
File "/usr/local/lib/python3.10/dist-packages/mmdeploy/codebase/base/task.py", line 125, in build_pytorch_model
model = revert_sync_batchnorm(model)
File "/usr/local/lib/python3.10/dist-packages/mmengine/model/utils.py", line 174, in revert_sync_batchnorm
from mmcv.ops import SyncBatchNorm
File "/usr/local/lib/python3.10/dist-packages/mmcv/ops/init.py", line 3, in
from .active_rotated_filter import active_rotated_filter
File "/usr/local/lib/python3.10/dist-packages/mmcv/ops/active_rotated_filter.py", line 10, in
ext_module = ext_loader.load_ext(
File "/usr/local/lib/python3.10/dist-packages/mmcv/utils/ext_loader.py", line 13, in load_ext
ext = importlib.import_module('mmcv.' + name)
File "/usr/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ImportError: /usr/local/lib/python3.10/dist-packages/mmcv/_ext.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN2at4_ops10zeros_like4callERKNS_6TensorEN3c108optionalINS5_10ScalarTypeEEENS6_INS5_6LayoutEEENS6_INS5_6DeviceEEENS6_IbEENS6_INS5_12MemoryFormatEEE
08/14 20:57:16 - mmengine - ERROR - /usr/local/lib/python3.10/dist-packages/mmdeploy/apis/core/pipeline_manager.py - pop_mp_output - 80 - mmdeploy.apis.pytorch2onnx.torch2onnx with Call id: 0 failed. exit.

Reproduction

python mmdeploy/tools/deploy.py
mmdeploy/configs/mmpretrain/classification_onnxruntime_dynamic.py
work_dirs/config-ResNet50/config-ResNet50.py
work_dirs/config-ResNet50/best_accuracy_top1_epoch_275.pth
demo.jpg
--work-dir work_dirs/onnx
--device cpu
--dump-info

Environment

pip3 install torch==2.2.0 torchvision==0.17.0 torchaudio==2.2.0 
    pip3 install onnxruntime-gpu==1.16.0 && \
    pip3 install onnx==1.16.1 

    pip3 install mmcv==2.1.0 
    pip3 install mmdet==3.2.0 && \
    pip3 install mmpretrain==1.2.0 && \
    pip3 install mmengine  && \
    pip3 install mmdeploy==1.2.0 && \
    pip3 install mmdeploy-runtime-gpu==1.2.0

Error traceback

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant