Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Questionnaire] Which model do you want to support in mmdeploy? #170

Open
4 tasks
VVsssssk opened this issue Feb 18, 2022 · 115 comments
Open
4 tasks

[Questionnaire] Which model do you want to support in mmdeploy? #170

VVsssssk opened this issue Feb 18, 2022 · 115 comments
Assignees

Comments

@VVsssssk
Copy link
Collaborator

VVsssssk commented Feb 18, 2022

Which model do you want to support in mmdeploy? Comment here. You can like a comment to increase model priority.

Model Codebase Backend Number of likes Released
1 Swin-Transformer mmcls tensorrt 2 v0.6.0
2 Second mmdet3d tensorrt 1
3 htc mmdet tensorrt 1
4 Segmenter mmseg tensorrt 1 v0.6.0
5 ConvNeXt mmcls tensorrt 1
6 MASTER mmocr tensorrt 1
7 ABINet mmocr tensorrt 1
8 BEVFormer mmdet3d tensorrt 1
9 Oriented R-CNN mmrotate tensorrt 1 v0.6.0
10 SAR mmocr tensorrt 1

Reference:

@VVsssssk
Copy link
Collaborator Author

E.g:
Swin-Transformer mmcls tensorrt

@OrcunCanDeniz
Copy link

second mmdet3d tensorrt

@haofanwang
Copy link
Contributor

htc mmdetection tensorrt

@haofanwang
Copy link
Contributor

haofanwang commented Feb 19, 2022

segmenter mmsegmentation tensorrt

@starhou
Copy link

starhou commented Feb 19, 2022

swin-transformer mmdet tensorrt

@starhou
Copy link

starhou commented Feb 21, 2022

I haven't tried the conversion yet and I just used your example fater-rcnn. Maybe now, need to try.

@RunningLeon
Copy link
Collaborator

segmenter mmsegmentation tensorrt

@haofanwang Have you tried segmenter with mmdeploy?

@haofanwang
Copy link
Contributor

segmenter mmsegmentation tensorrt

@haofanwang Have you tried segmenter with mmdeploy?

@RunningLeon , Yes, but I receive error messages as below

TypeError: upsample_bicubic2d() received an invalid combination of arguments - got (Tensor, NoneType, bool, list), but expected one of:
 * (Tensor input, tuple of ints output_size, bool align_corners, tuple of floats scale_factors)
      didn't match because some of the arguments have invalid types: (Tensor, NoneType, bool, list)
 * (Tensor input, tuple of ints output_size, bool align_corners, float scales_h, float scales_w, *, Tensor out)

And my command is

python ./tools/deploy.py \
    ./configs/mmseg/segmentation_tensorrt-fp16_dynamic-512x1024-2048x2048.py \
    ./mmsegmentation/configs/segmenter/segmenter_vit-l_mask_8x1_512x512_160k_ade20k.py \
    ./mmsegmentation/checkpoints/segmenter_vit-l_mask_8x1_512x512_160k_ade20k_20220105_162750-7ef345be.pth \
    ./mmsegmentation/demo/demo.png \
    --work-dir work_dir\
    --device cuda\
    --dump-in

@RunningLeon
Copy link
Collaborator

512x512
@haofanwang
Try with static config with 512x512. You can change in config file.

@eliabruni
Copy link

maskformer mmdet tensorrt

@isbrycee
Copy link

swin-transformer mmdet onnx

@Siannodel
Copy link

Swin-Transformer mmcls tensorrt

@shannanyinxiang
Copy link

SATRN mmocr tensorrt

@awesome01010
Copy link

SATRN for ocr, to onnx and tensorrt pls!

1 similar comment
@summerlvsong
Copy link

SATRN for ocr, to onnx and tensorrt pls!

@Mountchicken
Copy link

Mountchicken commented Mar 22, 2022

SATRN MMOCR tensorrt

@Xiangxu-0103
Copy link
Contributor

dynamic_voxelization mmdet3d onnxruntime and tensorrt

@YongboLiang
Copy link

Swin-Transformer Second Segmenter

@hust-lidelong
Copy link

Oriented R-CNNS2A-Net for mmrotate tensorrt, please!

@eliabruni
Copy link

mask2former mmdet tensorrt

@jianghaijun007
Copy link

monoflex tensorrt

@gzxy-0102
Copy link

mmtracking

@troyliu0105
Copy link

AssociativeEmbedding pose detection in mmpose

@SpaceView
Copy link

mmocr

@Zaughter
Copy link

Zaughter commented Sep 7, 2023

dab-detr onnx

@RunningLeon
Copy link
Collaborator

RunningLeon commented Sep 8, 2023

dab-detr onnx

@Zaughter @DuckJ @sibet-lb @RobinHan24 @idonahum1
hi, for DETR, DINO, Dab-DETR, Conditional-DETR, could try this PR##2410

@RobinHan24
Copy link

dab-detr onnx

@Zaughter @DuckJ @sibet-lb @RobinHan24 @idonahum1 hi, for DETR, DINO, Dab-DETR, Conditional-DETR, could try this PR##2410

I just tried it, but error.

2023-09-10:10:53:30 - root - ERROR - execute_async_v2(): incompatible function arguments. The following argument types are supported:
1. (self: tensorrt.tensorrt.IExecutionContext, bindings: List[int], stream_handle: int, input_consumed: capsule = None) -> bool

Invoked with: <tensorrt.tensorrt.IExecutionContext object at 0x7f01857b5bb0>, [139631165177856, None, 139634709371904, 139634709365760], 0
Traceback (most recent call last):
File "/mnt/sda/qilibin/project/mmdeploy-support_detr_multi-batch/mmdeploy/utils/utils.py", line 41, in target_wrapper
result = target(*args, **kwargs)
File "/mnt/sda/qilibin/project/mmdeploy-support_detr_multi-batch/mmdeploy/apis/visualize.py", line 72, in visualize_model
result = model.test_step(model_inputs)[0]
File "/mnt/sda/qilibin/miniconda3/envs/py3.9/lib/python3.9/site-packages/mmengine/model/base_model/base_model.py", line 145, in test_step
return self._run_forward(data, mode='predict') # type: ignore
File "/mnt/sda/qilibin/miniconda3/envs/py3.9/lib/python3.9/site-packages/mmengine/model/base_model/base_model.py", line 340, in _run_forward
results = self(**data, mode=mode)
File "/mnt/sda/qilibin/miniconda3/envs/py3.9/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/mnt/sda/qilibin/project/mmdeploy-support_detr_multi-batch/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 299, in forward
outputs = self.predict(inputs, shape_info)
File "/mnt/sda/qilibin/project/mmdeploy-support_detr_multi-batch/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 323, in predict
outputs = self.wrapper(inputs)
File "/mnt/sda/qilibin/miniconda3/envs/py3.9/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/mnt/sda/qilibin/project/mmdeploy-support_detr_multi-batch/mmdeploy/backend/tensorrt/wrapper.py", line 174, in forward
self.__trt_execute(bindings=bindings)
File "/mnt/sda/qilibin/project/mmdeploy-support_detr_multi-batch/mmdeploy/utils/timer.py", line 67, in fun
result = func(*args, **kwargs)
File "/mnt/sda/qilibin/project/mmdeploy-support_detr_multi-batch/mmdeploy/backend/tensorrt/wrapper.py", line 185, in __trt_execute
self.context.execute_async_v2(bindings,
TypeError: execute_async_v2(): incompatible function arguments. The following argument types are supported:
1. (self: tensorrt.tensorrt.IExecutionContext, bindings: List[int], stream_handle: int, input_consumed: capsule = None) -> bool

Invoked with: <tensorrt.tensorrt.IExecutionContext object at 0x7f01857b5bb0>, [139631165177856, None, 139634709371904, 139634709365760], 0
09/10 10:53:32 - mmengine - ERROR - /mnt/sda/qilibin/project/mmdeploy-support_detr_multi-batch/tools/deploy.py - create_process - 82 - visualize tensorrt model failed.

tensorrt version: 8.6.1.6

command:
python tools/deploy.py configs/mmdet/detection/detection_tensorrt_dynamic-320x320-1344x1344.py dino/dino-4scale_r50_8xb2-36e_coco.py dino-4scale_r50_8xb2-12e_coco_20221202_182705-55b2bba2.pth demo/resources/det.jpg --work-dir mmdeploy_models/mmdet/dino --device cuda --show --dump-info

could you please help me?

@RobinHan24
Copy link

RobinHan24 commented Sep 10, 2023

dab-detr onnx

@Zaughter @DuckJ @sibet-lb @RobinHan24 @idonahum1 hi, for DETR, DINO, Dab-DETR, Conditional-DETR, could try this PR##2410
thanks a lot, I have got the tensorrt model. But how to infer with trt model sdk?
my code:
from mmdeploy_runtime import Detector
import cv2
img = cv2.imread('./demo/resources/det.jpg')
detector = Detector(model_path='mmdeploy_models/mmdet/dino0910', device_name='cuda', device_id=0)
res = detector(img)
print (res)
indices = [i for i in range(len(bboxes))]
for index, bbox, label_id in zip(indices, bboxes, labels):
[left, top, right, bottom], score = bbox[0:4].astype(int), bbox[4]
if score < 0.3:
continue
cv2.rectangle(img, (left, top), (right, bottom), (0, 255, 0))
cv2.imwrite('output_detection.png', img)

error:
[2023-09-10 12:51:09.788] [mmdeploy] [error] [net_module.cpp:130] sample 0 missing key shape

can you help me, thanks.

@MarouaneMja
Copy link

MarouaneMja commented Sep 13, 2023

CO-DETR https://github.com/Sense-X/Co-DETR -> onnx and tensorrt

@x-yy0
Copy link
Contributor

x-yy0 commented Oct 19, 2023

mvxnet mmdet3d tensorrt

@mjehanzaib999
Copy link

slowonly(spatiotemporal) mmaction tensorrt

@DoctorDream
Copy link

Co-DETR please!

@Mayyyybe
Copy link

Mayyyybe commented Nov 9, 2023

Co-DETR to onnx please!!!

@MarouaneMja
Copy link

@DoctorDream @Mayyyybe I already mentionned CO-DETR, Could you please upvote my suggestion , so that it could increase this model priority?

@Baboom-l
Copy link

Grounding DINO mmdetection tensorrt

@xinlin-xiao
Copy link

CO-DETR https://github.com/Sense-X/Co-DETR -> onnx and tensorrt

CO-DETR->onnx ,TorchScript,tensorrt

@OSSome01
Copy link

OSSome01 commented Dec 7, 2023

KNet mmsegmentaion
ConvNext mmsegmentation

@WaseemDR
Copy link

TensorRT for Grounding Dino

@Jialeen
Copy link

Jialeen commented Jan 4, 2024

TensorRT for Detic

@iceriver97
Copy link

fcos3d mmdet3d tensorrt

这个目前实现了吗?

@chunxt
Copy link

chunxt commented Jan 27, 2024

tood. thanks

@Nieees
Copy link

Nieees commented Jan 31, 2024

htc Series models (scnet, detectRS) Is there any progress so far?

@Bovey0809
Copy link

MMRazor QAT

@monologuesmw
Copy link

TOOD model. Thanks!

@monkeycc
Copy link

YOLOv8
[Detect]
[Segment]
[Classify]
[Pose]
[OBB]

@ZJU-lishuang
Copy link

glip | mmdet | tensorrt

@chenzhutian
Copy link

chenzhutian commented Apr 19, 2024

3D pose (eg. MotionBert) in mmPose

@xlg-go
Copy link

xlg-go commented May 9, 2024

SVTR for ocr, to onnx and tensorRT pls!

@MarouaneMja
Copy link

CO-DETR https://github.com/Sense-X/Co-DETR -> onnx and tensorrt

Hello @VVsssssk ,

Any updates about the exportation of Co-DETR to onnx and tensorrt please?

Best regards,
Marouane

@songkemail
Copy link

superpoint superglue

@sjtu-cz
Copy link

sjtu-cz commented Jul 26, 2024

Swin-Transformer v2 mmpretrain tensorRT

@wostons
Copy link

wostons commented Aug 12, 2024

grounding_dino

@Liufanfanfan
Copy link

ddrnet-slim mmseg

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests