Error:SamAutomaticMaskGenerator function has a large memory footprint

[lstswb]

GPU：4090 24G
System：Ubuntu for WSL2
Model：sam_vit_h
Image_size：[1024,1024]
Parameter settings：
model=sam,
points_per_side=128,
points_per_batch = 64,
pred_iou_thresh=0.86,
stability_score_thresh=0.92,
crop_n_layers=3,
crop_n_points_downscale_factor=2,
min_mask_region_area=100,
process_batch_size=4
Issue：
When I use SamAutomaticMaskGenerator，GPU memory usage up to 55GB.

And there will be an error.
[torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 16.63 GiB. GPU 0 has a total capacity of 23.99 GiB of which 0 bytes is free. Including non-PyTorch memory, this process has 17179869184.00 GiB memory in use. Of the allocated memory 34.86 GiB is allocated by PyTorch, and 5.63 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
]

However, when using the original SAM code, this problem does not exist, and the GPU memory will not exceed 24GB.

[lyf6]

@lstswb have u solved this?

[lstswb]

@lstswb have u solved this?

Not yet

[cpuhrsch]

Does the code snippet from the example help?

In particular

segment-anything-fast/amg_example/amg_example.py

Lines 36 to 44 in 387488b

	from segment_anything_fast import sam_model_registry, sam_model_fast_registry, SamAutomaticMaskGenerator
	sam_checkpoint = "checkpoints/sam_vit_h_4b8939.pth"
	model_type = "vit_h"
	device = "cuda"
	sam = sam_model_fast_registry[model_type](checkpoint=sam_checkpoint)
	sam.to(device=device)
	mask_generator = SamAutomaticMaskGenerator(sam, process_batch_size=8)

note that you can adjust process_batch_size for a smaller memory footprint and note the use of sam_model_fast_registry

[lstswb]

Does the code snippet from the example help?

In particular

segment-anything-fast/amg_example/amg_example.py

Lines 36 to 44 in 387488b

	from segment_anything_fast import sam_model_registry, sam_model_fast_registry, SamAutomaticMaskGenerator
	sam_checkpoint = "checkpoints/sam_vit_h_4b8939.pth"
	model_type = "vit_h"
	device = "cuda"
	sam = sam_model_fast_registry[model_type](checkpoint=sam_checkpoint)
	sam.to(device=device)
	mask_generator = SamAutomaticMaskGenerator(sam, process_batch_size=8)

note that you can adjust process_batch_size for a smaller memory footprint and note the use of sam_model_fast_registry

I tried to adjust batch_size, and the GPU memory footprint was reduced, but it still far exceeded the original code.

[cpuhrsch]

Yes, the batch is larger, but should be faster. The original code uses batch size 1. You can try setting it to batch size 1.

[lstswb]

Yes, the batch is larger, but should be faster. The original code uses batch size 1. You can try setting it to batch size 1.

I tried to adjust batch_size=1, but still got GPU memory error.

[cpuhrsch]

Hm, I assume you're also using the GPU for the display manager? That will take up additional memory as well. Maybe the solution in #97 will help.

Can you use your onboard GPU (if you have one) for the display manager and the GPU for the model only? Does it work with vit_b?

[lstswb]

Hm, I assume you're also using the GPU for the display manager? That will take up additional memory as well. Maybe the solution in #97 will help.

Can you use your onboard GPU (if you have one) for the display manager and the GPU for the model only? Does it work with vit_b?

Vit_b can be used normally. Display takes up only a small portion of GPU memory. Setting vit_h equally works fine with the original code.

[cpuhrsch]

Hm, can you try setting the environment variable SEGMENT_ANYTHING_FAST_USE_FLASH_4 to 0?

[lstswb]

Hm, can you try setting the environment variable SEGMENT_ANYTHING_FAST_USE_FLASH_4 to 0?
SEGMENT_ANYTHING_FAST_USE_FLASH_4 has been set to 0. But the problem remains.

[cpuhrsch]

Hm, I'm not sure to be honest. It seems to work on other 4090s, but I think they're on Linux and not Windows.

[lstswb]

Hm, I'm not sure to be honest. It seems to work on other 4090s, but I think they're on Linux and not Windows.

Well. I try to do this using a linux system instead of using ubuntu based on WSL2