Target layers of ViT models

[WangYZ1608]

Hi @jacobgil. It's a nice work! Thanks to you and all the contributors behind it.

I tried replacing norm1 with block[-1].attn or norm2 without any other changes, but the gradient seems to be 0, as shown below (I test with GradCAM and LayerCAM. And I could get satisfactory result when using norm1). What is the reason for this?

Also, I would like to ask, how to visualize the class token? And how to use CAM in an image containing multiple instances?

original image

result

[zermatt-luo]

Hello, did you resolve this issue? I have a similar problem!

[sophmrtn]

I also have the same issue!