[Question] what is the difference between ManagedCollisionEmbeddingCollection and ITEPEmbeddingBagCollection

[tiankongdeguiji]

What is the difference between ManagedCollisionEmbeddingCollection and ITEPEmbeddingBagCollection, and when should I use ManagedCollisionEmbeddingCollection versus ITEPEmbeddingBagCollection?

[tiankongdeguiji]

Hi, @henrylhtsang @joshuadeng @PaulZhang12 @TroyGarden can you see this problem?

[henrylhtsang]

@YLGH any idea?

[tiankongdeguiji]

hi, @YLGH can you see this problem?

[PaulZhang12]

Hi @tiankongdeguiji, ManagedCollisionEmbeddingCollection is meant to represent ZCH embeddings, while ITEPEmbeddingBagCollection refers to pruning embeddings during training for memory savings.

[tiankongdeguiji]

hi, @PaulZhang12 , does eviction in ManagedCollisionEmbeddingCollection correspond to pruning in ITEPEmbeddingBagCollection? Additionally, I believe ManagedCollisionEmbeddingCollection is also memory-efficient because it eliminates the need for a large hash size to prevent hash collisions.