Information request about the competitions in each category in Figure 6

[JK-SHIN-PG]

Hello, thank you for your team's great work.

I have been working on classifying the competitions, but the numbers I came up with do not match those shown in Figure 6. Could you please provide the data used in Figure 6? Specifically, which category does each competition belong to? For example, what category does the 'Stanford Corona Vaccine' fall into?

Thank you

Jongkyung

[SamuelSchmidgall]

Hello, I am not one of the authors, but I believe you can find all of the split information here: https://github.com/openai/mle-bench/blob/main/experiments/splits/

[thesofakillers]

I assume they are looking for the categories like "Computer Vision" etc.

We are monitoring these issues/requests and will get back to you.

[JK-SHIN-PG]

Thank you @SamuelSchmidgall . I want to know about the competitions that fall into each category listed in Figure 6, such as 'Object Detection', 'Image Classification', and 'Tabular', as @thesofakillers said.

@thesofakillers , Thank you for your positive feedback. I believe that this information will be highly beneficial in analyzing potential differences in performance across various modalities and tasks.

[james-aung]

Hey, we'll share a csv of these on the repo later this week or next

[JK-SHIN-PG]

Thank you!!