You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am trying to reproduce the CIFAR-10 experiment with wide resnet. However I was only able to achieve 2.96% Test Error with SAM and 3.5% Test Error with SGD. Since I am using only 4 GPUs for training, I understand that the performance of SAM could be different. However in case of SGD, if all hyperparameters are same, I believe the performance should be similar.
I tried my best to replicate the hyperparameters mentioned in the paper. However some parameters were not so clearly stated. So it would be grateful, if you could verify my training details. Here is the hyperparameters I used for training:
Hi,
I am trying to reproduce the CIFAR-10 experiment with wide resnet. However I was only able to achieve 2.96% Test Error with SAM and 3.5% Test Error with SGD. Since I am using only 4 GPUs for training, I understand that the performance of SAM could be different. However in case of SGD, if all hyperparameters are same, I believe the performance should be similar.
I tried my best to replicate the hyperparameters mentioned in the paper. However some parameters were not so clearly stated. So it would be grateful, if you could verify my training details. Here is the hyperparameters I used for training:
learning rate : 0.1
epoch : 200 (and 400 in case of SGD)
batch size : 256
scheduler : cosine
weight decay : 0.0005
rho : 0.05
beta : 0.9
Nesterov : True
The text was updated successfully, but these errors were encountered: