-
Notifications
You must be signed in to change notification settings - Fork 398
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Load Custom Pre-Trained PyTorch MLP as Surrogate Model #2566
Comments
Fundamentally, you need to implement this in a way that's compliant with the basic Since you're using dropout I assume your model will produce a bunch of samples - for that you could use the Once you have that you should be able to use the model with the standard botorch acquisition function machinery. If you're re-sampling your dropouts in each pass then you'll need to be careful to not use deterministic optimization via Sample Average Approximation (see the botorch paper for details) and use a stochastic optimizer for optimizing your acquisition function instead. Alternatively you can fix the dropouts to generate a batched ensemble model in which case you can use Sample Average Approximation. |
Hi Balandat, thank you so much for the prompt reply! I would like to clarify my main objective: minimizing the time required to retrain the surrogate model, ideally keeping it to just a few minutes. I have several questions regarding this process:
Thanks! |
It appears your questions are mostly about the MLP with dropout and not so much about botorch, so I'm not sure how much I can help you with this.
I am not sure I understand. Yes you can always load pretrained weights, not sure why you'd want to do this in the forward method? The way I would think about this is that you train your MLP outside of the botorch Model API, and then stick it into an
It's your surrogate model so you can handle this as you like. It makes sense to me to fine-tune the model, especially later in the optimization process.
I don't understand this one - what hyperparameters are you referring to? Are these for the training of your MLP model? |
Hi everyone,
I have developed a custom Multi-Layer Perceptron (MLP) model using PyTorch, which has been pre-trained for my specific application. The model architecture has an input size of 579 features and produces an output of size 7. Additionally, I've incorporated dropout layers to enable probabilistic outputs. I would like assistance with loading this model and using it as a surrogate model.
Any guidance on how to implement this would be appreciated!
Thank you!
The text was updated successfully, but these errors were encountered: