You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@tianyu-l Some basic generation may help discover/test issues in checkpoint conversion, tokenization, validation before continued pre training etc.. I am working on adapting a minimal version from torchtune to support basic train-time evals during checkpointing
@jaysonfrancis Covering the very basic generation/inference sounds good, as long as the code can be kept simple. Please feel free to submit a PR, we can help review.
Though there are some related tickets, it would be great to see the steps to use the raw checkpoints for inference. https://github.com/pytorch/torchtitan/blob/main/docs/checkpoint.md
This helps to verify the training works well.
The text was updated successfully, but these errors were encountered: