add unbalanced param_sync example. #126
Open
+397
−120
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
haolin-nju
reviewed
Oct 17, 2024
SeaOfOcean
reviewed
Oct 18, 2024
|
|
||
| if __name__ == "__main__": | ||
| chatlearn.init() | ||
| args = chatlearn.get_args() |
There was a problem hiding this comment.
可以像https://github.com/alibaba/ChatLearn/blob/main/examples/megatron/tests/test_parameter_sync.py#L37 一样设置debug=True，另外 parameter_sync 文件中的 validate 函数是不是还不支持
There was a problem hiding this comment.
validate 函数单独提一个来做校验，这个用来测试第一个episode的str outputs是否正确
SeaOfOcean
reviewed
Oct 25, 2024
| reward_load_iteration=${REWARD_LOAD_ITERATION} \ | ||
| reward_load=${REWARD_LOAD} \ | ||
| tokenizer_model=${TOKENIZER_MODEL} \ | ||
| num_episode=${num_ppo_episode:-0} \ |
SeaOfOcean
reviewed
Oct 25, 2024
| reward_load=${REWARD_LOAD} \ | ||
| tokenizer_model=${TOKENIZER_MODEL} \ | ||
| num_episode=${num_ppo_episode:-0} \ | ||
| data_path=${DATASET_PATH} \ |
There was a problem hiding this comment.
设置环境变量 validate_param_sync 为True
There was a problem hiding this comment.
这个不是e2e的测试，只是测试一次param sync的正确性。e2e的得换成rlhf或其他alignment格式的逻辑。多个episode会有ppo_policy forward step参数不足的问题
There was a problem hiding this comment.
validate_param_sync 这个参数只是在parameter sync的时候触发 validate函数，和是否是e2e无关
SeaOfOcean
reviewed
Oct 25, 2024
SeaOfOcean
reviewed
Oct 25, 2024
haolin-nju
reviewed
Oct 28, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.