Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: broken merge_group tests #858

Closed
huitseeker opened this issue Nov 7, 2023 · 1 comment · Fixed by #861
Closed

CI: broken merge_group tests #858

huitseeker opened this issue Nov 7, 2023 · 1 comment · Fixed by #861
Assignees
Labels
bug Something isn't working

Comments

@huitseeker
Copy link
Member

huitseeker commented Nov 7, 2023

Issue

Merge group GPU benchmark tests are broken:
https://github.com/lurk-lab/lurk-rs/actions/runs/6777848351/job/18422227933

Run cat fc495987b972a3736da3d946e5615590c06036.json | criterion-table > BENCHMARKS.md
  cat fc495987b972a3736da3d9146e56155920c06036.json | criterion-table > BENCHMARKS.md
  shell: /usr/bin/bash -e {0}
  env:
    CACHE_ON_FAILURE: false
    CARGO_INCREMENTAL: 0
    BASE_REF: 25c170bf85cd0676fd012[3](https://github.com/lurk-lab/lurk-rs/actions/runs/6777848351/job/18422227933#step:17:3)7acdf[4](https://github.com/lurk-lab/lurk-rs/actions/runs/6777848351/job/18422227933#step:17:4)0e1632d39[5](https://github.com/lurk-lab/lurk-rs/actions/runs/6777848351/job/18422227933#step:17:5)9[8](https://github.com/lurk-lab/lurk-rs/actions/runs/6777848351/job/18422227933#step:17:8)
    LURK_BENCH_OUTPUT: commit-comment
An error occurred processing Criterion data: Duplicate column: fib-branch=gh-readonly-queue

Which is logically followed by Error: Missing comment 'body' or 'body-path'. on the next step (commit-comment)

Origin

Introduced in #852 and made visible by #857.

@huitseeker huitseeker added the bug Something isn't working label Nov 7, 2023
@huitseeker huitseeker changed the title CI: fix merge_group tests CI: broken merge_group tests Nov 7, 2023
@huitseeker
Copy link
Member Author

huitseeker commented Nov 7, 2023

I've added a gist of :

LURK_RC="100,600" LURK_BENCH_OUTPUT=commit-comment cargo criterion --bench fibonacci_lem --message-format=json | tee -a $(git rev-parse HEAD).json

On a machine that does not have enough RAM to run past rc=100. At:
https://gist.github.com/huitseeker/2b2ffc93d2963dda8e1d920726b375f8

Note the tee -a is load-bearing to reproduce this, which I believe is caused by nested checkouts: in the case of our gh-merge queue branch, the branch is likely the same both on base ref and PR.

Passing it to criterion-table does produce

An error occurred processing Criterion data: Duplicate column: fib-branch=master

Seemingly, using the bench group as a separator with the function name duplicated is not helpful when using a single json, contrarily to two (differential bench), which is what I used to demo the output in #852.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants