-
Notifications
You must be signed in to change notification settings - Fork 22
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
refactor: split data file into multiple
any benchmark file should end in _data.yaml
- Loading branch information
1 parent
88beff8
commit 123d231
Showing
12 changed files
with
979 additions
and
924 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,62 @@ | ||
# Top-level keys: benchmark modules | ||
# Values: list of dictionaries, each containing a test case | ||
# | ||
# Test case keys: | ||
# - input (for creating the test) | ||
# - expected (for asserting ourcomes and generating a score) | ||
# - case (for categorizing the test case) | ||
# | ||
# If any input is a dictionary itself, it will be expanded into separate test | ||
# cases, using the top-level key to create a concatenated test case purpose. | ||
|
||
api_calling: | ||
- case: oncokb:braf:melanoma | ||
input: | ||
prompt: | ||
exact_spelling: "What is the consequence of the V600E BRAF variant in Melanoma?" | ||
expected: | ||
parts_of_query: | ||
[ | ||
"https://demo.oncokb.org/api/v1/annotate/mutations/byProteinChange?", | ||
"hugoSymbol=BRAF", | ||
"alteration=V600E", | ||
"tumorType=Melanoma", | ||
] | ||
- case: oncokb:tp53:colon_adenocarcinoma | ||
input: | ||
prompt: | ||
exact_spelling: "What is the consequence of the R273C TP53 variant in Colon Adenocarcinoma?" | ||
expected: | ||
parts_of_query: | ||
[ | ||
"https://demo.oncokb.org/api/v1/annotate/mutations/byProteinChange?hugoSymbol=TP53", | ||
"alteration=R273C", | ||
"tumorType=Colon%20Adenocarcinoma", | ||
] | ||
- case: oncokb:braf:histiocytosis | ||
input: | ||
prompt: | ||
exact_spelling: "What is the consequence of the N486_P490del BRAF variant in Histiocytosis?" | ||
descriptive_spelling: "What is the consequence of an N486_P490 deletion in BRAF in Histiocytosis?" | ||
expected: | ||
parts_of_query: | ||
[ | ||
"https://demo.oncokb.org/api/v1/annotate/mutations/byProteinChange?", | ||
"hugoSymbol=BRAF", | ||
"alteration=N486_P490del", | ||
"tumorType=Histiocytosis", | ||
] | ||
- case: oncokb:ros1:lung_adenocarcinoma | ||
input: | ||
prompt: | ||
exact_spelling: "What is the consequence of the functional fusion of CD47 and ROS1 in Lung Adenocarcinoma?" | ||
expected: | ||
parts_of_query: | ||
[ | ||
"https://demo.oncokb.org/api/v1/annotate/structuralVariants?", | ||
"hugoSymbolA=CD74", | ||
"hugoSymbolB=ROS1", | ||
"structuralVariantType=FUSION", | ||
"isFunctionalFusion=true", | ||
"tumorType=Lung%20Adenocarcinoma", | ||
] |
Oops, something went wrong.