feat(api): reranking backend integrated in with rag

[CollectiveUnicorn]

Description

• Adds reranking to RAG pipeline
• Adds RAG configuration endpoint when in dev mode
• Adds additional logging
• Refactors the pytest's test_routes api tests
• Alters default RAG values into two steps, retrieval and ranking. With the retrieval results being set to 100 after ranking the results are filtered down to the user specified k value. If reranking is not enabled, the user specified k results is returned from the retrieval step.
• Adds Zarf configs to enable dev mode.

FlashRank Evals

• Rerank (ms-marco-TinyBERT-L-2-v2) + 100 top-k retrieval

Final Results:
INFO:root:Average Needle in a Haystack (NIAH) Retrieval: 1.0
INFO:root:Average Needle in a Haystack (NIAH) Response: 1.0
INFO:root:Average Correctness (GEval): 0.812
INFO:root:Average Answer Relevancy: 0.9590000000000001
INFO:root:Average Annotation Relevancy: 0.92
INFO:root:MMLU: 0.697979797979798
INFO:root:HumanEval: 0.96
INFO:root:Eval Execution Runtime (seconds): 3122.107043981552

• Rerank (ms-marco-TinyBERT-L-2-v2) + 10 top-k retrieval

Final Results:
INFO:root:Average Needle in a Haystack (NIAH) Retrieval: 1.0
INFO:root:Average Needle in a Haystack (NIAH) Response: 1.0
INFO:root:Average Correctness (GEval): 0.8140000000000001
INFO:root:Average Answer Relevancy: 0.9470000000000001
INFO:root:Average Annotation Relevancy: 0.92
INFO:root:MMLU: 0.697979797979798
INFO:root:HumanEval: 0.96
INFO:root:Eval Execution Runtime (seconds): 3119.72075009346

• No Rerank

Final Results:
INFO:root:Average Needle in a Haystack (NIAH) Retrieval: 1.0
INFO:root:Average Needle in a Haystack (NIAH) Response: 1.0
INFO:root:Average Correctness (GEval): 0.8299999999999998
INFO:root:Average Answer Relevancy: 0.9555128205128205
INFO:root:Average Annotation Relevancy: 0.92
INFO:root:MMLU: 0.697979797979798
INFO:root:HumanEval: 0.96
INFO:root:Eval Execution Runtime (seconds): 3112.184923171997

• Baseline - No Rerank

Final Results:
INFO:root:Average Needle in a Haystack (NIAH) Retrieval: 1.0
INFO:root:Average Needle in a Haystack (NIAH) Response: 1.0
INFO:root:Average Correctness (GEval): 0.81
INFO:root:Average Answer Relevancy: 0.9616666666666666
INFO:root:Average Annotation Relevancy: 0.92
INFO:root:MMLU: 0.695959595959596
INFO:root:HumanEval: 0.96
INFO:root:Eval Execution Runtime (seconds): 1961.5159051418304

Checklist before merging

• Tests, documentation, ADR added or updated as needed
• Followed the Contributor Guide Steps

[netlify]

✅ Deploy Preview for leapfrogai-docs canceled.

Name	Link
🔨 Latest commit	01e77df
🔍 Latest deploy log	https://app.netlify.com/sites/leapfrogai-docs/deploys/66fc411a06886c000814d563

[jalling97]

The Annotation Relevancy metric is what I was initially concerned with when I saw the results. I would have expected it to change (in either direction) after reranking was implemented. However, from what I can tell, the top_k chunks (with or without reranking) correspond to the same documents either way, resulting in no practical difference in annotations. Since the Annotation Relevancy metric isn't concerned with the order of the documents (just that the ones you expect are in the list of annotations), the results are pretty much unchanged.

This is good to note, as the next stage of evals will incorporate the chunk data thanks to #1164, so we can better evaluate the rank of the chunks themselves.

Overall I think this is a net gain and we'll have to see with the next wave of evals what practical difference the reranker has.