feat(api): openai compliant annotations with additional metadata

[gphorvath]

• Adds vector_content ids to metadata returned by runs endpoints
• Adds an endpoint for querying vector_content to leapfrogai namespace - which contains chunk text

[netlify]

✅ Deploy Preview for leapfrogai-docs canceled.

Name	Link
🔨 Latest commit	bacbd58
🔍 Latest deploy log	https://app.netlify.com/sites/leapfrogai-docs/deploys/66f6d6f93af5680008d5990f

[alekst23]

The annotations were not quite working with a simplistic test file that just said "Testing", so I had to augment it a little with some text data, and added a correlated question to the test file.

Perhaps there's better data we can use for this purpose.

[alekst23]

I created a conformance test method that will evaluate the same feature in a separate call to OpenAI and LeapfrogAI:
test_conf_chat_files.py > test_file_annotations

This method tries to evaluate Annotations returned from an Assistant query, but will inherently also evaluate the features that build the Assistant in the process:

• Create VectorStore
• Create Thread
• Create and Poll Runs
• Create Assistant

We should set these up as separate tests on their own to isolate errors.

Presently, the OpenAI call succeeds and returns the proper type structure.

LeapfrogAI call fails with a number of issues related to the structure of the message object in the parameter:

• id, missing
• content, must be list
• created_at, missing
• object, missing
• status, missing
• thread_id, missing

This fails on call to client.beta.threads.create with response:

"POST /openai/v1/threads HTTP/1.1" 422 Unprocessable Entity 

[alekst23]

I have moved conformance testing for individual components to another PR and removed the code from here.

What's left should be a more minimal test focusing on conformance of file annotations.

Some issues that were identified via testing:

• threads.runs.create_and_poll - This should only attach messages to the run that were generated by the run itself, not any previous messages.
• OpenAI annotations format is very simplistic and looks like this: 【4:0†source】
• LeapfrogAI annotations format is this: [file-uuid]

[alekst23]

TODO:
Reformat annotations:
src/leapfrogai_api/backend/converters.py:57

[gphorvath]

Closing in favor of:

#1165
#1164