Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow to override endpoints by configuration #2711

Open
2 tasks done
rmax opened this issue Oct 28, 2024 · 2 comments
Open
2 tasks done

Allow to override endpoints by configuration #2711

rmax opened this issue Oct 28, 2024 · 2 comments
Labels
area:configuration Relates to configuration options kind:enhancement Indicates a new feature request, imrovement, or extension

Comments

@rmax
Copy link

rmax commented Oct 28, 2024

Validations

  • I believe this is a way to improve. I'll try to join the Continue Discord for questions
  • I'm not able to find an open issue that requests the same enhancement

Problem

I’m trying to use CodeLlama deployed on databricks but the endpoint is /invocations rather than /completions, using OpenAI-compatible provided.

Solution

It would be useful if we could override the completion endpoint by configuration.

@dosubot dosubot bot added area:configuration Relates to configuration options kind:enhancement Indicates a new feature request, imrovement, or extension labels Oct 28, 2024
@RomneyDa RomneyDa added the needs-triage Waiting to be triaged label Oct 31, 2024
@RomneyDa
Copy link
Collaborator

RomneyDa commented Nov 1, 2024

Comment from duplicate #2763
https://discord.com/channels/1108621136150929458/1300903286060220469

Can't currently do fully custom endpoints that aren't compatible with e.g. openai, ollama without adding a provider in the codebase.
e.g. company.com/embeddings, company.com/conversation

baseUrl and apiUrl are nice but assume the endpoints match with e.g. ollama or openai

Solution
Add chatUrl, embeddingsUrl etc. or some other paradigm that would allow this

@RomneyDa RomneyDa assigned RomneyDa and unassigned Patrick-Erichsen Nov 1, 2024
@RomneyDa
Copy link
Collaborator

RomneyDa commented Nov 1, 2024

Falls under meta llm-info + config #2769

@RomneyDa RomneyDa removed the needs-triage Waiting to be triaged label Nov 1, 2024
@RomneyDa RomneyDa removed their assignment Nov 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:configuration Relates to configuration options kind:enhancement Indicates a new feature request, imrovement, or extension
Projects
None yet
Development

No branches or pull requests

3 participants