Managing OpenAI usage and costs

[ChristianIvicevic]

Greetings,

out of curiosity I tried out gptcommit yesterday for the first time with three commits. Two of them were a bit larger with 265 additions and 214 deletions according to git. Much to my surprise there were two dozen requests and got billed for $0.55. I'd like to ask whether it's expected that the amount of requests (and thus the price) is that high or whether this might indicate a bug?

[zurawiki]

Hey @ChristianIvicevic , you bring up a great point here.

In the current design, gptcommit issues N+2 prompts, where N is the number of modified files with diffs under the max_token_limit. The other prompts are the title and summary.

OpenAI Completions are billed by "tokens" that are both sent and generated. Pricing per token depends on the model used. The number of tokens generated are generally predictable (as a commit message is usually only so big) but gptcommit could be sending over a lot of tokens in the form of diff data.

Today, I see two low-hanging solutions for reducing cost:

• Switch to a different model using the openai.model configuration option
• Reduce the side of prompts and diff data sent to OpenAI

OpenAI's pricing page can be found at
https://openai.com/api/pricing/#faq-completions-pricing

[ChristianIvicevic]

Thanks for the summary. I might have a proposal that may be an option to take into consideration. As of right now I can technically create a patch for a specific commit and without its subject line ask ChatGPT about a descriptive commit message. Unlike gptcommit it would not summarize the changes of each touched file, but it's basically a single sentence or paragraph based on a patch file.

Given that GPT can't explain the reasoning behind certain changes summarizing the details of each file feels less valuable so I'd like to know whether the described approach with less output might be a feature of gptcommit to condense output and maybe reduce costs.

[zurawiki]

Contributions are very much welcome! Feel free to a pull request!

[zcf0508]

https://github.com/markuswt/gpt-commit/blob/f2a8a239bcc7cac36fb73dcd817d5a56782f7d98/gpt-commit.py#L8-L9

This repo uses a shorter prompt.

[zurawiki]

Given the new Chat GPT models and cheaper pricing, is this issue still relevant?