Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

$Dollar symbol in PREFECT_API_DATABASE_CONNECTION_URL setting causes ValueError when resolving default_database_connection_url #11067

Closed
4 tasks done
annshress opened this issue Nov 1, 2023 · 3 comments · Fixed by #14876
Assignees
Labels
bug Something isn't working

Comments

@annshress
Copy link

First check

  • I added a descriptive title to this issue.
  • I used the GitHub search to find a similar issue and didn't find it.
  • I searched the Prefect documentation for this issue.
  • I checked that this issue is related to Prefect and not one of its dependencies.

Bug summary

When prefect's db connection url contains$ followed by non-alphabetic character, it raises ValueError. Because, it tries to use python's string.template.substitute which behaves like so.

Example DB connection url: postgresql+asyncpg://yoda:?0YpAwMy$&adQqcjSERqkqqc6BnS@some-domain.amazonaws.com:5432/prefect2

Reproduction

docker run -p 4200:4200 -it --rm prefecthq/prefect:2-python3.9

# Inside the container

export PREFECT_API_DATABASE_CONNECTION_URL="postgresql+asyncpg://yoda:?0YpAwMy$&adQqcjSERqkqqc6BnS@some-domain.amazonaws.com:5432/prefect2"
prefect server start

Error

Traceback (most recent call last):
  File "/usr/local/bin/prefect", line 5, in <module>
    from prefect.cli import app
  File "/usr/local/lib/python3.9/site-packages/prefect/__init__.py", line 86, in <module>
    prefect.logging.configuration.setup_logging()
  File "/usr/local/lib/python3.9/site-packages/prefect/logging/configuration.py", line 75, in setup_logging
    config = load_logging_config(
  File "/usr/local/lib/python3.9/site-packages/prefect/logging/configuration.py", line 40, in load_logging_config
    {
  File "/usr/local/lib/python3.9/site-packages/prefect/logging/configuration.py", line 43, in <dictcomp>
    if setting.value() is not None
  File "/usr/local/lib/python3.9/site-packages/prefect/settings.py", line 142, in value
    return self.value_from(get_current_settings(), bypass_callback=bypass_callback)
  File "/usr/local/lib/python3.9/site-packages/prefect/settings.py", line 154, in value_from
    value = settings.value_of(self, bypass_callback=bypass_callback)
  File "/usr/local/lib/python3.9/site-packages/prefect/settings.py", line 1372, in value_of
    value = setting.value_callback(self, value)
  File "/usr/local/lib/python3.9/site-packages/prefect/settings.py", line 418, in default_database_connection_url
    return templater(settings, value)
  File "/usr/local/lib/python3.9/site-packages/prefect/settings.py", line 316, in templater
    return original_type(template.substitute(template_values))
  File "/usr/local/lib/python3.9/string.py", line 121, in substitute
    return self.pattern.sub(convert, self.template)
  File "/usr/local/lib/python3.9/string.py", line 118, in convert
    self._invalid(mo)
  File "/usr/local/lib/python3.9/string.py", line 101, in _invalid
    raise ValueError('Invalid placeholder in string: line %d, col %d' %
ValueError: Invalid placeholder in string: line 1, col 35

Versions

Version:             2.13.4
API version:         0.8.4
Python version:      3.9.18
Git commit:          48d9d9a2
Built:               Fri, Sep 29, 2023 11:26 AM
OS/Arch:             linux/aarch64
Profile:             default
Server type:         ephemeral
Server:
  Database:          sqlite
  SQLite version:    3.40.1

Additional context

No response

@annshress annshress added bug Something isn't working needs:triage labels Nov 1, 2023
@desertaxle
Copy link
Member

Thanks for the issue @annshress! We'll look into resolving this, but in the meantime, you can escape the $ character by providing $$ in its place.

@ashrielbrian
Copy link

Is there a timeline for this to get fixed? This has cropped up a few times when using RDS to autogenerate and manage the master password, which occasionally generates a $ in the password string.

@desertaxle
Copy link
Member

@ashrielbrian I have a fix in #14876 if you want to check it out!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants