Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Config file #162

Draft
wants to merge 45 commits into
base: master
Choose a base branch
from
Draft

Config file #162

wants to merge 45 commits into from

Conversation

burnout87
Copy link
Contributor

No description provided.

@burnout87 burnout87 linked an issue Apr 10, 2024 that may be closed by this pull request
@burnout87
Copy link
Contributor Author

burnout87 commented Apr 10, 2024

With this PR the following changes are introduced:

  • config using FlaskDynaconf and passing those to the NotebookAdapter init, the file settings.toml is used to load the config
  • check the file size before download

The config values for the init of the NotebookAdapter are currently passed only within the service module, do they have to be passed also elsewhere (eg ontology) @dsavchenko @volodymyrss ?

Copy link
Member

@dsavchenko dsavchenko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The config file would be useful also to

nb2workflow/__init__.py Outdated Show resolved Hide resolved
@burnout87
Copy link
Contributor Author

* set ontology path (it's used in deploy now, but will be needed in context of [Support for multiline parameter definitions. Basic type annotations. #155](https://github.com/oda-hub/nb2workflow/pull/155) to get parameter type from annotation)

I now extract the path from the toml file for the deploy module, I guess the analogous approach should be implemented within galaxy ?

Copy link
Member

@dsavchenko dsavchenko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some suggestions on config structure.

Also, it would be good to have an ability to set the path to the config file in cli commands.
As before, I suggest using --settings for file path and rename the arg used to set config options directly from cli (and add it to other cli then). Alternatively, --settings_path is aalso ok

nb2workflow/__init__.py Outdated Show resolved Hide resolved
settings.toml Outdated
Comment on lines 4 to 8
max_download_size = 1000000000
n_download_max_tries = 10
download_retry_sleep = 0.5
sentry_url = "https://63ae106793010d836c74830fa75b300c@o264756.ingest.sentry.io/4506186624335872"
ontology_path = "https://odahub.io/ontology/ontology.ttl"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These configs belong not only to service module. Probably better to have some separate blocks, e.g. [global] and [service] ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so, basically only host an port are specific for the service?

settings.toml Outdated
@@ -0,0 +1,8 @@
[default.service]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why "default" in the block name? Does it have any special meaning?

Copy link
Contributor Author

@burnout87 burnout87 Apr 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is needed when using the FlaskDynaConf, extension of DynaConf specific for Flask

https://www.dynaconf.com/flask/#settings-files

It needs to have an environment associated to that configuration, and default is the default

@burnout87
Copy link
Contributor Author

Some suggestions on config structure.

Alternatively, --settings_path is aalso ok

This would be mainly used for the service, correct?

@dsavchenko
Copy link
Member

This would be mainly used for the service, correct?

Mainly used: yes. But we need the ability to set the path to the config file in the whatever module that uses this config

@burnout87
Copy link
Contributor Author

The number of changes to be introduced with the config file are far more than what I expected, and requires some more discussions.

I am going to split this PR in the two, where the first only tackles the "max size download" matter.

@burnout87 burnout87 changed the title Config file and max size download Config file May 1, 2024
@volodymyrss
Copy link
Member

@burnout87 it's still waiting in requested changes. Do you want to complete this? If it is still needed?

@burnout87 burnout87 marked this pull request as draft July 29, 2024 14:11
@burnout87
Copy link
Contributor Author

@burnout87 it's still waiting in requested changes. Do you want to complete this? If it is still needed?

It requires a number of changes that should perhaps be discussed. Also, it'll take some time, so it has to be decided how much priority this actually has.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants