Skip to content

Commit

Permalink
ENG-428: Support flow.username for both live and batch modes (#1175)
Browse files Browse the repository at this point in the history
  • Loading branch information
asvetlov authored Oct 23, 2024
1 parent 64c04af commit 909d5aa
Show file tree
Hide file tree
Showing 8 changed files with 31 additions and 18 deletions.
4 changes: 4 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,10 @@ repos:
# hooks:
# - id: pyupgrade
# args: ['--py37-plus']
- repo: https://github.com/twisted/towncrier
rev: 23.11.0 # run 'pre-commit autoupdate' to update
hooks:
- id: towncrier-check
- repo: https://github.com/PyCQA/flake8
rev: '7.1.0'
hooks:
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.D/1175.feature.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Provide `flow.username` in both *live* and *batch* modes.
36 changes: 18 additions & 18 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,14 +32,14 @@ No significant changes.

```
images:
image_a:
ref: image:imagea
context: dir
dockerfile: dir/Dockerfile
extra_kaniko_args: >-
--reproducible
--cache-ttl=1h
--single-snapshot
image_a:
ref: image:imagea
context: dir
dockerfile: dir/Dockerfile
extra_kaniko_args: >-
--reproducible
--cache-ttl=1h
--single-snapshot
```

More details on available arguments could be found in [official Kaniko documentation](https://github.com/GoogleContainerTools/kaniko?tab=readme-ov-file#additional-flags). ([#1110](https://github.com/neuro-inc/neuro-flow/issues/1110))
Expand All @@ -65,7 +65,7 @@ No significant changes.
kind: batch
life_span: 30d
tasks:
...
...
``` ([#1075](https://github.com/neuro-inc/neuro-flow/issues/1075))
Expand Down Expand Up @@ -97,7 +97,7 @@ Features
- Added `hash_files_relative` function to expression, it works same as `hash_files` but requires additional leading
parameters that defines directory to glob over. It can be used to glob over action files:
```
${{ hash_files_relative(flow.action_path, "**/pattern/here/**/*.py", "other/**/pattern")
${{ hash_files_relative(flow.action_path, "**/pattern/here/**/*.py", "other/**/pattern")
``` ([#904](https://github.com/neuro-inc/neuro-flow/issues/904))
Expand Down Expand Up @@ -188,16 +188,16 @@ Features
```
matrix:
old_way_key: ["1", "2", "3"]
new_way_key: ${{ ["1", "2", "3"] }}
old_way_key: ["1", "2", "3"]
new_way_key: ${{ ["1", "2", "3"] }}
```
This can be helpful when used together with new `range()` function:
```
matrix:
old_way_key: [0, 1, 2, 3]
new_way_key: ${{ range(4) }}
old_way_key: [0, 1, 2, 3]
new_way_key: ${{ range(4) }}
```
The `range()` function supports same parameters as python's `range()`, but it returns list.
Expand All @@ -209,16 +209,16 @@ Features
```
matrix:
old_way_key: ["k1", "k2", "k3"]
new_way_key: ${{ [fmt("k{}", str(it)) for it in range(1, 4)] }}
old_way_key: ["k1", "k2", "k3"]
new_way_key: ${{ [fmt("k{}", str(it)) for it in range(1, 4)] }}
```
You can also filter some values in comprehension same way as in python:
```
matrix:
old_way_key: [0, 4, 16]
new_way_key: ${{ [it * it for it in range(1, 5) if it % 2 == 0] }}
old_way_key: [0, 4, 16]
new_way_key: ${{ [it * it for it in range(1, 5) if it % 2 == 0] }}
``` ([#741](https://github.com/neuro-inc/neuro-flow/issues/741))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ The `flow`context contains information about the workflow: its ID, title, etc.
| `flow.project_id` | `str` | The project's ID. It is automatically generated based on the name of the flow folder. You can override it using [`project.id`](../project-configuration-syntax.md#id) attribute. Check [the project configuration](../project-configuration-syntax.md) for details. |
| `flow.workspace` | `LocalPath` | A path to the workspace (the root folder of the flow). |
| `flow.title` | `str` | The workflow's title. Set the [`flow.title`](./#title) attribute to override the auto-calculated value. |
| `flow.username` | `str` | Name of user who executed the workflow run. |

### `project` context

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ The `flow` context contains information about the workflow: its id, title, etc.
| `flow.project_id` | `str` | The project's ID. It is automatically generated based on the name of the flow folder. You can override it using [`project.id`](../project-configuration-syntax.md#id) attribute. Check [the project configuration](../project-configuration-syntax.md) for details. |
| `flow.workspace` | `LocalPath` | A path to the workspace (the root folder of the flow). |
| `flow.title` | `str` | The workflow title. Set the [`flow.title`](./#title) attribute to override the auto-generated value. |
| `flow.username` | `str` | Name of user who executed the workflow run. |

### `project` context

Expand Down
4 changes: 4 additions & 0 deletions src/apolo_flow/context.py
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,7 @@ class FlowCtx:
project_id: str
workspace: LocalPath
title: str
username: str

@property
def id(self) -> str:
Expand All @@ -133,6 +134,7 @@ def with_action(self, action_path: LocalPath) -> "ActionFlowCtx":
project_id=self.project_id,
workspace=self.workspace,
title=self.title,
username=self.username,
action_path=action_path,
)

Expand Down Expand Up @@ -792,6 +794,7 @@ async def setup_flow_ctx(
project_id=project.id,
workspace=config_loader.workspace,
title=flow_title or flow_id,
username=config_loader.client.config.username,
)


Expand All @@ -809,6 +812,7 @@ async def setup_batch_flow_ctx(
project_id=base_flow.project_id,
workspace=base_flow.workspace,
title=base_flow.title,
username=config_loader.client.config.username,
life_span=life_span,
)

Expand Down
1 change: 1 addition & 0 deletions tests/unit/test_context.py
Original file line number Diff line number Diff line change
Expand Up @@ -1389,6 +1389,7 @@ async def test_batch_action_path(batch_config_loader: ConfigLoader) -> None:
project_id="unit",
workspace=ANY,
title="batch_action_call",
username=batch_config_loader.client.config.username,
action_path=here,
)

Expand Down
1 change: 1 addition & 0 deletions tests/unit/test_functions.py
Original file line number Diff line number Diff line change
Expand Up @@ -224,6 +224,7 @@ def _factory(
project_id="test",
workspace=assets,
title="unit test flow",
username="username",
),
git=GitCtx(None),
env={},
Expand Down

0 comments on commit 909d5aa

Please sign in to comment.