Release 3.0.0rc1
Pre-releaseRelease 3.0.0rc1
We're excited to announce the release candidate of Prefect 3.0. It's the most flexible, powerful, fastest version of Prefect yet. Prefect 3.0 includes several exciting new features. Install it by running pip install prefect==3.0.0rc1
and check out the docs here.
Run tasks independently of flows
You can now run and serve tasks outside of flows and inside of other tasks.
from prefect import task
@task
def my_background_task(name: str):
print(f"Hello, {name}!")
if __name__ == "__main__":
my_background_task.delay("ford")
Transactional semantics
Use rollback and commit hooks to facilitate idempotent python code.
from prefect import flow, task
from prefect.transactions import transaction
@task
def first_task():
print('first')
@first_task.on_rollback
def roll(txn):
print('rolling back')
@task
def second_task():
raise RuntimeError("oopsie")
@flow
def txn_flow():
with transaction():
first_task()
second_task()
if __name__ == "__main__":
txn_flow()
Open source Events and Automations
Trigger actions, such as sending notifications, pausing schedules, starting flow runs and more in response to Prefect events.
More flexible variables and new artifact types
Variables can now be any JSON compatible type including dicts, lists, and integers. Progress and Image artifacts make it easy to add visual annotations to your flow run graph.
Faster and richer CLI
Improved CLI speed and several added commands and conveniences.
Updated navigation, styling, and interaction design
The new Runs page displays both flow and task run information, and an improved sidebar and switcher makes navigating Prefect simpler than ever.
Enhancements
- Create artifact for unsuccessful dbt task runs — #13348
- Add filter on
task_run.expected_start_time
— #13491 - Add utilities to serialize context to a dictionary and hydrate context from a dictionary — #13529
- Add API endpoints for deployment count and next flow run — #13544
- Allow flow parameter schema generation when dependencies are missing — #13315
- Change the default value for
enforce_parameter_schema
fromFalse
toTrue
— #13594 - Migrate schemas to pydantic v2 — #13574
- Removes block auto-instrumentation — #13407
- Migrate all uses of the banned characters validation to a self-validator — #13370
- Ignore and warn on unrecognized settings - #13624
Fixes
- Remove unnecessary flow run infrastructure override access checks — #13401
- Enforce False case when flow run id is null — #13464
- Fix workspace variable hydration to allow for JSON data — #13548
- Remove unused settings/experimental work pool flags:
PREFECT_EXPERIMENTAL_ENABLE_WORK_POOLS
andPREFECT_EXPERIMENTAL_WARN_WORK_POOLS
— #13144 - Pin
pydantic>=2.7
forSecret
— #13613 - Skip on cancellation hooks if runner can't load flow — #13660
- Refactor lazy imports to avoid accidental eager imports — #13296
- Allow block registration to use client schemas for server model creation — #13602
- Replace our customized
Duration
types with plaintimedelta
s — #13603
Experimental
- Add
prefect.yaml
and cli support for new schedule fields — #13318
Documentation
- Transition documentation hosting from Netlify to Mintlify — #13634
- Add Python 3.12 to list of Docker images — #13321
- Update
index.md
— #13353 - Improve tutorial section — #13297
- Fix jinja template in automations doc — #13422
- Update development section docs — #13247
- Update Ray integration docs — #13467
- Update Variables docs to include JSON types — #13493
- Update quickstart guide for usability — #13562
- Remove
deployments-block-based
concept page and refs for 3.0 — #13626 - Remove
infrastructure
concept page and refs for 3.0 — #13629 - Update docs image paths and remove outdated images — #13666
- Remove references to
prefect.software
from docs — #13382 - Update
host.md
— #13351 - Simplify rate limits page — #13689
- Removing references to deprecated block types and add disclaimer — #13651
- Update guides — #13253
- Remove
storage
concept page and refs - #13630
Integrations
- Migrate
prefect-dbt
to pydantic v2 - #13718 - Migrate
prefect-email
to pydantic v2 — #13654 - Migrate
prefect-slack
to pydantic v2 — #13673 - Migrate
prefect-shell
to pydantic v2 — #13675 - Migrate
prefect-gcp
to pydantic v2 — #13650 - Migrate
prefect-github
to pydantic v2 — #13655 - Migrate
prefect-gitlab
to pydantic v2 — #13656 - Migrate
prefect-docker
to pydantic v2 - #13697 - Migrate
prefect-sqlalchemy
to pydantic v2 - #13700 - Add
PrefectDistributedClient
toprefect-dask
— #13537 - Update
RayTaskRunner
for compatibility with new engine — #13575 - Update
DaskTaskRunner
for compatibility with the updated engine — #13555 - prefect-dbt artifact consolidation and markdown fixes — #13379
- prefect-dbt - Cause unsuccessful dbt tasks to fail — #13405
- DBT Tasks extra_command_args Fix — #13308
- Update dbt-core dependency — #13394
Breaking Changes
- Remove
prefect deployment build
CLI frommain
— #13366 - Remove
prefect agent
CLI frommain
— #13365 - Remove
prefect deployment apply
CLI frommain
— #13367 - Remove
PrefectAgent
class — #13374 - Remove
prefect.software
— #13375 - Remove
deployments
module — #13373 - Remove
EcsTask
frommain
— #13417 - Remove
AzureContainerInstanceJob
frommain
— #13418 - Remove
VertexAICustomTrainingJob
frommain
— #13419 - Remove
CloudRunJob
frommain
— #13420 - Remove infrastructure blocks from
main
— #13424 - Remove
Infrastructure
,BlockWorker
frommain
— #13430 - Remove deprecated storage blocks from
main
— #13410 - Remove
prefect-agent
as a possible work pool type — #13444 - Remove old engine — #13542
- Remove Python 3.8 support — #13331
- Remove
deprecated
module and its references — #13345 - Remove old task runners and futures modules — #13593
- Remove
is_state
— #13569 - Remove deprecated options from
prefect work-queue
and refs to agents - #13638
Contributors
All changes: 2.19.0...3.0.0rc1
See release notes for details.