Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve type hinting and favor statically typed parameters for Modality and Platform #6

Merged
merged 7 commits into from
Jun 21, 2024

Conversation

bruno-f-cruz
Copy link
Collaborator

This PR refactors the ManifestConfig class by improving the type hinting for some of the fields.

This allows easier validation (without the need for custom validators) as well as auto-completion.

@bruno-f-cruz bruno-f-cruz changed the title Improve type hints and favor statically typed parameters for Modality and Platform Improve type hinting and favor statically typed parameters for Modality and Platform May 28, 2024
@bruno-f-cruz bruno-f-cruz linked an issue Jun 3, 2024 that may be closed by this pull request
@bruno-f-cruz bruno-f-cruz changed the base branch from main to development June 3, 2024 17:23
Copy link
Collaborator

@arielleleon arielleleon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bruno-f-cruz Take a look at my comments. From what I understand you cannot type a platform (or modality) attribute as a Platform model type.

@@ -44,7 +44,7 @@ class ManifestConfig(BaseModel):
description="where to send data to on VAST",
title="VAST destination and maybe S3?",
)
modalities: Dict[str, List[str]] = Field(
modalities: Dict[Modality, List[str]] = Field(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change Modality to Modality.ONE_OF. This should allow users to input a string that the Modality model can validate

@@ -30,11 +30,11 @@ class ManifestConfig(BaseModel):
description="Transfer time to schedule copy and upload. If None defaults to trigger the transfer immediately", # noqa
title="APScheduler transfer time",
)
platform: str = Field(description="Platform type", title="Platform type")
platform: Platform = Field(description="Platform type", title="Platform type")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change Platform to Platform.ONE_OF. This should allow users to input a string that the Platform model can validate

@bruno-f-cruz
Copy link
Collaborator Author

bruno-f-cruz commented Jun 12, 2024

Relevant to this problem:

AllenNeuralDynamics/aind-data-schema#960

I think we should code our own wrapper around the aind-data-schema for now that uses the same syntax

class Modality(Enum, str):
    """Modality classes"""

    BEHAVIOR = modaloties.Modality.Behavior().name
    BEHAVIOR_VIDEOS = modaloties.Modality.BehaviorVideos().name

This solves a few problems:

  1. Type hinting for users
  2. easier runtime validation
  3. Will likely be backwards compatible for users if aind-data-schema fixes the bug

It has a few cons:
Still requires a cast from our local type to the aind-data-schema-models type
If the problem in aind-data-schemas is not fixed, we will need to maintain this enumerator and keep adding entries to it.

@arielleleon
Copy link
Collaborator

I will add this wrapper to another issue. Does that work?

@arielleleon arielleleon merged commit ea37cff into development Jun 21, 2024
@arielleleon arielleleon deleted the feat-statically-type-platform-and-modality branch June 21, 2024 19:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add platform and modality objects to manifest config
2 participants