patterns for collecting providers

[RonnyPfannschmidt]

in a application im developing, plugins bring in new details,
im wondering if there are some common/recommended patterns for providers, or whether i have to create a own one

[Tishka17]

From my own point of view, I recommend following you application structure when splitting providers. So, if you have layers - have a provider for objects in the same layer. If you split app vertically - split providers in same manner.

Having alternative implementations of providers may be usefull for testing/debugging purposes, but you do not need many of them. You do not need container in unit tests (except pytest, of cause), and for integration tests you should have container as much close to production one as it possible.

[RonnyPfannschmidt]

in my particular case the application is split into plugins in a manner matching the structure of other applications - depending on circumstances there's easily between 40 and 60 parts active with a provider each (representing components or global features (each of the plugins represents frontend parts or backend services + additional plugins for shared internals)

[Tishka17]

We also had a discussion about collecting providers based on entrypoints (in the same manner pyetest does) but decided to postpone that feature as there are a lot of questions

[RonnyPfannschmidt]

Dynamic collection and composition of providers is most likely a minefield

Utilities to ease it should be provided in a manner that allows downstreams to make informed decisions

I'd like to explore Ways to integrate provider discovery that allows lazy import of components (sometimes of the dozens of components only one is needed,and importing them all is expensive)

[Tishka17]

Sounds a bit like container which provides more containers. Maybe, with generic providers. I need to think about it, probably we can construct something new here.

Can you, please, provide minimal example with couple of classes and details describing which information (names, components, classes) are you going to avoid in main container and which we can use there.

[RonnyPfannschmidt]

sorry for typos: this is a first type-down (in the web form) of the utilities i believe i need

first a trimmed down lineup of the mess i have at my hand

# base_framework.py

class Application:
   hostname: str
   ...

   def __getattr__(self, key) -> ApplicationPlugin:
      # get or create instance - key is a entrypoint name
     return ...
     
     
     class ApplicationPlugin:
          ...

# some_plugin.py

# registred under the name inventory
class InventoryApplicationPlugin:
     # left out the declaration here
      api = RestService.declare("inventory_openapi_package") # lazy property that provides a namespace with api objects declared in that openapi package
      
      
      def some_helper(self):
          self.api.some_apiname.some_operation()

i want to be able to request components and their apis based on the type/name lazily

# base_framework

# base_framework.py

class Application:
  hostname: str
  _container: Container

  def __getattr__(self, key) -> ApplicationPlugin:
    return self._container.get(ApplicationPlugin. component="key"
    
 class ApplicationPlugin:
         ...

class RestServiceDeclaration(...):
    
  


# some_plugin.py



# registred under the name inventory
class InventoryApplicationPlugin:
    # left out the declaration here
     api = RestService.declare("inventory_openapi_package") # lazy property that provides a namespace with api objects declared in that openapi package
     
     some_api = DeclaredApi(api, "some_apiname") # there must be a better way
     
     def some_helper(self):
         self.some_api.some_operation()

now i want to be able to request application from the di container before and without importing some_plugin, in fact i want to avoid importing some_plugin until after it is requested from the api

same goes for importing the openapi packages - i want to avoid importing them until the api object is actually requested (as impoting them is slow )

[Tishka17]

It is still a bit confusing. Can we reduce the example only to part with scopes and container?

[RonnyPfannschmidt]

im still working on that, the current design is a bit involved and im not yet at a place where i can safely map it