Editor agnostic configuration

[erikjuhani]

Is your feature request related to a problem? Please describe.
Currently, Contextive heavily relies on the .vscode folder to exist in order to manage the path location for Contextive definitions files. However, this approach does not work well, if at all, with other editors.

Describe the solution you'd like
I think to solve this, one must approach this problem in an agnostic way. One option could be to have a separate contextive.json file stored in the repository root. This file would contain the behavior-altering configurations, such as where to look for the definitions path for the Contextive language server. Instead of setting up language server client settings, the Contextive configuration would be handled separately.

Additional context
It may not be feasible to configure this within the language server client options, considering that many people use editors that don't rely on the .vscode directory. Moreover, it becomes impractical when working across multiple repositories with different path locations. In practical terms, this would require me to adjust my editor configuration every time I switch to a repository with a distinct Contextive path location.

[chrissimon-au]

Hi @erikjuhani - in theory, the language server should already be agnostic - it does rely on the LSP configuration methods, which may have mixed support in different IDEs. Specifically, https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#workspace_configuration.

The intention is that the setting is stored in the repository using whatever the standard IDE's configuration mechanism is. In VSCode it's a settings.json file in the .vscode folder, but in other IDEs it could be in another file - as long as the IDE supports storing a level of config in the repo it should work.

e.g. in neovim, as far as I can tell according to https://neovim.io/doc/user/lsp.html#vim.lsp.start_client(), there is a parameter called settings that:

settings: Map with language server specific settings. These are returned to the language server if requested via workspace/configuration. Keys are case-sensitive.

When working across different repositories, each repository should have it's own configuration pointing to the location of the file in that repository.

One issue with this approach is that if there are a mix of IDEs being used within a team, then each IDE would have it's own config storage... but I'm not sure yet how common a scenario that would be, or how bothersome teams would find it given that all other LSPs use this same approach for config.

Hope that clarifies the intended approach?

[erikjuhani]

Thanks @chrissimon-au for clarification!

In our environment, we have a diverse group of individuals using various IDEs, including both those we are familiar with and those we may not be aware of. The current approach necessitates configuring each repository in advance, requiring us to be well-informed about the editors developers are using and create multiple configurations for each editor, depending on their support. This can pose a significant challenge, particularly in larger mono-repositories with over 40 developers working on them.

I believe that it would be more efficient if we adopt a separation of concerns approach. Many other tools manage their configurations through runtime config files, as an example eslint.

[erikjuhani]

The approach of utilizing a separate configuration file aligns with the Project Local Config solution. This method is generally more developer-friendly, as the alternative options introduce additional tooling or personal settings configuration.

[chrissimon-au]

Hi @erikjuhani , ok, I can see the rationale now. That Project Local Config page seems to offer 4 options, one of which is Use native server file configs, which is the approach you're requesting here, right?

If I understand correctly the other options don't align well with the goal of having the settings stored in the repo and having them apply without needing to do anything special for each developer with respect to how neovim manages settings?

And I agree, a native server file config would help with teams that are not all using the same tooling.

However, one of the biggest challenges with having a 'well-known file location' for a config file is you do have to support a number of options for finding it - e.g. in a multi-root workspace, which workspace should hold the file? You have to scan all workspaces? If each workspace has it's own file, that works fine because you could apply the settings per workspace, but if only one workspace has the file, is it intuitive that those settings should apply across all workspaces?

The LSP configuration mechanism includes a scope field that delegates the resolution of scope and settings location to the IDE to manage, as it is responsible for the multi-root workspace management.

There is some discussion on #38 about this challenge.

This whole conversation has further made me even question of having the path setting... if we have a fixed known location for a settings file, why not just make the definitions file in a fixed known location and remove all the configuration complexity? At this time there are no other settings, and this extra indirection wouldn't add much value other than allowing you to put the definitions file in an arbitrary location (but at the expense of requiring a separate settings file in a fixed known location)...

I'll continue to ponder this. I'm also currently working on the intellij platform extension, so I'll investigate further how that manages config to ensure we have a solution that works across all IDEs.

[erikjuhani]

Hi @erikjuhani , ok, I can see the rationale now. That Project Local Config page seems to offer 4 options, one of which is Use native server file configs, which is the approach you're requesting here, right?

Yes exactly. The first option would be the most favorable out of the other ones presented there.

If I understand correctly the other options don't align well with the goal of having the settings stored in the repo and having them apply without needing to do anything special for each developer with respect to how neovim manages settings?

Yeah that is the problem. It's not really feasible to manage multiple editors / IDE for every repository separately. Instead it would be easier and more developer friendly to just have a "contextive" specific runtime configuration file.

And I agree, a native server file config would help with teams that are not all using the same tooling.

However, one of the biggest challenges with having a 'well-known file location' for a config file is you do have to support a number of options for finding it - e.g. in a multi-root workspace, which workspace should hold the file? You have to scan all workspaces? If each workspace has it's own file, that works fine because you could apply the settings per workspace, but if only one workspace has the file, is it intuitive that those settings should apply across all workspaces?

Yes it has challenges for sure, but I don't think it would add that much complexity. For exampleeslint prioritizes the closest runtime configuration file and uses "extends" to merge configurations.

So how it would work in theory:

1. Check if contextive.json is found in the current working directory

• If yes stop and read configuration file

2. Traverse directory upwards until next contextive.json is found and repeat step 1
3. Stop traversal when root uri is hit

The LSP configuration mechanism includes a scope field that delegates the resolution of scope and settings location to the IDE to manage, as it is responsible for the multi-root workspace management.

There is some discussion on #38 about this challenge.

This whole conversation has further made me even question of having the path setting... if we have a fixed known location for a settings file, why not just make the definitions file in a fixed known location and remove all the configuration complexity? At this time there are no other settings, and this extra indirection wouldn't add much value other than allowing you to put the definitions file in an arbitrary location (but at the expense of requiring a separate settings file in a fixed known location)...

I'll continue to ponder this. I'm also currently working on the intellij platform extension, so I'll investigate further how that manages config to ensure we have a solution that works across all IDEs.

How would it work if the project is using some centralized contextive definitions solution and want to change the "default" location e.g. previously using the path setting? I think it's a great thing to have configurability as different projects have very different needs.

[chrissimon-au]

@erikjuhani, thanks again for this suggestion - having thought about this further, I do like this idea of a file in each folder that effectively contains a 'pointer' to the definitions file that applies to files in that folder (and subfolder).

However, we need to consider it carefully in relation to a variety of bounded context <-> repository/disk mapping layouts that exist and the current definitions.yml file paths setting. These are:

1. Monorepo - all contexts in a single repo, usually separated by paths, but some paths needing to using definitions from multiple contexts
2. Repo per context
3. Multiple Repos per context

Current Approach Recap

Repo per context is probably the 'recommended' approach', but I do also like to support the monorepo approach as I think it's very valid in many scenarios. Option 3 is strongly not recommended but is unfortunately very common.

The current definitions file includes a list of contexts, and each context has a list of paths it applies to. This is to support option 1. Because the definition of which paths each context applies to is independent, multiple contexts can be easily applied to the same path (which supports the overlapping contexts scenario).

Option 2 is generally applied by just having a single context in the list, and each repo gets its own definitions file.

Option 3 is a struggle - I don't like the idea of encoding in one repo a relative or fixed path of a definitions file outside the repo - it presupposes an unmanaged disk layout that could be different on every machine.

So, the current support is via the suggestion that the canonical file is in a repo that houses a shared library or some common code that is distributed to the code in other repos via packages. The definitions file should be included in the package and the consuming repos can configure their settings to indicate the location of the definitions file in the local packages folder (e.g. node_modules, or nuget packages or whatever). This is the purpose of the go example repository, to illustrate this pattern.

Settings file in subfolders

Considering option 1, the monorepo, if we had a contextive.json file in an arbitrary folder, with a setting pointing at a definitions file, how should this work when the definitions file has multiple contexts in it?

Should we still only use contexts that have a paths field that includes the current path? This could get very confusing if contextive.json pointed at a definitions file where none of the contexts included the current path.

Should the fact that a contextive.json file points at that definitions file make it implicity part of the paths setting? If so, for which contexts in the file? Is it only meaningful for a single-context definitions file?

I'm kind of ok with this - we could move towards a structure where each context is in a separate file, and the contextive.json is present in the folders that point to that file. This would make it even simpler for option 2, as there would only ever be a single context in a file.

How then to handle overlapping contexts? perhaps contextive.json needs to contain a list of files, not just a single file.

Considering option 2, in theory every file in the repo should have the same definitions file - so there is limited value in allowing contextive.json to exist in subfolders, as there should be no subfolders that point to different definitions files.

Considering option 3, again, if the context is distributed across multiple repos, at least a single repo should belong to just that context - so in theory, limited value in allowing contextive.json to exist in subfolders.

Next steps

Given the above, it's not yet clear to me what will be the simplest approach. Now that we can offer a custom definitions file location with neovim (see #58), I think this buys some time to consider the correct approach here.

So for now, I won't be making any immediate changes, but will continue to consider how to better handle the scenario where teams have mixed tooling/IDEs but need consistent configuration and may have a custom definitions file location. Will leave this issue open until further decisions made.

[chrissimon-au]

Thinking further about the above, I'd like to propose the following approach:

• Any folder can contain one or more %contextName%.contextive.yml definitions files which will contain only one context
• The definitions automatically apply to any file in that folder or subfolders
• If there is another *.contextive.yml file in a parent folder, then they merge (or the child-most folder takes precedence? TBD)
• The file may include an import statement with the location of another definitions file - absolute, relative or even URL path

Examples

MonoRepo

.
├── ContextA/
│   ├── src/
│   └── contextA.contextive.yml
├── ContextB/
│   ├── src/
│   │   ├── antiCorruptionLayer/
│   │   │   ├── aclWithContextA.cs
│   │   │   └── importContextA.contextive.yml # contains an import with relative path to ../../../ContextA/contextA.contextive.yml
│   │   └── file.cs
│   └── contextB.contextive.yml
└── README.md

Repo per Context

ContextA repo

.
├── src/
├── contextA.contextive.yml
└── README.md

ContextB repo

.
├── src/
│   ├── antiCorruptionLayer/
│   │   ├── aclWithContextA.cs
|   │   └── importContextA.contextive.yml # contains an import with url to hosted contextA definitions
│   └── file.cs
├── contextB.contextive.yml
└── README.md

Distributed Context

Defintions Repo - contents need to get published to a url somehow

.
├── contextA.contextive.yml
└── README.md

Component 1 in Context A repo

.
├── src/
├── contextA.contextive.yml # contains an import to the url that the definitions are published to
└── README.md

Component 2 in Context A repo

.
├── src/
├── contextA.contextive.yml # contains an import to the url that the definitions are published to
└── README.md

Backwards compatibility

Contextive is currently major version 1.

I'd propose to implement this within major version 1 but retain support for existing .contextive/definitions.yml file as the priority, with a fallback to looking for files that look like the above approach if that file can't be found.

I'd also propose an automated approach to migrate the old definitions file structure to the new one, which should be fairly straight forward for the monorepo and repo-per context approach, but will likely need to be tackled manually for the distributed context approach.

We'd remove support for the original configuration approach with a major version bump to version 2.

Feedback welcome

Feedback welcome from any interested parties :)