Allow filesystem registry from git repository

[bansan85]

Just set "kind" to "filesystem-from-git" and use the same parameters of "git".

            "kind": "filesystem-from-git",
            "baseline": "1603f26e9c52e8fe9dcee9266d939aa14e010221",
            "reference": "main",
            "repository": "https://git.example.com/repository.git",

What do you think about this feature ?

[vicroms]

Hi @bansan85

Can you describe what motivated this feature?

I don't see how this improves on just doing git clone and pointing a filesystem registry at the cloned instance. But I can see this feature creating confusion for would-be-users.

[bansan85]

Sure.

I have a project. It has a few external dependencies (vtk, hdf5, Qt, etc...) and internal ones (requires our own registries, one is public for our open source libraries, the other is private for our proprietary libraries).

I'm trying to convince my colleagues to migrate from an old-fashioned CMake (an ExternalProject_Add for each external dependency and a submodule for each internal dependency) to vcpkg.

Gradually, I migrated from ExternalProject_Add to vcpkg ports and created ports for our internal dependencies. I centralized everything in a vcpkg-configuration.json file for ease of use of vcpkg.

I like the “git” registry because there's no need to clone manually and you can specify the commit precisely (although I don't like the fact that you have to specify a branch, which seems to me to be a duplication and causes problems cf. microsoft/vcpkg#42300). On the other hand, the procedure for modifying commits is a bit laborious (“git add files”, “git commit”, “vcpkg format-manifest”, “vcpkg x-add-version”, “git add versions”, “git commit”, “git push”). My colleagues don't want to spend months like me to master vcpkg.

I like filesystem registries because modifying a port consists in duplicating a folder, incrementing the version and updating the version/baseline.json / package.json files. No need for complicated git commands. But it does require manual cloning of our two registries. My colleagues don't understand the point of deleting submodules (our internal dependencies) and replacing them with others (our registries).

My “filesystem-from-git” has the advantage of both: automatic cloning and easy updating.

Yes, it may add confusion (a better name perhaps?). But I think that “git” for "registry from a git repository" and “filesystem” for "registry from a folder" are already confusing. It doesn't indicate that the architecture of the “ports” folders is not the same without having read the documentation.

[vicroms]

You can do that now without needing a new feature.

Add your filesystem registry's repository as a submodule on your project and then have the vcpkg-configuration.json point to the submodule directory using a relative path.

[bansan85]

But as I said, I'd rather not add submodule in project.

I have about ten repositories. One is the main projet, the other are librairies. CI is configured for each repo. I would like not to add a submodule in each repository but a single vcpkg-configuration.json is fine.

[bansan85]

Another solution is to allow "baseline", "reference" and "repository" for "kind": "filesystem". It still requires my patch but it will be more transparent for the user.

[BillyONeal]

What do you think about this feature ?

Paraphrasing @ras0219-msft , it's difficult to evaluate how we feel about this feature because we don't have a great understanding of the problems the feature is intended to solve. Adding an entirely new registry format is not a small undertaking in terms of product complexity, documentation that would need to be written, and potentially tooling added to support that (e.g. does x-add-version need to be taught about this new thing?) so the problems it solves would need to be big.

I like the “git” registry because there's no need to clone manually and you can specify the commit precisely (although I don't like the fact that you have to specify a branch, which seems to me to be a duplication and causes problems cf. microsoft/vcpkg#42300). On the other hand, the procedure for modifying commits is a bit laborious (“git add files”, “git commit”, “vcpkg format-manifest”, “vcpkg x-add-version”, “git add versions”, “git commit”, “git push”). My colleagues don't want to spend months like me to master vcpkg.

It seems like "editing git registries needs to be made easier" is the real feedback here, and we need to fix that by, for example, making x-add-version format the manifest for you, and not require committing first, would solve the problem without introducing a new on-disk format?

I like filesystem registries because modifying a port consists in duplicating a folder, incrementing the version and updating the version/baseline.json / package.json files. No need for complicated git commands. But it does require manual cloning of our two registries. My colleagues don't understand the point of deleting submodules (our internal dependencies) and replacing them with others (our registries).

If it's a filesystem registry you would also need to duplicate an entire baseline key which seems more difficult to do than needing to commit first...