Consolidating ibc-rs's structure

[seanchen1991]

I had a good discussion with @adizere where we discussed what a more cleanly organized ibc-rs codebase would look like. I think the main issue right now is that there is not a clear logical distinction between the modules crate, Hermes, and the protobuf utilities.

I initially floated the idea that these three crates be split off into their own distinct GitHub repos, but there's some concern there that if we did that, that would result in looser integrations between the three pieces, which is something Adi would like to avoid.

Ultimately, I think we'd still be splitting ibc-rs into modules, relayer, and proto crates, but they'd be congregated into a single clearly-delineated repository, perhaps as a Cargo workspace, with the following structure:

ibc-rs
  |_ modules
  |_ relayer
  |_ proto

What do folks think about this consolidation?

[adizere]

We currently have a "monorepo" mindset of conflating everything related to IBC (in Rust) into this single repository. This approach is helpful as it keeps the crates closer to each other, and so we avoid the need for explicit basing on and off master among dependencies (such as we need to do currently with tm-rs, e.g., #1339).

The disadvantage of this approach is that is a bit unwelcoming: onboarding is more tough because there are 6 crates and their relations are unclear. Our naming is also inconsistent, which does not help: the modules directory corresponds to the ibc crate, directory telemetry corresponds to crate ibc-telemetry, and directory relayer-rest corresponds to crate ibc-relayer-rest. Additionally, a repository-wide architecture.md is not going to have full impact on helping newcomers, since there are no architectural invariants and tactics that apply globally to our whole repository.

I like the proposed solution of splitting into 3 logical parts. I can't think of any disadvantage aside from the initial discomfort this can create to devs that are used with the existing layout.

[romac]

Isn't the proposed structure the very same one we have already? Or is the suggestion to move everything aside from  the ibc, ibc-relayer, and ibc-proto crates in their own repository? If the latter, I am not so keen on doing that as this significantly complicates the workflow, by having to deal with submodules and/or patch overrides in Cargo.toml, etc.

Can we not address this by just restructuring the repo a little bit? eg. move proto-compiler under a tools directory, and maybe ibc-telemetry and ibc-relayer-rest into an services directory, etc.?

On the naming issue, I agree that we should rename the folders to the full name of the crate they contain.

[adizere]

We would keep the same (single) repo. We're talking about the directory (folder) layout of the repository. Crate boundaries would also remain pretty much unchanged.

[seanchen1991]

move proto-compiler under a tools directory, and maybe ibc-telemetry and ibc-relayer-rest into an services directory, etc.

Yes, this is essentially what I had in mind, though the structure I was considering looks a bit different. The overarching logical components I'm thinking of are modules, proto, and relayer. In addition we'd probably also need tools or utilities, as well as testing (which I think is a more fitting name for the e2e directory). So the structure would look like this:

ibc-rs
  |_ ci
  |_ testing     <-- renamed from `e2e`
  |_ modules
  |_ relayer     <-- includes the main relayer logic as well as `relayer-cli`, `relayer-rest`, and `telemetry`
  |_ proto       <-- includes the `proto` types as well as `proto-compiler`
  |_ utilities   <-- includes `scripts` and any other utility repos

I think this would be the cleanest way to structure the repo. Each sub-crate would then have its own docs, and the current guide directory would be moved into its own repo. Would this make sense?

[adizere]

Nitpick: telemetry probably belongs into relayer. Otherwise I like the proposed structure.