Proposal: support the notary protocol

[arschles]

Is your feature request related to a problem? Please describe.

Google just released a proposal to add a "notary" protocol to the Go toolchain. Very roughly speaking, the intention is to add security for public modules in the go ecosystem.

The proposal lays out two options to allow for Athens (and any internal proxy) to serve private modules (see here). The "lightweight" option doesn't require us to do anything in this project, but has a few drawbacks:

• Every developer in the organization needs to set, and update configuration on their local environment to indicate which packages to ignore the notary for. If people are opting out, I think it's too easy to just opt-out of everything
• Internal modules get no verification, and you have to pull them directly from the VCS

Describe the solution you'd like

So, it would be nice if Athens could still be a proxy for internal modules, and still be a proxy for public modules. I think we'd need to add support for the notary protocol in Athens to do that.

Describe alternatives you've considered

The "lightweight" section in the privacy section is one alternative, but it's not really as good.

Additional context

Not right now

[FiloSottile]

Your point about it being too easy to just opt-out of everything is good, but there's a core catch-22 here that I don't think we can do much about: if the client is willing to trust the proxy for all modules, public and private, then GONOVERIFY=* is ok, there's HTTPS providing authentication for the proxy already; if the client wants to trust the proxy only for private modules, we are back to having to configure which ones are private.

So, it would be nice if Athens could still be a proxy for internal modules, and still be a proxy for public modules. I think we'd need to add support for the notary protocol in Athens to do that.

It can! The client just needs to know which ones are internal and which ones are external, unavoidably.

About adding support for the notary protocol, you don't have to, but if you do you can provide better privacy for your clients. It doesn't change the fact that the notary can only list public modules.

[arschles]

@FiloSottile thanks for that. is it true that Athens will need to support the notary protocol if a developer wants to fetch private code from Athens, while not having to configure GONOVERIFY on their machine?

[marwan-at-work]

Added blocked label until details of the proposal is finalized in golang/go#25530

@FiloSottile

The client just needs to know which ones are internal and which ones are external, unavoidably.

There's a considerable point of failure that is mentioned by both me and @leitzler in the upstream issue.

It might be best to keep the conversation consistent over there and have this issue be purely about the Athens implementation of the outcome of the upstream issue?

[arschles]

Thanks @marwan-at-work. @FiloSottile essentially I'm asking the same question as golang/go#25530 (comment), so I'll move it over to golang/go#25530

[FiloSottile]

+1 on moving the discussion over there, but I'll answer this question here because I don't think it's the same one.

is it true that Athens will need to support the notary protocol if a developer wants to fetch private code from Athens, while not having to configure GONOVERIFY on their machine?

No, GONOVERIFY will work without Athens support, and even if Athens supports the notary protocol, GONOVERIFY is required for private packages. Athens support is only needed for more privacy preservation, or to bypass outbound firewalls.

This is because the proxy is not trusted (we'll also run one, and we don't want clients to trust it blindly without the accountability of the notary log) so it can't influence the decision the client makes about what to verify.

[rsc]

@arschles asked me to comment here. I agree that mostly the general discussion should stay on golang/go#25530. But I wanted to point out where I think Athens can add value and what might be some productive ways to work together.

For a company running an Athens server, my expectation is that the Athens server is secured about as well as their main source code servers and that host-level TLS authentication should suffice for getting private modules. (If TLS certs within your own company, on your own VPN, are no good, maybe you have other problems?) At some point in the future we might investigating being able to layer a company-wide notary summarizing private modules on top of the public notary summarizing public modules, but the two settings really are about as different as they could possibly be. It makes sense to tackle them separately, one at a time, and the public internet is the scarier place.

An internal Athens instance must know which modules come from the internet and which are private.

I would suggest that by default, an internal Athens should serve a 200 OK from /notary/supported and proxy /notary/notary.golang.org, but 404 /notary/notary.golang.org/lookup/<privatemodule>@<anything> itself. That way, Athens provides companies with a no-leaking guarantee. If GONOVERIFY is set wrong, the notary lookup will fail (without getting to the actual notary), and the go command will print a good error that will cause people to double-check that setting.

I would also suggest that, not by default but as an optional mode, an internal Athens should be able to prefetch the entire notary database and then serve all the actual notary requests from that database itself. This will take some doing on the Go notary side as well - we don't have a bulk record download spec'ed out yet. But once we get that done, Athens could download whatever couple GB of notary data exists and then work nearly completely without notary.golang.org connectivity. It would still need to reconnect to notary.golang.org to incrementally upload its mirrored database as new records are added.

Those of us on the Go team working on the notary would love to work with you to make those two modes really great for Athens users. The spec is still in flux but we'll get there. I wouldn't start implementing yet, unless you're just doing it for exploration and willing to throw the code away as things change. We should keep talking through design.

And then we still need to figure out the best way to get GONOVERIFY set correctly for private modules. There is some good discussion on golang/go#25530 and I'm sure that will continue. If we did move to a company-wide notary layer then we'd need an equivalent to GONOVERIFY to let us control which notary to use for which packages, so any effort figuring out how to specify GONOVERIFY well would not be wasted. But having Athens run in a "nothing leaks" mode by default would go a long way toward making GONOVERIFY misconfiguration less of a big deal.

[marwan-at-work]

@rsc everything you said makes a lot of sense and I'm happy to participate in early experimental implementations as well as spec designs.

The only issue that I see from your above proposal is that a company would need to have a strong communication tool to ensure that all of their developers use a GOPROXY and don't directly go to the notary themselves. Because if a company employee doesn't have GOPROXY set, then private modules will leak. I'm not sure what's the best way to achieve that yet.

Enjoy your time off :)

[arschles]

@marwan-at-work did #1208 close this?

[marpio]

@arschles we can think about downloading the sumdb and letting Athens serve the sumdb requests instead of just proxying them.
We need to wait for that until the go team provides a spec for bulk download though.

[FiloSottile]

The endpoint for bulk download is now specified at https://go.googlesource.com/proposal/+/master/design/25530-sumdb.md#checksum-database (the /tile/H/data/K[.p/W] one), and will soon be supported by sum.golang.org.

You can also see an example implementation here:

https://github.com/golang/exp/blob/efd6b22b2522a48712e1fd0b3c1ab34d243f5245/sumdb/internal/sumweb/server.go#L131-L160

[arschles]

@marpio the bulk download is probably a good next step, yes

[arschles]

@marpio do you think you still want to do this one? No pressure as always, just let me know what you're thinking ✌️ 😄

If you are thinking about doing it still, I've heard a few requests to have Athens serve sum DB entries. I think a bulk download could help here. If it did, then one open question would be - how do we allow an operator to keep the local DB up to date?

[FiloSottile]

how do we allow an operator to keep the local DB up to date?

If you are doing a full mirror, you should poll /latest and fetch new tiles and records as they show up.

[arschles]

@FiloSottile yup, makes sense. I was referring to how we let an operator configure Athens to do that. Since sum.golang.org/latest doesn't (seem to) stream new tiles & records, Athens can poll it as you said, fall back to the sum DB if a tile doesn't exist for a module@version, we can allow an Athens operator to manually trigger an update, or some combination of those.

What do you & others think about those options?

[FiloSottile]

Presuming your users will only select this mode if they are unwilling to leak lookups to the upstream, I would advise against falling back if you don't have a record in your index. What you can try is kicking off a pull to check if /latest moved, and index the new records while the client waits.

[arschles]

I don't think we know all the use cases for why folks want this functionality. It looks like we have some requests for a read through cache (#1468 ). @wayjam would this polling strategy work for you?

cc/ @marwan-at-work @marpio

[wayjam]

Would athens 's local sumdb data left behind by upstrem or split brain?

[arschles]

the global sum DB is append-only in all cases that I know of. in that case, athens is a pure mirror and will only fall behind

[arschles]

we seem to have landed on caching in here, and we have a more specific use case in #1468. I am closing this now in favor of discussing specifics there