Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MF-1676 - Make Bootstrap fetch Certs data on demand #1669

Open
wants to merge 81 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
e4bddb1
Update Go version and dependencies (#1663)
dborovcanin Oct 26, 2022
0d2c639
Add : auto renew certficate
arvindh123 Nov 16, 2022
0e9be3f
Add : redis event stream and bug fix
arvindh123 Nov 18, 2022
786e2b5
Add : empty mock
arvindh123 Nov 18, 2022
3a6a902
Add : logging and metrics
arvindh123 Nov 18, 2022
716d1e8
Fix : certs BSClient
arvindh123 Nov 18, 2022
85da2b2
add: bsclient in service_test
arvindh123 Nov 18, 2022
c6fd44c
add: bsclient in service
arvindh123 Nov 18, 2022
f2d0e62
add: bsclient default values
arvindh123 Nov 18, 2022
765f2f5
fix: bsclient in service_test
arvindh123 Nov 18, 2022
ba92450
fix: env names
arvindh123 Nov 18, 2022
8f0b5e7
fix: env names
arvindh123 Nov 18, 2022
6fd08c2
fix: bsclient
arvindh123 Nov 18, 2022
526e7ce
fix: comments typo
arvindh123 Nov 18, 2022
a60bab3
change: things with multi cert
arvindh123 Nov 21, 2022
c3790bc
fix: cert query
arvindh123 Nov 21, 2022
46a7685
add : multierror
arvindh123 Nov 21, 2022
56e146a
remove: unused variables
arvindh123 Nov 21, 2022
1f42221
remove: AutRenew function from service
arvindh123 Nov 21, 2022
e3b0da2
remove: unused commented function
arvindh123 Nov 21, 2022
1ca7630
remove: unused commented function
arvindh123 Nov 21, 2022
f44b428
add: comment to exported variable
arvindh123 Nov 21, 2022
2929401
remove: AutRenew function from service
arvindh123 Nov 21, 2022
47dd5d9
remove: AutRenew function from service
arvindh123 Nov 21, 2022
031d2af
remove: multierror in pkg
arvindh123 Nov 21, 2022
f9dc560
Merge branch 'master' into certs-auto-renew
dborovcanin Nov 21, 2022
3fb74f1
change: logic to filter exipry certificate
arvindh123 Nov 21, 2022
19cba2b
Merge branch 'certs-auto-renew' of https://github.com/arvindh123/main…
arvindh123 Nov 21, 2022
d6eacf1
fix: sql query params
arvindh123 Nov 21, 2022
09fae2b
fix: ci errors
arvindh123 Nov 21, 2022
b23b77a
fix: certs auto removal during renew process
arvindh123 Nov 22, 2022
010f95e
fix: certs auto removal during renew process
arvindh123 Nov 22, 2022
820a628
fix: BS response for not found
arvindh123 Nov 22, 2022
0327c96
change: default auto cert new to false
arvindh123 Nov 22, 2022
8e94dfb
fix: certs sdk
arvindh123 Nov 22, 2022
c812cf9
add: default key type in provision
arvindh123 Nov 22, 2022
acf0ea3
bug fix: SDK cert request struct
arvindh123 Nov 22, 2022
aafc39b
bug fix: certs exipre date
arvindh123 Nov 22, 2022
daa0314
bug fix: SDK certs issue cert
arvindh123 Nov 22, 2022
23bf3e5
add: sdk errors
arvindh123 Nov 22, 2022
af1e3b0
add: sdk errors
arvindh123 Nov 23, 2022
7084283
add: sdk errors
arvindh123 Nov 23, 2022
c5be5ec
add: export update cert error in bootstrap
arvindh123 Nov 23, 2022
4c04fe4
add: sdk error fine tuned
arvindh123 Nov 23, 2022
b40f634
add: bootstrap update certs sdk with sdk error
arvindh123 Nov 23, 2022
2a5293b
add: certs bootstrap sdk
arvindh123 Nov 23, 2022
cc8fe33
fix: certs bootstrap sdk url
arvindh123 Nov 23, 2022
3200910
remove: unused error
arvindh123 Nov 23, 2022
8d048c7
add: sdk errors
arvindh123 Nov 23, 2022
4b37fa4
remove: unnecessary prints
arvindh123 Nov 23, 2022
d143ff3
fix: code format
arvindh123 Nov 23, 2022
b4b64e4
remove:unsed in certs BS client
arvindh123 Nov 23, 2022
ed4caa0
update : env variables
arvindh123 Nov 23, 2022
1bf134a
update : sdk errors
arvindh123 Nov 24, 2022
6bac4db
fix: CheckError
arvindh123 Nov 24, 2022
f2617eb
add:pagination for RenewCerts & ThingCertsRevokeHn
arvindh123 Nov 29, 2022
7be1b4d
add:pagination for RenewCerts & ThingCertsRevokeHn
arvindh123 Nov 29, 2022
3ecf644
Merge branch 'master' into certs-auto-renew
dborovcanin Dec 8, 2022
ffe822c
Merge branch 'master' into certs-auto-renew
arvindh123 Dec 14, 2022
6928b84
Merge branch 'master' into certs-auto-renew
arvindh123 Jan 3, 2023
56ee1d6
draft certs logic modifed
arvindh123 Jan 9, 2023
8a83e88
add things event stream client
arvindh123 Jan 10, 2023
e922aa9
Merge branch 'master' into certs-auto-renew
arvindh123 Jan 11, 2023
6a13247
add certs
arvindh123 Jan 16, 2023
85d9794
rebase with mainflux/master
arvindh123 Jan 16, 2023
a370c4b
rebase with mainflux/master
arvindh123 Jan 16, 2023
ef7205f
add certs
arvindh123 Jan 16, 2023
ebcbe2d
remove : unused variable
arvindh123 Jan 17, 2023
ef84ebf
update: certs postgres migration
arvindh123 Jan 17, 2023
cc1f369
fix: IssueCert
arvindh123 Jan 17, 2023
e41f56e
add: test for Issue certificate
arvindh123 Jan 17, 2023
22a5d3f
fix: cert issue endpoint
arvindh123 Jan 17, 2023
35b662c
fix: cert issue endpoint
arvindh123 Jan 17, 2023
32f409e
fix: cert CRUD endpoints
arvindh123 Jan 17, 2023
ebd72d2
Merge branch 'master' into certs-auto-renew
arvindh123 Feb 9, 2023
7841d81
Update renew, revoke remove certificate by thing id
arvindh123 Feb 10, 2023
98cd2ad
Update event handers
arvindh123 Feb 10, 2023
310d18d
Update event handers
arvindh123 Feb 10, 2023
1457353
Update event handers
arvindh123 Feb 10, 2023
824ca8a
add: certs api spec
arvindh123 Mar 22, 2023
26c24a8
add: certs api spec
arvindh123 Mar 22, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
324 changes: 259 additions & 65 deletions api/openapi/certs.yml

Large diffs are not rendered by default.

5 changes: 3 additions & 2 deletions bootstrap/service.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ var (

// ErrBootstrap indicates error in getting bootstrap configuration.
ErrBootstrap = errors.New("failed to read bootstrap configuration")
// ErrUpdateCert indicates error in updating the certificates
ErrUpdateCert = errors.New("failed to update cert")

errAddBootstrap = errors.New("failed to add bootstrap configuration")
errUpdateConnections = errors.New("failed to update connections")
Expand All @@ -40,7 +42,6 @@ var (
errDisconnectThing = errors.New("failed to disconnect thing")
errCheckChannels = errors.New("failed to check if channels exists")
errConnectionChannels = errors.New("failed to check channels connections")
errUpdateCert = errors.New("failed to update cert")
)

var _ Service = (*bootstrapService)(nil)
Expand Down Expand Up @@ -191,7 +192,7 @@ func (bs bootstrapService) UpdateCert(ctx context.Context, token, thingID, clien
return err
}
if err := bs.configs.UpdateCert(owner, thingID, clientCert, clientKey, caCert); err != nil {
return errors.Wrap(errUpdateCert, err)
return errors.Wrap(ErrUpdateCert, err)
}
return nil
}
Expand Down
107 changes: 79 additions & 28 deletions certs/api/endpoint.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,30 +16,23 @@ func issueCert(svc certs.Service) endpoint.Endpoint {
if err := req.validate(); err != nil {
return nil, err
}
res, err := svc.IssueCert(ctx, req.token, req.ThingID, req.TTL)
res, err := svc.IssueCert(ctx, req.token, req.ThingID, req.Name, req.TTL)
if err != nil {
return certsRes{}, err
}

return certsRes{
CertSerial: res.Serial,
ThingID: res.ThingID,
ClientCert: res.ClientCert,
ClientKey: res.ClientKey,
Expiration: res.Expire,
created: true,
}, nil
return CertToCertResponse(res, true), nil
}
}

func listSerials(svc certs.Service) endpoint.Endpoint {
func listCerts(svc certs.Service) endpoint.Endpoint {
return func(ctx context.Context, request interface{}) (interface{}, error) {
req := request.(listReq)
if err := req.validate(); err != nil {
return nil, err
}

page, err := svc.ListSerials(ctx, req.token, req.thingID, req.offset, req.limit)
page, err := svc.ListCerts(ctx, req.token, req.certID, req.thingID, req.serial, req.name, req.certStatus, req.offset, req.limit)
if err != nil {
return certsPageRes{}, err
}
Expand All @@ -53,9 +46,7 @@ func listSerials(svc certs.Service) endpoint.Endpoint {
}

for _, cert := range page.Certs {
cr := certsRes{
CertSerial: cert.Serial,
}
cr := CertToCertResponse(cert, true)
res.Certs = append(res.Certs, cr)
}
return res, nil
Expand All @@ -64,39 +55,99 @@ func listSerials(svc certs.Service) endpoint.Endpoint {

func viewCert(svc certs.Service) endpoint.Endpoint {
return func(ctx context.Context, request interface{}) (interface{}, error) {
req := request.(viewReq)
req := request.(viewRevokeRenewRemoveReq)
if err := req.validate(); err != nil {
return nil, err
}

cert, err := svc.ViewCert(ctx, req.token, req.serialID)
cert, err := svc.ViewCert(ctx, req.token, req.certID)
if err != nil {
return certsPageRes{}, err
}

certRes := certsRes{
CertSerial: cert.Serial,
ThingID: cert.ThingID,
ClientCert: cert.ClientCert,
Expiration: cert.Expire,
return CertToCertResponse(cert, false), nil
}
}

func revokeCert(svc certs.Service) endpoint.Endpoint {
return func(ctx context.Context, request interface{}) (interface{}, error) {
req := request.(viewRevokeRenewRemoveReq)
if err := req.validate(); err != nil {
return nil, err
}
return emptyCertRes{}, svc.RevokeCert(ctx, req.token, req.certID)
}
}

return certRes, nil
func renewCert(svc certs.Service) endpoint.Endpoint {
return func(ctx context.Context, request interface{}) (interface{}, error) {
req := request.(viewRevokeRenewRemoveReq)
if err := req.validate(); err != nil {
return nil, err
}
cert, err := svc.RenewCert(ctx, req.token, req.certID)
if err != nil {
return certsPageRes{}, err
}
return CertToCertResponse(cert, false), nil
}
}

func revokeCert(svc certs.Service) endpoint.Endpoint {
func removeCert(svc certs.Service) endpoint.Endpoint {
return func(ctx context.Context, request interface{}) (interface{}, error) {
req := request.(viewRevokeRenewRemoveReq)
if err := req.validate(); err != nil {
return nil, err
}
if err := svc.RemoveCert(ctx, req.token, req.certID); err != nil {
return nil, err
}
return emptyCertRes{}, nil

}
}

func revokeThingCerts(svc certs.Service) endpoint.Endpoint {
return func(ctx context.Context, request interface{}) (interface{}, error) {
req := request.(revokeRenewRemoveThingIDReq)
if err := req.validate(); err != nil {
return nil, err
}
c, err := svc.RevokeThingCerts(ctx, req.token, req.thingID, req.limit)
if err != nil {
return nil, err
}
rc := map[string]interface{}{"remaining": c}
return rc, nil
}
}

func renewThingCerts(svc certs.Service) endpoint.Endpoint {
return func(ctx context.Context, request interface{}) (interface{}, error) {
req := request.(revokeRenewRemoveThingIDReq)
if err := req.validate(); err != nil {
return nil, err
}
c, err := svc.RenewThingCerts(ctx, req.token, req.thingID, req.limit)
if err != nil {
return nil, err
}
rc := map[string]interface{}{"remaining": c}
return rc, nil
}
}

func removeThingCerts(svc certs.Service) endpoint.Endpoint {
return func(ctx context.Context, request interface{}) (interface{}, error) {
req := request.(revokeReq)
req := request.(revokeRenewRemoveThingIDReq)
if err := req.validate(); err != nil {
return nil, err
}
res, err := svc.RevokeCert(ctx, req.token, req.certID)
c, err := svc.RemoveThingCerts(ctx, req.token, req.thingID, req.limit)
if err != nil {
return nil, err
}
return revokeCertsRes{
RevocationTime: res.RevocationTime,
}, nil
rc := map[string]interface{}{"remaining": c}
return rc, nil
}
}
80 changes: 66 additions & 14 deletions certs/api/logging.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ func NewLoggingMiddleware(svc certs.Service, logger log.Logger) certs.Service {
return &loggingMiddleware{logger, svc}
}

func (lm *loggingMiddleware) IssueCert(ctx context.Context, token, thingID, ttl string) (c certs.Cert, err error) {
func (lm *loggingMiddleware) IssueCert(ctx context.Context, token, thingID, name, ttl string) (c certs.Cert, err error) {
defer func(begin time.Time) {
message := fmt.Sprintf("Method issue_cert for token: %s and thing: %s took %s to complete", token, thingID, time.Since(begin))
if err != nil {
Expand All @@ -36,57 +36,109 @@ func (lm *loggingMiddleware) IssueCert(ctx context.Context, token, thingID, ttl
lm.logger.Info(fmt.Sprintf("%s without errors.", message))
}(time.Now())

return lm.svc.IssueCert(ctx, token, thingID, ttl)
return lm.svc.IssueCert(ctx, token, thingID, name, ttl)
}

func (lm *loggingMiddleware) ListCerts(ctx context.Context, token, thingID string, offset, limit uint64) (cp certs.Page, err error) {
func (lm *loggingMiddleware) ListCerts(ctx context.Context, token, certID, thingID, serial, name string, status certs.Status, offset, limit uint64) (cp certs.Page, err error) {
defer func(begin time.Time) {
message := fmt.Sprintf("Method list_certs for token: %s and thing id: %s took %s to complete", token, thingID, time.Since(begin))
message := fmt.Sprintf("Method list_certs for token: %s, cert ID: %s thing id: %s serial: %s name: %s took %s to complete", token, certID, thingID, serial, name, time.Since(begin))
if err != nil {
lm.logger.Warn(fmt.Sprintf("%s with error: %s.", message, err))
return
}
lm.logger.Info(fmt.Sprintf("%s without errors.", message))
}(time.Now())

return lm.svc.ListCerts(ctx, token, thingID, offset, limit)
return lm.svc.ListCerts(ctx, token, certID, thingID, serial, name, status, offset, limit)
}

func (lm *loggingMiddleware) ListSerials(ctx context.Context, token, thingID string, offset, limit uint64) (cp certs.Page, err error) {
func (lm *loggingMiddleware) ViewCert(ctx context.Context, token, certID string) (c certs.Cert, err error) {
defer func(begin time.Time) {
message := fmt.Sprintf("Method list_serials for token: %s and thing id: %s took %s to complete", token, thingID, time.Since(begin))
message := fmt.Sprintf("Method view_cert for token: %s and certificate id: %s took %s to complete", token, certID, time.Since(begin))
if err != nil {
lm.logger.Warn(fmt.Sprintf("%s with error: %s.", message, err))
return
}
lm.logger.Info(fmt.Sprintf("%s without errors.", message))
}(time.Now())

return lm.svc.ListSerials(ctx, token, thingID, offset, limit)
return lm.svc.ViewCert(ctx, token, certID)
}

func (lm *loggingMiddleware) ViewCert(ctx context.Context, token, serialID string) (c certs.Cert, err error) {
func (lm *loggingMiddleware) RevokeCert(ctx context.Context, token, certID string) (err error) {
defer func(begin time.Time) {
message := fmt.Sprintf("Method view_cert for token: %s and serial id %s took %s to complete", token, serialID, time.Since(begin))
message := fmt.Sprintf("Method revoke_cert for token: %s and certificate id: %s took %s to complete", token, certID, time.Since(begin))
if err != nil {
lm.logger.Warn(fmt.Sprintf("%s with error: %s.", message, err))
return
}
lm.logger.Info(fmt.Sprintf("%s without errors.", message))
}(time.Now())

return lm.svc.ViewCert(ctx, token, serialID)
return lm.svc.RevokeCert(ctx, token, certID)
}

func (lm *loggingMiddleware) RevokeCert(ctx context.Context, token, thingID string) (c certs.Revoke, err error) {
func (lm *loggingMiddleware) RenewCert(ctx context.Context, token, certID string) (c certs.Cert, err error) {
defer func(begin time.Time) {
message := fmt.Sprintf("Method revoke_cert for token: %s and thing: %s took %s to complete", token, thingID, time.Since(begin))
message := fmt.Sprintf("Method renew_certs for token: %s and certificate id: %s took %s to complete", token, certID, time.Since(begin))
if err != nil {
lm.logger.Warn(fmt.Sprintf("%s with error: %s.", message, err))
return
}
lm.logger.Info(fmt.Sprintf("%s without errors.", message))
}(time.Now())

return lm.svc.RenewCert(ctx, token, certID)
}

func (lm *loggingMiddleware) RemoveCert(ctx context.Context, token, certID string) (err error) {
defer func(begin time.Time) {
message := fmt.Sprintf("Method renew_certs for token: %s and certificate id: %s took %s to complete", token, certID, time.Since(begin))
if err != nil {
lm.logger.Warn(fmt.Sprintf("%s with error: %s.", message, err))
return
}
lm.logger.Info(fmt.Sprintf("%s without errors.", message))
}(time.Now())

return lm.svc.RevokeCert(ctx, token, thingID)
return lm.svc.RemoveCert(ctx, token, certID)
}

func (lm *loggingMiddleware) RevokeThingCerts(ctx context.Context, token, thingID string, limit int64) (c uint64, err error) {
defer func(begin time.Time) {
message := fmt.Sprintf("Method revoke_cert for token: %s and thing: %s took %s to complete", token, thingID, time.Since(begin))
if err != nil {
lm.logger.Warn(fmt.Sprintf("%s with error: %s.", message, err))
return
}
lm.logger.Info(fmt.Sprintf("%s without errors. %d remaining certificates to revoke ", message, c))
}(time.Now())

return lm.svc.RevokeThingCerts(ctx, token, thingID, limit)
}

func (lm *loggingMiddleware) RenewThingCerts(ctx context.Context, token, thingID string, limit int64) (c uint64, err error) {
defer func(begin time.Time) {
message := fmt.Sprintf("Method renew_certs token: %s and thing: %s took %s to complete", token, thingID, time.Since(begin))
if err != nil {
lm.logger.Warn(fmt.Sprintf("%s with error: %s.", message, err))
return
}
lm.logger.Info(fmt.Sprintf("%s without errors. %d remaining certificates to renew ", message, c))
}(time.Now())

return lm.svc.RenewThingCerts(ctx, token, thingID, limit)
}

func (lm *loggingMiddleware) RemoveThingCerts(ctx context.Context, token, thingID string, limit int64) (c uint64, err error) {
defer func(begin time.Time) {
message := fmt.Sprintf("Method remove_certs for token: %s and thing: %s took %s to complete", token, thingID, time.Since(begin))
if err != nil {
lm.logger.Warn(fmt.Sprintf("%s with error: %s.", message, err))
return
}
lm.logger.Info(fmt.Sprintf("%s without errors. %d remaining certificates to remove ", message, c))
}(time.Now())

return lm.svc.RemoveThingCerts(ctx, token, thingID, limit)
}
Loading