Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DEVHAS-488]annotation to force generate gitops #410

Merged

Conversation

yangcao77
Copy link
Contributor

@yangcao77 yangcao77 commented Nov 7, 2023

What does this PR do?:

this PR checks for annotation forceGitopsGeneration in component CR to force generate the gitops resources upon reconcilation being hit

verified in log, once the annotation set to true, the re-generation will be triggered

{"level":"info","ts":"2023-11-07T18:43:50.703Z","msg":"forceGenerateGitopsResource is true","controller":"component","controllerGroup":"appstudio.redhat.com","controllerKind":"Component","Component":{"name":"devfile-sample-go-basic3","namespace":"application-service-system"},"namespace":"application-service-system","name":"devfile-sample-go-basic3","reconcileID":"897da944-25ef-4df4-b06a-858e5fb8a7e4"}
{"level":"info","ts":"2023-11-07T18:43:50.703Z","logger":"controllers.Application","msg":"API Resource changed: Update","namespace":"application-service-system","audit":"true","name":"devfile-sample-go-basic3","controllerKind":"Application","action":"Update"}
{"level":"info","ts":"2023-11-07T18:43:50.764Z","msg":"Starting reconcile loop for application-service-system/devfile-sample-go-basic3","controller":"component","controllerGroup":"appstudio.redhat.com","controllerKind":"Component","Component":{"name":"devfile-sample-go-basic3","namespace":"application-service-system"},"namespace":"application-service-system","name":"devfile-sample-go-basic3","reconcileID":"897da944-25ef-4df4-b06a-858e5fb8a7e4"}
{"level":"info","ts":"2023-11-07T18:43:50.764Z","msg":"Re-attempting GitOps generation for devfile-sample-go-basic3","controller":"component","controllerGroup":"appstudio.redhat.com","controllerKind":"Component","Component":{"name":"devfile-sample-go-basic3","namespace":"application-service-system"},"namespace":"application-service-system","name":"devfile-sample-go-basic3","reconcileID":"897da944-25ef-4df4-b06a-858e5fb8a7e4"}
{"level":"info","ts":"2023-11-07T18:43:50.768Z","msg":"reading the kubernetes inline from component kubernetes-deploy","controller":"component","controllerGroup":"appstudio.redhat.com","controllerKind":"Component","Component":{"name":"devfile-sample-go-basic3","namespace":"application-service-system"},"namespace":"application-service-system","name":"devfile-sample-go-basic3","reconcileID":"897da944-25ef-4df4-b06a-858e5fb8a7e4"}
{"level":"info","ts":"2023-11-07T18:43:51.164Z","msg":"GitOps re-generation successful for devfile-sample-go-basic3","controller":"component","controllerGroup":"appstudio.redhat.com","controllerKind":"Component","Component":{"name":"devfile-sample-go-basic3","namespace":"application-service-system"},"namespace":"application-service-system","name":"devfile-sample-go-basic3","reconcileID":"897da944-25ef-4df4-b06a-858e5fb8a7e4"}
{"level":"info","ts":"2023-11-07T18:43:51.178Z","msg":"Re-attempting GitOps generation for devfile-sample-go-basic3","controller":"component","controllerGroup":"appstudio.redhat.com","controllerKind":"Component","Component":{"name":"devfile-sample-go-basic3","namespace":"application-service-system"},"namespace":"application-service-system","name":"devfile-sample-go-basic3","reconcileID":"897da944-25ef-4df4-b06a-858e5fb8a7e4"}
{"level":"info","ts":"2023-11-07T18:43:51.184Z","msg":"reading the kubernetes inline from component kubernetes-deploy","controller":"component","controllerGroup":"appstudio.redhat.com","controllerKind":"Component","Component":{"name":"devfile-sample-go-basic3","namespace":"application-service-system"},"namespace":"application-service-system","name":"devfile-sample-go-basic3","reconcileID":"897da944-25ef-4df4-b06a-858e5fb8a7e4"}
{"level":"info","ts":"2023-11-07T18:43:51.518Z","msg":"GitOps re-generation successful for devfile-sample-go-basic3","controller":"component","controllerGroup":"appstudio.redhat.com","controllerKind":"Component","Component":{"name":"devfile-sample-go-basic3","namespace":"application-service-system"},"namespace":"application-service-system","name":"devfile-sample-go-basic3","reconcileID":"897da944-25ef-4df4-b06a-858e5fb8a7e4"}

Which issue(s)/story(ies) does this PR fixes:

https://issues.redhat.com/browse/DEVHAS-488

PR acceptance criteria:

  • Unit/Functional tests

  • Documentation

  • Client Impact

How to test changes / Special notes to the reviewer:

Signed-off-by: Stephanie <[email protected]>
Copy link

codecov bot commented Nov 7, 2023

Codecov Report

Attention: 281 lines in your changes are missing coverage. Please review.

Comparison is base (5005e34) 83.98% compared to head (c2ab406) 81.41%.
Report is 27 commits behind head on main.

Files Patch % Lines
controllers/componentdetectionquery_controller.go 75.42% 45 Missing and 13 partials ⚠️
pkg/spi/spi.go 25.39% 47 Missing ⚠️
controllers/component_controller.go 62.83% 39 Missing and 3 partials ⚠️
cdq-analysis/pkg/componentdetectionquery.go 68.64% 33 Missing and 4 partials ⚠️
...pplicationsnapshotenvironmentbinding_controller.go 13.88% 30 Missing and 1 partial ⚠️
cdq-analysis/pkg/devfile.go 76.31% 20 Missing and 7 partials ⚠️
cdq-analysis/pkg/errors.go 22.22% 21 Missing ⚠️
cdq-analysis/pkg/util.go 76.92% 5 Missing and 4 partials ⚠️
cdq-analysis/pkg/detect.go 78.57% 6 Missing ⚠️
controllers/application_controller.go 75.00% 2 Missing ⚠️
... and 1 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #410      +/-   ##
==========================================
- Coverage   83.98%   81.41%   -2.58%     
==========================================
  Files          31       32       +1     
  Lines        4259     4772     +513     
==========================================
+ Hits         3577     3885     +308     
- Misses        514      698     +184     
- Partials      168      189      +21     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@johnmcollier johnmcollier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments

@@ -234,6 +243,7 @@ func (r *ComponentReconciler) Reconcile(ctx context.Context, req ctrl.Request) (
return ctrl.Result{}, err
} else {
log.Info(fmt.Sprintf("GitOps re-generation successful for %s", component.Name))
setForceGenerateGitopsAnnotation(&component, "false")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The person or the tool that sets the annotations can handle un setting the annotation. I don't think we need a dedicated function for setting it to false.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking the person or HAC can enable the force generation, but they are not going to track it? and after the regeneration is done, should this be set to false? or the expected usage will be after enabling, the gitops resource going to attempt regeneration in every following reconcile?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the expected usage will be after enabling, the gitops resource going to attempt regeneration in every following reconcile

I think the expected usage should be that as long as the annotation is present, every reconcile will re-generate the gitops resources. It's up to the user or tool to remove it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have discussed in team meeting, setting the annotation to false when a force generation is done

@@ -754,3 +764,21 @@ func (r *ComponentReconciler) incrementCounterAndRequeue(log logr.Logger, ctx co
return ctrl.Result{}, componentErr
}
}

// getForceGenerateGitopsAnnotation gets the internal annotation on the component whether to force generate the gitops resource
func getForceGenerateGitopsAnnotation(component appstudiov1alpha1.Component) (bool, error) {
Copy link
Member

@johnmcollier johnmcollier Nov 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can simplify this a fair bit by avoiding casting the string to bool (which means we also don't need to return an err)

func getForceGenerateGitopsAnnotation(component appstudiov1alpha1.Component) bool {
    if compAnnotations != nil && compAnnotations[forceGenerationAnnotation] == "true" {
        return true
    }
    return false
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

Signed-off-by: Stephanie <[email protected]>
Signed-off-by: Stephanie <[email protected]>
Signed-off-by: Stephanie <[email protected]>
Copy link
Member

@johnmcollier johnmcollier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good to me. Though I think tests need to be added? (Now that the status update works)

@yangcao77
Copy link
Contributor Author

@johnmcollier The status update still not working. I found out any update to the CR causes the reconcile being entered twice. Since we are resetting the annotation back to false after the first generation, the second time will overwrite the status to be the one without force generation information.

I'm not sure why the reconcile is executed twice, verified I see this behavior on main as well. Assuming this is a bug, but should we hold the force annotation until that bug being fixed? or I can check the timestamp of the gitopsGeneration status in the test to verify the gitops resource has been regenerated

@johnmcollier
Copy link
Member

johnmcollier commented Nov 14, 2023

I don’t think we need to hold the PR. The multiple reconciles is from when we set the statuses on the resource I think. I don’t think there’s a way to avoid it.

I would instead introduce a new status type for the force generation, as that will not be overwritten. I was thinking about the statuses earlier, and it may be good to have two separate statuses anyways, so that we can see when the last force generation was, even if other gitops generation occurred since then.

Signed-off-by: Stephanie <[email protected]>
Signed-off-by: Stephanie <[email protected]>
Signed-off-by: Stephanie <[email protected]>
Signed-off-by: Stephanie <[email protected]>
@yangcao77
Copy link
Contributor Author

/retest

1 similar comment
@yangcao77
Copy link
Contributor Author

/retest

@yangcao77
Copy link
Contributor Author

@johnmcollier just pushed new commits to update the condition type and also controller tests. please help take another look when you have time. thx

@yangcao77
Copy link
Contributor Author

/retest

Copy link

openshift-ci bot commented Nov 15, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: johnmcollier, yangcao77

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [johnmcollier,yangcao77]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@yangcao77 yangcao77 merged commit c3e6afd into redhat-appstudio:main Nov 15, 2023
7 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants