Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement stagedUpdateRun initialization #975

Merged
merged 6 commits into from
Dec 11, 2024

Conversation

jwtty
Copy link
Contributor

@jwtty jwtty commented Nov 26, 2024

Description of your changes

Implement the initialization stage of the clusterStagedUpdateRun controller.

Fixes #

I have:

  • Run make reviewable to ensure this PR is ready for review.

How has this code been tested

Special notes for your reviewer

@jwtty jwtty force-pushed the stagerun-init branch 3 times, most recently from d8825f2 to 5022c97 Compare November 26, 2024 07:19
@jwtty jwtty changed the title feat: implememnt stagedUpdateRun initialization feat: implement stagedUpdateRun initialization Nov 26, 2024
pkg/controllers/updaterun/initialization.go Show resolved Hide resolved
pkg/controllers/updaterun/suite_test.go Outdated Show resolved Hide resolved
pkg/controllers/updaterun/initialization.go Outdated Show resolved Hide resolved
pkg/controllers/updaterun/initialization.go Show resolved Hide resolved
pkg/controllers/updaterun/initialization.go Outdated Show resolved Hide resolved
pkg/controllers/updaterun/initialization.go Show resolved Hide resolved
@jwtty jwtty force-pushed the stagerun-init branch 3 times, most recently from 6727c44 to f2d71b3 Compare December 5, 2024 08:14
pkg/controllers/updaterun/initialization.go Outdated Show resolved Hide resolved
pkg/controllers/updaterun/initialization.go Outdated Show resolved Hide resolved
pkg/controllers/updaterun/initialization.go Outdated Show resolved Hide resolved
pkg/controllers/updaterun/initialization.go Outdated Show resolved Hide resolved
pkg/controllers/updaterun/initialization.go Show resolved Hide resolved
if toBeUpdatedBindings, toBeDeletedBindings, err = r.initialize(ctx, &updateRun); err != nil {
klog.ErrorS(err, "Failed to initialize the clusterStagedUpdateRun", "clusterStagedUpdateRun", runObjRef)
// errInitializedFailed cannot be retried.
if errors.Is(err, errInitializedFailed) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we also should not retry if the error is "unexpected"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe I wrapped all terminating errors including unexpected ones as errInitializedFailed. I think adding another case would make things more complicated.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, I miss read the code. However, since the outer error is always failedToInitialize, can we unwrap when we return the error. It might be easier to classify the error and create alert/monitor that way, especially now we emit error types like user error, unexpected or API error.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmmm but this error is not returned/reported in the controller, it's written to the clusterStagedUpdateRun condition.

@jwtty jwtty force-pushed the stagerun-init branch 2 times, most recently from d318d7a to ad9bda9 Compare December 9, 2024 01:08
if toBeUpdatedBindings, toBeDeletedBindings, err = r.initialize(ctx, &updateRun); err != nil {
klog.ErrorS(err, "Failed to initialize the clusterStagedUpdateRun", "clusterStagedUpdateRun", runObjRef)
// errInitializedFailed cannot be retried.
if errors.Is(err, errInitializedFailed) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, I miss read the code. However, since the outer error is always failedToInitialize, can we unwrap when we return the error. It might be easier to classify the error and create alert/monitor that way, especially now we emit error types like user error, unexpected or API error.

pkg/controllers/updaterun/controller_integration_test.go Outdated Show resolved Hide resolved
})
})

func validateFailedInitCondition(ctx context.Context, updateRun *placementv1alpha1.ClusterStagedUpdateRun, message string) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need to add this in this PR but I think it will be good to also verify the type of error (user error, unexpected, or api error) in this function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one is related to above discussion. The initialization errors are not returned by the reconciler but just written to the initialization condition. The controller returns nil for these errors to avoid retrying. We can only check the error by checking the condition.

@ryanzhang-oss ryanzhang-oss merged commit 1adcb20 into Azure:main Dec 11, 2024
12 checks passed
@jwtty jwtty deleted the stagerun-init branch December 11, 2024 01:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants