Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for incremental stamping #175

Merged
merged 13 commits into from
Jul 8, 2024
Merged

Conversation

randrei-adobe
Copy link
Member

@randrei-adobe randrei-adobe commented Apr 29, 2024

Description

Modify the create_gitops_pr utility to optionally implement the following algorithm:

  • Assume the service.yaml file is an output of the service.gitops target
  • Run service.gitops > service.yaml
  • Calculate service.yaml digest > service.yaml.digest
  • Check if the contents of the service.yaml.digest are the same as those committed into git
    • Stop if there are no changes to the service.yaml.digest file
  • Stamp (replace placeholders) service.yaml
  • Commit and push the changes to service.yaml.digest and service.yaml

Related Issue

Motivation and Context

The existing GitOps PR process creates a new GitOps PR when a ConfigMap object that exposes a volatile file changes even if the deployment artifact doesn't change.

How Has This Been Tested?

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • I have signed the Adobe Open Source CLA.
  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

variables = "--variable=NAMESPACE={namespace}".format(
namespace = namespace,
)
variables += " --variable=GIT_REVISION=\"$(git rev-parse HEAD)\""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not do stamping of stamps in .show and .apply commands.

  1. The git command will not work because the current directory is not the source directory.
  2. We will be breaking the idempotency properties of .show and apply commands, which is no backward compatible behavior change.

Comment on lines 75 to 76
dryRun = flag.Bool("dry_run", false, "Do not create PRs, just print what would be done")
stamp = flag.Bool("stamp", false, "Stamp results of gitops targets with volatile information")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
dryRun = flag.Bool("dry_run", false, "Do not create PRs, just print what would be done")
stamp = flag.Bool("stamp", false, "Stamp results of gitops targets with volatile information")
stamp = flag.Bool("stamp", false, "Stamp results of gitops targets with volatile information")
dryRun = flag.Bool("dry_run", false, "Do not create PRs, just print what would be done")

Historically we kept dryRun as a last argument definition.

}

// split by newline and ignore empty strings
func SplitFunc(c rune) bool {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function doesn't seem to belong to the public interface of git module. Please make it private or inline.

if err != nil {
log.Fatalf("ERROR: %s", err)
}
return strings.FieldsFunc(files, SplitFunc)
Copy link
Member

@kzadorozhny kzadorozhny Apr 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer the use of the Scanner. While it will be more verbose it will be more obvious what empty lines re not returned.

    var files []string
    sc := bufio.NewScanner(strings.NewReader(s))
    for sc.Scan() {
        files = append(files, sc.Text())
    }
    return lines

@@ -103,6 +104,34 @@ func (r *Repo) Commit(message, gitopsPath string) bool {
return true
}

// RemoveDiff removes the changes made to a specific file in the repository
func (r *Repo) RemoveDiff(fileName string) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
func (r *Repo) RemoveDiff(fileName string) {
func (r *Repo) RestoreFile(fileName string) {

Align with the terminology used by Git. For example:

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   src/index.ts

@@ -100,6 +103,40 @@ func bazelQuery(query string) *analysis.CqueryResult {
return qr
}

func getContext(workdir *git.Repo, branchName string) map[string]interface{} {
Copy link
Member

@kzadorozhny kzadorozhny Apr 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
func getContext(workdir *git.Repo, branchName string) map[string]interface{} {
func getGitStatusDict(workdir *git.Repo, branchName string) map[string]interface{} {

getContext is too generic.


ctx := getContext(workdir, branchName)

stampedTemplate := fasttemplate.ExecuteString(string(template), "{{", "}}", ctx)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fasttemplate.Execute function writes directly into file.

Suggested change
stampedTemplate := fasttemplate.ExecuteString(string(template), "{{", "}}", ctx)
outf, err = os.OpenFile(fullPath, os.O_RDWR|os.O_CREATE|os.O_TRUNC, perm)
if err != nil {
log.Fatalf("Unable to create output file %s: %v", output, err)
}
defer outf.Close()
_, err = fasttemplate.Execute(string(template), "{{", "}}", ctx)

@randrei-adobe randrei-adobe marked this pull request as ready for review May 2, 2024 09:02
@randrei-adobe randrei-adobe requested a review from a team as a code owner May 2, 2024 09:02
@randrei-adobe randrei-adobe requested a review from kzadorozhny May 2, 2024 09:02
@kzadorozhny kzadorozhny merged commit 5bcb981 into main Jul 8, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants