Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add 'experimental/python' support #2052

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

kanterov
Copy link
Contributor

@kanterov kanterov commented Dec 30, 2024

Changes

Add experimental/python section replacing experimental/pydabs.

Add 2 new mutators into existing pipeline:

  • ApplyPythonMutator(load_resources) - loads resources from Python code
  • ApplyPythonMutator(apply_mutators) - transforms existing resources defined in Python/YAML

Example:

experimental:
  python:
    resources:
    - "resources:load_resources"
    mutators:
    - "mutators:add_email_notifications"

Tests

Unit tests and manually

@@ -441,9 +547,9 @@ func createInitOverrideVisitor(ctx context.Context) merge.OverrideVisitor {
}

func isOmitemptyDelete(left dyn.Value) bool {
// PyDABs can omit empty sequences/mappings in output, because we don't track them as optional,
// Python can omit empty sequences/mappings in output, because we don't track them as optional,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find the original sentence more clear, "Python can omit empty sequences/mappings in output" does not make sense, as I assume Python is CPython binary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 We are getting rid of "PyDABs" name, so we can't use it anymore. I've rephrased it the same way as it was in the unit test.

Copy link

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/cli

Inputs:

  • PR number: 2052
  • Commit SHA: 937f46f1e18d9a84b19d6061018ab2a4f9a3dc93

Checks will be approved automatically on success.

@kanterov
Copy link
Contributor Author

@pietern @denik please review, for some reason I can't assign reviewer through UI

@andrewnester andrewnester requested a review from pietern December 30, 2024 15:19
"pattern": "\\$\\{(var(\\.[a-zA-Z]+([-_]?[a-zA-Z0-9]+)*(\\[[0-9]+\\])*)+)\\}"
}
]
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please execute "make schema" to update this.

Copy link
Contributor

@lennartkats-db lennartkats-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor comments, otherwise LGTM

}

// PyDABs is deprecated use Python instead
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: deprecation should also be mentioned in the field above

@@ -29,6 +30,7 @@ import (
"github.com/databricks/cli/libs/process"
)

// phase is a phase of the Python mutator.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: this doesn't really help understand the field; the above is what the code already says.

}

return *b.Config.Experimental
pydabsEnabled := !reflect.DeepEqual(experimental.PyDABs, config.PyDABs{})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I'm happy with the reflect call here. But I can see how a maintainable alternative is difficult to pull off. Could you add a comment as to why reflection is used inline for maintainers?

pietern added a commit that referenced this pull request Jan 2, 2025
## Changes

I noticed a diff in the schema in #2052.

This check should be performed automatically.

## Tests

This PR includes a commit that changes the schema to check that the
workflow actually fails.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants