Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ISSUE] Account Client groups, users, and service_principal list method now raises KeyError: 'content-type'. #820

Open
Feromond opened this issue Nov 12, 2024 · 0 comments

Comments

@Feromond
Copy link

Feromond commented Nov 12, 2024

Description
A clear and concise description of what the bug is.

I have been using some automated scripts to grab data out of the account level information. My last successful pull was November 6th and since then, I have not had success on the 7th and onwards. I have been using the Databricks-sdk version that is included within the Databricks runtime environment for Serverless in Databricks right now. With this specific version, I am able to authenticate and then create an account client object. From there I try get get any of the user data such as "users", "groups", or "service_principals" and I run into the same problem for all three.

The error I get looks like this:
KeyError: 'content-type'

If I try to force the upgrade on the pip install of the Databricks SDK to use the latest version, I seem to run into some value error now on the AccountClient initialization that says

ValueError: default auth: oauth-m2m: Not Found. Config: host=[REDACTED], account_id=[REDACTED], client_id=[REDACTED], client_secret=***

I think this could likely be a break caused by some change in the Databricks api itself and it now not properly being handled by this sdk but I am not sure. Hoping to see if anything can fix the issue as this endpoint was very useful before.

The documentation I referenced for this code is here (and In users and service_principals too):
https://databricks-sdk-py.readthedocs.io/en/latest/account/iam/groups.html#databricks.sdk.service.iam.AccountGroupsAPI.list

Reproduction
A minimal code sample demonstrating the bug.

account_client = AccountClient(                      # <--- Or this line fails if I try to upgrade the SDK version.
  host          = dbutils.secrets.get(scope=scope, key="host"),
  account_id    = dbutils.secrets.get(scope=scope, key="account_id"),
  client_id     = dbutils.secrets.get(scope=scope, key="client_id"),
  client_secret = dbutils.secrets.get(scope=scope, key="secret")
)

group_list = account_client.groups.list() 

account_group_list = []
for group in group_list:  # <--- THIS LINE FAILS

Expected behavior
A clear and concise description of what you expected to happen.

The expected behaviour here is that I am able to iterate through the group_list which is an iterator object. This is true once again for not just groups but for users and also the service_principals as far as I am aware. This was working without any issue previously and just stopped on November 7th.

Is it a regression?
Did this work in a previous version of the SDK? If so, which versions did you try?

I tried version 0.36 and I believe 0.35 but I cannot see what the default version is in the included library within the Databricks runtime used on Serverless. I do not see any way to print the version for this library.

Debug Logs
The SDK logs helpful debugging information when debug logging is enabled. Set the log level to debug by adding logging.basicConfig(level=logging.DEBUG) to your program, and include the logs here.

logs.log

Other Information

  • OS: Databricks runtime 15.4 or higher (Serverless compute in Databricks)
  • Version: 0.35-0.36

Additional context
I only check this endpoint once daily at 12 PM MST. The last successful 12PM MST run was on November 6th, and then November 7th onwards I have had these issues.

These issues are on all account level user related areas such as groups, users, and service_principals.
This appears to only be an issue when running Serverless Databricks computes. Standard computes are not having any of these issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant