Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wrong users list and empty export csv file #64

Open
labrude91 opened this issue Jul 1, 2021 · 7 comments
Open

wrong users list and empty export csv file #64

labrude91 opened this issue Jul 1, 2021 · 7 comments
Assignees
Labels
bug Something isn't working investigate Has to be checked/confirmed

Comments

@labrude91
Copy link

Hello,

When I try to display a list of users with table or csv, there is a lot of empty values and I don't find all my users.
(screenshot1) If I try to Download CSV, this one is empty : 0 Ko
For information I use LDAP backend and I have 12000 users in here.
(Downloaded: 12234 users and 1 groups in 149.0 seconds)
Nextcloud_Userexport

I try 2 differents configuration : local Apache2/Php webserver in my ldaptop in MacOS Big Sur and in a webserver (Debian).
Some Help, please ?

@labrude91
Copy link
Author

NC : 21.0.2
Nextcloud Userexport v1.2.1

(i can give you an export of occ list users with user_id if it helps)

@bpcurse
Copy link
Owner

bpcurse commented Jul 1, 2021

(i can give you an export of occ list users with user_id if it helps)

No need for a full list, yet. Thank you for offering this.
But could you post one or more of the IDs that are missing in the list?

Do you see any commonalities in the affected (not listed) user accounts? Even independent of the user ID?
Very long displaynames or special chars/emojis in them, etc.

FTR: A quick test on a Nextcloud 20.0.10 instance with a user id set to "_1602" did not provoke an issue. So the leading underscores I suspected first probably aren't related.

Could you add which PHP version is used on the server/laptop hosting the userexport script?
Is it correct, that all displayed user IDs in the screenshot have never logged in before, or is this part of the issue?

And thank you for posting the user/groups/download time info!
I was always curious about the performance when the script is confronted with larger setups :)

Ref: Seems to be similar to #61
I should be able to test against Nextcloud v21.x.x this weekend.

@bpcurse bpcurse self-assigned this Jul 1, 2021
@bpcurse bpcurse added bug Something isn't working investigate Has to be checked/confirmed labels Jul 1, 2021
@bpcurse
Copy link
Owner

bpcurse commented Jul 1, 2021

@labrude91 You could also test the API response directly from a shell with cURL:

curl -u adminname:password -X GET 'https://cloud.example.com/ocs/v1.php/cloud/users/username' -H "OCS-APIRequest: true"

Simply replace adminname, password and the cloud URL with valid admin data and replace username with one of the affected user IDs. Then check if correct user data is returned.

Repeat this using a non-affected user ID and compare the two outputs. Maybe you can find something of interest in the diff.

@labrude91
Copy link
Author

labrude91 commented Jul 7, 2021

Thank you for your quick answer.

But could you post one or more of the IDs that are missing in the list?
*** Deleted by bpcurse
*** Deleted by bpcurse
*** Deleted by bpcurse

No commonalities in the list of missing users or long displayname or emojis..
I try with the command curl for these 3 accounts and all informations are ok

@labrude91
Copy link
Author

My PHP version on my laptop is : PHP 7.3.24
"Is it correct, that all displayed user IDs in the screenshot have never logged in before, or is this part of the issue?"

I have since reduced the number of users. I now have 4,186.
In the export : table display , for few users I see a connection date. Most of them did not connect but some yes, and for some I can find them in the export.

For the stat :
Execution times
Fetch userlist 68.8 s
Fetch grouplist 0.2 s
Fetch groupfolders 0.2 s
Fetch userdata 54.1 s

@bpcurse
Copy link
Owner

bpcurse commented Jul 7, 2021

Thank you for checking and adding the information.

Interesting is the execution time for fetching the userlist (68,8 s for a single API call). This seems far too long for such a simple list, especially as it takes even longer than transferring the whole detailed userdata (in your case 4186 separate API calls).
Something might be causing trouble at this step.

EDIT: The long query time could be linked to LDAP. I do not know how Nextcloud handles this in the background, but if it checks the userlist with the LDAP server first before serving the API request, this might be the reason.

I am preparing a debug version/patch, that will give us more insight into the process. It will be accessible in a debug branch. Will give instructions on how to use it as soon as it's ready.

@bpcurse
Copy link
Owner

bpcurse commented Feb 9, 2022

Maybe related and possible solution: #65

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working investigate Has to be checked/confirmed
Projects
None yet
Development

No branches or pull requests

2 participants