Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Performance] Opening search for the first time on a big account is causing the app to hang massively #46590

Open
1 of 6 tasks
hannojg opened this issue Jul 31, 2024 · 26 comments · May be fixed by #54681
Open
1 of 6 tasks
Assignees
Labels
Engineering Reviewing Has a PR in review Weekly KSv2

Comments

@hannojg
Copy link
Contributor

hannojg commented Jul 31, 2024

If you haven’t already, check out our contributing guidelines for onboarding and email [email protected] to request to join our Slack channel!


What performance issue do we need to solve?

In a customers meeting the customer was trying to open the search for the first time in NewDot. It caused the app to hang up, and it was a very bad user experience (see the recording):

Screen.Recording.2024-07-31.at.16.25.28.mov

What is the impact of this on end-users?

User perception of the app is very bad, as it's super laggy, basically not usable (the browser even warns that the app is hanging).

List any benchmarks that show the severity of the issue

The customer shared a profile trace with us:

Firefox 2024-07-25 10.42 profile.json.gz

(note: the trace also contains other test cases as well)

The customer had ~15k reports loaded in onyx when these tests were conducted, although in focus mode only 6 chats were displayed.

Proposed solution (if any)

None yet, I will go through the profile and see what can be optimised, what exactly caused those lags.

List any benchmarks after implementing the changes to show impacts of the proposed solution (if any)

not available yet

Platforms:

Which of our officially supported platforms is this issue occurring on?

  • Android: Native
  • Android: mWeb Chrome
  • iOS: Native
  • iOS: mWeb Safari
  • MacOS: Chrome / Safari
  • MacOS: Desktop

Version Number: v9.0.11-5
Reproducible in staging?: not tested
Reproducible in production?: yes
Email or phone of affected tester (no customers): customer
Logs: See performance file
Notes/Photos/Videos: See attached video
Expensify/Expensify Issue URL: n/a
Issue reported by: @hannojg
Slack conversation: https://expensify.slack.com/archives/C05LX9D6E07/p1721919928992729

View all open jobs on Upwork

Copy link

melvin-bot bot commented Jul 31, 2024

Auto-assigning issues to engineers is no longer supported. If you think this issue should receive engineering attention, please raise it in #whatsnext.

@hannojg
Copy link
Contributor Author

hannojg commented Jul 31, 2024

cc @sakluger (feel free to assign me as I (or someone from my team) will work on this ticket!)

@marcaaron
Copy link
Contributor

Curious what is happening with this especially to see if it is related to large numbers of reports (seems likely as there are 15k reports). Gonna assign myself in case @hannojg needs any assistance.

@marcaaron marcaaron self-assigned this Aug 2, 2024
@melvin-bot melvin-bot bot added the Overdue label Aug 5, 2024
@sakluger
Copy link
Contributor

sakluger commented Aug 5, 2024

No updates

@melvin-bot melvin-bot bot added Overdue and removed Overdue labels Aug 5, 2024
@sakluger
Copy link
Contributor

sakluger commented Aug 8, 2024

No update

@melvin-bot melvin-bot bot added Overdue and removed Overdue labels Aug 8, 2024
@hannojg
Copy link
Contributor Author

hannojg commented Aug 12, 2024

A large part of this hanging is due to us building the search options the moment we open the search page:

  • const searchOptions = useMemo(() => {
    if (!areOptionsInitialized || !isScreenTransitionEnd) {
    return {
    recentReports: [],
    personalDetails: [],
    userToInvite: null,
    currentUserOption: null,
    categoryOptions: [],
    tagOptions: [],
    taxRatesOptions: [],
    headerMessage: '',
    };
    }
    const optionList = OptionsListUtils.getSearchOptions(options, '', betas ?? []);

I am thinking about how to solve this here, by moving the operations in a first step to a separate thread/web worker:

@hannojg
Copy link
Contributor Author

hannojg commented Aug 14, 2024

I just opened a PR hat shows a PoC for using web workers for running the search, which will unblock the main thread:

I am now seeking approval on this new technical design here.

(Note: I am OOO for the rest of the week and will continue next week Monday 19.08)

@sakluger sakluger added Weekly KSv2 and removed Daily KSv2 labels Aug 15, 2024
@sakluger
Copy link
Contributor

Moving to weekly for now since Hanno is out until Monday.

@hannojg
Copy link
Contributor Author

hannojg commented Aug 19, 2024

Can you please also assign @chrispader? he will work on that as well (Chris please comment 🫶 )

@chrispader
Copy link
Contributor

Commenting for assignment :)

@sakluger
Copy link
Contributor

Thanks for helping, @chrispader. I'll be OOO for the next week. Hopefully we'll have some more details from the investigation when I return!

@hannojg
Copy link
Contributor Author

hannojg commented Aug 22, 2024

We were able to reproduce the bigger lags when running on a virtual machine, where the hardware is more constraint than on our recent MacBook Pro models directly. (note: we tested against a recent adhoc build that contained some search improvements).

The thing that takes the most time when opening the search page / chat finder page is:

  • calling createOptionsList
  • calling getSearchOptions (on the results of createOptionsList)
  • calling triggerUnreadUpdate also took ~1-2 seconds

Screenshot 2024-08-22 at 16 56 57

Screenshot 2024-08-22 at 16 57 03

Screenshot 2024-08-22 at 16 58 09

Screenshot 2024-08-22 at 16 58 15

We will come up with solutions for improving the performance of these functions (in the current single threaded architecture).
(Some of these solutions might blend with the solutions we are working on here)

@melvin-bot melvin-bot bot added the Overdue label Aug 30, 2024
@hannojg
Copy link
Contributor Author

hannojg commented Oct 3, 2024

We were able to get another, cleaner, sample of that problem here which relates to creating the options (which causes the lag):

I am sure now that we have a cleaner recording we can come up with good solutions to make this faster!

@melvin-bot melvin-bot bot added the Overdue label Oct 11, 2024
@sakluger
Copy link
Contributor

No updates.

@melvin-bot melvin-bot bot removed the Overdue label Oct 14, 2024
@melvin-bot melvin-bot bot added the Overdue label Oct 22, 2024
@sakluger
Copy link
Contributor

No updates.

@melvin-bot melvin-bot bot removed the Overdue label Oct 22, 2024
@melvin-bot melvin-bot bot added the Overdue label Oct 31, 2024
@sakluger
Copy link
Contributor

sakluger commented Nov 1, 2024

No updates

@melvin-bot melvin-bot bot removed the Overdue label Nov 1, 2024
@melvin-bot melvin-bot bot added the Overdue label Nov 11, 2024
@sakluger
Copy link
Contributor

No updates

@melvin-bot melvin-bot bot removed the Overdue label Nov 12, 2024
@melvin-bot melvin-bot bot added the Overdue label Nov 20, 2024
@sakluger
Copy link
Contributor

No updates

@melvin-bot melvin-bot bot removed the Overdue label Nov 20, 2024
@melvin-bot melvin-bot bot added the Overdue label Nov 29, 2024
@sakluger
Copy link
Contributor

sakluger commented Dec 2, 2024

No updates

@melvin-bot melvin-bot bot removed the Overdue label Dec 2, 2024
@melvin-bot melvin-bot bot added the Overdue label Dec 11, 2024
@sakluger
Copy link
Contributor

No update

@melvin-bot melvin-bot bot removed the Overdue label Dec 13, 2024
@chrispader
Copy link
Contributor

I will try to look into this issue this week!

@melvin-bot melvin-bot bot added the Overdue label Dec 26, 2024
@hannojg
Copy link
Contributor Author

hannojg commented Dec 31, 2024

We've identified one place where the performance can be improved.
When opening the search page we need to construct search options. Part of that step is to order the options.

Opening the full search page takes on my Mac:

Timing:development.new.expensify.open_search 416.6999999284744

For that, we need to create the search options, which makes up for ~70ms of the 416ms:

Timing:development.new.expensify.load_search_options 37.700000047683716
Timing:development.new.expensify.load_search_options 34.200000047683716

Of those 70ms half of the time is spend on ordering the items.

We use lodash order functions, which are known to be slow. We should replace the lodash sort with our own function that is optimized for performance.


Note: the numbers may not seem that drastically as those are from my high end MacBook. The numbers for android on a lower end device are:

Timing:development.new.expensify.load_search_options 790.5101539939642
Ordering options took 475.9423850029707ms
Timing:development.new.expensify.open_search 6194.899076998234

This is only one part of where the time goes for creating the search options, we will continue to where the other time is spend and how it can be optimized.


Edit: It was suspicious that load_search_options was called twice, as it's quite expensive. We put up a PR to address the issue here:

@melvin-bot melvin-bot bot removed the Overdue label Dec 31, 2024
@hannojg hannojg linked a pull request Dec 31, 2024 that will close this issue
48 tasks
@melvin-bot melvin-bot bot added Reviewing Has a PR in review Weekly KSv2 and removed Weekly KSv2 labels Dec 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Engineering Reviewing Has a PR in review Weekly KSv2
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants