Prepare the script to run tests on Android emulator #4387
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/4387
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 007633e with merge base 11b2fcb ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
huydhn
changed the title
|
@huydhn has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
huydhn
changed the title
|
@huydhn has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
| # Reuse the script that install Android on ET Docker image | ||
| sudo -E bash .ci/docker/common/install_android.sh | ||
|
|
||
| # NB: It takes about 10m to cold boot the emulator here |
There was a problem hiding this comment.
Is the emulator configurable? Like the RAM allocated to it. I know typically we would only use it to validate storiesM, but if it's configurable, it can be potentially used on a non-critical path to validate large GenAI model first before actual benchmarking on device.
There was a problem hiding this comment.
I think it has something to do with our Linux runner, it should be faster when hardware accelerator is support according to https://github.com/ReactiveCircus/android-emulator-runner. Let me try to use GitHub runner and compare the perf.
There was a problem hiding this comment.
https://github.com/ReactiveCircus/android-emulator-runner?tab=readme-ov-file#configurations
ram-size maybe 12GB?
Also emulator-options: -no-snapshot-save -no-window -gpu swiftshader_indirect -noaudio -no-boot-anim -camera-back none
disable-animations: true
might be helpful
There was a problem hiding this comment.
Yeah, it's worth to try out these options, the doc also mentions some caching setup that could make it faster. Quouting https://github.com/ReactiveCircus/android-emulator-runner
We can significantly reduce emulator startup time by setting up AVD snapshot caching:
There was a problem hiding this comment.
Thank you! Current cold startup time is slower than on device testing
|
@huydhn has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@huydhn has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
To document my findings so far:
As nothing has been run on the emulator yet, we can follow up on this later to see how bad the perf is. A reference point is PT Android test https://github.com/pytorch/pytorch/actions/runs/10083807511/job/27881235211 which takes only 20+ minutes to finish, which is fine. It's not as sophisticated nor comprehensive as ET though. |
Unlike other Linux job where we can use Docker images, Android emulator needs to be run directly on our Linux runner as I don't know if Docker supports KVM which is needed to emulate the devices.