{{ message }}
gh-116622: Add Android test script#121595
Merged
freakboy3742 merged 40 commits intoAug 16, 2024
Merged
Conversation
Member
Sorry, something went wrong.
Member
Author
|
@freakboy3742: Please review, and test that it works on your own machine. Instructions are in README.md. Some of the actual tests are currently failing, but I'll deal with that in a separate PR. |
Sorry, something went wrong.
freakboy3742
requested changes
Jul 24, 2024
freakboy3742
left a comment
freakboy3742
left a comment
Contributor
There was a problem hiding this comment.
Mostly makes sense; and once I got it working, it did exactly what it said it would.
A couple of quirks I hit working through these instructions:
- The docs make no mention of
JAVA_HOME; I got an error until I set that as well. I didn't need to havejavaon the path as the instructions suggest. - I can see there's an earlier reference to building x86_64, but I wasn't expecting that as a dependency for testing - especially in terms of maintaining a fast testing cycle. Ideally, the
testtarget should only build the current architecture by default. - I had a few difficulties getting the test suite to run. On my first attempts, with both the
--managedand--connectedoptions, I got the following in the console:
...
> Task :app:createDebugAndroidTestApkListingFileRedirect UP-TO-DATE
> Task :app:targetSdkSetup
> Task :app:targetSdkDebugAndroidTest
Serial: emulator-5556
Boot completed
but then no further output for 5 minutes. From some debugging, it looks like it was getting stuck interrogating for a UID of the test package; it's getting stuck in the loop polling adb shell pm. It might be worth adding a safety catch to this loop so that it will actually stop (eventually) if something goes wrong.
- I'm not sure if the delay was due to a weird state in my emulator (It was behaving really weird, so I can't rule that out), or due to a "first run" delay with Gradle. I know from the Briefcase experience that the first Android project run can take ~10 mins to download the emulator images; and the way that the wrapper script is eating console output means even the (woefully inadequate) status bars that Gradle provides aren't visible.
- However, after a reboot, the test suite started fine with
--connectedwith the Briefcase default emulator. However - it got to 127/478, and then crashed with an OOM error. I guess there's a need for guidance on what the minimum emulator specification is. - The
--managedoption was successful; it created an emulator, and started it without any problems; 4 test failures in the overall suite (test_android test_capi test_pathlib test_posixpath), but not for reasons that seem to be related to the runner itself.
I've dropped a couple of other comments inline.
In summary: most of the issues I'm seeing can be addressed with documentation, or a little more verbosity in the test runner script (or both).
One other minor style inconsistency - there's a couple of files with no trailing newline; they all seem to be Java-ecosystem files.
Sorry, something went wrong.
|
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated. Once you have made the requested changes, please leave a comment on this pull request containing the phrase |
Sorry, something went wrong.
Member
Author
I think either of them will work; I've updated the README to say that.
Good point – this has since been fixed in #122487.
From the name of the task, this must have been
I wasn't able to reproduce this. What did the error look like? And are you sure you're using the default emulator from the current version of Briefcase? I created one with the Briefcase main branch, and it shows the following settings in Android Studio: 
Fixed. |
Sorry, something went wrong.
Member
Author
I've now added a |
Sorry, something went wrong.
Contributor
|
I haven't been able to reproduce the OOM issue I saw last time; I think I'll have to chalk it up to the emulator being in an odd state (which it was, AFAICT). One UX note - When I ran this time with a warm, connected emulator, there was a delay of several minutes in the test startup. The delay was after the On second startup, the delay was only a couple of seconds - so I'm guessing the delay was gradle downloading something with no feedback to the user. It's not clear to me what might be downloaded at that point (or what point is triggering the download), but another "this might take a few minutes" log message might be called for. Managed emulators are another matter: I get no error messages, but no test suite either. |
Sorry, something went wrong.
…n in non-verbose mose
46 hidden items
Load more…
Member
Author
|
I did some experimentation with
|
Sorry, something went wrong.
Member
Author
|
In one particular random order I also hit the OOM error you mentioned above. Increasing the emulator RAM from 2 GB to 4 GB was enough to allow it to pass, so I've added a note about this to the README. This occurred on commit 7a3d674, with arguments LogBased on these issues with |
Sorry, something went wrong.
Member
Author
|
I've got one more error handling improvement to push, and then this should be ready to merge. |
Sorry, something went wrong.
freakboy3742
approved these changes
Aug 16, 2024
freakboy3742
left a comment
freakboy3742
left a comment
Contributor
There was a problem hiding this comment.
Looks good to me; passes every test run I've been able to throw at it, with appropriate messaging when things are slow due to initial installs etc.
Sorry, something went wrong.
|
Thanks @mhsmith for the PR, and @freakboy3742 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13. |
Sorry, something went wrong.
|
Sorry, @mhsmith and @freakboy3742, I could not cleanly backport this to |
Sorry, something went wrong.
freakboy3742
added
needs backport to 3.13
bugs and security fixes
and removed
needs backport to 3.13
bugs and security fixes
labels
Aug 16, 2024
|
Thanks @mhsmith for the PR, and @freakboy3742 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13. |
Sorry, something went wrong.
miss-islington
pushed a commit
to miss-islington/cpython
that referenced
this pull request
Aug 16, 2024
Adds a script for running the test suite on Android emulator devices. Starting with a fresh install of the Android Commandline tools; the script manages installing other requirements, starting the emulator (if required), and retrieving results from that emulator. (cherry picked from commit f84cce6) Co-authored-by: Malcolm Smith <smith@chaquo.com>
freakboy3742
pushed a commit
that referenced
this pull request
Aug 16, 2024
gh-116622: Add Android test script (GH-121595) Adds a script for running the test suite on Android emulator devices. Starting with a fresh install of the Android Commandline tools; the script manages installing other requirements, starting the emulator (if required), and retrieving results from that emulator. (cherry picked from commit f84cce6) Co-authored-by: Malcolm Smith <smith@chaquo.com>
⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️Hi! The buildbot AMD64 Ubuntu NoGIL Refleaks 3.13 has failed when building commit cf6d14b. What do you need to do:
You can take a look at the buildbot page here: https://buildbot.python.org/#builders/1431/builds/376 Failed tests:
Summary of the results of the build (if available): == Click to see traceback logsremote: Enumerating objects: 64, done.
remote: Counting objects: 1% (1/64)
remote: Counting objects: 3% (2/64)
remote: Counting objects: 4% (3/64)
remote: Counting objects: 6% (4/64)
remote: Counting objects: 7% (5/64)
remote: Counting objects: 9% (6/64)
remote: Counting objects: 10% (7/64)
remote: Counting objects: 12% (8/64)
remote: Counting objects: 14% (9/64)
remote: Counting objects: 15% (10/64)
remote: Counting objects: 17% (11/64)
remote: Counting objects: 18% (12/64)
remote: Counting objects: 20% (13/64)
remote: Counting objects: 21% (14/64)
remote: Counting objects: 23% (15/64)
remote: Counting objects: 25% (16/64)
remote: Counting objects: 26% (17/64)
remote: Counting objects: 28% (18/64)
remote: Counting objects: 29% (19/64)
remote: Counting objects: 31% (20/64)
remote: Counting objects: 32% (21/64)
remote: Counting objects: 34% (22/64)
remote: Counting objects: 35% (23/64)
remote: Counting objects: 37% (24/64)
remote: Counting objects: 39% (25/64)
remote: Counting objects: 40% (26/64)
remote: Counting objects: 42% (27/64)
remote: Counting objects: 43% (28/64)
remote: Counting objects: 45% (29/64)
remote: Counting objects: 46% (30/64)
remote: Counting objects: 48% (31/64)
remote: Counting objects: 50% (32/64)
remote: Counting objects: 51% (33/64)
remote: Counting objects: 53% (34/64)
remote: Counting objects: 54% (35/64)
remote: Counting objects: 56% (36/64)
remote: Counting objects: 57% (37/64)
remote: Counting objects: 59% (38/64)
remote: Counting objects: 60% (39/64)
remote: Counting objects: 62% (40/64)
remote: Counting objects: 64% (41/64)
remote: Counting objects: 65% (42/64)
remote: Counting objects: 67% (43/64)
remote: Counting objects: 68% (44/64)
remote: Counting objects: 70% (45/64)
remote: Counting objects: 71% (46/64)
remote: Counting objects: 73% (47/64)
remote: Counting objects: 75% (48/64)
remote: Counting objects: 76% (49/64)
remote: Counting objects: 78% (50/64)
remote: Counting objects: 79% (51/64)
remote: Counting objects: 81% (52/64)
remote: Counting objects: 82% (53/64)
remote: Counting objects: 84% (54/64)
remote: Counting objects: 85% (55/64)
remote: Counting objects: 87% (56/64)
remote: Counting objects: 89% (57/64)
remote: Counting objects: 90% (58/64)
remote: Counting objects: 92% (59/64)
remote: Counting objects: 93% (60/64)
remote: Counting objects: 95% (61/64)
remote: Counting objects: 96% (62/64)
remote: Counting objects: 98% (63/64)
remote: Counting objects: 100% (64/64)
remote: Counting objects: 100% (64/64), done.
remote: Compressing objects: 5% (1/18)
remote: Compressing objects: 11% (2/18)
remote: Compressing objects: 16% (3/18)
remote: Compressing objects: 22% (4/18)
remote: Compressing objects: 27% (5/18)
remote: Compressing objects: 33% (6/18)
remote: Compressing objects: 38% (7/18)
remote: Compressing objects: 44% (8/18)
remote: Compressing objects: 50% (9/18)
remote: Compressing objects: 55% (10/18)
remote: Compressing objects: 61% (11/18)
remote: Compressing objects: 66% (12/18)
remote: Compressing objects: 72% (13/18)
remote: Compressing objects: 77% (14/18)
remote: Compressing objects: 83% (15/18)
remote: Compressing objects: 88% (16/18)
remote: Compressing objects: 94% (17/18)
remote: Compressing objects: 100% (18/18)
remote: Compressing objects: 100% (18/18), done.
remote: Total 37 (delta 15), reused 27 (delta 8), pack-reused 0 (from 0)
From https://github.com/python/cpython
* branch 3.13 -> FETCH_HEAD
Note: switching to 'cf6d14b96656baa911fa7a793ffa085e1ce6f328'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:
git switch -c <new-branch-name>
Or undo this operation with:
git switch -
Turn off this advice by setting config variable advice.detachedHead to false
HEAD is now at cf6d14b966 [3.13] gh-116622: Add Android test script (GH-121595) (#123061)
Switched to and reset branch '3.13'
configure: WARNING: no system libmpdecimal found; falling back to bundled libmpdecimal (deprecated and scheduled for removal in Python 3.15)
make: *** [Makefile:2264: buildbottest] Error 2 |
Sorry, something went wrong.
Contributor
|
As far as I can make out, this buildbot failure is unrelated to this PR; the same test has failed on other PRs recently (eg #122998). |
Sorry, something went wrong.
blhsing
pushed a commit
to blhsing/cpython
that referenced
this pull request
Aug 22, 2024
Adds a script for running the test suite on Android emulator devices. Starting with a fresh install of the Android Commandline tools; the script manages installing other requirements, starting the emulator (if required), and retrieving results from that emulator.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.
This PR builds on #117878 by adding an
android.py testcommand, which can be used to run the Android testbed either in a buildbot or interactively during development.This initial version uses two threads: one to run Gradle and one to run
adb logcat. But it has a number of bugs around interleaved output and unclean shutdowns. So I'm currently refactoring it to use asyncio instead.@freakboy3742: FYI. I'll request a review after I've done the refactor.