I verified that the double return bug is gone. I will recheck other stuff tomorrow. And also look at the test failure. If the difference of behavior on Windows does not affect Shell's behavior, then maybe the test is not needed.

With the main blocker I know of gone, I am hopeful we can get this in the May 3 beta. It would not have to be perfect because I do not expect to immediately backport for the 3.9.5 and 3.8.10 releases on the same day. (Hence, I removed the automatic backport labels.) At this point, I don't mind not putting this in 3.8 (May 4 is its last maintenance release), or delaying putting it in 3.9 (next is 3.9.6 on June 26) until tested and improved. In particular, I would want tab indents gone so users see an immediate benefit that merits having their visual comfort initially disrupted.

If you pull a couple of things into separate issues and PRs, then I will deal with news and merging, once satisfied, and baby-sitting the backports.

My thinking now is that even if a bug is discovered during and its quick fix is motivated by another issue, it should be done as a separate issue and PR as long as it is applicable to and an upgrade to IDLE as it is. (In otherwords, if we would want it even without the main issue.) Then merge it into the original PR. I should have said something more earlier.

@terryjreedy, I suggest that you wait and let me take another look at the tests. I should find the time for that in the next few days, and hopefully will find a solution that works.

If I don't, we can indeed temporarily disable the offending tests, since the issue is entirely with the testing method being fragile (rather than an actual problem with IDLE.)

AFAICT after reviewing everything here and the latest fixes and cleanups, there are no other outstanding issues, and there is no need for separate PRs.

CONFIGDIALOG is disabled because it is still calling update_colors. That line must not be covered in its test. (probably should use some mock_funcs).

@terryjreedy
I have made the requested changes; please review again.

I am about to get more sleep before continuing. Ignoring the issue of making this work with #25678, discussed on the tracker, do you consider this PR ready to commit for a beta release?

do you consider this PR ready to commit for a beta release?

Yes!

I'd like to follow up quickly with implementing mouse interaction with the sidebar (similar to the line-numbers sidebar), including specifically a context-menu (i.e. right-click) option to copy the contents with prompts, which would be useful for doctests for example.

I checked this out and played with it, but while the prompt in the sidebar looks nice, indentation is still with tabs, which gives it a decidedly 1990s feel. :-)

@gvanrossum, thanks for giving this a whirl!

Changing indentation from tabs to spaces is a quick followup that @terryjreedy has begun working on (see PR GH-25678).

@terryjreedy, I've got a local branch which refactors ShellSidebar to also inherit from BaseSidebar, moves more common logic to BaseSidebar, and adds support for selection via mouse events to the shell sidebar. Let me know if you'd like those in this PR or in a followup PR.

#25678 works to change indents when applied on top of this. I have done a thorough manual testing of the combination. While I want to review these changes more, I am merging this now so we can move on. I will then merge master in #25678 and merge that. Given Tal preference, Guido's comments, and the fact that the changes here are, from a bug-finding viewpoint, more in need of user testing than those in #25678, the sidebar should be in place first. I will look at the new sidebar code after Tal posts a new PR. While trying to make the sidebar optional after adding a 'usesidebar' attribute, I will look at the pyshell changes. Tal, if I have trouble, I will ask for help.

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Fedora Clang 3.x has failed when building commit 15d3861.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/3/builds/65) and take a look at the build logs.
4. Check if the failure is related to this commit (15d3861) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/3/builds/65

Failed tests:

• test_nntplib

Failed subtests:

• test_with_statement - test.test_nntplib.NetworkedNNTP_SSLTests

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

410 tests OK.

10 slowest tests:

• test_concurrent_futures: 2 min 39 sec
• test_peg_generator: 2 min 23 sec
• test_multiprocessing_spawn: 1 min 6 sec
• test_multiprocessing_forkserver: 1 min
• test_multiprocessing_fork: 54.5 sec
• test_asyncio: 51.3 sec
• test_signal: 47.3 sec
• test_io: 33.8 sec
• test_tokenize: 33.3 sec
• test_unparse: 32.4 sec

1 test failed:
test_nntplib

15 tests skipped:
test_devpoll test_ioctl test_kqueue test_msilib test_nis
test_ossaudiodev test_readline test_startfile test_tix test_tk
test_ttk_guionly test_winconsoleio test_winreg test_winsound
test_zipfile64

1 re-run test:
test_nntplib

Total duration: 5 min 16 sec

Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/test/test_nntplib.py", line 250, in wrapped
    meth(self)
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/test/test_nntplib.py", line 293, in test_with_statement
    if re.search(r'(?i)KEY.TOO.SMALL', ssl_err.reason):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/re.py", line 178, in search
    return _compile(pattern, flags).search(string)
TypeError: expected string or bytes-like object


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/test/test_nntplib.py", line 277, in test_with_statement
    server = self.NNTP_CLASS(self.NNTP_HOST,
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/nntplib.py", line 1025, in __init__
    super().__init__(host, port, user, password, readermode,
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/nntplib.py", line 334, in __init__
    self.sock = self._create_socket(timeout)
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/nntplib.py", line 1031, in _create_socket
    sock = _encrypt_on(sock, self.ssl_context, self.host)
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/nntplib.py", line 292, in _encrypt_on
    return context.wrap_socket(sock, server_hostname=hostname)
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/ssl.py", line 518, in wrap_socket
    return self.sslsocket_class._create(
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/ssl.py", line 1070, in _create
    self.do_handshake()
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/ssl.py", line 1339, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:969)