Issue 5845: rlcompleter should be enabled automatically

Messages (66) msg86570 - (view) Author: Cherniavsky Beni (cben) * Date: 2009-04-25 23:27

An interactive prompt should offer working completion out-of-the-box,
without requiring every Python user on earth to create a $PYTHONSTARTUP
with '''import readline; readline.parse_and_bind("tab: complete")'''.

Note that it should work not only when Python is run without arguments,
but also when running with the -i option.
And ideally, it should only install the completion function when
dropping to the interpreter, not from the start, so that code that runs
before dropping to the interpreter is not affected.  But this is really
not important.

Safety: if 'rlcompleter' or 'readline' are not availiable (e.g. broken
python install), the user must still get a working interpreter, just
without completion.

Portability: there won't be completion on windows (as long as pyreadline
or similar is not part of Python).  But that shouldn't delay unix support!

msg86632 - (view) Author: Éric Araujo (eric.araujo) * Date: 2009-04-27 00:38

What would be the right place for this to happen? I first thought of
site.py, but -S would then turn off readline support. Does it have to be
done somewhere in C land?

But perhaps tab isn’t the right key to bind. I think inputrc could set
it to something different, perhaps shell rc files too. Is there an API
to get this setting, does readline.so already do the right thing, or
does it have to be added?

Moreover, in my PYTHONSTARTUP I not only bind tab to complete, but also
set a history file. Should this be automatic too? UNIX rules would
suggest a standard ~/.pyhistory file, or we could follow newer
freedesktop.org specifications [1].

So, I don’t know the history and choices behind readline.so, but from a
user point of view, being fed up with lack of tab completion finally had
me put up a PYTHONSTARTUP file with these settings, and then some useful
functions, and so on. This lack of completion was ultimately a good
thing for me. Still, completion could be made to work out of the box,
and the man page could link to the doc explaining how to customize this
behavior.

[1] XDG Base Directory Specification:
http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html

Cheers, Merwok

msg86660 - (view) Author: Antoine Pitrou (pitrou) * Date: 2009-04-27 12:48

He, adding completion is also something I find myself adding on every
box on which I use Python...

msg123696 - (view) Author: R. David Murray (r.david.murray) * Date: 2010-12-09 20:49

for what it is worth, I am +1 on having completion and history file work by default.  The sqlite3 command line does this, for example.  I think it is what unix user expect nowadays, and I think it is reasonable.

Looking at my home directory, it would appear that the de-facto standard history file name would be .python_history.  This is based on .mysql_history, .psql_history, and .sqlite_history, none of which I configured explicitly to support history saving.

msg123697 - (view) Author: Éric Araujo (eric.araujo) * Date: 2010-12-09 20:54

Okay, that’s one question answered.  Still to solve: How to bind the right key? (“But perhaps tab isn’t the right key to bind. I think inputrc could set it to something different, perhaps shell rc files too. Is there an API to get this setting, does readline.so already do the right thing, or does it have to be added?”)

msg123698 - (view) Author: Antoine Pitrou (pitrou) * Date: 2010-12-09 20:58

I think TAB is the key expected by most people, so let's make it the default.
As for the location, site.py is an adequate one IMO.

msg123703 - (view) Author: Ned Deily (ned.deily) * Date: 2010-12-09 21:48

Keep in mind that the Python readline module may be linked to either GNU readline or the BSD editline (libedit) library and they have different command strings.  Note the warning here:

http://docs.python.org/dev/py3k/library/readline.html

Here's a snippet of what I have today in my startup file:

    import rlcompleter
    if 'libedit' in readline.__doc__:
        readline.parse_and_bind("bind ^I rl_complete")
    else:
        readline.parse_and_bind("tab: complete")

See Issue10666 for more details and some possible changes.

msg123705 - (view) Author: Ned Deily (ned.deily) * Date: 2010-12-09 22:30

Nosying Martin: any Windows installer concerns?

msg123709 - (view) Author: Martin v. Löwis (loewis) * Date: 2010-12-09 23:12

There is no readline support on Windows at all, so I don't think the Windows installer can be affected. I'm uncertain what the proposed change to Python is at this point, though.

msg132117 - (view) Author: Éric Araujo (eric.araujo) * Date: 2011-03-25 17:08

Ned, does readline.read_init_file() with libedit?

msg132120 - (view) Author: Ned Deily (ned.deily) * Date: 2011-03-25 17:39

readline.read_init_file() does work with libedit.  The directives read have to be in libedit format.

msg132158 - (view) Author: Éric Araujo (eric.araujo) * Date: 2011-03-25 21:28

Please ignore the generated patches, left here for Martin’s benefit, and review the manually attached fix-5845.diff.

I have used read_init_file instead of hard-coding tab (it’s anyway the default for readline).

I may have repeated the same info in too much doc files, but I preferred to have more than one place where people could learn about the new behavior.  Advice welcomed.

If the basic approach looks good, especially if site still feels the right place for this, I will add tests.

msg142371 - (view) Author: Éric Araujo (eric.araujo) * Date: 2011-08-18 17:05

Updated patch (not using Mercurial, looks like I haven’t enough bandwidth to push all those changesets).

msg142393 - (view) Author: Antoine Pitrou (pitrou) * Date: 2011-08-18 18:32

> Updated patch (not using Mercurial, looks like I haven’t enough
> bandwidth to push all those changesets).

I think there's a bug in the way you are detecting interactive mode:

$ ./python -c "import sys; print(sys.stdin.isatty())"
True

msg142395 - (view) Author: Antoine Pitrou (pitrou) * Date: 2011-08-18 18:37

Interestingly, there's already the following code in Modules/main.c:

    if ((Py_InspectFlag || (command == NULL && filename == NULL && module == NULL)) &&
        isatty(fileno(stdin))) {
        PyObject *v;
        v = PyImport_ImportModule("readline");
        if (v == NULL)
            PyErr_Clear();
        else
            Py_DECREF(v);
    }


...meaning readline already gets imported automatically when desired. And indeed:

$ ./python -S
Python 3.3.0a0 (default:50f1922bc1d5, Aug 18 2011, 00:09:47) 
[GCC 4.5.2] on linux2
>>> import sys
>>> sys.modules['readline']
<module 'readline' from '/home/antoine/cpython/default/build/lib.linux-x86_64-3.3-pydebug/readline.cpython-33dm.so'>


So perhaps we could simply change this code to import another, private module (e.g. "_setupinteractive.py") which would setup readline and rlcompleter?

msg143548 - (view) Author: Éric Araujo (eric.araujo) * Date: 2011-09-05 16:30

We can’t just decide to enable completion by checking “'readline' in sys.modules” in site, because it always returns False.  site is imported in Python/pythonrun.c, and I guess that Modules/main.c is run afterwards.  More importantly, I think the import of readline in Modules/main.c is a CPython implementation detail, and I’d prefer to have code in site that is directly useful for other VMs.  (I’d go as far as proposing to remove the import readline from main.c, when my patch makes it obsolete.)

I re-read the docs for sys.argv and the -m switch and decided to translate to C checks (“command == NULL && filename == NULL && module == NULL”) to “if not sys.argv[0]”.  Unfortunately, sys.argv is not available when site is imported, so this is a dead end :(

So, I could try your _setupinteractive module idea, even though I think it’s a bit unclean, or drink the cool aid and set up completion in main.c.

msg143554 - (view) Author: Cherniavsky Beni (cben) * Date: 2011-09-05 18:29

Easily detecting interactive mode is of general interest for customization.

1. What if C also set sys.flags.interactive in "python" mode,
   or exposed sys.flags.implicit_interactive (but with better name)?

2. It's more useful to have a hook called when entering interactive mode,
   rather than a flag that's set from the beginning:

   $ python -i -c 'import sys; print sys.flags.interactive'
   1
   >>> 

   For this, importing _setupinteractive is a step forward;
   calling e.g. sys.__interactivehook__ sounds even better.
   (site.py would set it by default to a function that enables
   rlcompleter, user can always override...)

BTW, drawback of doing any such setup in site.py: "python -S" would 
be unfriendly!

msg143608 - (view) Author: Éric Araujo (eric.araujo) * Date: 2011-09-06 14:51

> Easily detecting interactive mode is of general interest for
> customization.
Thanks for the feedback.  We could open a feature request for that, and/or ask python-ideas.

> What if C also set sys.flags.interactive in "python" mode, or exposed
> sys.flags.implicit_interactive (but with better name)?
An attribute in the sys module sounds like a good idea.  It would not be an attribute of sys.flags though, as those are strictly command-line arguments.

> It's more useful to have a hook called when entering interactive mode, rather than a flag
> that's set from the beginning:
We already have such a hook: $PYTHONSTARTUP

> calling e.g. sys.__interactivehook__ sounds even better.
That’s another interesting idea.

> BTW, drawback of doing any such setup in site.py: "python -S" would be unfriendly!
People using -S don’t want the customizations done in site, so I don’t think there’s a problem here.

msg143609 - (view) Author: Antoine Pitrou (pitrou) * Date: 2011-09-06 14:54

> > It's more useful to have a hook called when entering interactive mode, rather than a flag
> > that's set from the beginning:
> We already have such a hook: $PYTHONSTARTUP

$PYTHONSTARTUP doesn't work with -i

msg143618 - (view) Author: Cherniavsky Beni (cben) * Date: 2011-09-06 16:05

[sorry, html mail was bad idea]

On Tue, Sep 6, 2011 at 17:54, Antoine Pitrou <report@bugs.python.org> wrote:

Éric Araujo <merwok@netwok.org> added the comment:
> > It's more useful to have a hook called when entering interactive mode,
> > rather than a flag
> > that's set from the beginning:
> We already have such a hook: $PYTHONSTARTUP

Good point!
It covers the user's desire customization very well (esp. if it worked with -i).
sys.__interactivehook__ has the benefit of being cleanly settable from python code.
But it might well be a YAGNI idea.

> $PYTHONSTARTUP doesn't work with -i
 
Perhaps it should?
I can't think of a thing that makes sense in $PYTHONSTARTUP that I wouldn't want with -i.
(and if there is one, one can add a test for sys.flags.interactive, or run with env PYTHONSTARTUP='')
 
Point to watch out for: errors in $PYTHONSTARTUP.
One of the uses of "python -i script.py" is doing pdb.pm() on an exception thrown by the script;
ideally a broken $PYTHONSTARTUP would not overr

> > BTW, drawback of doing any such setup in site.py: "python -S" 
> > would be unfriendly!
> People using -S don’t want the customizations done in site, so I 
> don’t think there’s a problem here.

python -S doesn't disable readline.  What makes completions more of a "customization" than editing?
The fact that it'd be implemented in site.py?
Yes, obviously, if it's implemented in site.py, -S should disable it.  My point was that it doesn't have to be implemented there.  You could drink the cool aid instead :-)

msg143621 - (view) Author: Éric Araujo (eric.araujo) * Date: 2011-09-06 16:14

> sys.__interactivehook__ has the benefit of being cleanly settable from python code.
> But it might well be a YAGNI idea.
I’ll ask python-dev about that.  For the moment, I prefer the idea of a new sys.interactive attribute (boolean).

>> $PYTHONSTARTUP doesn't work with -i
> Perhaps it should?
-i means two things: interactive and inspect.  The second meaning is used when you want python to execute a script or module just as usual but without exiting afterward, to let you inspect the state of the program.  Supporting PYTHONSTARTUP would change these use cases and be an unacceptable change.

 
>> BTW, drawback of doing any such setup in site.py: "python -S" 
>> would be unfriendly!
> People using -S don’t want the customizations done in site, so I 
> don’t think there’s a problem here.

> python -S doesn't disable readline.
> What makes completions more of a "customization" than editing?
> The fact that it'd be implemented in site.py?
If you go back to the first messages of this report, you’ll see that I wasn’t sure what a good location would be, and decided site because it did not look out of place.  There isn’t a stronger rationale than that.  One could argue that even now, site mixes disparate functionality: one program could wish to use -S to disable sys.path munging but run into bugs because the builtin exit is not defined anymore for example.

> Yes, obviously, if it's implemented in site.py, -S should disable it.
We agree.

> My point was that it doesn't have to be implemented there.
I really want to explore all options before renouncing to implement it in site.  I’d like the implementation to be maintainable and shareable with other Python VMs.

msg143961 - (view) Author: Éric Araujo (eric.araujo) * Date: 2011-09-13 12:39

FTR, I tried checking sys.ps1 instead of argv but it’s the same problem.

msg188241 - (view) Author: Antoine Pitrou (pitrou) * Date: 2013-05-01 20:31

In the spirit of pushing this forward, here is an updated patch using the sys.__interactivehook__ approach. I didn't add any tests since it doesn't seem very easy to write any.

If nobody objects, I would like to commit this soon.

msg188385 - (view) Author: Roundup Robot (python-dev) Date: 2013-05-04 18:08

New changeset d5ef330bac50 by Antoine Pitrou in branch 'default':
Issue #5845: Enable tab-completion in the interactive interpreter by default, thanks to a new sys.__interactivehook__.
http://hg.python.org/cpython/rev/d5ef330bac50

msg188386 - (view) Author: Antoine Pitrou (pitrou) * Date: 2013-05-04 18:11

End-users will hopefully rejoice :)

msg188505 - (view) Author: Mark Dickinson (mark.dickinson) * Date: 2013-05-06 11:06

On OS X 10.6, I now get the following message at interpreter startup:


iwasawa:cpython mdickinson$ ./python.exe
Python 3.4.0a0 (default:d5ef330bac50, May  6 2013, 13:05:57) 
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Failed calling sys.__interactivehook__
Traceback (most recent call last):
  File "/Users/mdickinson/Python/cpython/Lib/site.py", line 481, in register_readline
    readline.read_init_file()
OSError: [Errno -1] Unknown error: -1

msg188511 - (view) Author: Mark Dickinson (mark.dickinson) * Date: 2013-05-06 11:21

More information: readline.read_init_file() produced the same traceback before this commit, so all that's changed is that we're now calling this at interpreter startup.  It looks like I'm using the system libedit:

iwasawa:cpython mdickinson$ otool -L build/lib.macosx-10.6-x86_64-3.4-pydebug/readline.so 
build/lib.macosx-10.6-x86_64-3.4-pydebug/readline.so:
	/usr/lib/libedit.2.dylib (compatibility version 2.0.0, current version 2.11.0)
	/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.11)
iwasawa:cpython mdickinson$

msg188517 - (view) Author: Mark Dickinson (mark.dickinson) * Date: 2013-05-06 12:20

So I'm failing to find any documentation for libedit, but it looks as though this error occurs if rl_read_init_file fails to find an .editrc file in the appropriate place.  If I create an empty .editrc file in my home directory, the error disappears.  (Having an .inputrc file doesn't seem to make a difference either way.)

Perhaps the solution is to ignore an OSError from readline.read_init_file in register_readline---i.e., add a try / except OSError there.  The other option would be to ignore a nonzero errno from rl_read_init_file, but that doesn't sound so good: I'd expect someone deliberately calling readline.read_init_file to get an exception if the initialization file isn't found.

msg188521 - (view) Author: Antoine Pitrou (pitrou) * Date: 2013-05-06 12:27

> So I'm failing to find any documentation for libedit, but it looks as
> though this error occurs if rl_read_init_file fails to find an
> .editrc file in the appropriate place.  If I create an empty .editrc
> file in my home directory, the error disappears.  (Having an
> .inputrc file doesn't seem to make a difference either way.)
> 
> Perhaps the solution is to ignore an OSError from
> readline.read_init_file in register_readline---i.e., add a try /
> except OSError there.

This sounds fine to me. Can you propose a patch? I'm unlikely to ever have
an OS X machine.

msg188523 - (view) Author: Mark Dickinson (mark.dickinson) * Date: 2013-05-06 12:34

The attached fixes the issue for me.  I'm not sure whether the try / except should only be done on Apple, though.  What's the behaviour on Linux if there's no .inputrc file?

msg188524 - (view) Author: Antoine Pitrou (pitrou) * Date: 2013-05-06 12:42

> The attached fixes the issue for me.  I'm not sure whether the try /
> except should only be done on Apple, though.  What's the behaviour
> on Linux if there's no .inputrc file?

Everything works fine under Linux (as usual :-)).
There's no need to restrict the try / except to Apple platforms, though.

msg188525 - (view) Author: Steven D'Aprano (steven.daprano) * Date: 2013-05-06 12:46

I'm not able to test the patch at the moment, but since it essentially just uses the recipe in the docs, I expect it will have the same side-effect. Namely, it prevents you using the tab key to indent in the interactive interpreter.

Now I don't know if I'm missing something painfully obvious, but having to bang out space-space-space-space for every indent is surely not going to be a win for usability ;-) Even if I am missing something, surely so will a lot of other users.

I use a readline completer that (in my opinion) does the Right Thing: at the start of the line, hitting TAB inserts a tab, otherwise it does completion. It also sets a history file, adds a couple of key bindings that I sometimes find useful, and wraps it all up in a class. (Attached.) If you think there is any value in this, I'm happy to update it to Python 3.3 and polish it up as a patch to the readline module. Or someone can just mine it for ideas.

msg188532 - (view) Author: Antoine Pitrou (pitrou) * Date: 2013-05-06 13:24

> Now I don't know if I'm missing something painfully obvious, but
> having to bang out space-space-space-space for every indent is
> surely not going to be a win for usability ;-) Even if I am missing
> something, surely so will a lot of other users.

What *looks* painfully obvious to me is that Python doesn't force you
to use 4 spaces (or tabs) for indents :-)

> I use a readline completer that (in my opinion) does the Right Thing:
> at the start of the line, hitting TAB inserts a tab, otherwise it
> does completion. It also sets a history file, adds a couple of key
> bindings that I sometimes find useful, and wraps it all up in a
> class.

The "indent at beginning of line" thing sounds useful. The rest seems
to conflate the concept of a completer with the readline module itself.
Even though it's called rlcompleter, the rlcompleter module is
actually generic and could be used with other input schemes, so I don't
think it's the right place to set history options or key bindings.

In any case, you should post your patch as part of a separate issue, IMO.

msg188533 - (view) Author: Roundup Robot (python-dev) Date: 2013-05-06 13:39

New changeset 82e92da929eb by Mark Dickinson in branch 'default':
Issue #5845: avoid an exception at startup on OS X if no .editrc file exists.
http://hg.python.org/cpython/rev/82e92da929eb

msg188534 - (view) Author: Mark Dickinson (mark.dickinson) * Date: 2013-05-06 13:40

Applied the OS X fix;  reclosing.

msg188535 - (view) Author: R. David Murray (r.david.murray) * Date: 2013-05-06 13:44

The tab-doesn't-indent still needs to be fixed before 3.4 is released.

msg188540 - (view) Author: Antoine Pitrou (pitrou) * Date: 2013-05-06 14:06

> The tab-doesn't-indent still needs to be fixed before 3.4 is
> released.

I don't really understand why this would be a release blocker?
The interpreter prompt is a convenience, we don't guarantee
compatibility like with stdlib APIs.

msg188542 - (view) Author: R. David Murray (r.david.murray) * Date: 2013-05-06 14:14

It is a release blocker because it is a major usability regression.

msg188543 - (view) Author: R. David Murray (r.david.murray) * Date: 2013-05-06 14:17

Just to be clear how important I consider this, I would advocate for backing out this patch rather than releasing 3.4 with a broken tab key at the interactive prompt.  But I'd rather have the patch *and* a working tab key.

msg188549 - (view) Author: Antoine Pitrou (pitrou) * Date: 2013-05-06 14:40

> It is a release blocker because it is a major usability regression.

Really? You can press the space key to indent, it works as well
as the tab key...

msg188552 - (view) Author: R. David Murray (r.david.murray) * Date: 2013-05-06 14:44

Well, post it to python-dev and see what reaction you get :)

I could be wrong, but I don't think I am (obviously).

msg188553 - (view) Author: Ronald Oussoren (ronaldoussoren) * Date: 2013-05-06 14:45

I expect that a lot of users use the tab key to indent in the repl (as well as in editors, smart enough editors can convert the tab presses to spaces)

msg188555 - (view) Author: Antoine Pitrou (pitrou) * Date: 2013-05-06 14:52

> Well, post it to python-dev and see what reaction you get :)

I'm not interested in python-dev reactions here. For this
kind of issue, we'll have hundreds of messages for and against
the change without any useful content.

The commit adds an often-requested feature (to the point that many
people were already taking the pain of activating it manually).
But, yes, we could add Steven's refinement so that all bases are
covered.

msg188557 - (view) Author: Antoine Pitrou (pitrou) * Date: 2013-05-06 14:57

> I expect that a lot of users use the tab key to indent in the repl
> (as well as in editors, smart enough editors can convert the tab
> presses to spaces)

The interpreter prompt is not a text editor at all. You can misuse
it as one, but that's a loss of time and energy IMO.

Really, everyone I've ever showed tab-completion to has always
been enthusiastic about it ("why isn't this enabled by default?").
None has ever raised the concern of being able to use the tab key
to indent code under the prompt.

(probably because noone is crazy enough to type long chunks of
code under the interactive prompt anyway)

msg188560 - (view) Author: Alexander Belopolsky (belopolsky) * Date: 2013-05-06 15:34

I am in AP's camp on the tab issue, but I think we can preserve "tab inserts tab" behavior at the continuation prompt.  I don't like "indent at beginning of line."  I have rlcompleter enabled in Python 2.6 and i get the following when I press tab:

Python 2.6.6 (r266:84292, Feb 22 2013, 00:00:18)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> <tab>
Display all 173 possibilities? (y or n) y
ArithmeticError(            abs(                        input(
AssertionError(             all(                        int(
AttributeError(             and                         intern(
BaseException(              any(                        is
..

I find this rather useful.

At the ... prompt, however, a tab (or better four spaces) is arguably the right completion at the beginning of the line.

msg188589 - (view) Author: Antoine Pitrou (pitrou) * Date: 2013-05-06 20:36

Take it with a large grain of Internet salt, but Ezio pointed me to the following reactions:
http://www.reddit.com/r/Python/comments/1dq7sh/python_repl_finally_gets_tab_completion_by_default/

msg188592 - (view) Author: Tshepang Lekhonkhobe (tshepang) * Date: 2013-05-06 20:55

Also made it to the front-page of Hacker News[1], with better quality comments than the reddit thread.

[1] https://news.ycombinator.com/item?id=5658062

msg193981 - (view) Author: Larry Hastings (larry) * Date: 2013-07-31 07:14

This issue seems to have stalled.  But it's still marked as a release blocker, which means I can't release Python 3.4a1 in two days if this issue is still open.

One of three things will happen:
1) This issue is marked "closed".
2) This issue is downgraded from "release blocker".
3) The alpha slips.

IMO the optimal solution is that tab preceded by only whitespace indents, and tab preceded by any non-whitespace character attempts to complete.  Can we goad readline into behaving this way?

msg193986 - (view) Author: Steven D'Aprano (steven.daprano) * Date: 2013-07-31 08:04

On 31/07/13 17:14, Larry Hastings wrote:

> IMO the optimal solution is that tab preceded by only whitespace indents, and tab preceded by any non-whitespace character attempts to complete.  Can we goad readline into behaving this way?

Yes we can. Attached are a pair of lightweight modules I've used for this for the last 3-4 years, and it works fine in Python 2.4 through 3.3. I put something like this in my startup.py:

import completer
import history
history = history.History()
completer = completer.Completer(
         bindings=(r'"\C-xo": overwrite-mode',
                   r'"\C-xd": dump-functions',
                  ))

Originally they were in a single class, but I was persuaded that it was cleaner to separate them.

msg194006 - (view) Author: Martin v. Löwis (loewis) * Date: 2013-07-31 18:03

If R. David agrees (assuming no patch is coming forward), it could be degrated to deferred-blocker, and the alpha phases could be used to see how annoyed people are with "tab not working".

msg194010 - (view) Author: R. David Murray (r.david.murray) * Date: 2013-07-31 19:42

Yeah, deferred blocker is fine with me.

msg194427 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * Date: 2013-08-04 21:15

Do you know that the readline module first force rebind of TAB to insert-tab?

msg198636 - (view) Author: Xavier de Gaye (xdegaye) * Date: 2013-09-29 19:41

There is a backward compatibility issue with changeset d5ef330bac50
that enables tab-completion in the interactive interpreter by default.

When a user is not aware of this new feature and has been implementing
up to now her/his PYTHONSTARTUP file with the first example given in
python 3.3 documentation at

http://docs.python.org/3/library/readline.html?highlight=readline#example

then each new interactive python 3.4 session reads (appending them)
both history files which are saved on exit. Thus those files double
their size after each interactive python 3.4 session and pretty soon
become few mega bytes long with a python start up time that becomes
quite noticeable.

msg198640 - (view) Author: Antoine Pitrou (pitrou) * Date: 2013-09-29 20:00

> When a user is not aware of this new feature and has been implementing
> up to now her/his PYTHONSTARTUP file with the first example given in
> python 3.3 documentation at
> 
> http://docs.python.org/3/library/readline.html?highlight=readline#example
> 
> then each new interactive python 3.4 session reads (appending them)
> both history files which are saved on exit. Thus those files double
> their size after each interactive python 3.4 session and pretty soon
> become few mega bytes long with a python start up time that becomes
> quite noticeable.

Interesting. A way to avoid doing this would be to only load history if
no history is already present. Since the sys.__interactivehook__ is
executed after PYTHONSTARTUP, this would allow the latter to take
precedence.

msg198642 - (view) Author: Antoine Pitrou (pitrou) * Date: 2013-09-29 20:06

Attaching patch to only load history file if no history exists.

msg198645 - (view) Author: Roundup Robot (python-dev) Date: 2013-09-29 20:19

New changeset 687dd81cee3b by Antoine Pitrou in branch 'default':
Issue #5845: In site.py, only load readline history from ~/.python_history if no history has been read already.  This avoids double writes to the history file at shutdown.
http://hg.python.org/cpython/rev/687dd81cee3b

msg198648 - (view) Author: Xavier de Gaye (xdegaye) * Date: 2013-09-29 20:43

The patch fixes the problem on my setup. A very minor glitch: after
manually emptying or removing the PYTHONSTARTUP history file, the
history is loaded with the content of ~/.python_history on the next
session.

msg198653 - (view) Author: Antoine Pitrou (pitrou) * Date: 2013-09-29 21:13

> The patch fixes the problem on my setup. A very minor glitch: after
> manually emptying or removing the PYTHONSTARTUP history file, the
> history is loaded with the content of ~/.python_history on the next
> session.

Yes, the only way to know if a history file has already been loaded with
readline seems to be to query the history itself. So, if the history is
empty, we consider no file has been loaded.

msg205569 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * Date: 2013-12-08 15:02

What is the status of this issue?

msg205595 - (view) Author: Antoine Pitrou (pitrou) * Date: 2013-12-08 19:18

I would say it's now closed. If there's some fine tuning needed, separate issues should be opened.

msg209472 - (view) Author: Jason R. Coombs (jaraco) * Date: 2014-01-27 20:13

Added issue20411, revealed in 3.4b2.

msg212972 - (view) Author: Roundup Robot (python-dev) Date: 2014-03-09 19:17

New changeset 69c451851c71 by R David Murray in branch 'default':
whatsnew: sys.__interactivehook__. (#5845)
http://hg.python.org/cpython/rev/69c451851c71

msg215784 - (view) Author: David Beazley (dabeaz) Date: 2014-04-09 00:17

Funny thing, this feature breaks the interactive interpreter in the most basic way on OS X systems.   For example, the tab key won't even work to indent.  You can't even type the most basic programs into the interactive interpreter. For example:

>>> for i in range(10):
...      print(i)

Oh sure, you can make it work by typing the space bar a bunch of times, but it's extremely annoying.  

The only way I was able to get a working interactive interpreter on my machine was to manually edit site.py and remove the call to enablerlcompleter() from main().

I hope someone reconsiders this feature and removes it as default behavior.

msg215975 - (view) Author: R. David Murray (r.david.murray) * Date: 2014-04-12 16:09

This issue should have gone back to being a release blocker after the alpha release to fix the tab-as-indent issue, but obviously that didn't happen (I forgot about it myself).  Please open a new issue requesting a fix for this bug (that tab doesn't work as indent at the ... prompt), referencing the discussion in this issue.

msg245325 - (view) Author: Victor Varvariuc (Victor.Varvariuc) Date: 2015-06-13 19:42

Here is the new issue (putting the link here for reference): http://bugs.python.org/issue22086

msg245328 - (view) Author: R. David Murray (r.david.murray) * Date: 2015-06-13 20:09

The active issue for this is now 23441.

History Date User Action Args 2022-04-11 14:56:48adminsetgithub: 50095 2015-06-13 20:09:00r.david.murraysetmessages: + msg245328 2015-06-13 19:42:50Victor.Varvariucsetnosy: + Victor.Varvariuc
messages: + msg245325
2014-04-12 16:09:43r.david.murraysetmessages: + msg215975 2014-04-09 00:17:44dabeazsetnosy: + dabeaz
messages: + msg215784
2014-03-09 19:17:41python-devsetmessages: + msg212972 2014-01-27 20:13:40jaracosetnosy: + jaraco
messages: + msg209472
2013-12-08 19:18:07pitrousetstatus: open -> closed

messages: + msg205595

2013-12-08 15:02:34serhiy.storchakasetmessages: + msg205569 2013-09-29 21:13:22pitrousetmessages: + msg198653 2013-09-29 20:43:44xdegayesetmessages: + msg198648 2013-09-29 20:19:15python-devsetmessages: + msg198645 2013-09-29 20:06:57pitrousetkeywords: - needs review
resolution: fixed
messages: + msg198642

files: + rl_history_guard.patch

2013-09-29 20:00:50pitrousetmessages: + msg198640 2013-09-29 19:41:03xdegayesetnosy: + xdegaye
messages: + msg198636
2013-08-04 21:15:42serhiy.storchakasetmessages: + msg194427 2013-07-31 19:42:20r.david.murraysetpriority: release blocker -> deferred blocker

messages: + msg194010

2013-07-31 18:03:41loewissetmessages: + msg194006 2013-07-31 08:04:14steven.dapranosetfiles: + completer.py, history.py

messages: + msg193986

2013-07-31 07:14:13larrysetmessages: + msg193981 2013-06-26 10:36:36techtoniksetnosy: + techtonik
2013-05-06 20:55:20tshepangsetmessages: + msg188592 2013-05-06 20:36:36pitrousetmessages: + msg188589 2013-05-06 15:34:45belopolskysetmessages: + msg188560 2013-05-06 14:57:07pitrousetmessages: + msg188557 2013-05-06 14:52:07pitrousetmessages: + msg188555 2013-05-06 14:45:27ronaldoussorensetmessages: + msg188553 2013-05-06 14:44:02r.david.murraysetmessages: + msg188552 2013-05-06 14:40:06pitrousetmessages: + msg188549 2013-05-06 14:17:58r.david.murraysetmessages: + msg188543 2013-05-06 14:14:42r.david.murraysetmessages: + msg188542 2013-05-06 14:06:52pitrousetmessages: + msg188540 2013-05-06 13:44:31r.david.murraysetstatus: closed -> open
priority: normal -> release blocker

nosy: + larry
messages: + msg188535

resolution: fixed -> (no value)

2013-05-06 13:40:47mark.dickinsonsetstatus: open -> closed
resolution: fixed
messages: + msg188534
2013-05-06 13:39:47python-devsetmessages: + msg188533 2013-05-06 13:24:02pitrousetmessages: + msg188532 2013-05-06 12:46:53steven.dapranosetfiles: + completer.py

messages: + msg188525

2013-05-06 12:42:32pitrousetmessages: + msg188524 2013-05-06 12:34:14mark.dickinsonsetfiles: + issue5845_osx_fix.patch

messages: + msg188523

2013-05-06 12:27:25pitrousetmessages: + msg188521 2013-05-06 12:20:34mark.dickinsonsetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg188517
2013-05-06 11:21:21mark.dickinsonsetmessages: + msg188511 2013-05-06 11:06:12mark.dickinsonsetnosy: + mark.dickinson
messages: + msg188505
2013-05-04 18:11:40pitrousetstatus: open -> closed
resolution: fixed
messages: + msg188386

stage: patch review -> resolved

2013-05-04 18:08:43python-devsetnosy: + python-dev
messages: + msg188385
2013-05-02 07:37:39serhiy.storchakasetnosy: + serhiy.storchaka
2013-05-01 20:31:41pitrousetfiles: + defaultreadline.patch

messages: + msg188241

2013-05-01 19:35:41pitrousetfiles: + c43e264256e4.diff 2013-03-21 23:51:05tshepangsetversions: + Python 3.4, - Python 3.3 2013-03-21 23:49:41tshepangsetnosy: + tshepang
2013-03-21 12:23:54floxsetnosy: + flox
2013-03-21 09:58:20giampaolo.rodolasetnosy: + giampaolo.rodola
2012-10-05 15:43:57steven.dapranosetnosy: + steven.daprano
2011-09-13 12:39:18eric.araujosetmessages: + msg143961 2011-09-08 10:20:25ezio.melottisetnosy: + ezio.melotti
2011-09-06 16:14:42eric.araujosetmessages: + msg143621 2011-09-06 16:06:06eric.araujosetmessages: - msg143617 2011-09-06 16:05:46eric.araujosetfiles: - unnamed 2011-09-06 16:05:05cbensetmessages: + msg143618 2011-09-06 15:54:26cbensetfiles: + unnamed

messages: + msg143617

2011-09-06 14:54:39pitrousetmessages: + msg143609 2011-09-06 14:51:23eric.araujosetmessages: + msg143608 2011-09-05 18:29:38cbensetmessages: + msg143554 2011-09-05 16:30:44eric.araujosetmessages: + msg143548 2011-08-18 18:37:53pitrousetmessages: + msg142395 2011-08-18 18:32:07pitrousetmessages: + msg142393 2011-08-18 17:05:24eric.araujosetfiles: + fix-5845.diff

messages: + msg142371

2011-08-18 17:04:40eric.araujosetfiles: - fix-5845.diff 2011-03-25 23:25:04eric.araujosetfiles: - d1cbf0347eb4.diff 2011-03-25 23:25:01eric.araujosetfiles: - c43e264256e4.diff 2011-03-25 23:24:56eric.araujosetfiles: - a5bded1b8db4.diff 2011-03-25 23:18:48eric.araujosetfiles: + d1cbf0347eb4.diff 2011-03-25 22:21:16loewissetfiles: + c43e264256e4.diff 2011-03-25 22:15:42loewissetfiles: - c43e264256e4.diff 2011-03-25 21:28:17eric.araujosetkeywords: + needs review
files: + fix-5845.diff
messages: + msg132158

stage: patch review

2011-03-25 21:16:17eric.araujosetfiles: + c43e264256e4.diff 2011-03-25 21:15:09eric.araujosetfiles: + a5bded1b8db4.diff
keywords: + patch 2011-03-25 20:26:05eric.araujosethgrepos: + hgrepo9 2011-03-25 17:39:41ned.deilysetmessages: + msg132120 2011-03-25 17:08:32eric.araujosetmessages: + msg132117
versions: - Python 3.2 2010-12-09 23:12:51loewissetmessages: + msg123709 2010-12-09 22:30:22ned.deilysetnosy: + loewis
messages: + msg123705
2010-12-09 21:48:17ned.deilysetmessages: - msg123702 2010-12-09 21:48:10ned.deilysetmessages: + msg123703 2010-12-09 21:47:21ned.deilysetnosy: + ronaldoussoren, ned.deily
messages: + msg123702
2010-12-09 20:58:58pitrousetnosy: cben, belopolsky, pitrou, eric.araujo, r.david.murray, lesmana
messages: + msg123698
components: + Library (Lib), - Interpreter Core
versions: + Python 3.2 2010-12-09 20:54:53eric.araujosetmessages: + msg123697
versions: + Python 3.3, - Python 3.1, Python 2.7 2010-12-09 20:49:07r.david.murraysetnosy: + r.david.murray
messages: + msg123696
2010-06-12 01:45:23belopolskysetnosy: + belopolsky
2010-06-06 15:32:04lesmanasetnosy: + lesmana
2009-04-27 12:48:49pitrousetnosy: + pitrou
messages: + msg86660
2009-04-27 00:38:22eric.araujosetnosy: + eric.araujo
messages: + msg86632
2009-04-25 23:28:27cbensettype: enhancement
components: + Interpreter Core
versions: + Python 3.1, Python 2.7 2009-04-25 23:27:18cbencreate