Readline and Python in GDB: please tell me more

Hi Tom,

I am having a look at the possible work required to get rid of PyOS_ReadlineFunctionPointer in 
gdb/python/py-gdb-readline.c, and spent some time digging into CPython code base, PEPs and the GDB 
mailing list history.

[NB: I didn't add Khoo Yit Phang in CC as he does not seem to have contributed to the GDB project 
since 2012, and the email might not be up to date. Feel free to add him in CC if you think otherwise.]

In [1], Khoo Yit Phang mentioned conflicts when enabling readline support under "python" or 
"python-interactive".

> This is somewhat of a workaround: unfortunately, there is no easy way to make GDB and Python use
> libreadline without conflicting with each other, since libreadline is configured essentially using
> global variables. Using the standard Python readline module will lead to strange conflicts, e.g.,
> changing key bindings or tab-completion.

IOW, global states inside libreadline can be an issue if a program tries to configure twice 
libreadline. The second configuration (import of readline module from Python) overwrites the first 
one (the original configuration in GDB), causing inconsistencies in behavior before and after Python 
imported libreadline. Inconsistencies can manifest themselves as key-bindings or tab-completion 
issues. Another one, and more problematic I think, is the signals handling.

Would using rl_save_state() and rl_restore_state() [2] at entry and exit of 
python_interactive_command() help for the first part of the issue, i.e. key-bindings or 
tab-completion [3] ?
This would require importing artificially readline before it is imported by the user.
Not sure however how this would work with the history between GDB and Python.

Today, gdbpy_readline_wrapper() handles GDB exceptions and Ctrl-C and Ctrl-D signals.
Also, from what I can read in [4], SIGWINCH is the only signal set up by readline's Python code. If 
it is acceptable to ignore it, we could disable all signals handling in readline's config. This way, 
the swapping of setups in python_interactive_command(), would be restricted to key-bindings, history 
and tab-completion.

If this config swapping approach does not work, I am not sure that there is any easy solution here 
except using PyOS_ReadlineFunctionPointer.

Is my understanding above correct ?

> I've thought of two possible solutions which are both
> rather complicated: 1) either reimplement the readline module that carefully swaps configurations
 > between GDB and Python; 2) or run Python's readline in a separate controlling terminal with a
 > proxy readline module in GDB.

For (1), it seems similar to what I explained above.
I don't understand (2).


Additionally, I cannot find any alternative in the limited C API to PyOS_ReadlineFunctionPointer.
PyOS_InputHook is part of the limited API, so I don't really understand why
pyOS_ReadlineFunctionPointer should not be as well.
FYI, I am in hope of opening a discussion with some Python maintainers, particularly Victor Stinner, 
to understand their constraints and perspectives on future versions of Python regarding what should 
be part of the stable API or not.

Same issue as PyOS_ReadlineFunctionPointer with PyRun_InteractiveLoop(). This function should be 
part of the limited API. I don't see any other way to do that with the limited API.

In the meantime, while Python maintainers are deciding what they would like to expose in their 
limited API, couldn't we create a GDB header exposing those symbols so that they are defined when 
compiling with the limited API ?
If the prototype of such a function does not change, and no Python internals are exposed, the ABI 
should be considered stable. I think that it could be an acceptable workaround with a relatively low 
maintenance cost. What do you think ?

Regards,
Matthieu

[1]: https://inbox.sourceware.org/gdb-patches/[email protected]/#r
[2]: https://tiswww.case.edu/php/chet/readline/readline.html#Utility-Functions-1
[3]: 
https://github.com/python/cpython/blob/0c7a75aeef4dae87f02536ed4c42a57c13ef20e2/Modules/readline.c#L1337-L1450
[4]: https://github.com/python/cpython/blob/main/Modules/readline.c and
      https://github.com/python/cpython/blob/main/Modules/clinic/readline.c.h
[5]: 
https://github.com/python/cpython/blob/3364e7e62fa24d0e19133fb0f90b1c24ef1110c5/Parser/myreadline.c#L390-L392