keypad(TRUE) mode not immediately restored after SIGTSTP/SIGCONT

Hi,

Working with the dialog utility (version 1.3-20250116 in Debian trixie)
I noticed something unexpected.  Simple reproduction:
1. Start `dialog --menu Test 15 40 4 1 One 2 Two 3 Three 4 Four`.
2. Press Ctrl-Z to background dialog, then run `fg` at the shell prompt.
3. Press Down to move to the next menu item.
4. Instead of advancing the selection, dialog immediately exits as if
   Esc had been pressed.

I worked with an AI agent on the phenomenon and came up with a sensible
bug report for ncurses.  I tested the provided minimal reproduction code
and it works as expected.  I'm not familiar with the ncurses code base,
but I found the reasoning below plausible, hope you find it helpful.  I
cannot judge the suggested fix direction, so take it with a bigger grain
of salt, but the described workaround worked for me.  I'm willing to
provide further details if needed.

Regards,
Feri.

------------------------------------------------------------------------

ncurses version: 6.5+20250216 (libncursesw6 6.5+20250216-2, Debian 13 "trixie")
Also traced against the current ncurses/master sources on
https://github.com/mirror/ncurses (a read-only mirror), so this is not
specific to the packaged snapshot.

TERM=xterm (also reproduces with xterm-256color).

Summary
-------
After a curses program using keypad(win, TRUE) is suspended with Ctrl-Z
(SIGTSTP) and resumed with `fg` (SIGCONT), the terminal is left out of
application keypad mode. The *first* special key pressed after resume
(e.g. an arrow key) is misreported: getch() returns a bare KEY_ESCAPE
(27), typically followed immediately by the escape sequence's trailing
bytes read back as ordinary characters, instead of the expected KEY_*
code. Every special key pressed *after* that first one is reported
correctly again -- ncurses silently self-heals, but one keystroke too
late. The screen itself is redrawn correctly on resume, so this is
easy to miss unless you specifically press a special key right after
resuming.

This is user-visible and disruptive in interactive full-screen tools:
in dialog(1)-based menus, for instance, the resulting bare Escape is
interpreted as Cancel, so the very first keypress after resuming a
suspended dialog(1) session unexpectedly cancels/exits the program
instead of moving the menu selection.

Root cause
----------
Traced in the current sources (ncurses/tty/lib_tstp.c,
ncurses/tinfo/lib_options.c, ncurses/base/lib_getch.c):

1. On SIGTSTP, handle_SIGTSTP() (ncurses/tty/lib_tstp.c) calls
   endwin() before actually stopping the process, so the terminal is
   usable by the shell while the program is suspended.

2. endwin() (ncurses/base/lib_endwin.c) calls reset_shell_mode()
   (ncurses/tinfo/lib_ttyflags.c; the "reset_shell_mode" branch of
   drv_mode() in ncurses/tinfo/tinfo_driver.c on term-driver builds
   does the equivalent), which unconditionally calls
   _nc_keypad(sp, FALSE):

       if (SP_PARM) {
           _nc_keypad(SP_PARM, FALSE);
           _nc_flush();
       }

   This call is necessary and deliberate here: while the program is
   suspended the shell owns the terminal, and the shell does not want
   function keys delivered in application-keypad encoding, so
   `keypad_local` (rmkx) genuinely must be sent every time. The bug is
   not that this call happens, but what it happens to clobber as a
   side effect (see step 3).

3. _nc_keypad(sp, FALSE) (ncurses/tinfo/lib_options.c) is the single
   function used both to physically toggle the terminal's keypad
   transmission mode (by sending keypad_xmit/keypad_local) and to
   record the last-requested state, as one and the same flag:

       if (flag) {
           (void) NCURSES_PUTP2_FLUSH("keypad_xmit", keypad_xmit);
       } else if (keypad_local) {
           (void) NCURSES_PUTP2_FLUSH("keypad_local", keypad_local);
       }
       ...
       sp->_keypad_on = flag;

   Sending rmkx to the terminal is correct and required (step 2), but
   because it goes through the same _nc_keypad() call that also
   updates sp->_keypad_on, it has the side effect of clobbering the
   one persistent record that reset_prog_mode() (step 4) consults to
   decide whether keypad_xmit needs to be re-sent on resume. A
   separate, per-window flag recording the application's actual
   intent does exist -- win->_use_keypad, set once by
   keypad(win, TRUE) and otherwise unchanged by the suspend/resume
   cycle (see point 5) -- but reset_prog_mode() cannot consult it: it
   runs at the SCREEN level, before any specific window is back in
   hand, and a SCREEN may have several windows with independently
   different keypad settings, so there is no single per-window value
   it could even validly substitute for sp->_keypad_on here. Only
   _nc_wgetch()'s self-heal check (step 5) ever compares the two
   flags to each other, and by then it is one keystroke too late.

4. After SIGCONT, handle_SIGTSTP()'s resume path calls doupdate(),
   whose own comment says: "This relies on the fact that doupdate()
   will restore the program-mode tty state, and issue enter_ca_mode
   if need be." That happens via reset_prog_mode(), which -- in the
   "reset_prog_mode" branch of drv_mode() (tinfo_driver.c), mirrored
   in ncurses/tinfo/lib_ttyflags.c -- does:

       if (sp->_keypad_on)
           _nc_keypad(sp, TRUE);

   Since step 3 already cleared `_keypad_on`, this condition is now
   false, so `keypad_xmit` (smkx) is not re-sent here.

5. Why only the *first* subsequent key is affected: _nc_wgetch()
   (ncurses/base/lib_getch.c) independently self-heals this, but only
   as each read is serviced. At the very top of every call, before
   anything is read:

       if (win->_use_keypad != sp->_keypad_on)
           _nc_keypad(sp, win->_use_keypad);

   `win->_use_keypad` is still TRUE (set once at startup and never
   touched by the suspend/resume path), while `sp->_keypad_on` is
   FALSE (cleared in step 3), so the very first getch() call after
   resume detects the mismatch and calls _nc_keypad(sp, TRUE), which
   re-sends `keypad_xmit` and sets `_keypad_on` back to TRUE. But this
   happens only once that first getch() call has already started
   servicing the read -- the terminal had already transmitted the
   user's pending keypress (typed right after `fg`) in *normal*
   (non-application) mode, before this correction reaches it. So that
   one keystroke is misparsed as a bare Escape plus literal trailing
   bytes, using the stale terminal state; every keystroke after it
   arrives once the terminal is correctly back in application mode
   and is reported correctly.

Net effect: `_keypad_on` is being used for two different things --
"should the terminal currently be sending function-key escape
sequences" (a physical/shell-mode concern) and "did the application
ask for keypad mode" (a program-mode concern, already tracked
separately per-window as `_use_keypad`) -- and the shell-mode
transition clobbers the flag the program-mode transition and the
per-getch() self-heal both depend on, delaying the correction by
exactly one keystroke.

This matches independently reported instances of the same symptom
that predate this analysis:
  - https://stackoverflow.com/questions/1189708 ("ncurses to external
    shell and back messing with keys" -- "ncurses thinks ^[ and A are
    seen respectively if I press the up arrow twice"; the accepted
    answer's workaround is to explicitly call keypad(win, TRUE) again,
    which works because keypad() calls _nc_keypad() unconditionally,
    bypassing the stale flag)
  - https://stackoverflow.com/questions/3328528 (same symptom pattern
    after a def_prog_mode()/endwin()/.../reset_prog_mode()/refresh()
    round trip)

How to reproduce
-----------------
Minimal, self-contained reproducer attached (repro.c),
using nothing but ncurses' own default SIGTSTP/SIGCONT handling (no
application-installed signal handler):

    cc -o repro repro.c -lncursesw
    ./repro

Steps:
  1. Press an arrow key a few times: getch() correctly reports
     KEY_UP / KEY_DOWN / KEY_LEFT / KEY_RIGHT.
  2. Press Ctrl-Z, then run `fg` in the shell to resume.
  3. Press an arrow key once: getch() reports a bare KEY_ESCAPE (27)
     instead of the expected KEY_* code (the sequence's trailing
     bytes, e.g. 'O' and 'B' for xterm's Down arrow, are read back
     immediately afterward as ordinary characters).
  4. Press an arrow key again: getch() now correctly reports the
     expected KEY_* code again, without any further Ctrl-Z/fg cycle.

Expected: arrow keys are recognized identically before and after a
Ctrl-Z/fg cycle, with no one-keystroke "misfire" window.

Actual: exactly the first special key pressed after resume is
misparsed as a bare Escape plus literal trailing bytes; every key
after that is fine again.

Suggested fix
-------------
sp->_keypad_on is overloaded: reset_shell_mode() must unconditionally
turn the terminal's *physical* keypad mode off for the shell's sake,
but it does so via _nc_keypad(), which also happens to overwrite the
one persistent record (sp->_keypad_on) that reset_prog_mode() later
relies on to know whether keypad mode should be turned back on. A
per-window record of the application's actual intent already exists
(win->_use_keypad), but reset_prog_mode() runs before any window is
back in hand and can't use it directly.

A fix along the following lines would let reset_prog_mode() do the
right thing immediately, instead of leaving it to be discovered one
keystroke late by _nc_wgetch()'s self-heal check: save
sp->_keypad_on's value before reset_shell_mode() clears it (e.g. in
def_prog_mode(), analogous to how it already saves other tty state
across this same round trip via Nttyb), and restore it afterward, so
that reset_prog_mode()'s existing
`if (sp->_keypad_on) _nc_keypad(sp, TRUE);` check sees the correct
pre-suspend value and can re-send keypad_xmit right away, before the
user's next keystroke is even typed.

Workaround in the meantime, for applications that cannot patch
ncurses: install a SIGCONT handler that unconditionally re-sends the
terminfo `smkx` capability (or calls keypad(win, TRUE) again) as soon
as the process resumes, before the user's next keystroke can reach the
terminal in the wrong mode.

Happy to provide more detail, a pty-based automated reproduction
script, or test a candidate patch.

Thank you for maintaining ncurses.

/*
 * Minimal reproducer for: after suspending an ncurses program with
 * Ctrl-Z (SIGTSTP) and resuming it with `fg` (SIGCONT), the terminal is
 * left out of application keypad mode, so the next special key (e.g. an
 * arrow key) is reported back as a bare KEY_ESCAPE followed by the
 * literal trailing bytes of the escape sequence, instead of the
 * expected KEY_* code.
 *
 * Build:   cc -o keypad_sigtstp_repro keypad_sigtstp_repro.c -lncursesw
 * Run:     ./keypad_sigtstp_repro
 * Steps:   1. Press an arrow key a few times -- observe correct KEY_UP /
 *             KEY_DOWN / ... reports.
 *          2. Press Ctrl-Z, then run `fg` to resume.
 *          3. Press an arrow key again -- it is now reported as
 *             KEY_ESCAPE (or an unrecognized raw byte sequence),
 *             instead of the expected KEY_* code.
 *
 * This relies only on ncurses' own default SIGTSTP handling -- no
 * application-level signal handler is installed.
 */
#include <curses.h>
#include <stdlib.h>

int
main(void)
{
    initscr();
    cbreak();
    noecho();
    keypad(stdscr, TRUE);

    printw("Press arrow keys; Ctrl-Z then `fg` to reproduce; 'q' to quit.\n");
    refresh();

    for (;;) {
        int ch = getch();
        if (ch == 'q')
            break;

        printw("getch() returned %d", ch);
        if (ch == KEY_UP)
            printw(" (KEY_UP)");
        else if (ch == KEY_DOWN)
            printw(" (KEY_DOWN)");
        else if (ch == KEY_LEFT)
            printw(" (KEY_LEFT)");
        else if (ch == KEY_RIGHT)
            printw(" (KEY_RIGHT)");
        else if (ch == 27)
            printw(" (bare ESC -- bug reproduced if this follows Ctrl-Z/fg "
                   "and an arrow key was pressed)");
        printw("\n");
        refresh();
    }

    endwin();
    return EXIT_SUCCESS;
}