Proc-Spawner: what is open, in one place
========================================

0.12, 2026-11-13.  Every postponed thing, by who has to act.  The full
records behind each line: doc/pspawn-known-issues.md (X..., section "Open"),
doc/large-subset/large-subset-state.md (sections 4-6),
doc/large-subset/large-subset-environment.md (X27), doc/pspawn-spawner.md
(section 8: Windows), Changes.  Keep this file current: when an item is
decided or done, remove it here and record it where it belongs.

A. WAITING FOR THE USER'S DECISION

 1. X27: which samples of the behaviour run (large-subset-state.md,
    section 7) compare meaningfully, in the user's four steps (2026-11-10):
    (1) find the samples the environment cannot affect; (2) for the
    others, describe exactly how they are affected, and run them in a few
    example environments (where only one command of a snippet is
    affected); (3) recognize more such samples by simple programmatic
    tests; (4) leave the rest to deep AI analysis.
    Step 1, done (2026-11-07): the levels from the parse trees alone
    (doc/large-subset/large-subset-environment.md, sections 1-4): strict
    365, held 2, almost strict 34,291 (if programs behave alike in each
    run), undecided 5,752 (before step 3).  Record per sample:
    large-subset-provenance.tsv.
    Step 2, done in part (2026-11-08; section 9): each undecided sample has
    codes naming what kind of thing it depends on (files, vars, path-tmp,
    ...: section 3, "At a glance"), not exactly how (files does not say
    which file).  Example environments were run where one switch of the
    whole environment flips the case (-env of large-subset-runcompare.pl):
    POSIXLY_CORRECT, locale, SIGPIPE, /proc, a fresh /tmp, a fixed
    /dev/urandom, empty /home, /mnt, /var, a private /dev, every logged
    program failing (STANDIN_STATUS) or printing a chosen text
    (STANDIN_OUTPUT), fixed file times (FIXTURE_MTIME); each on every
    sample with its code, not only those with one affected command.  The
    samples that changed: tools/large-subset-env-variants.tsv.  Decided:
    such runs are footnotes to the level totals (EV1); the published table
    is the run with a fresh /tmp, a fixed /dev/urandom and fixed file
    times (A1(b)).
    Step 2 for the codes inside the run, done (2026-11-10, 11; branch
    x27-step2; large-subset-environment.md, section 10):
    large-subset-blocks.pl -d names each undecided sample's dependences
    exactly; the 2,901 samples that depend only on files, variables,
    standard input, arguments, cd or HOME by exact names (the partially
    decided) ran in example setups: 3 each, every combination (9) for two
    dependences, 3 + 3m for more (the user's "3^m tries ... 3+3*m"):
    1,547 kept their class, 1,350 changed it with ours still on the side
    of a shell, 4 moved to where ours agrees with neither (explained).
    Nine bugs of ours found on the way (X39-X47), all fixed.  Record:
    tools/large-subset-env-setups.tsv.
    The match across environments (the user, 2026-11-10 and 11; section
    11): two trees (the user's layout): every snippet by whether dash and
    bash agree and with which ours sides; columns by builtin-vs-coreutils,
    checked command by command (every command ours ran, in every
    environment, run alone by coreutils, dash's builtin, bash's builtin
    and ours: ours = coreutils in all 155); the second tree splits each
    box by legalization.  Messages count nowhere, wherever a redirection
    sends them (the user: compensated before the analysis; the former
    comparison: large-subset-match-raw.md).  Run again post-0.11
    (2026-11-13: the code with X39 to X56 found, the runner waiting for
    background jobs, step 3's environments counted too): of the 1,793
    snippets where ours is not as both shells: legalized by rulings 1,620,
    awaiting A12 88, chance 40, not legalized 45 (items 13 and 17; X56,
    and cd -).  large-subset-matchreport.pl prints it all;
    tools/large-subset-match.tsv per snippet.
    Step 3, done (2026-11-12; section 12): large-subset-blocks.pl follows
    every value from the environment to where it lands; of the 5,741 left
    undecided by the codes: pass 664 and split 1,476 (only passing through
    the shell, some split and globbed into a program's words) are almost
    strict now (the user's (a): assumptions passthrough and splitglob,
    probed by make probe-values), so almost strict 36,442, undecided
    3,601; control 2,525 (only through what the run can set: a switch whose
    other state a ruling covers, a program's answer, /tmp); affects-shell
    1,076 (a value from the environment affects what the shell itself
    does).
    Checked against step 2's runs: no counterexample left unexplained.
    Step 2 for the control class, done (2026-11-12; section 12; the user:
    with any program a stand-in): 2,441 samples in 8,862 setups, each run
    in a root of its own (-env cowroot), the programs' answer one more
    dependence; where the environment changed the class, only into a
    difference a ruling legalizes (the one not legalized, cd with extra
    words, is the same in every environment).  Partially decided now
    2,838 of the 3,601 undecided.  Record: large-subset-env-control.tsv.
    Step 2 for the class affects-shell, done (2026-11-12; section 12): 948
    samples, with the same $0 for the three and a PID namespace per run;
    of the 410 where ours is not as both shells: legalized 301, awaiting
    A12 67, chance 39 (33 races; 6 persistent results, item 17), not legalized 3
    ([^...], item 13).  X53 found and fixed.  Names with an expansion
    (the user: choose the value and make the file of that name together;
    2026-11-12): 119 of 149 samples got setups (control 34, affects-shell
    85); not legalized among them 31 (cd with extra words 30, a two-digit
    descriptor 1: item 13); X54 found and fixed.  The 33 races, run 20 times
    each (make corpus-races): 14 were a background job that ours finishes
    before it ends and the run killed in the shells; decided (A18, the
    user, 2026-11-12): the runner waits for them too, up to 5 s (-after).
    Then 27 the same in every run, 6 races (chance); X55 found and fixed.  Partially decided now
    3,508 of the 3,601 undecided; the other 93, which the setups left out,
    are classified by Claude reading them (item 19).
 3. P2 and P4 (large-subset-state.md, sections 4-5): ${...} forms both
    shells refuse; messages with our name and line, and $-.
 5. X2: $! of a background job that starts no process (design question K4).
 6. X7: run() ignores SIGINT and SIGQUIT while it runs, as Perl's system
    does: say if not wanted.
 7. Perl's Windows bugs drafted in doc/perl-win32-system-bugs.md (argv
    quoting, the cmd.exe retry, batch files, inherited handles): file them, or ask.
 8. Delete the branch run of this repository on GitHub (the proxy refuses
    deletions here); its contents are in ilyaz-ai/run, whose 0.07 is on
    CPAN (the user, 2026-11-10).
 9. X22 (bash's own bug): report it to bug-bash, if wanted.
10. M2: where there is neither posix_spawn nor fork, a missing Run makes
    spawn die (at its first call, not at load).  Act: keep it, or fail
    with $! as an unsupported setting does.
11. Perl's own writes into the graph's own pipes reopen /proc/self/fd/N for
    every write.  Keeping that handle for the pipe's life (closed with the
    end) is safe for the graph's pipes only (not the caller's fds, which
    the module never closes); measured: at most 2.5% even for a loop of
    builtins writing into a pipeline.  Act: do it, or leave it.
12. X27, section 11: ~ with HOME empty.  dash 0.5.12 drops the word (and
    ~/x loses its prefix): ❰BUG❱ dash, as POSIX treats the result of ~ as
    quoted; bash and ours give an empty field.  The rule table marks ours'
    way "proposed" (62 samples).  Act: confirm (then
    also report it to dash, if wanted), or decide otherwise.
13. X27, section 11: 7 snippets where bash = ours != dash and no ruling
    legalizes it (and 2 with cd -, in B): [^...] in a pattern (2) and
    return outside a function (1), as rule E but not yet listed under it;
    echo foo 42>/dev/null (dash: one-digit fd numbers only; also, in
    step 3, head -<$LINENUM + 10> filename, NB:6724); echo $LINENO
    after a #! line (dash: empty); cd with extra words (dash ignores them;
    in step 3 also 30 samples where a program's answer gives cd its
    operands, cd $(find . -name config): bash and ours, since X54, "too
    many arguments", status 1); [^...] also in step 3 (mv
    /mnt/usbdisk/[^l]* ...);
    races between the elements of a pipeline: ls -a | tee output.file,
    and since the runs of post-0.11 cat lol | sed ... > lol (sed's
    redirection empties lol before or after cat reads it) and NB:12523,
    find $DiskName ... | sort ... >> $TMPFILE given "two  words *"
    (find's pattern sees the file sort's redirection makes, or not); each
    came out otherwise in another run; and, found by the fuzzer, not a
    snippet (the user, 2026-11-12: join it to this list): return || shift
    at the top level, where ours follows bash for return (X48) and dash
    for the failed shift (A5) and so ends as neither shell (status 2; both
    give 0).  Act: decide them one by one, with the rules (the user,
    2026-11-11: later).
15. MetaCPAN shows the .md files rendered (/view/doc/...; relative links
    between them work, checked by the author 2026-11-11) but not their
    tables; the POD's links to them (Proc::Spawner::Docs, to /source/:
    the raw text) did not work.  The experiments for 0.12 (the user,
    2026-11-11 and 12), written post-0.11: Proc::Spawner::Docs links
    each document twice, its name to the rendered file of the newest
    release (/dist/Proc-Spawner/view/doc/...) and, in an =for html
    paragraph, the same file of the same release (../../../doc/... from
    /release/.../view/lib/Proc/Spawner/Docs.pod; it works from
    /release/.../view/ and /dist/.../view/, not from /pod/...); the test
    page doc/metacpan-tables.md has one table as a pipe table, raw HTML,
    preformatted text indented and fenced, and a list.  Act, after 0.12
    is on MetaCPAN: look at Docs and the test page, check both kinds of
    link, choose the table form.
16. The documents: large-subset-state.md is a log grown by appending
    (paragraphs of 2,000 characters, its history inside), hardly usable
    (the user, 2026-11-11).  Rewrite it as the current state, its
    history to Changes; links into the private repository (item 14) with
    it.  Postponed (the user: not critical).
17. X27, step 3: persistent results (the same in every run, which the
    environment does not change), found in the class affects-shell,
    that no ruling covers yet (section 12): the
    environment env and printenv list (5 snippets, since IN1 with v=5 env |
    less: bash adds SHLVL and _; dash and ours list the same variables,
    in another order); /proc/$$/cmdline (1: each shell's own command line,
    ours a Perl one); times (1: ours, Perl, has used CPU time where dash
    shows 0); and, found post-0.11 (20 runs: ours' own result in each, so
    not a race), (echo ...; sleep 1; echo ...) | zenity with zenity
    missing: the shells' builtin echo dies of SIGPIPE and ends the
    subshell, ours' echo (inside Perl, as the program before IP1) fails
    alone and the subshell goes on (one sleep 1 more).  Act: rule on them
    (they are built into the shells and into ours; the environment does
    not change them).
19. X27, step 3: the 93 undecided samples the setups left out, classified
    by Claude (an AI) reading them (2026-11-13; the user's categories: AI- and a category used
    before, and AI-PATH-twister; large-subset-environment.md, section 12;
    record doc/large-subset/tools/large-subset-ai-classes.tsv): AI-strict
    27 (an unmovable stone 18, a device working as expected 9), AI-names
    with an expansion 26, AI-values passed on unchanged 7, AI-values passed
    on split 7, AI-tilde 6, AI-A5 5, AI-glob 4, AI-shells' own variables 4,
    AI-PATH-twister 4, AI-noise 3.  Act: accept each category, or move a
    sample (the work each takes: B).
20. win_spawn.h for Perl's core (post-0.12; another session, prhx, is
    drafting a patch that takes a copy of it into Perl as its posix_spawn
    on Windows; doc/pspawn-spawner.md, section 8, "Done post-0.12").
    Taken from that patch, with changes: the options WIN_OPT_CMD_RAW,
    WIN_OPT_ARGV_RAW, WIN_OPT_KEEP_QUOTED, which Proc::Spawner does not
    set.  Act: keep them, or leave them to Perl's copy.
    Also, a batch file: cmd.exe takes the first word of its command line
    both as what it runs and as %0, so posix_spawn's two names (FILE, what
    runs; argv[0], what the program is called) cannot both reach a batch
    file.  With system { 'rm' } 'swissarmy', 'foo' and rm.cmd found as
    C:\bin\rm.cmd, that first word can be: (1) the path found,
    "C:\bin\rm.cmd" (now; %0 shows the path); (2) FILE as written, rm
    (cmd.exe looks it up again, in the current directory first, then PATH:
    another file may run than the one found and checked); (3) argv[0],
    swissarmy (cmd.exe runs whatever swissarmy names, not FILE at all).
    Perl's own spawn: (3), as far as Claude knows (its command line starts
    with argv[0], and Windows hands a batch file's command line to cmd.exe
    as it is; not checked on real Windows, and under Wine Perl's batch
    files fail); its t/win32/system.t, which expects %0 as written, has
    FILE and argv[0] the same, where (2) and (3) agree.  Act: (1), (2), or
    (2) as an option (Perl's copy, for its old %0).  Claude's view
    (2026-11-14): (1), (2) only as an option, never (3).

21. X61's rest: `read` with backslashes in the line, where dash and bash
    differ from each other (found 2026-11-13 with X61's random cases):
    for the user to decide case by case (pspawn-known-issues.md, X61).

B. WORK (no decision needed)

 - FIRST IN THE NEXT CYCLE (the user, 2026-11-18: after the release, which
   is made in the configuration as tested; all found reading the code, none
   of it run):
   1. X67 (ours): Proc::Spawner::spawn_unsupported (Spawner.pm, the line
      "if (grep $m{$_}, qw(dup2 close open closefrom))"): where the libc
      lacks one fd action, every fd action of every spawn is done in Perl's
      own process around the call, even when the call asks for none of the
      missing kind.  Hit: a libc without addclosefrom (glibc before 2.34,
      musl), and t/flavors.pl's level 2, which so tests posix_spawn with
      every fd action done in the parent, not "the redirections only".
      Fix: decide per call (only an action of a missing kind asked for).
   2. A test of closefrom: no test file calls it, and nothing in the
      distribution uses it (only Proc::Spawner's interface).
   3. Where closefrom alone is missing, a wrapper around posix_spawn: the
      closefrom replaced by a close for each fd from N up open in the child
      at that point (Perl's inheritable fds, and the earlier dup2 and open
      targets, minus the earlier closes), as system1 does already
      (run_redirs); the fds from /proc/self/fd where present, not one fcntl
      per fd up to the open-files limit (which may be 1,048,576).  Low
      value until something uses closefrom.
   4. F1, the user's decision: the fork primitive (__spawn_nowait, M2) does
      nothing between fork and exec but the signal mask, as a plain spawn
      would, so setsid, pgroup => 0, tcsetpgrp, sched and cgroup are
      refused, and rlimit, sigignore, nice are flipped in the parent around
      the call (not thread-safe), though the child could do them all
      itself.  (a) keep it; (b) every setting in the child, between fork
      and exec, errors through the exec pipe; (c) (b), and a test-only
      level keeping today's plain spawn (Claude's view).  Only the fork
      level reaches __spawn_nowait; its other paths (emulate_actions, the
      settings around the call) are reached by level 2 and the matrix too.
   5. system1 on Linux refuses rlimit, sigignore and nice, which fork does
      (found 2026-11-18 by t/flavors-outcomes.pl: 6 tests skipped, and 2
      of win_priority): their wrappers in C (M1, around_begin and
      around_end in Spawner.xs) are called around posix_spawn and fork,
      not around Run's start.  Calling them there too would make system1
      as capable (not thread-safe, as for the other two).

 - W6 and W7 (post-0.12; doc/pspawn-spawner.md, section 8): try them with a
   real Ctrl-C and Ctrl-Break (under Wine with a pseudo-terminal, as the
   tricks topic on Wine says, or on Windows): the foreground child gets the
   Ctrl-C, a background job does not; and what Perl's kill TERM does to a
   child in Perl's group (it leads no process group).

 - win_spawn.h and Perl (post-0.12; A20): once the patch to Perl exists,
   use Perl's posix_spawn where Perl has it (by its %Config entry) and keep
   win_spawn.h for older Perls; section 5 of doc/perl-win32-system-bugs.md
   then becomes per Perl version.  Where Perl has system(1, LIST) without
   fork, M2's fallback (system1, through Run) can use it.  Until then,
   after any change to win_spawn.h tell the user, so that Perl's copy
   follows (the interface and names there are not settled).

 - The parser's work list (large-subset-state.md, section 4, 44 samples):
   the real gaps alias :=, export ${foo?}, command -, unset -f ls /bin/ls.
 - cd - with OLDPWD unset (dash prints the directory; ours prints nothing);
   exec foo's message lacks "exec: " (X5's kind).
 - X30: $LINENO makes parsing quadratic (count the lines once).
 - X1's rest (K5, post-0.12: at exit ours waits only while it has
   something left to start for a job, then leaves the running processes,
   as the shells do; PROC_SHELLSPAWN_END): a job whose next command ours
   still has to start keeps ours from exiting until then (the shells leave
   a copy of themselves for it).  To exit at once as they do, ours would
   run such a job in a Perl process of its own from the start (sh as Perl,
   SH1), handing it the state it needs (variables, functions, $1..., the
   options, the open descriptors); on Linux only the parent gets a child's
   status, so taking a job over at exit is not possible; on Windows a
   helper could take over the job's process handles (DuplicateHandle).
 - The choices now set by environment variables (PROC_SHELLSPAWN_END,
   _ECHO, _SH, _BIN, _INPROC, _ENVNAMES; PROC_SPAWNER_MISSING): a stopgap (the user,
   2026-11-15); eventually each is an argument of the API, with no action
   at a distance.
 - X3 brace expansion; X4 redir_strict; X6 no default PATH; X9 the virtual
   directory passed as a path; X15 reference cycles left in the shell layer;
   X17 messages inside eval; X24 a trap that re-sends its own signal.
 - Settings posix_spawn lacks (M1, M2; pspawn-spawner.md, section 8): a
   helper program that sets in the child what cannot be done around the
   call (setsid, pgroup 0, tcsetpgrp, sched, cgroup).
 - Windows (doc/pspawn-shell-windows.md): kill of another process and time
   (X21's rest); ulimit; a test on real Windows.  The snippet library under
   Wine, now that the shell layer runs there: ours alone on the samples
   where it calls no program but those done inside Perl (23,898 by the
   logging run of post-0.09), against ours on Linux (large-subset-state.md,
   section 7, "W5": the driver was a scratch script, not kept).  Of the
   user's follow-ups to WS1 (2026-11-09) one remains: cat, sort, grep and
   the like have so far only BIN1, a temporary workaround (the user,
   2026-11-10; PROC_SHELLSPAWN_BIN, else Git for Windows' or MSYS2's
   programs; doc/pspawn-shell-windows.md): what replaces it is open.
 - X10: only Linux is tested (and Windows under Wine).
 - "Through Run" (large-subset-state.md, section 7) was run before IP1,
   with the coreutils programs; the table itself is rebuilt for 0.10's
   in-process default, and W5 checked over it on Linux (2026-11-09: no
   change).
 - X32: without /proc, Perl's own writes into a pipe the caller gave can
   block if a child fills the pipe between select and the write (possible,
   not observed; no test yet).
 - Messages: section 11 of large-subset-environment.md counts them nowhere
   (post-0.11), section 7 of large-subset-state.md and README.md's run table
   still count a message on standard output or in a file as a difference.
   Bring section 7 in line: large-subset-runreport.pl as matchreport.pl
   (drop the shells' message lines from standard output; files by
   files_nomsg, which needs a new full run); then make docs-check.
 - X56: exec with a builtin's name runs ours' builtin, where dash and bash
   look only for a program (not found, 127).  Fix in the base grammar
   (refuse after exec what the large one refuses); the parse results are
   then measured again.
 - large-subset-difffuzz.pl, what is left after seed 3 (-j 4, two hours,
   315,187 strings, 285,676 accepted; it found X50 and X51): of the 826
   where dash and bash agree and ours not, 683 have a background job's
   EXIT trap (its output comes after the shell ended, or not at all:
   chance), 119 other strings with & (X2's effects: wait $! gives 0 for a
   job started without a process), 24 without & (X50 and X51, fixed, and
   return || shift, A13); a background job that sends a signal to $$
   ends with the main shell in ours, while in the shells it is another
   process and goes on (X2's kind).  Not looked at: the 456 where all three differ.
 - X27, step 3, the 93 classified by Claude reading them (A19), what each category
   takes in the setups (large-subset-setups.pl): AI-names with an
   expansion, a value the text itself sets (a for or read variable, an
   assignment: for f in */; do cd "$f"; find | while read d; do cd "$d";
   tmpdir=$(mktemp -d) && cd $tmpdir; 19): follow the value through the
   text (a glob's matches, a stand-in's answer, the fixed word); a program
   at a fixed path not on PATH (2): a stand-in placed there; $HOME/... (5):
   as ~/...; AI-tilde: users added to the run's own /etc/passwd; AI-A5:
   names with .. in the run's own root; AI-shells' own variables: OLDPWD
   varied; AI-PATH-twister: the inherited PATH varied (unset, empty, a
   missing directory); AI-glob: example names for patterns with non-ASCII,
   a / inside, system directories; AI-strict on a device: a private /dev
   with the device absent, a file, not writable.

C. WATCH (seen once, not reproduced)

 - t/programs.t, SH1 (10 strings with sh), fails as a user other than root when
   PATH has a directory that user may not search (here one under /root):
   dash says "Permission denied" for a command found nowhere, ours "not
   found" (seen 2026-11-10, branch x27-step2; not from its changes).
 - X8, X13, X20.

D. REMINDERS (the user, 2026-11-17: "should we keep these as they are?";
   mention them now and then, do not insist)

 - JK1 to JK3, the decision page "Executor Idiom Review": its last three
   items (JK1: its text is on the page only).
 - JK2 (the user, 2026-11-15): the
   shell layer counts by hand who still needs each open file (hold,
   release, release_all, and $tail: the tables to release once the last
   process has started; 42 calls).  The page said this cannot be left to
   Perl, since the rule of 2026-10-30 (pspawn-design.md, section 5) is to
   forget an open file once every process that could use it has started,
   which comes before the last Perl reference goes (the code waiting for
   the child still refers to the table).  The user thinks it might be,
   with weak references: to investigate whether the waiting code could
   hold the tables weakly, so that the last strong reference goes when
   the last process has started, and what commands would then see a pipe
   closed at another moment ({ false || cat /etc/hostname; } | wc -l is
   the document's example); also X15, the reference cycles left there.
 - JK3 (the user, 2026-11-15): whether to rewrite test and [ in
   a Perlish way (Proc::ShellSpawn::Programs).  Today their eight
   functions mirror coreutils 9.4's test.c one for one (posixtest, expr,
   and, term, unary_operator, binary_operator, advance, beyond), which
   keeps the messages (')' expected, found ']') and oddities (-l STRING)
   the same; a rewrite must keep them, checked against the program.
