Skip to content

Commit d1ea9b3

Browse files
committed
gh-158633: say that subprocess text mode guesses the child's encoding
Text mode documents which encoding is used and not that the value is a guess about the child process, so a wrong guess reads as a bug in this module: the output is either silently mojibake, or a UnicodeDecodeError raised while the stream is read, reported from inside subprocess rather than from the call that is missing encoding=. The warning keeps to what was asked for on gh-105312: the default is a guess, pass encoding= on every platform, and on Windows more than one default is in force at once -- the ANSI code page and the console output code page -- so a console child is read with the console page while a Python child can be told what to write through PYTHONUTF8 / PYTHONIOENCODING.
1 parent 1a85213 commit d1ea9b3

1 file changed

Lines changed: 23 additions & 0 deletions

File tree

‎Doc/library/subprocess.rst‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -308,6 +308,29 @@ default values. The arguments that are most commonly needed are:
308308
If text mode is not used, *stdin*, *stdout* and *stderr* will be opened as
309309
binary streams. No encoding or line ending conversion is performed.
310310

311+
.. warning::
312+
313+
In text mode, the encoding used when *encoding* is not given is a guess
314+
about the child process rather than information about it. If the guess is
315+
wrong, the output is decoded incorrectly: either silently, producing
316+
mojibake, or as a :exc:`UnicodeDecodeError` raised while the stream is
317+
read, which is reported from inside this module rather than from the call
318+
that is missing the argument. Passing *encoding* explicitly, chosen for
319+
the program being run, is recommended on every platform, and this stays
320+
true where the default is UTF-8 (see :pep:`686`): what the child writes is
321+
the child's choice, not the parent's.
322+
323+
A wrong guess is most likely on Windows, where more than one default is in
324+
force at once -- the ANSI code page that
325+
:func:`locale.getpreferredencoding` reports, and the console output code
326+
page that console programs write in, reported by the Windows
327+
``GetConsoleOutputCP`` API. These are commonly different values, so no
328+
single encoding is correct for every child of one process: a console
329+
program such as :program:`cmd` is read with the console output code page,
330+
while a Python child can be told which encoding to write through the
331+
:envvar:`PYTHONUTF8` and :envvar:`PYTHONIOENCODING` environment variables
332+
in its *env*.
333+
311334
.. versionchanged:: 3.6
312335
Added the *encoding* and *errors* parameters.
313336

0 commit comments

Comments
 (0)