subprocess stdin buffer not flushing on newline with bufsize=1

Ryan Haining

I have two small python files, the first reads a line using input and then prints another line

a = input()
print('complete')

The second attempts to run this as a subprocess

import subprocess

proc = subprocess.Popen('./simp.py',
                        stdout=subprocess.PIPE,
                        stdin=subprocess.PIPE,
                        bufsize=1)
print('writing')
proc.stdin.write(b'hey\n')
print('reading')
proc.stdout.readline()

The above script will print "writing" then "reading" but then hang. At first I thought this was a stdout buffering issue, so I changed bufsize=1 to bufsize=0, and this does fix the problem. However, it seems it's the stdin that's causing the problem.

With bufsize=1, if I add proc.stdin.flush() below the write, the process continues. Both of these approaches seem clumsy since (1) unbuffered streams are slow (2) adding flushes everywhere is error-prone. Why does the above write not flush on a newline? The docs say that bufsize is used when creating stdin, stdout, and stderr stream for the subprocess, so what's causing the write to not flush on the newline?

tdelaney

From the docs: "1 means line buffered (only usable if universal_newlines=True i.e., in a text mode)". This works:

import subprocess

proc = subprocess.Popen('./simp.py',
                        stdout=subprocess.PIPE,
                        stdin=subprocess.PIPE,
                        bufsize=1,
                        universal_newlines=True)

print('writing')
proc.stdin.write('hey\n')
print('reading')
proc.stdout.readline()

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related