Decorate CLI program with subprocess.Popen

Question

I want to decorate python.exe. For example, it can be just Input:\n when we write to stdin and Output:\n when we read from stdout prefixes in interactive mode:

Original python.exe:

$ python
Python 3.6.1 |Anaconda custom (64-bit)| (default, Mar 22 2017, 20:11:04) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> print(2)
2
>>> 2 + 2
4
>>>

Excepted decorated python.exe:

$ decorated_python
Output:
Python 3.6.1 |Anaconda custom (64-bit)| (default, Mar 22 2017, 20:11:04) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> 
Input:
print(2)
Output:
2
>>> 
Input:
2 + 2
Output:
4
>>>

I think it should look like this:

import subprocess

pid = subprocess.Popen("python".split(), stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

while True:
    pid.stdin.write(input('Input:\n').encode())
    print('Output:\n' + pid.stdout.readlines())

But pid.stdout.readlines() never done.

I also tried to use communicate method but it work only first time:

import subprocess

pid = subprocess.Popen("python".split(), stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

while True:
    print('Output:\n', pid.communicate(input('Input:\n').encode()))

Test:

Input:
print(1)
Output:
 (b'1\r\n', b'')
Input:
pritn(1)
Traceback (most recent call last):
  File "C:/Users/adr-0/OneDrive/Projects/Python/AdrianD/temp/tmp.py", line 6, in <module>
    print('Output:\n', pid.communicate(input('Input:\n').encode()))
  File "C:\Users\adr-0\Anaconda3.6\lib\subprocess.py", line 811, in communicate
    raise ValueError("Cannot send input after starting communication")
ValueError: Cannot send input after starting communication

Maybe I just miss something, because if I put just 2 in python I will get 2. But I can not get this 2 with communicate method:

Pure python:

>>> 2
2

Decorated with communicate method:

Input:
2
Output:
 (b'', b'')

score 1 · Answer 1 · answered Oct 15 '17 at 22:01

if you look at python documentation you can find that when using stdin/stdout=PIPE it is hardly advised to not use read/write actions on those streams asit can cause deadlock - which you actually experienced while doing readlines: https://docs.python.org/2/library/subprocess.html#popen-objects

The next problem is due

"Popen.communicate() is a helper method that does a one-time write of data to stdin and creates threads to pull data from stdout and stderr. It closes stdin when its done writing data and reads stdout and stderr until those pipes close. You can't do a second communicate because the child has already exited by the time it returns." @tdelaney

more at: Multiple inputs and outputs in python subprocess communicate

Generally doing interactive child process is hard, you may try to use https://pexpect.readthedocs.io/en/stable/

Decorate CLI program with subprocess.Popen

1 Answers1