Feature request: handle streaming output

[rayburgemeestre]

Feature

For longer-running processes that produce output continuously, I think it would be nice to have the ability to process each line of output on the fly. Additionally the ability to cleanly terminate the process as soon as some predicate is True.

Why

I'm thinking that this is useful for the following scenario that came up in my case while writing integration test code:

• start monitoring some log files with tail -f foobar.log.
• execute something that should result in a log message in foobar.log
• if XYZ is in the standard output of the tail process --- then: we found the confirmation we were looking for and we can stop the tail command.
• if XYZ was not found in the standard output --- then: we can throw an error after X amount of time.

Example

This is a demonstration of what I have in mind with subprocess:

import subprocess


def test():
    # setup tail
    with subprocess.Popen(
        "touch /tmp/test.log && timeout 10s tail -f /tmp/test.log",
        shell=True,
        stdout=subprocess.PIPE,
    ) as proc:

        # do something (in this case fake some log activity)
        subprocess.check_output("echo Hello world > /tmp/test.log", shell=True)

        while True:
            line = proc.stdout.readline().decode("utf-8")
            if not line:
                break

            if "Hello" in line:
                # we found what we were looking for
                return True

        return False


print(test())  # True in this case