How to get argparse to read arguments from a file with an option after positional arguments

Question

I would like to be able to put command-line arguments in a file, and then pass them to a python program, with argparse, using an option rather than a prefix character, for instance: $ python myprogram.py 1 2 --foo 1 -A somefile.txt --bar 2 This is almost the same as this question, except that I need to have some positional arguments at the start; when that solution calls parse_args, it fails if the file does not have the positional arguments in it.

Answer 1

If somefile.txt contains

one
two
three

then

$ python myprogram.py 1 2 --foo 1 @somefile.txt --bar 2

using the prefix char is effectively the same as

$ python myprogram.py 1 2 --foo 1 one two three --bar 2

In other words, right at the start of parsing, the @ file is read, and its contents are spliced into the argv list. From there parsing occurs normally.

One thing I suggested in the other SO question was to implement that splicing yourself, prior to parsing.

The answer that you are referring to uses a custom Action; at the point where the usual Action just places the value in the namespace, this action reads and parses the file:

parser.parse_args(f.read().split(), namespace)

A variant parses into a new namespace and selectively copies values to the main namespace.

Apparently your problem is that your parser has some required arguments, and this parse_args raises an error if the file does not contain those. That's not surprising.

One solution is to use a different parser for this file, one that is designed to work with just the content of the file. I would just define it globally.

alt_parser.parse_args(f.read().split(), namespace)

In other words, you don't have to use the parser that was passed in as a parameter.

A variation on this is to put the filename in the namespace, and handle it after the first parsing:

args = parser.parse_args()
if args.A:
    argv = open(args.A).read().split()
    args1 = alt_parser.parse_args(argv)

But you might ask, can't we tell what arguments have already been parsed, and take that into account in handling -A? The parser is not a state-machine; parsing does not alter any of its attributes. The only variable that reflects parsing so far is the namespace. All other parsing variables are local to the calling code.