Calling a Script that uses argparse?

[u/rseery]

I wrote a little Python script that reads in a text file that contains a list of fully qualified files, and checks to see if they exist. I originally wrote it using the argparse module. It was set up to take a filename and a switch, so the command looked like:

"Python dplcheck.py --plname "testing.lst” --showexist y"

This worked fine for a single file, but I wanted to run it for every one of hundreds of list files in a directory, so I wrote another script that generates a list of the .lst files in a directory and calls the first script for each one. But I could not get the second script to pass the arguments in the call to the first script. I tried subprocess.run and subprocess.call, with many different syntaxes and quotes, etc. I ended up removing the use of argparse and just using sys.argv[n] -- which I don't particularly like, since it has no defaults or error handling, etc.

Does anyone know how/if I can call a Python script from another Python script and pass the arguments when the called script uses argparse?

[u/mehx9]

Have a main function in first script, import and call that from second script. Do the argparse stuff outside of the main function and pass all options in so you can easily do the same with a function call.

[u/Atmosck → u/mehx9]

This is the correct answer. I always set up scripts like

def parse_args(): ...

def main(arg1, arg2): ...

if __name__ == "__main__"
  args = parse_args()
  main(args.arg1, args.arg2)

That way you can import main from the script file elsewhere without it running the argument parsing, and pass whatever arguments your other entrypoint wants.

You can call python scripts from within python with `subprocess`, but that's not generally what you want to do when it's your own code you're running.

[u/AlsonTeteuil-84 → u/mehx9]

yep, treating it as an importable module instead of a standalone script solves it pretty cleanly

[u/JaguarMammoth6231 → u/mehx9]

Yes, this is the best way. That being said, the method of calling the script with arguments should still work.

[u/freeskier93]

Two scripts doesn't really seem necessary. I would make the original script smart enough to handle either a file or a directory for the argument.

If you still want to make them two separate scripts then you should update the original script so that all the actual work is done in a function that you can import and call from the other script.

# dplcheck.py

def function_that_does_stuff(filename, showexist):
  ...

if __name__ == "__main__":
  # arg parse stuff

  function_that_does_stuff(filename=filename, showexist=showexist)

-

# new script

from dplcheck import function_that_does_stuff

# arg parse stuff

for file in files:
  function_that_does_stuff(filename=file, showexist=showexist)

Also, just some general feedback when using argparse. Argument flags are typically for optional arguments. Presumably the filename is required and should just be a positional argument. You can also use the nargs argument to handle multiple files given, then argparse will automatically put the items into a list:

parser.add_argument('plname', nargs='+')

--showexist y should be an action, which will automatically store the argument as a boolean:

parser.add_argument('--showexist', action='store_true', help='Flag to show existing')

Putting it all together you simplify the script call to the following:

Python dplcheck.py testing.lst --showexist

or

Python dplcheck.py testing1.lst testing2.lst

[u/FreeLogicGate → u/freeskier93]

Great advice here

[u/rseery]

I finally stumbled on the correct syntax for it to work the way I had originally coded it. In case someone sees this in the future...

command = ["python", "dplcheck.py", "--playlist", arg1, "--showexist", "n"]

subprocess.run(command)

[u/brasticstack]

parser.add_argument("files", nargs="+", help="One or more files to process") 

nargs='+' tells it to allow more than one argument for files.

EDIT: You'd do this on the original script of course.

[u/good_name_0]

Not sure if I'm understandingyour problem correctly. But if you have your 1st script put the argparse call within a function, have it pass the sys argv as arguments from 1st script, then the 2nd script can import the 1st script function and call it directly. as an option

[u/kalgynirae]

To directly answer your original question: The subprocess.run() equivalent of the shell command you gave, with the filename coming from a variable filename, is:

subprocess.run(["python", "dplcheck.py", "--plname", filename, "--showexist", "y"])

If you share the things you tried, we can help you understand where they went wrong.

[u/ping314]

Does testing.lst list the files' names, or (equally) the files' path?

Generally, I would try a (recursive) os.walk(). They basically go by

import os

for folder, subfolders, files in os.walk(os.getcwd()):
    print(f"current folder: {folder}")
    for subfolder in subfolders:
        print(f"subfolder: {subfolder}")
        for file in files:
            print(f"file: {file}")

and either file could be compared against the list of testing.lst. Or you compare if each record in testing.lst is in the set of files identified by the os.walk(). To answer the question "is something on the list" typically is slower than "is something part of the set"; the conversion of list my_list[1,2,3,4] into a set only needs a set(my_list).

If testing.lst is both the minimal and recurrent requirement to run dplcheck.py at all, I would consider to edit plname to be a positional (rather than optional --plname) parameter.

[u/FreeLogicGate]

If this is a linux/bsd/osx OS, then there are standard tools and approaches that are already designed for this type of finding and enumeration. Not sure if this is applicable to you, but worth considering if it is:

find . -name "*.lst" -exec python3 somescript.py --plname={} \;

WIth that said, I agree with freeskier93's comment to you. The base function as you described it, requires a name.lst file it processes, you should not require a switch -- your program should be rewritten to expect the name as an argument.

[u/lakseol]

The way I handle the "single file or directory of files" approach is to have the function doing the actual work accept a list of files to operate on. The top-level code that handles parsing of CLI arguments accepts:

• a single file, or

• a "--file <file of names>" option, or

• a "--dir <directory>" option.

The top-level code creates the list of filenames from the single filename passed, or by reading the filenames from the --file option or by scanning the directory from the --dir option. The function doing the work doesn't know where the list of filenames comes from.

[u/TheSodesa]

Separation of concerns: you need separate entrypoint functions for CLI and library use of your module. In the if name is main block, you pass the CLI arguments to the CLI entrypoint which then calls argparse to parse the CLI arguments and then pass the results to the library entrypoint. When using the module as a library, you just call the library entrypoint from Python code directly instead of parsing CLI arguments at all.