4

I have a config file with the following contents.

[Default] file_list = ['dataset1/*.wav', 'dataset2/*.wav', 'dataset3/*.wav', 'dataset4/*.wav'] 

which is a line break that most of the time works in Python. I parse it with this

import configargparse p = configargparse.ArgParser() p.add('-c', '--my-config', required=True, is_config_file=True, help='config file path') p.add('--file_list', action="append", required=False, help='path to genome file') args = p.parse_args() 

by making the appropriate call

python file.py --my-config=config.ini 

The problem is (not surprisingly):

file.py: error: unrecognized arguments: --'dataset4/*.wav'] true 

I assume this happens because it expects a new arg in the config file. I also tried different line continuation methods.

How can I input multiple line lists using configargparse? If I can't, what is a minimal alternative solution to config file list inputs?

2
  • 1
    Did you ever manage to find an answer to your question? Commented Feb 28, 2020 at 12:04
  • Unfortunately no Commented Feb 28, 2020 at 13:18

2 Answers 2

3

I came across the same problem. I decided to modify the default parser a bit and create my parser which supports multiple lines. I only added a few lines of code. It looks something like this:

import configargparse from collections import OrderedDict import re class LongLineConfigFileParser(configargparse.ConfigFileParser): """Based on a simplified subset of INI and YAML formats. Here is the supported syntax: # this is a comment ; this is also a comment (.ini style) --- # lines that start with --- are ignored (yaml style) ------------------- [section] # .ini-style section names are treated as comments # how to specify a key-value pair (all of these are equivalent): name value # key is case sensitive: "Name" isn't "name" name = value # (.ini style) (white space is ignored, so name = value same as name=value) name: value # (yaml style) --name value # (argparse style) # how to set a flag arg (eg. arg which has action="store_true") --name name name = True # "True" and "true" are the same # how to specify a list arg (eg. arg which has action="append") fruit = [apple, orange, lemon] indexes = [1, 12, 35 , 40] # how to break a long line into multiple lines: fruit = [apple, \ orage, \ lemon] """ def parse(self, stream): """Parses the keys + values from a config file.""" items = OrderedDict() pre_line = '' # Used to support multiple lines for i, line in enumerate(stream): line = pre_line + line line = line.strip() if not line or line[0] in ["#", ";", "["] or line.startswith("---"): continue # Used to support multiple lines if line.endswith('\\'): pre_line = line.replace('\\', '').strip() continue else: pre_line = '' white_space = "\\s*" key = r"(?P<key>[^:=;#\s]+?)" value = white_space+r"[:=\s]"+white_space+"(?P<value>.+?)" comment = white_space+"(?P<comment>\\s[;#].*)?" key_only_match = re.match("^" + key + comment + "$", line) if key_only_match: key = key_only_match.group("key") items[key] = "true" continue key_value_match = re.match("^"+key+value+comment+"$", line) if key_value_match: key = key_value_match.group("key") value = key_value_match.group("value") if value.startswith("[") and value.endswith("]"): # handle special case of lists value = [elem.strip() for elem in value[1:-1].split(",")] items[key] = value continue raise configargparse.ConfigFileParserException("Unexpected line {} in {}: {}".format(i, getattr(stream, 'name', 'stream'), line)) return items 

To use it you only need to add config_file_parser_class=LongLineConfigFileParser when calling the ArgParser:

parser = configargparse.ArgParser(default_config_files=['DefaultParams.txt'], config_file_parser_class=LongLineConfigFileParser) 

Hope this helps

Sign up to request clarification or add additional context in comments.

1 Comment

I will make sure to try this in one of my projects, but thanks for taking the time to solve this! I marked it as answer for now.
-1

You'd probably want use nargs arg:

parser.add_argument('--my-arg', nargs="*", default=[]) 

* for 0 or more args

+ for 1 or more

? for 0 or 1

in shell you can use:

python scrypt.py --my-arg 1 2 3 bla 

this approach wasn't correctly work in conf files and env until 2021 but seems fixed now

for config files or environment variables you should pass:

MY_ARG = "[1, 2, 3, bla]" 

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.