pelutils.job_parser package

Typed argument and configuration parsing that treats a run as a job.

argparse handles command-line flags but knows nothing about config files, so the moment you want to save a run’s settings, reproduce it later, or launch a sweep of several runs, you end up hand-rolling a config file parser, merging it with the CLI args by hand, and writing your own record of what was actually run. JobParser does all of that: you declare your arguments once with a type, and it resolves values from the command line and an INI config file into one or more JobDescription objects — CLI values overriding config values, which override declared defaults. A single config file can describe many named jobs, and every resolved job can write out an exact, human-readable record of how it was invoked.

Quick start

from pathlib import Path
from pelutils.job_parser import Flag, JobParser, OptionalArg, RequiredArg

parser = JobParser(
    RequiredArg("data-path", help="Training data directory"),
    OptionalArg("learning-rate", default=1e-4, type=float, help="Optimizer learning rate"),
    Flag("fp16", help="Use mixed precision"),
    multiple_jobs=True,
)

for job in parser.parse_jobs():
    # Names are --kebab-case on the CLI and snake_case attributes on the job
    print(job.name, job.data_path, job.learning_rate, job.fp16)
    job.write_documentation(Path("runs") / job.name / "arguments.ini")
    ...  # Run your application with the resolved job values

Declare RequiredArg for values every job must supply, OptionalArg for values with a default, and Flag for booleans. Point the parser at a config file with --config-file config.ini; a file may contain a shared [DEFAULT] section plus named sections that each become a job. Target a single section directly with --config-file config.ini:section-name. A config file for the parser above might look like:

[DEFAULT]
data-path = /data/train    # Shared by every job below

[baseline]
# Inherits the default learning rate; fp16 as a bare key means True
fp16

[high-lr]
learning-rate = 5e-4

For the common single-job case, drop multiple_jobs=True and call JobParser.parse_job() to get one JobDescription back. See JobParser for the full API and JobDescription.write_documentation() for auto-documenting a run.

exception pelutils.job_parser.ConfigError

Bases: JobParserError

Raised when a configuration file cannot be resolved into the requested jobs.

class pelutils.job_parser.Flag(name: str, *, abbrev: str | None = None, help: str | None = None, **kwargs: Any)

Bases: _AbstractArgument

Declare a boolean command-line flag.

A flag defaults to False and becomes True when supplied on the CLI. INI files may use a bare key for True or an explicit True/False value.

Parameters:

• name (str) – Long option name without leading dashes.

• abbrev (str | None, optional) – Optional single-letter short option, without its leading dash.

• help (str | None, optional) – Help text shown by --help.

property default: bool

The default value for a Flag is always False.

class pelutils.job_parser.JobDescription(name: str, explicit_args: set[str], docfile_content: str, **kwargs: Any)

Bases: Namespace

Values resolved for one job by JobParser.

Parsed values are available as attributes and through job["option-name"]; dash-separated and underscore-separated keys are interchangeable. name is derived from a configuration section, an explicit --name, or a timestamp. explicit_args records values supplied by the configuration or CLI.

given_args_to_dict() → dict[str, Any]

Return the resolved public job values as a dictionary.

Parser bookkeeping and private attributes are not included in the result.

write_documentation(path: str | Path, *, append: bool = True)

Write the resolved invocation and configuration to path.

Parent directories are created automatically. Set append=False to replace an existing file; by default, documentation is appended.

class pelutils.job_parser.JobParser(*arguments: RequiredArg | OptionalArg | Flag, description: str | None = None, multiple_jobs: bool = False)

Resolve typed command-line arguments and INI configuration files into jobs.

Declare values with RequiredArg, OptionalArg, and Flag. Values supplied on the command line override configuration values, which override optional defaults. A configuration file may contain a [DEFAULT] section and named sections; named sections become individual jobs.

Use parse_job() when exactly one job is expected. Set multiple_jobs=True and use parse_jobs() when a configuration can select multiple named sections.

Parameters:

• *arguments (RequiredArg | OptionalArg | Flag) – Application-specific argument declarations.

• description (str | None, optional) – Description displayed by the generated --help command.

• multiple_jobs (bool, optional) – Allow configurations with multiple named sections. Use parse_jobs() to retrieve their job descriptions if this is True, otherwise parse_job().

parse_job() → JobDescription

Parse command-line and configuration input into exactly one job.

Use this method for a normal one-job invocation. It also works with a selected single section when multiple_jobs=True. Command-line values take precedence over configuration values.

Raises:

ConfigError – If the selected configuration resolves to multiple jobs.

parse_jobs() → list[JobDescription]

Parse command-line and configuration input into one or more jobs.

This method requires multiple_jobs=True when the configuration resolves to multiple named sections. A command-line-only invocation returns a one-item list.

property reserved_abbreviations: set[str]

Argument abbreviations which are reserved.

property reserved_names: set[str]

Argument names which are reserved.

exception pelutils.job_parser.JobParserError

Bases: Exception

Raised when command-line or configuration input is invalid.

class pelutils.job_parser.OptionalArg(name: str, *, default: _T | None = None, type: Callable[[str], _T] | None = None, abbrev: str | None = None, help: str | None = None, metavar: str | tuple[str, ...] | None = None, nargs: int | None = None, **kwargs: Any)

Bases: _AbstractArgument

Declare a command-line argument with a default value.

The value type is inferred from default when type is omitted. For list values, provide nargs and ensure every default item has the same type.

Parameters:

• name (str) – Long option name without leading dashes.

• default (_T | None, optional) – Value used when the option is absent from both the CLI and configuration. If nargs is set, this must be a tuple or list.

• type (Callable[[str], _T] | None, optional) – Function used to convert supplied values. Inferred as the type of default when omitted. If nargs is set, it refers to the type of each element.

• abbrev (str | None, optional) – Optional single-letter short option, without its leading dash.

• help (str | None, optional) – Help text shown by --help.

• metavar (str | tuple[str, ...] | None, optional) – Value label displayed in command-line help.

• nargs (int | None, optional) – Exact number of values required. Set to 0 to accept any number of values. If set, arguments should be given as space-separated values. Each element is converted from a string to the correct type using the type callable. default should either be None or a list or tuple.

class pelutils.job_parser.RequiredArg(name: str, *, type: ~collections.abc.Callable[[str], ~pelutils.job_parser._structs._T] = <class 'str'>, abbrev: str | None = None, help: str | None = None, metavar: str | tuple[str, ...] | None = None, nargs: int | None = None, **kwargs: ~typing.Any)

Bases: _AbstractArgument

Declare a command-line argument which every job must provide.

Values can be supplied by a command-line option or an INI configuration field. The argument is exposed as --<name> and, when possible, receives an automatic single-letter abbreviation.

Parameters:

• name (str) – Long option name without leading dashes. Dashes are converted to underscores in the resulting JobDescription attribute.

• type (Callable[[str], _T], optional) – Function used to convert supplied values. If nargs is set, it refers to the type of each element.

• abbrev (str | None, optional) – Optional single-letter short option, without its leading dash.

• help (str | None, optional) – Help text shown by --help.

• metavar (str | tuple[str, ...] | None, optional) – Value label displayed in command-line help.

• nargs (int | None, optional) – Exact number of values required. Set to 0 to accept any number of values. If set, arguments should be given as space-separated values. Each element is converted from a string to the correct type using the type callable.