snakemake package

Submodules

snakemake.benchmark module

snakemake.benchmark.BENCHMARK_INTERVAL = 30

Interval (in seconds) between measuring resource usage

snakemake.benchmark.BENCHMARK_INTERVAL_SHORT = 0.5

Interval (in seconds) between measuring resource usage before BENCHMARK_INTERVAL

class snakemake.benchmark.BenchmarkRecord(running_time=None, max_rss=None, max_vms=None, max_uss=None, max_pss=None, io_in=None, io_out=None, cpu_seconds=None)[源代码]

基类:object

Record type for benchmark times

cpu_seconds = None

Count of CPU seconds, divide by running time to get mean load estimate

first_time = None

First time when we measured CPU load, for estimating total running time

classmethod get_header()[源代码]
io_in = None

I/O read in bytes

io_out = None

I/O written in bytes

max_pss = None

Maximal PSS in MB

max_rss = None

Maximal RSS in MB

max_uss = None

Maximal USS in MB

max_vms = None

Maximal VMS in MB

prev_time = None

Previous point when measured CPU load, for estimating total running time

running_time = None

Running time in seconds

to_tsv()[源代码]

Return str with the TSV representation of this record

class snakemake.benchmark.BenchmarkTimer(pid, bench_record, interval=30)[源代码]

基类:snakemake.benchmark.ScheduledPeriodicTimer

Allows easy observation of a given PID for resource usage

bench_record = None

BenchmarkRecord to write results to

pid = None

PID of observed process

work()[源代码]

Write statistics

class snakemake.benchmark.DaemonTimer(interval, function, args=None, kwargs=None)[源代码]

基类:threading.Thread

Variant of threading.Timer that is deaemonized

cancel()[源代码]

Stop the timer if it hasn’t finished yet.

run()[源代码]

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

class snakemake.benchmark.ScheduledPeriodicTimer(interval)[源代码]

基类:object

Scheduling of periodic events

Up to self._interval, schedule actions per second, above schedule events in self._interval second gaps.

cancel()[源代码]

Call to cancel any events

start()[源代码]

Start the intervalic timer

work()[源代码]

Override to perform the action

snakemake.benchmark.benchmarked(pid=None, benchmark_record=None, interval=30)[源代码]

Measure benchmark parameters while within the context manager

Yields a BenchmarkRecord with the results (values are set after leaving context).

If pid is None then the PID of the current process will be used. If benchmark_record is None then a new BenchmarkRecord is created and returned, otherwise, the object passed as this parameter is returned.

Usage:

with benchmarked() as bench_result:
    pass
snakemake.benchmark.print_benchmark_records(records, file_)[源代码]

Write benchmark records to file-like object

snakemake.benchmark.write_benchmark_records(records, path)[源代码]

Write benchmark records to file at path

snakemake.checkpoints module

class snakemake.checkpoints.Checkpoint(rule, checkpoints)[源代码]

基类:object

checkpoints
get(**wildcards)[源代码]
rule
class snakemake.checkpoints.CheckpointJob(rule, output)[源代码]

基类:object

output
rule
class snakemake.checkpoints.Checkpoints[源代码]

基类:object

A namespace for checkpoints so that they can be accessed via dot notation.

register(rule)[源代码]

snakemake.common module

class snakemake.common.Mode[源代码]

基类:object

Enum for execution mode of Snakemake. This handles the behavior of e.g. the logger.

cluster = 2
default = 0
subprocess = 1
snakemake.common.escape_backslash(path)[源代码]
snakemake.common.get_container_image()[源代码]
snakemake.common.get_last_stable_version()[源代码]
snakemake.common.get_uuid(name)[源代码]
class snakemake.common.lazy_property(method)[源代码]

基类:property

cached
static clean(instance, method)[源代码]
method
snakemake.common.log_location(msg)[源代码]
snakemake.common.num_if_possible(s)[源代码]

Convert string to number if possible, otherwise return string.

snakemake.common.strip_prefix(text, prefix)[源代码]

snakemake.conda module

class snakemake.conda.Conda(singularity_img=None)[源代码]

基类:object

bin_path()[源代码]
instances = {}
prefix_path()[源代码]
shellcmd(env_path, cmd)[源代码]
class snakemake.conda.Env(env_file, dag, singularity_img=None)[源代码]

基类:object

Conda environment from a given specification file.

archive_file

Path to archive of the conda environment, which may or may not exist.

content
content_hash
create(dryrun=False)[源代码]

Create the conda enviroment.

create_archive()[源代码]

Create self-contained archive of environment.

classmethod get_singularity_envvars()[源代码]
hash
path

Path to directory of the conda environment.

First tries full hash, if it does not exist, (8-prefix) is used as default.

singularity_img_url
snakemake.conda.content(env_file)[源代码]

snakemake.cwl module

snakemake.cwl.cwl(path, basedir, input, output, params, wildcards, threads, resources, log, config, rulename, use_singularity, bench_record, jobid)[源代码]

Load cwl from the given basedir + path and execute it.

snakemake.cwl.dag_to_cwl(dag)[源代码]

Convert a given DAG to a CWL workflow, which is returned as JSON object.

snakemake.cwl.job_to_cwl(job, dag, outputs, inputs)[源代码]

Convert a job with its dependencies to a CWL workflow step.

snakemake.dag module

class snakemake.dag.DAG(workflow, rules=None, dryrun=False, targetfiles=None, targetrules=None, forceall=False, forcerules=None, forcefiles=None, priorityfiles=None, priorityrules=None, untilfiles=None, untilrules=None, omitfiles=None, omitrules=None, ignore_ambiguity=False, force_incomplete=False, ignore_incomplete=False, notemp=False, keep_remote_local=False)[源代码]

基类:object

Directed acyclic graph of jobs.

archive(path)[源代码]

Archives workflow such that it can be re-run on a different system.

Archiving includes git versioned files (i.e. Snakefiles, config files, …), ancestral input files and conda environments.

bfs(direction, *jobs, stop=<function DAG.<lambda>>)[源代码]

Perform a breadth-first traversal of the DAG.

check_and_touch_output(job, wait=3, ignore_missing_output=False, no_touch=False, force_stay_on_remote=False)[源代码]

Raise exception if output files of job are missing.

check_dynamic()[源代码]

Check dynamic output and update downstream rules if necessary.

check_incomplete()[源代码]

Check if any output files are incomplete. This is done by looking up markers in the persistence module.

check_periodic_wildcards(job)[源代码]

Raise an exception if a wildcard of the given job appears to be periodic, indicating a cyclic dependency.

checkpoint_jobs
clean(only_temp=False, dryrun=False)[源代码]

Removes files generated by the workflow.

cleanup()[源代码]
close_remote_objects()[源代码]

Close all remote objects.

collect_potential_dependencies(job)[源代码]

Collect all potential dependencies of a job. These might contain ambiguities.

create_conda_envs(dryrun=False, forceall=False, init_only=False, quiet=False)[源代码]
d3dag(max_jobs=10000)[源代码]
delete_job(job, recursive=True, add_dependencies=False)[源代码]

Delete given job from DAG.

delete_omitfrom_jobs()[源代码]

Removes jobs downstream of jobs specified by –omit-from.

dfs(direction, *jobs, stop=<function DAG.<lambda>>, post=True)[源代码]

Perform depth-first traversal of the DAG.

dot()[源代码]
downstream_of_omitfrom()[源代码]

Returns the downstream of –omit-from rules or files.

dynamic(job)[源代码]

Return whether a job is dynamic (i.e. it is only a placeholder for those that are created after the job with dynamic output has finished.

dynamic_output_jobs

Iterate over all jobs with dynamic output files.

file2jobs(targetfile)[源代码]
finish(job, update_dynamic=True)[源代码]

Finish a given job (e.g. remove from ready jobs, mark depending jobs as ready).

finished(job)[源代码]

Return whether a job is finished.

finished_jobs

Iterate over all jobs that have been finished.

get_jobs_or_groups()[源代码]
handle_log(job, upload_remote=True)[源代码]
handle_pipes()[源代码]

Use pipes to determine job groups. Check if every pipe has exactly one consumer

handle_protected(job)[源代码]

Write-protect output files that are marked with protected().

handle_remote(job, upload=True)[源代码]

Remove local files if they are no longer needed and upload.

handle_temp(job)[源代码]

Remove temp files if they are no longer needed. Update temp_mtimes.

handle_touch(job)[源代码]

Touches those output files that are marked for touching.

in_omitfrom(job)[源代码]

Return whether given job has been specified via –omit-from.

in_until(job)[源代码]

Return whether given job has been specified via –until.

incomplete_external_jobid(job)[源代码]

Return the external jobid of the job if it is marked as incomplete.

Returns None, if job is not incomplete, or if no external jobid has been registered or if force_incomplete is True.

incomplete_files

Return list of incomplete files.

init(progress=False)[源代码]

Initialise the DAG.

jobid(job)[源代码]

Return job id of given job.

jobs

All jobs in the DAG.

level_bfs(direction, *jobs, stop=<function DAG.<lambda>>)[源代码]

Perform a breadth-first traversal of the DAG, but also yield the level together with each job.

list_untracked()[源代码]

List files in the workdir that are not in the dag.

local_needrun_jobs

Iterate over all jobs that need to be run and are marked as local.

missing_temp(job)[源代码]

Return whether a temp file that is input of the given job is missing.

needrun(job)[源代码]

Return whether a given job needs to be executed.

needrun_jobs

Jobs that need to be executed.

new_job(rule, targetfile=None, format_wildcards=None)[源代码]

Create new job for given rule and (optional) targetfile. This will reuse existing jobs with the same wildcards.

new_wildcards(job)[源代码]

Return wildcards that are newly introduced in this job, compared to its ancestors.

newversion_files

Return list of files where the current version is newer than the recorded version.

noneedrun_finished(job)[源代码]

Return whether a given job is finished or was not required to run at all.

omitfrom_jobs()[源代码]

Returns a generator of jobs specified by omitfromjobs.

postprocess()[源代码]

Postprocess the DAG. This has to be invoked after any change to the DAG topology.

priority(job)[源代码]

Return priority of given job.

pull_singularity_imgs(dryrun=False, forceall=False, quiet=False)[源代码]
ready_jobs

Jobs that are ready to execute.

reason(job)[源代码]

Return the reason of the job execution.

replace_job(job, newjob, recursive=True)[源代码]

Replace given job with new job.

requested_files(job)[源代码]

Return the files a job requests.

rule2job(targetrule)[源代码]

Generate a new job from a given rule.

rule_dot()[源代码]
rule_dot2()[源代码]
set_until_jobs()[源代码]

Removes jobs downstream of jobs specified by –omit-from.

specialize_rule(rule, newrule)[源代码]

Specialize the given rule by inserting newrule into the DAG.

stats()[源代码]
summary(detailed=False)[源代码]
temp_input(job)[源代码]
temp_size(job)[源代码]

Return the total size of temporary input files of the job. If none, return 0.

unshadow_output(job, only_log=False)[源代码]

Move files from shadow directory to real output paths.

until_jobs()[源代码]

Returns a generator of jobs specified by untiljobs.

update(jobs, file=None, visited=None, skip_until_dynamic=False, progress=False)[源代码]

Update the DAG by adding given jobs and their dependencies.

update_(job, visited=None, skip_until_dynamic=False, progress=False)[源代码]

Update the DAG by adding the given job and its dependencies.

update_checkpoint_dependencies(jobs=None)[源代码]

Update dependencies of checkpoints.

update_checkpoint_outputs()[源代码]
update_dynamic(job)[源代码]

Update the DAG by evaluating the output of the given job that contains dynamic output files.

update_groups()[源代码]
update_jobids()[源代码]
update_needrun()[源代码]

Update the information whether a job needs to be executed.

update_output_index()[源代码]

Update the OutputIndex.

update_priority()[源代码]

Update job priorities.

update_ready(jobs=None)[源代码]

Update information whether a job is ready to execute.

Given jobs must be needrun jobs!

snakemake.decorators module

snakemake.decorators.dec_all_methods(decorator, prefix='test_')[源代码]

snakemake.exceptions module

exception snakemake.exceptions.AmbiguousRuleException(filename, job_a, job_b, lineno=None, snakefile=None)[源代码]

基类:snakemake.exceptions.RuleException

exception snakemake.exceptions.ChildIOException(parent=None, child=None, wildcards=None, lineno=None, snakefile=None, rule=None)[源代码]

基类:snakemake.exceptions.WorkflowError

exception snakemake.exceptions.ClusterJobException(job_info, jobid)[源代码]

基类:snakemake.exceptions.RuleException

exception snakemake.exceptions.CreateCondaEnvironmentException(*args, lineno=None, snakefile=None, rule=None)[源代码]

基类:snakemake.exceptions.WorkflowError

exception snakemake.exceptions.CreateRuleException(message=None, include=None, lineno=None, snakefile=None, rule=None)[源代码]

基类:snakemake.exceptions.RuleException

exception snakemake.exceptions.CyclicGraphException(repeatedrule, file, rule=None)[源代码]

基类:snakemake.exceptions.RuleException

exception snakemake.exceptions.DropboxFileException(msg, lineno=None, snakefile=None)[源代码]

基类:snakemake.exceptions.RuleException

exception snakemake.exceptions.FTPFileException(msg, lineno=None, snakefile=None)[源代码]

基类:snakemake.exceptions.RuleException

exception snakemake.exceptions.HTTPFileException(msg, lineno=None, snakefile=None)[源代码]

基类:snakemake.exceptions.RuleException

exception snakemake.exceptions.IOException(prefix, rule, files, include=None, lineno=None, snakefile=None)[源代码]

基类:snakemake.exceptions.RuleException

exception snakemake.exceptions.IOFileException(msg, lineno=None, snakefile=None)[源代码]

基类:snakemake.exceptions.RuleException

exception snakemake.exceptions.ImproperOutputException(rule, files, include=None, lineno=None, snakefile=None)[源代码]

基类:snakemake.exceptions.IOException

exception snakemake.exceptions.ImproperShadowException(rule, lineno=None, snakefile=None)[源代码]

基类:snakemake.exceptions.RuleException

exception snakemake.exceptions.IncompleteCheckpointException(rule, targetfile)[源代码]

基类:Exception

exception snakemake.exceptions.IncompleteFilesException(files)[源代码]

基类:snakemake.exceptions.RuleException

exception snakemake.exceptions.InputFunctionException(msg, wildcards=None, lineno=None, snakefile=None, rule=None)[源代码]

基类:snakemake.exceptions.WorkflowError

exception snakemake.exceptions.MissingInputException(rule, files, include=None, lineno=None, snakefile=None)[源代码]

基类:snakemake.exceptions.IOException

exception snakemake.exceptions.MissingOutputException(message=None, include=None, lineno=None, snakefile=None, rule=None)[源代码]

基类:snakemake.exceptions.RuleException

exception snakemake.exceptions.MissingRuleException(file, lineno=None, snakefile=None)[源代码]

基类:snakemake.exceptions.RuleException

exception snakemake.exceptions.NCBIFileException(msg, lineno=None, snakefile=None)[源代码]

基类:snakemake.exceptions.RuleException

exception snakemake.exceptions.NoRulesException(lineno=None, snakefile=None)[源代码]

基类:snakemake.exceptions.RuleException

exception snakemake.exceptions.PeriodicWildcardError(message=None, include=None, lineno=None, snakefile=None, rule=None)[源代码]

基类:snakemake.exceptions.RuleException

exception snakemake.exceptions.ProtectedOutputException(rule, files, include=None, lineno=None, snakefile=None)[源代码]

基类:snakemake.exceptions.IOException

exception snakemake.exceptions.RemoteFileException(msg, lineno=None, snakefile=None)[源代码]

基类:snakemake.exceptions.RuleException

exception snakemake.exceptions.RuleException(message=None, include=None, lineno=None, snakefile=None, rule=None)[源代码]

基类:Exception

Base class for exception occuring within the execution or definition of rules.

messages
exception snakemake.exceptions.S3FileException(msg, lineno=None, snakefile=None)[源代码]

基类:snakemake.exceptions.RuleException

exception snakemake.exceptions.SFTPFileException(msg, lineno=None, snakefile=None)[源代码]

基类:snakemake.exceptions.RuleException

exception snakemake.exceptions.SpawnedJobError[源代码]

基类:Exception

exception snakemake.exceptions.TerminatedException[源代码]

基类:Exception

exception snakemake.exceptions.UnexpectedOutputException(rule, files, include=None, lineno=None, snakefile=None)[源代码]

基类:snakemake.exceptions.IOException

exception snakemake.exceptions.UnknownRuleException(name, prefix='', lineno=None, snakefile=None)[源代码]

基类:snakemake.exceptions.RuleException

exception snakemake.exceptions.WebDAVFileException(msg, lineno=None, snakefile=None)[源代码]

基类:snakemake.exceptions.RuleException

exception snakemake.exceptions.WildcardError(*args, lineno=None, snakefile=None, rule=None)[源代码]

基类:snakemake.exceptions.WorkflowError

exception snakemake.exceptions.WorkflowError(*args, lineno=None, snakefile=None, rule=None)[源代码]

基类:Exception

static format_arg(arg)[源代码]
exception snakemake.exceptions.XRootDFileException(msg, lineno=None, snakefile=None)[源代码]

基类:snakemake.exceptions.RuleException

snakemake.exceptions.cut_traceback(ex)[源代码]
snakemake.exceptions.format_error(ex, lineno, linemaps=None, snakefile=None, show_traceback=False)[源代码]
snakemake.exceptions.format_traceback(tb, linemaps)[源代码]
snakemake.exceptions.get_exception_origin(ex, linemaps)[源代码]
snakemake.exceptions.log_verbose_traceback(ex)[源代码]
snakemake.exceptions.print_exception(ex, linemaps)[源代码]

Print an error message for a given exception.

Arguments ex – the exception linemaps – a dict of a dict that maps for each snakefile

the compiled lines to source code lines in the snakefile.

snakemake.executors module

class snakemake.executors.AbstractExecutor(workflow, dag, printreason=False, quiet=False, printshellcmds=False, printthreads=True, latency_wait=3)[源代码]

基类:object

cancel()[源代码]
get_default_remote_provider_args()[源代码]
handle_job_error(job)[源代码]
handle_job_success(job)[源代码]
print_job_error(job, msg=None, **kwargs)[源代码]
printjob(job)[源代码]
rule_prefix(job)[源代码]
run(job, callback=None, submit_callback=None, error_callback=None)[源代码]
shutdown()[源代码]
class snakemake.executors.CPUExecutor(workflow, dag, workers, printreason=False, quiet=False, printshellcmds=False, use_threads=False, latency_wait=3, cores=1)[源代码]

基类:snakemake.executors.RealExecutor

cancel()[源代码]
handle_job_error(job)[源代码]
handle_job_success(job)[源代码]
job_args_and_prepare(job)[源代码]
run(job, callback=None, submit_callback=None, error_callback=None)[源代码]
run_group_job(job)[源代码]

Run a pipe group job.

This lets all items run simultaneously.

run_single_job(job)[源代码]
shutdown()[源代码]
spawn_job(job)[源代码]
class snakemake.executors.ClusterExecutor(workflow, dag, cores, jobname='snakejob.{name}.{jobid}.sh', printreason=False, quiet=False, printshellcmds=False, latency_wait=3, cluster_config=None, local_input=None, restart_times=None, exec_job=None, assume_shared_fs=True, max_status_checks_per_second=1)[源代码]

基类:snakemake.executors.RealExecutor

cancel()[源代码]
cluster_params(job)[源代码]

Return wildcards object for job from cluster_config.

cluster_wildcards(job)[源代码]
default_jobscript = 'jobscript.sh'
format_job(pattern, job, **kwargs)[源代码]
get_jobscript(job)[源代码]
handle_job_error(job)[源代码]
handle_job_success(job)[源代码]
print_cluster_job_error(job_info, jobid)[源代码]
shutdown()[源代码]
tmpdir
write_jobscript(job, jobscript, **kwargs)[源代码]
class snakemake.executors.DRMAAClusterJob(job, jobid, callback, error_callback, jobscript)

基类:tuple

callback

Alias for field number 2

error_callback

Alias for field number 3

job

Alias for field number 0

jobid

Alias for field number 1

jobscript

Alias for field number 4

class snakemake.executors.DRMAAExecutor(workflow, dag, cores, jobname='snakejob.{rulename}.{jobid}.sh', printreason=False, quiet=False, printshellcmds=False, drmaa_args='', drmaa_log_dir=None, latency_wait=3, cluster_config=None, restart_times=0, assume_shared_fs=True, max_status_checks_per_second=1)[源代码]

基类:snakemake.executors.ClusterExecutor

cancel()[源代码]
run(job, callback=None, submit_callback=None, error_callback=None)[源代码]
shutdown()[源代码]
class snakemake.executors.DryrunExecutor(workflow, dag, printreason=False, quiet=False, printshellcmds=False, printthreads=True, latency_wait=3)[源代码]

基类:snakemake.executors.AbstractExecutor

class snakemake.executors.GenericClusterExecutor(workflow, dag, cores, submitcmd='qsub', statuscmd=None, cluster_config=None, jobname='snakejob.{rulename}.{jobid}.sh', printreason=False, quiet=False, printshellcmds=False, latency_wait=3, restart_times=0, assume_shared_fs=True, max_status_checks_per_second=1)[源代码]

基类:snakemake.executors.ClusterExecutor

cancel()[源代码]
register_job(job)[源代码]
run(job, callback=None, submit_callback=None, error_callback=None)[源代码]
class snakemake.executors.GenericClusterJob(job, jobid, callback, error_callback, jobscript, jobfinished, jobfailed)

基类:tuple

callback

Alias for field number 2

error_callback

Alias for field number 3

job

Alias for field number 0

jobfailed

Alias for field number 6

jobfinished

Alias for field number 5

jobid

Alias for field number 1

jobscript

Alias for field number 4

class snakemake.executors.KubernetesExecutor(workflow, dag, namespace, envvars, container_image=None, jobname='{rulename}.{jobid}', printreason=False, quiet=False, printshellcmds=False, latency_wait=3, cluster_config=None, local_input=None, restart_times=None)[源代码]

基类:snakemake.executors.ClusterExecutor

cancel()[源代码]
register_secret()[源代码]
run(job, callback=None, submit_callback=None, error_callback=None)[源代码]
shutdown()[源代码]
unregister_secret()[源代码]
class snakemake.executors.KubernetesJob(job, jobid, callback, error_callback, kubejob, jobscript)

基类:tuple

callback

Alias for field number 2

error_callback

Alias for field number 3

job

Alias for field number 0

jobid

Alias for field number 1

jobscript

Alias for field number 5

kubejob

Alias for field number 4

class snakemake.executors.RealExecutor(workflow, dag, printreason=False, quiet=False, printshellcmds=False, latency_wait=3, assume_shared_fs=True)[源代码]

基类:snakemake.executors.AbstractExecutor

format_job_pattern(pattern, job=None, **kwargs)[源代码]
handle_job_error(job, upload_remote=True)[源代码]
handle_job_success(job, upload_remote=True, handle_log=True, handle_touch=True, ignore_missing_output=False)[源代码]
register_job(job)[源代码]
class snakemake.executors.SynchronousClusterExecutor(workflow, dag, cores, submitcmd='qsub', cluster_config=None, jobname='snakejob.{rulename}.{jobid}.sh', printreason=False, quiet=False, printshellcmds=False, latency_wait=3, restart_times=0, assume_shared_fs=True)[源代码]

基类:snakemake.executors.ClusterExecutor

invocations like “qsub -sync y” (SGE) or “bsub -K” (LSF) are synchronous, blocking the foreground thread and returning the remote exit code at remote exit.

cancel()[源代码]
run(job, callback=None, submit_callback=None, error_callback=None)[源代码]
class snakemake.executors.SynchronousClusterJob(job, jobid, callback, error_callback, jobscript, process)

基类:tuple

callback

Alias for field number 2

error_callback

Alias for field number 3

job

Alias for field number 0

jobid

Alias for field number 1

jobscript

Alias for field number 4

process

Alias for field number 5

class snakemake.executors.TouchExecutor(workflow, dag, printreason=False, quiet=False, printshellcmds=False, latency_wait=3, assume_shared_fs=True)[源代码]

基类:snakemake.executors.RealExecutor

handle_job_success(job)[源代码]
run(job, callback=None, submit_callback=None, error_callback=None)[源代码]
snakemake.executors.change_working_directory(directory=None)[源代码]

Change working directory in execution context if provided.

snakemake.executors.run_wrapper(job_rule, input, output, params, wildcards, threads, resources, log, benchmark, benchmark_repeats, conda_env, singularity_img, singularity_args, use_singularity, linemaps, debug, shadow_dir, jobid)[源代码]

Wrapper around the run method that handles exceptions and benchmarking.

Arguments job_rule – the job.rule member input – list of input files output – list of output files wildcards – so far processed wildcards threads – usable threads log – list of log files shadow_dir – optional shadow directory root

snakemake.executors.sleep()[源代码]

snakemake.gui module

snakemake.gui.dag()[源代码]
snakemake.gui.dryrun()[源代码]
snakemake.gui.get_args()[源代码]
snakemake.gui.index()[源代码]
snakemake.gui.log(id)[源代码]
snakemake.gui.progress()[源代码]
snakemake.gui.register(run_snakemake, args)[源代码]
snakemake.gui.run()[源代码]
snakemake.gui.run_snakemake(**kwargs)[源代码]
snakemake.gui.set_args()[源代码]
snakemake.gui.status()[源代码]
snakemake.gui.targets()[源代码]

snakemake.io module

class snakemake.io.AnnotatedString(value)[源代码]

基类:str

class snakemake.io.IOCache[源代码]

基类:object

clear()[源代码]
deactivate()[源代码]
snakemake.io.IOFile(file, rule=None)[源代码]
class snakemake.io.InputFiles(toclone=None, fromdict=None, plainstr=False, strip_constraints=False, custom_map=None)[源代码]

基类:snakemake.io.Namedlist

size
class snakemake.io.Log(toclone=None, fromdict=None, plainstr=False, strip_constraints=False, custom_map=None)[源代码]

基类:snakemake.io.Namedlist

class snakemake.io.Namedlist(toclone=None, fromdict=None, plainstr=False, strip_constraints=False, custom_map=None)[源代码]

基类:list

A list that additionally provides functions to name items. Further, it is hashable, however the hash does not consider the item names.

add_name(name)[源代码]

Add a name to the last item.

Arguments name – a name

allitems()[源代码]
clone()[源代码]
get(key, default_value=None)[源代码]
get_names()[源代码]

Get the defined names as (name, index) pairs.

insert_items(index, items)[源代码]
items()[源代码]
keys()[源代码]
plainstrings()[源代码]
set_name(name, index, end=None)[源代码]

Set the name of an item.

Arguments name – a name index – the item index

stripped_constraints()[源代码]
take_names(names)[源代码]

Take over the given names.

Arguments names – the given names as (name, index) pairs

class snakemake.io.OutputFiles(toclone=None, fromdict=None, plainstr=False, strip_constraints=False, custom_map=None)[源代码]

基类:snakemake.io.Namedlist

class snakemake.io.Params(toclone=None, fromdict=None, plainstr=False, strip_constraints=False, custom_map=None)[源代码]

基类:snakemake.io.Namedlist

class snakemake.io.PeriodicityDetector(min_repeat=20, max_repeat=100)[源代码]

基类:object

is_periodic(value)[源代码]

Returns the periodic substring or None if not periodic.

class snakemake.io.ReportObject(caption, category)

基类:tuple

caption

Alias for field number 0

category

Alias for field number 1

class snakemake.io.Resources(toclone=None, fromdict=None, plainstr=False, strip_constraints=False, custom_map=None)[源代码]

基类:snakemake.io.Namedlist

class snakemake.io.Wildcards(toclone=None, fromdict=None, plainstr=False, strip_constraints=False, custom_map=None)[源代码]

基类:snakemake.io.Namedlist

snakemake.io.ancient(value)[源代码]

A flag for an input file that shall be considered ancient; i.e. its timestamp shall have no effect on which jobs to run.

snakemake.io.apply_wildcards(pattern, wildcards, fill_missing=False, fail_dynamic=False, dynamic_fill=None, keep_dynamic=False)[源代码]
snakemake.io.checkpoint_target(value)[源代码]
snakemake.io.contains_wildcard(path)[源代码]
snakemake.io.contains_wildcard_constraints(pattern)[源代码]
snakemake.io.directory(value)[源代码]

A flag to specify that an output is a directory, rather than a file or named pipe.

snakemake.io.dynamic(value)[源代码]

A flag for a file that shall be dynamic, i.e. the multiplicity (and wildcard values) will be expanded after a certain rule has been run

snakemake.io.expand(*args, **wildcards)[源代码]

Expand wildcards in given filepatterns.

Arguments *args – first arg: filepatterns as list or one single filepattern,

second arg (optional): a function to combine wildcard values (itertools.product per default)
**wildcards – the wildcards as keyword arguments
with their values as lists
snakemake.io.flag(value, flag_type, flag_value=True)[源代码]
snakemake.io.get_flag_value(value, flag_type)[源代码]
snakemake.io.get_git_root(path)[源代码]
参数:path – (str) Path a to a directory/file that is located inside the repo
返回:path to root folder for git repo
snakemake.io.get_git_root_parent_directory(path, input_path)[源代码]

This function will recursively go through parent directories until a git repository is found or until no parent directories are left, in which case a error will be raised. This is needed when providing a path to a file/folder that is located on a branch/tag no currently checked out.

参数:
  • path – (str) Path a to a directory that is located inside the repo
  • input_path – (str) origin path, used when raising WorkflowError
返回:

path to root folder for git repo

snakemake.io.get_wildcard_names(pattern)[源代码]
snakemake.io.git_content(git_file)[源代码]

This function will extract a file from a git repository, one located on the filesystem. Expected format is git+file:///path/to/your/repo/path_to_file@@version

参数:env_file (str) – consist of path to repo, @, version and file information Ex: git+file:////home/smeds/snakemake-wrappers/bio/fastqc/wrapper.py@0.19.3
返回:file content or None if the expected format isn’t meet
snakemake.io.glob_wildcards(pattern, files=None, followlinks=False)[源代码]

Glob the values of the wildcards by matching the given pattern to the filesystem. Returns a named tuple with a list of values for each wildcard.

snakemake.io.is_callable(value)[源代码]
snakemake.io.is_flagged(value, flag)[源代码]
snakemake.io.lchmod(f, mode)[源代码]
snakemake.io.limit(pattern, **wildcards)[源代码]

Limit wildcards to the given values.

Arguments: **wildcards – the wildcards as keyword arguments

with their values as lists
snakemake.io.load_configfile(configpath)[源代码]

Loads a JSON or YAML configfile as a dict, then checks that it’s a dict.

snakemake.io.local(value)[源代码]

Mark a file as local file. This disables application of a default remote provider.

snakemake.io.lstat(f)[源代码]
snakemake.io.lutime(f, times)[源代码]
snakemake.io.not_iterable(value)[源代码]
snakemake.io.pipe(value)[源代码]
snakemake.io.protected(value)[源代码]

A flag for a file that shall be write protected after creation.

snakemake.io.regex(filepattern)[源代码]
snakemake.io.remove(file, remove_non_empty_dir=False)[源代码]
snakemake.io.repeat(value, n_repeat)[源代码]

Flag benchmark records with the number of repeats.

snakemake.io.report(value, caption=None, category=None)[源代码]

Flag output file as to be included into reports.

snakemake.io.split_git_path(path)[源代码]
snakemake.io.strip_wildcard_constraints(pattern)[源代码]

Return a string that does not contain any wildcard constraints.

snakemake.io.temp(value)[源代码]

A flag for an input or output file that shall be removed after usage.

snakemake.io.temporary(value)[源代码]

An alias for temp.

snakemake.io.touch(value)[源代码]
snakemake.io.unpack(value)[源代码]
snakemake.io.update_wildcard_constraints(pattern, wildcard_constraints, global_wildcard_constraints)[源代码]

Update wildcard constraints

参数:
  • pattern (str) – pattern on which to update constraints
  • wildcard_constraints (dict) – dictionary of wildcard:constraint key-value pairs
  • global_wildcard_constraints (dict) – dictionary of wildcard:constraint key-value pairs
snakemake.io.wait_for_files(files, latency_wait=3, force_stay_on_remote=False, ignore_pipe=False)[源代码]

Wait for given files to be present in filesystem.

snakemake.jobs module

class snakemake.jobs.AbstractJob[源代码]

基类:object

download_remote_input()[源代码]
is_group()[源代码]
log_error(msg=None, **kwargs)[源代码]
log_info(skip_dynamic=False)[源代码]
properties(omit_resources=['_cores', '_nodes'], **aux_properties)[源代码]
remove_existing_output()[源代码]
class snakemake.jobs.GroupJob(id, jobs)[源代码]

基类:snakemake.jobs.AbstractJob

all_products
attempt
check_protected_output()[源代码]
cleanup()[源代码]
dag
download_remote_input()[源代码]
dynamic_input
expanded_output

Yields the entire expanded output of all jobs

finalize()[源代码]
format_wildcards(string, **variables)[源代码]

Format a string with variables from the job.

get_targets()[源代码]
get_wait_for_files()[源代码]
groupid
input
inputsize
is_branched
is_checkpoint
is_group()[源代码]
is_local
is_updated
jobid
jobs
log
log_error(msg=None, **kwargs)[源代码]
log_info(skip_dynamic=False)[源代码]
merge(other)[源代码]
name
needs_singularity
output
postprocess(error=False, **kwargs)[源代码]
priority
products
properties(omit_resources=['_cores', '_nodes'], **aux_properties)[源代码]
register()[源代码]
remove_existing_output()[源代码]
resources
restart_times
rules
threads
class snakemake.jobs.Job(rule, dag, wildcards_dict=None, format_wildcards=None, targetfile=None)[源代码]

基类:snakemake.jobs.AbstractJob

HIGHEST_PRIORITY = 9223372036854775807
archive_conda_env()[源代码]

Archive a conda environment into a custom local channel.

attempt
b64id
benchmark
benchmark_repeats
check_protected_output()[源代码]
cleanup()[源代码]

Cleanup output files.

close_remote()[源代码]
conda_env
conda_env_file
conda_env_path
dag
dependencies
download_remote_input()[源代码]
dynamic_input
dynamic_output
dynamic_wildcards

Return all wildcard values determined from dynamic output.

empty_remote_dirs
existing_output
existing_remote_input
existing_remote_output
expand_dynamic(pattern)[源代码]

Expand dynamic files.

expanded_output

Iterate over output files while dynamic output is expanded.

files_to_download
files_to_upload
format_wildcards(string, **variables)[源代码]

Format a string with variables from the job.

get_targets()[源代码]
get_wait_for_files()[源代码]
group
input
input_maxtime

Return newest input file.

inputsize

Return the size of the input files. Input files need to be present.

is_branched
is_checkpoint
is_cwl
is_group()[源代码]
is_local
is_norun
is_run
is_script
is_shadow
is_shell
is_valid()[源代码]

Check if job is valid

is_wrapper
jobid
local_input
local_output
log
log_error(msg=None, indent=False, **kwargs)[源代码]
log_info(skip_dynamic=False, indent=False, printshellcmd=True)[源代码]
message

Return the message for this job.

missing_input

Return missing input files.

missing_output(requested=None)[源代码]

Return missing output files.

missing_remote_input
missing_remote_output
name
needs_singularity
output
output_mintime

Return oldest output file.

output_mintime_local
outputs_older_than_script()[源代码]

return output that’s older than script, i.e. script has changed

params
postprocess(upload_remote=True, handle_log=True, handle_touch=True, handle_temp=True, error=False, ignore_missing_output=False, assume_shared_fs=True, latency_wait=None)[源代码]
prepare()[源代码]

Prepare execution of job. This includes creation of directories and deletion of previously created dynamic files. Creates a shadow directory for the job if specified.

priority
products
properties(omit_resources=['_cores', '_nodes'], **aux_properties)[源代码]
protected_output
register()[源代码]
remote_input
remote_input_newer_than_local
remote_input_older_than_local
remote_output
remote_output_newer_than_local
remote_output_older_than_local
remove_existing_output()[源代码]

Clean up both dynamic and regular output before rules actually run

resources
restart_times
rmdir_empty_remote_dirs()[源代码]
rule
rules
shadow_dir
shadowed_path(f)[源代码]

Get the shadowed path of IOFile f.

shellcmd

Return the shell command.

singularity_img
singularity_img_path
singularity_img_url
subworkflow_input
targetfile
temp_output
threads
touch_output
unique_input
updated()[源代码]
wildcards
wildcards_dict
class snakemake.jobs.Reason[源代码]

基类:object

derived
forced
incomplete_output
missing_output
noio
nooutput
updated_input
updated_input_run
snakemake.jobs.format_files(job, io, dynamicio)[源代码]
snakemake.jobs.jobfiles(jobs, type)[源代码]

snakemake.logging module

class snakemake.logging.ColorizingStreamHandler(nocolor=False, stream=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'>, use_threads=False, mode=0)[源代码]

基类:logging.StreamHandler

BLACK = 0
BLUE = 4
BOLD_SEQ = '\x1b[1m'
COLOR_SEQ = '\x1b[%dm'
CYAN = 6
GREEN = 2
MAGENTA = 5
RED = 1
RESET_SEQ = '\x1b[0m'
WHITE = 7
YELLOW = 3
can_color_tty(mode)[源代码]
colors = {'CRITICAL': 1, 'DEBUG': 4, 'ERROR': 1, 'INFO': 2, 'WARNING': 3}
decorate(record)[源代码]
emit(record)[源代码]

Emit a record.

If a formatter is specified, it is used to format the record. The record is then written to the stream with a trailing newline. If exception information is present, it is formatted using traceback.print_exception and appended to the stream. If the stream has an ‘encoding’ attribute, it is used to determine how to do the output to the stream.

is_tty
class snakemake.logging.Logger[源代码]

基类:object

cleanup()[源代码]
d3dag(**msg)[源代码]
dag_debug(msg)[源代码]
debug(msg)[源代码]
error(msg)[源代码]
get_logfile()[源代码]
group_error(**msg)[源代码]
group_info(**msg)[源代码]
handler(msg)[源代码]
info(msg, indent=False)[源代码]
job_error(**msg)[源代码]
job_finished(**msg)[源代码]
job_info(**msg)[源代码]
location(msg)[源代码]
logfile_hint()[源代码]
progress(done=None, total=None)[源代码]
remove_logfile()[源代码]
resources_info(msg)[源代码]
rule_info(**msg)[源代码]
run_info(msg)[源代码]
set_level(level)[源代码]
set_stream_handler(stream_handler)[源代码]
setup_logfile()[源代码]
shellcmd(msg, indent=False)[源代码]
text_handler(msg)[源代码]

The default snakemake log handler.

Prints the output to the console.

参数:msg (dict) – the log message dictionary
warning(msg)[源代码]
snakemake.logging.format_dict(dict, omit_keys=[], omit_values=[])[源代码]
snakemake.logging.format_resource_names(resources, omit_resources=['_cores', '_nodes'])[源代码]
snakemake.logging.format_resources(dict, *, omit_keys={'_cores', '_nodes'}, omit_values=[])
snakemake.logging.format_wildcards(dict, omit_keys=[], *, omit_values={'__snakemake_dynamic__'})
snakemake.logging.setup_logger(handler=None, quiet=False, printshellcmds=False, printreason=False, debug_dag=False, nocolor=False, stdout=False, debug=False, use_threads=False, mode=0)[源代码]

snakemake.output_index module

class snakemake.output_index.OutputIndex(rules)[源代码]

基类:object

match(targetfile)[源代码]

snakemake.parser module

class snakemake.parser.AbstractCmd(snakefile, rulename, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.Run

args()[源代码]
block_content(token)[源代码]
decorate_end(token)[源代码]
end()[源代码]
end_func = None
is_block_end(token)[源代码]
overwrite_block_content(token)[源代码]
overwrite_cmd = None
start()[源代码]
start_func = None
class snakemake.parser.Benchmark(snakefile, base_indent=0, dedent=0, root=True, rulename=None)[源代码]

基类:snakemake.parser.RuleKeywordState

class snakemake.parser.CWL(snakefile, rulename, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.Script

args()[源代码]
end_func = 'cwl'
start_func = '@workflow.cwl'
class snakemake.parser.Checkpoint(snakefile, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.Rule

start()[源代码]
class snakemake.parser.Conda(snakefile, base_indent=0, dedent=0, root=True, rulename=None)[源代码]

基类:snakemake.parser.RuleKeywordState

class snakemake.parser.Configfile(snakefile, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.GlobalKeywordState

class snakemake.parser.DecoratorKeywordState(snakefile, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.KeywordState

args = []
decorator = None
end()[源代码]
start()[源代码]
class snakemake.parser.GlobalKeywordState(snakefile, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.KeywordState

start()[源代码]
class snakemake.parser.GlobalSingularity(snakefile, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.GlobalKeywordState

keyword
class snakemake.parser.GlobalWildcardConstraints(snakefile, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.GlobalKeywordState

keyword
class snakemake.parser.Group(snakefile, base_indent=0, dedent=0, root=True, rulename=None)[源代码]

基类:snakemake.parser.RuleKeywordState

class snakemake.parser.Include(snakefile, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.GlobalKeywordState

class snakemake.parser.Input(snakefile, base_indent=0, dedent=0, root=True, rulename=None)[源代码]

基类:snakemake.parser.RuleKeywordState

class snakemake.parser.KeywordState(snakefile, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.TokenAutomaton

block(token)[源代码]
block_content(token)[源代码]
colon(token)[源代码]
decorate_end(token)[源代码]
end()[源代码]
is_block_end(token)[源代码]
keyword
prefix = ''
yield_indent(token)[源代码]
class snakemake.parser.Localrules(snakefile, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.GlobalKeywordState

block_content(token)[源代码]
class snakemake.parser.Log(snakefile, base_indent=0, dedent=0, root=True, rulename=None)[源代码]

基类:snakemake.parser.RuleKeywordState

class snakemake.parser.Message(snakefile, base_indent=0, dedent=0, root=True, rulename=None)[源代码]

基类:snakemake.parser.RuleKeywordState

class snakemake.parser.OnError(snakefile, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.DecoratorKeywordState

args = ['log']
decorator = 'onerror'
class snakemake.parser.OnStart(snakefile, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.DecoratorKeywordState

args = ['log']
decorator = 'onstart'
class snakemake.parser.OnSuccess(snakefile, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.DecoratorKeywordState

args = ['log']
decorator = 'onsuccess'
class snakemake.parser.Output(snakefile, base_indent=0, dedent=0, root=True, rulename=None)[源代码]

基类:snakemake.parser.RuleKeywordState

class snakemake.parser.Params(snakefile, base_indent=0, dedent=0, root=True, rulename=None)[源代码]

基类:snakemake.parser.RuleKeywordState

class snakemake.parser.Priority(snakefile, base_indent=0, dedent=0, root=True, rulename=None)[源代码]

基类:snakemake.parser.RuleKeywordState

class snakemake.parser.Python(snakefile, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.TokenAutomaton

python(token)[源代码]
subautomata = {'checkpoint': <class 'snakemake.parser.Checkpoint'>, 'configfile': <class 'snakemake.parser.Configfile'>, 'include': <class 'snakemake.parser.Include'>, 'localrules': <class 'snakemake.parser.Localrules'>, 'onerror': <class 'snakemake.parser.OnError'>, 'onstart': <class 'snakemake.parser.OnStart'>, 'onsuccess': <class 'snakemake.parser.OnSuccess'>, 'report': <class 'snakemake.parser.Report'>, 'rule': <class 'snakemake.parser.Rule'>, 'ruleorder': <class 'snakemake.parser.Ruleorder'>, 'singularity': <class 'snakemake.parser.GlobalSingularity'>, 'subworkflow': <class 'snakemake.parser.Subworkflow'>, 'wildcard_constraints': <class 'snakemake.parser.GlobalWildcardConstraints'>, 'workdir': <class 'snakemake.parser.Workdir'>}
class snakemake.parser.Report(snakefile, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.GlobalKeywordState

class snakemake.parser.Resources(snakefile, base_indent=0, dedent=0, root=True, rulename=None)[源代码]

基类:snakemake.parser.RuleKeywordState

class snakemake.parser.Rule(snakefile, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.GlobalKeywordState

block_content(token)[源代码]
dedent
end()[源代码]
name(token)[源代码]
start(aux='')[源代码]
subautomata = {'benchmark': <class 'snakemake.parser.Benchmark'>, 'conda': <class 'snakemake.parser.Conda'>, 'cwl': <class 'snakemake.parser.CWL'>, 'group': <class 'snakemake.parser.Group'>, 'input': <class 'snakemake.parser.Input'>, 'log': <class 'snakemake.parser.Log'>, 'message': <class 'snakemake.parser.Message'>, 'output': <class 'snakemake.parser.Output'>, 'params': <class 'snakemake.parser.Params'>, 'priority': <class 'snakemake.parser.Priority'>, 'resources': <class 'snakemake.parser.Resources'>, 'run': <class 'snakemake.parser.Run'>, 'script': <class 'snakemake.parser.Script'>, 'shadow': <class 'snakemake.parser.Shadow'>, 'shell': <class 'snakemake.parser.Shell'>, 'singularity': <class 'snakemake.parser.Singularity'>, 'threads': <class 'snakemake.parser.Threads'>, 'version': <class 'snakemake.parser.Version'>, 'wildcard_constraints': <class 'snakemake.parser.WildcardConstraints'>, 'wrapper': <class 'snakemake.parser.Wrapper'>}
class snakemake.parser.RuleKeywordState(snakefile, base_indent=0, dedent=0, root=True, rulename=None)[源代码]

基类:snakemake.parser.KeywordState

start()[源代码]
class snakemake.parser.Ruleorder(snakefile, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.GlobalKeywordState

block_content(token)[源代码]
class snakemake.parser.Run(snakefile, rulename, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.RuleKeywordState

block_content(token)[源代码]
end()[源代码]
is_block_end(token)[源代码]
start()[源代码]
class snakemake.parser.Script(snakefile, rulename, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.AbstractCmd

args()[源代码]
end_func = 'script'
start_func = '@workflow.script'
class snakemake.parser.Shadow(snakefile, base_indent=0, dedent=0, root=True, rulename=None)[源代码]

基类:snakemake.parser.RuleKeywordState

class snakemake.parser.Shell(snakefile, rulename, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.AbstractCmd

args()[源代码]
end_func = 'shell'
start_func = '@workflow.shellcmd'
class snakemake.parser.Singularity(snakefile, base_indent=0, dedent=0, root=True, rulename=None)[源代码]

基类:snakemake.parser.RuleKeywordState

class snakemake.parser.Snakefile(path, rulecount=0)[源代码]

基类:object

exception snakemake.parser.StopAutomaton(token)[源代码]

基类:Exception

class snakemake.parser.Subworkflow(snakefile, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.GlobalKeywordState

block_content(token)[源代码]
end()[源代码]
name(token)[源代码]
subautomata = {'configfile': <class 'snakemake.parser.SubworkflowConfigfile'>, 'snakefile': <class 'snakemake.parser.SubworkflowSnakefile'>, 'workdir': <class 'snakemake.parser.SubworkflowWorkdir'>}
class snakemake.parser.SubworkflowConfigfile(snakefile, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.SubworkflowKeywordState

class snakemake.parser.SubworkflowKeywordState(snakefile, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.KeywordState

end()[源代码]
prefix = 'Subworkflow'
start()[源代码]
class snakemake.parser.SubworkflowSnakefile(snakefile, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.SubworkflowKeywordState

class snakemake.parser.SubworkflowWorkdir(snakefile, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.SubworkflowKeywordState

class snakemake.parser.Threads(snakefile, base_indent=0, dedent=0, root=True, rulename=None)[源代码]

基类:snakemake.parser.RuleKeywordState

class snakemake.parser.TokenAutomaton(snakefile, base_indent=0, dedent=0, root=True)[源代码]

基类:object

consume()[源代码]
dedent
effective_indent
error(msg, token)[源代码]
indentation(token)[源代码]
subautomata = {}
subautomaton(automaton, *args, **kwargs)[源代码]
class snakemake.parser.Version(snakefile, base_indent=0, dedent=0, root=True, rulename=None)[源代码]

基类:snakemake.parser.RuleKeywordState

class snakemake.parser.WildcardConstraints(snakefile, base_indent=0, dedent=0, root=True, rulename=None)[源代码]

基类:snakemake.parser.RuleKeywordState

keyword
class snakemake.parser.Workdir(snakefile, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.GlobalKeywordState

class snakemake.parser.Wrapper(snakefile, rulename, base_indent=0, dedent=0, root=True)[源代码]

基类:snakemake.parser.Script

args()[源代码]
end_func = 'wrapper'
start_func = '@workflow.wrapper'
snakemake.parser.format_tokens(tokens)[源代码]
snakemake.parser.is_colon(token)[源代码]
snakemake.parser.is_comma(token)[源代码]
snakemake.parser.is_comment(token)[源代码]
snakemake.parser.is_dedent(token)[源代码]
snakemake.parser.is_eof(token)[源代码]
snakemake.parser.is_greater(token)[源代码]
snakemake.parser.is_indent(token)[源代码]
snakemake.parser.is_name(token)[源代码]
snakemake.parser.is_newline(token, newline_tokens={4, 56})[源代码]
snakemake.parser.is_op(token)[源代码]
snakemake.parser.is_string(token)[源代码]
snakemake.parser.lineno(token)[源代码]
snakemake.parser.parse(path, overwrite_shellcmd=None, rulecount=0)[源代码]

snakemake.persistence module

class snakemake.persistence.Persistence(nolock=False, dag=None, conda_prefix=None, singularity_prefix=None, shadow_prefix=None, warn_only=False)[源代码]

基类:object

all_inputfiles()[源代码]
all_outputfiles()[源代码]
cleanup(job)[源代码]
cleanup_conda()[源代码]
cleanup_locks()[源代码]
cleanup_metadata(path)[源代码]
cleanup_shadow()[源代码]
code(path)[源代码]
code_changed(job, file=None)[源代码]

Yields output files with changed code of bool if file given.

deactivate_cache()[源代码]
external_jobids(job)[源代码]
files
finished(job)[源代码]
incomplete(job)[源代码]
input(path)[源代码]
input_changed(job, file=None)[源代码]

Yields output files with changed input of bool if file given.

lock()[源代码]
lock_warn_only()[源代码]
locked
log(path)[源代码]
metadata(path)[源代码]
noop(*args)[源代码]
params(path)[源代码]
params_changed(job, file=None)[源代码]

Yields output files with changed params of bool if file given.

rule(path)[源代码]
shellcmd(path)[源代码]
started(job, external_jobid=None)[源代码]
unlock(*args)[源代码]
version(path)[源代码]
version_changed(job, file=None)[源代码]

Yields output files with changed versions of bool if file given.

snakemake.persistence.pickle_code(code)[源代码]

snakemake.rules module

class snakemake.rules.Rule(*args, lineno=None, snakefile=None, restart_times=0)[源代码]

基类:object

apply_default_remote(item)[源代码]
apply_input_function(func, wildcards, incomplete_checkpoint_func=<function Rule.<lambda>>, **aux_params)[源代码]
benchmark
check_output_duplicates()[源代码]

Check Namedlist for duplicate entries and raise a WorkflowError on problems.

check_wildcards(wildcards)[源代码]
conda_env
dynamic_branch(wildcards, input=True)[源代码]
expand_benchmark(wildcards)[源代码]
expand_conda_env(wildcards)[源代码]
expand_group(wildcards)[源代码]

Expand the group given wildcards.

expand_input(wildcards)[源代码]
expand_log(wildcards)[源代码]
expand_output(wildcards)[源代码]
expand_params(wildcards, input, output, resources, omit_callable=False)[源代码]
expand_resources(wildcards, input, attempt)[源代码]
static get_wildcard_len(wildcards)[源代码]

Return the length of the given wildcard values.

Arguments wildcards – a dict of wildcards

get_wildcards(requested_output)[源代码]

Return wildcard dictionary by matching regular expression output files to the requested concrete ones.

Arguments requested_output – a concrete filepath

has_wildcards()[源代码]

Return True if rule contains wildcards.

input
is_producer(requested_output)[源代码]

Returns True if this rule is a producer of the requested output.

log
output
params
products
register_wildcards(wildcard_names)[源代码]
set_input(*input, **kwinput)[源代码]

Add a list of input files. Recursive lists are flattened.

Arguments input – the list of input files

set_log(*logs, **kwlogs)[源代码]
set_output(*output, **kwoutput)[源代码]

Add a list of output files. Recursive lists are flattened.

After creating the output files, they are checked for duplicates.

Arguments output – the list of output files

set_params(*params, **kwparams)[源代码]
set_wildcard_constraints(**kwwildcard_constraints)[源代码]
singularity_img
update_wildcard_constraints()[源代码]
version
wildcard_constraints
wildcard_names
class snakemake.rules.RuleProxy(rule)[源代码]

基类:object

benchmark
input
log
output
params
class snakemake.rules.Ruleorder[源代码]

基类:object

add(*rulenames)[源代码]

Records the order of given rules as rule1 > rule2 > rule3, …

compare(rule1, rule2)[源代码]

Return whether rule2 has a higher priority than rule1.

snakemake.scheduler module

class snakemake.scheduler.DummyRateLimiter[源代码]

基类:contextlib.ContextDecorator

class snakemake.scheduler.JobScheduler(workflow, dag, cores, local_cores=1, dryrun=False, touch=False, cluster=None, cluster_status=None, cluster_config=None, cluster_sync=None, drmaa=None, drmaa_log_dir=None, kubernetes=None, kubernetes_envvars=None, container_image=None, jobname=None, quiet=False, printreason=False, printshellcmds=False, keepgoing=False, max_jobs_per_second=None, max_status_checks_per_second=100, latency_wait=3, greediness=1.0, force_use_threads=False, assume_shared_fs=True)[源代码]

基类:object

calc_resource(name, value)[源代码]
candidate(job)[源代码]

Return whether a job is a candidate to be executed.

exit_gracefully(*args)[源代码]
get_executor(job)[源代码]
job_reward(job)[源代码]
job_selector(jobs)[源代码]
Using the greedy heuristic from “A Greedy Algorithm for the General Multidimensional Knapsack

Problem”, Akcay, Li, Xu, Annals of Operations Research, 2012

Args:
jobs (list): list of jobs
job_weight(job)[源代码]
open_jobs

Return open jobs.

progress()[源代码]

Display the progress.

rule_weight(rule)[源代码]
run(job)[源代码]
schedule()[源代码]

Schedule jobs that are ready, maximizing cpu usage.

stats
snakemake.scheduler.cumsum(iterable, zero=[0])[源代码]

snakemake.script module

class snakemake.script.JuliaEncoder[源代码]

基类:object

Encoding Pyton data structures into Julia.

classmethod encode_dict(d)[源代码]
classmethod encode_items(items)[源代码]
classmethod encode_list(l)[源代码]
classmethod encode_namedlist(namedlist)[源代码]
classmethod encode_positional_items(namedlist)[源代码]
classmethod encode_value(value)[源代码]
class snakemake.script.REncoder[源代码]

基类:object

Encoding Pyton data structures into R.

classmethod encode_dict(d)[源代码]
classmethod encode_items(items)[源代码]
classmethod encode_list(l)[源代码]
classmethod encode_namedlist(namedlist)[源代码]
classmethod encode_numeric(value)[源代码]
classmethod encode_value(value)[源代码]
class snakemake.script.Snakemake(input, output, params, wildcards, threads, resources, log, config, rulename, bench_iteration, scriptdir=None)[源代码]

基类:object

log_fmt_shell(stdout=True, stderr=True, append=False)[源代码]

Return a shell redirection string to be used in shell() calls

This function allows scripts and wrappers support optional log files specified in the calling rule. If no log was specified, then an empty string “” is returned, regardless of the values of stdout, stderr, and append.

参数:
  • stdout (bool) – Send stdout to log
  • stderr (bool) – Send stderr to log
  • append (bool) – Do not overwrite the log file. Useful for sending output of multiple commands to the same log. Note however that the log will not be truncated at the start.
  • following table describes the output (The) –
  • -------- -------- ----- ------------- (--------) –
  • stderr append log return value (stdout) –
  • -------- -------- ----- ------------ (--------) –
  • True True fn >> fn 2>&1 (True) –
  • False True fn >> fn (True) –
  • True True fn 2>> fn (False) –
  • True False fn > fn 2>&1 (True) –
  • False False fn > fn (True) –
  • True False fn 2> fn (False) –
  • any any None "" (any) –
  • -------- -------- ----- ----------- (--------) –
snakemake.script.get_source(path, basedir=None)[源代码]
snakemake.script.script(path, basedir, input, output, params, wildcards, threads, resources, log, config, rulename, conda_env, singularity_img, singularity_args, bench_record, jobid, bench_iteration, shadow_dir)[源代码]

Load a script from the given basedir + path and execute it. Supports Python 3 and R.

snakemake.shell module

class snakemake.shell.shell[源代码]

基类:object

classmethod check_output(cmd, **kwargs)[源代码]
classmethod cleanup()[源代码]
classmethod executable(cmd)[源代码]
classmethod get_executable()[源代码]
static iter_stdout(proc, cmd)[源代码]
classmethod kill(jobid)[源代码]
classmethod prefix(prefix)[源代码]
classmethod suffix(suffix)[源代码]

snakemake.singularity module

class snakemake.singularity.Image(url, dag)[源代码]

基类:object

hash
is_local
path
pull(dryrun=False)[源代码]
snakemake.singularity.shellcmd(img_path, cmd, args='', envvars=None, shell_executable=None, container_workdir=None)[源代码]

Execute shell command inside singularity container given optional args and environment variables to be passed.

snakemake.stats module

class snakemake.stats.Stats[源代码]

基类:object

file_stats
overall_runtime
report_job_end(job)[源代码]
report_job_start(job)[源代码]
rule_stats
to_json(path)[源代码]

snakemake.utils module

class snakemake.utils.AlwaysQuotedFormatter(quote_func=None, *args, **kwargs)[源代码]

基类:snakemake.utils.QuotedFormatter

Subclass of QuotedFormatter that always quotes.

Usage is identical to QuotedFormatter, except that it always acts like “q” was appended to the format spec.

format_field(value, format_spec)[源代码]
class snakemake.utils.QuotedFormatter(quote_func=None, *args, **kwargs)[源代码]

基类:string.Formatter

Subclass of string.Formatter that supports quoting.

Using this formatter, any field can be quoted after formatting by appending “q” to its format string. By default, shell quoting is performed using “shlex.quote”, but you can pass a different quote_func to the constructor. The quote_func simply has to take a string argument and return a new string representing the quoted form of the input string.

Note that if an element after formatting is the empty string, it will not be quoted.

format_field(value, format_spec)[源代码]
snakemake.utils.R(code)[源代码]

Execute R code.

This is deprecated in favor of the script directive. This function executes the R code given as a string. The function requires rpy2 to be installed.

参数:code (str) – R code to be executed
class snakemake.utils.SequenceFormatter(separator=' ', element_formatter=<string.Formatter object>, *args, **kwargs)[源代码]

基类:string.Formatter

string.Formatter subclass with special behavior for sequences.

This class delegates formatting of individual elements to another formatter object. Non-list objects are formatted by calling the delegate formatter’s “format_field” method. List-like objects (list, tuple, set, frozenset) are formatted by formatting each element of the list according to the specified format spec using the delegate formatter and then joining the resulting strings with a separator (space by default).

format_element(elem, format_spec)[源代码]

Format a single element

For sequences, this is called once for each element in a sequence. For anything else, it is called on the entire object. It is intended to be overridden in subclases.

format_field(value, format_spec)[源代码]
class snakemake.utils.Unformattable(errormsg='This cannot be used for formatting')[源代码]

基类:object

snakemake.utils.argvquote(arg, force=True)[源代码]

Returns an argument quoted in such a way that that CommandLineToArgvW on Windows will return the argument string unchanged. This is the same thing Popen does when supplied with an list of arguments. Arguments in a command line should be separated by spaces; this function does not add these spaces. This implementation follows the suggestions outlined here: https://blogs.msdn.microsoft.com/twistylittlepassagesallalike/2011/04/23/everyone-quotes-command-line-arguments-the-wrong-way/

snakemake.utils.available_cpu_count()[源代码]

Return the number of available virtual or physical CPUs on this system. The number of available CPUs can be smaller than the total number of CPUs when the cpuset(7) mechanism is in use, as is the case on some cluster systems.

Adapted from http://stackoverflow.com/a/1006301/715090

snakemake.utils.format(_pattern, *args, stepout=1, _quote_all=False, **kwargs)[源代码]

Format a pattern in Snakemake style.

This means that keywords embedded in braces are replaced by any variable values that are available in the current namespace.

snakemake.utils.linecount(filename)[源代码]

Return the number of lines of given file.

参数:filename (str) – the path to the file
snakemake.utils.listfiles(pattern, restriction=None, omit_value=None)[源代码]

Yield a tuple of existing filepaths for the given pattern.

Wildcard values are yielded as the second tuple item.

参数:
  • pattern (str) – a filepattern. Wildcards are specified in snakemake syntax, e.g. “{id}.txt”
  • restriction (dict) – restrict to wildcard values given in this dictionary
  • omit_value (str) – wildcard value to omit
Yields:

tuple – The next file matching the pattern, and the corresponding wildcards object

snakemake.utils.makedirs(dirnames)[源代码]

Recursively create the given directory or directories without reporting errors if they are present.

snakemake.utils.min_version(version)[源代码]

Require minimum snakemake version, raise workflow error if not met.

snakemake.utils.read_job_properties(jobscript, prefix='# properties', pattern=re.compile('# properties = (.*)'))[源代码]

Read the job properties defined in a snakemake jobscript.

This function is a helper for writing custom wrappers for the snakemake –cluster functionality. Applying this function to a jobscript will return a dict containing information about the job.

snakemake.utils.report(text, path, stylesheet='/home/docs/checkouts/readthedocs.org/user_builds/snakemakecn/envs/latest/lib/python3.7/site-packages/snakemake/report.css', defaultenc='utf8', template=None, metadata=None, **files)[源代码]

Create an HTML report using python docutils.

This is deprecated in favor of the –report flag.

Attention: This function needs Python docutils to be installed for the python installation you use with Snakemake.

All keywords not listed below are intepreted as paths to files that shall be embedded into the document. They keywords will be available as link targets in the text. E.g. append a file as keyword arg via F1=input[0] and put a download link in the text like this:

report('''
==============
Report for ...
==============

Some text. A link to an embedded file: F1_.

Further text.
''', outputpath, F1=input[0])

Instead of specifying each file as a keyword arg, you can also expand
the input of your rule if it is completely named, e.g.:

report('''
Some text...
''', outputpath, **input)
参数:
  • text (str) – The “restructured text” as it is expected by python docutils.
  • path (str) – The path to the desired output file
  • stylesheet (str) – An optional path to a css file that defines the style of the document. This defaults to <your snakemake install>/report.css. Use the default to get a hint how to create your own.
  • defaultenc (str) – The encoding that is reported to the browser for embedded text files, defaults to utf8.
  • template (str) – An optional path to a docutils HTML template.
  • metadata (str) – E.g. an optional author name or email address.
snakemake.utils.simplify_path(path)[源代码]

Return a simplified version of the given path.

snakemake.utils.update_config(config, overwrite_config)[源代码]

Recursively update dictionary config with overwrite_config.

See http://stackoverflow.com/questions/3232943/update-value-of-a-nested-dictionary-of-varying-depth for details.

参数:
  • config (dict) – dictionary to update
  • overwrite_config (dict) – dictionary whose items will overwrite those in config
snakemake.utils.validate(data, schema, set_default=True)[源代码]

Validate data with JSON schema at given path.

参数:
  • data (object) – data to validate. Can be a config dict or a pandas data frame.
  • schema (str) – Path to JSON schema used for validation. The schema can also be in YAML format. If validating a pandas data frame, the schema has to describe a row record (i.e., a dict with column names as keys pointing to row values). See http://json-schema.org. The path is interpreted relative to the Snakefile when this function is called.
  • set_default (bool) – set default values defined in schema. See http://python-jsonschema.readthedocs.io/en/latest/faq/ for more information

snakemake.workflow module

class snakemake.workflow.RuleInfo(func)[源代码]

基类:object

class snakemake.workflow.Rules[源代码]

基类:object

A namespace for rules so that they can be accessed via dot notation.

class snakemake.workflow.Subworkflow(workflow, name, snakefile, workdir, configfile)[源代码]

基类:object

snakefile
target(paths)[源代码]
targets(dag)[源代码]
workdir
class snakemake.workflow.Workflow(snakefile=None, jobscript=None, overwrite_shellcmd=None, overwrite_config={}, overwrite_workdir=None, overwrite_configfile=None, overwrite_clusterconfig={}, config_args=None, debug=False, use_conda=False, conda_prefix=None, use_singularity=False, singularity_prefix=None, singularity_args='', shadow_prefix=None, mode=0, wrapper_prefix=None, printshellcmds=False, restart_times=None, attempt=1, default_remote_provider=None, default_remote_prefix='', run_local=True, default_resources={})[源代码]

基类:object

add_rule(name=None, lineno=None, snakefile=None, checkpoint=False)[源代码]

Add a rule.

benchmark(benchmark)[源代码]
check()[源代码]
check_localrules()[源代码]
concrete_files
conda(conda_env)[源代码]
config
configfile(fp)[源代码]

Update the global config with data from the given file.

current_basedir

Basedir of currently parsed Snakefile.

cwl(cwl)[源代码]
docstring(string)[源代码]
execute(targets=None, dryrun=False, touch=False, cores=1, nodes=1, local_cores=1, forcetargets=False, forceall=False, forcerun=None, until=[], omit_from=[], prioritytargets=None, quiet=False, keepgoing=False, printshellcmds=False, printreason=False, printdag=False, cluster=None, cluster_sync=None, jobname=None, immediate_submit=False, ignore_ambiguity=False, printrulegraph=False, printd3dag=False, drmaa=None, drmaa_log_dir=None, kubernetes=None, kubernetes_envvars=None, container_image=None, stats=None, force_incomplete=False, ignore_incomplete=False, list_version_changes=False, list_code_changes=False, list_input_changes=False, list_params_changes=False, list_untracked=False, list_conda_envs=False, summary=False, archive=None, delete_all_output=False, delete_temp_output=False, detailed_summary=False, latency_wait=3, wait_for_files=None, nolock=False, unlock=False, resources=None, notemp=False, nodeps=False, cleanup_metadata=None, cleanup_conda=False, cleanup_shadow=False, subsnakemake=None, updated_files=None, keep_target_files=False, keep_shadow=False, keep_remote_local=False, allowed_rules=None, max_jobs_per_second=None, max_status_checks_per_second=None, greediness=1.0, no_hooks=False, force_use_threads=False, create_envs_only=False, assume_shared_fs=True, cluster_status=None, report=None, export_cwl=False)[源代码]
get_rule(name)[源代码]

Get rule by name.

Arguments name – the name of the rule

get_sources()[源代码]
global_singularity(singularity_img)[源代码]
global_wildcard_constraints(**content)[源代码]

Register global wildcard constraints.

group(group)[源代码]
include(snakefile, overwrite_first_rule=False, print_compilation=False, overwrite_shellcmd=None)[源代码]

Include a snakefile.

input(*paths, **kwpaths)[源代码]
is_local(rule)[源代码]
is_rule(name)[源代码]

Return True if name is the name of a rule.

Arguments name – a name

list_resources()[源代码]
list_rules(only_targets=False)[源代码]
localrules(*rulenames)[源代码]
log(*logs, **kwlogs)[源代码]
message(message)[源代码]
norun()[源代码]
onerror(func)[源代码]

Register onerror function.

onstart(func)[源代码]

Register onstart function.

onsuccess(func)[源代码]

Register onsuccess function.

output(*paths, **kwpaths)[源代码]
params(*params, **kwparams)[源代码]
priority(priority)[源代码]
report(path)[源代码]

Define a global report description in .rst format.

resources(*args, **resources)[源代码]
rule(name=None, lineno=None, snakefile=None, checkpoint=False)[源代码]
ruleorder(*rulenames)[源代码]
rules
run(func)[源代码]
script(script)[源代码]
shadow(shadow_depth)[源代码]
shellcmd(cmd)[源代码]
singularity(singularity_img)[源代码]
subworkflow(name, snakefile=None, workdir=None, configfile=None)[源代码]
subworkflows
threads(threads)[源代码]
version(version)[源代码]
wildcard_constraints(*wildcard_constraints, **kwwildcard_constraints)[源代码]
workdir(workdir)[源代码]

Register workdir.

wrapper(wrapper)[源代码]
snakemake.workflow.format_resources(dict, *, omit_keys={'_cores', '_nodes'}, omit_values=[])
snakemake.workflow.srcdir(path)[源代码]

Return the absolute path, relative to the source directory of the current Snakefile.

snakemake.wrapper module

snakemake.wrapper.find_extension(path, extensions=['.py', '.R', '.Rmd', '.jl'])[源代码]
snakemake.wrapper.get_conda_env(path, prefix=None)[源代码]
snakemake.wrapper.get_path(path, prefix=None)[源代码]
snakemake.wrapper.get_script(path, prefix=None)[源代码]
snakemake.wrapper.is_git_path(path)[源代码]
snakemake.wrapper.is_local(path)[源代码]
snakemake.wrapper.is_script(path)[源代码]
snakemake.wrapper.wrapper(path, input, output, params, wildcards, threads, resources, log, config, rulename, conda_env, singularity_img, singularity_args, bench_record, prefix, jobid, bench_iteration, shadow_dir)[源代码]

Load a wrapper from https://bitbucket.org/snakemake/snakemake-wrappers under the given path + wrapper.(py|R|Rmd) and execute it.

Module contents

snakemake.bash_completion(snakefile='Snakefile')[源代码]

Entry point for bash completion.

snakemake.get_appdirs()[源代码]
snakemake.get_argument_parser(profile=None)[源代码]

Generate and return argument parser.

snakemake.get_profile_file(profile, file, return_default=False)[源代码]
snakemake.main(argv=None)[源代码]

Main entry point.

snakemake.parse_config(args)[源代码]

Parse config from args.

snakemake.parse_default_resources(resources_args)[源代码]

Parse default resource definition args.

snakemake.parse_resources(resources_args, fallback=None)[源代码]

Parse resources from args.

snakemake.snakemake(snakefile, report=None, listrules=False, list_target_rules=False, cores=1, nodes=1, local_cores=1, resources={}, default_resources={}, config={}, configfile=None, config_args=None, workdir=None, targets=None, dryrun=False, touch=False, forcetargets=False, forceall=False, forcerun=[], until=[], omit_from=[], prioritytargets=[], stats=None, printreason=False, printshellcmds=False, debug_dag=False, printdag=False, printrulegraph=False, printd3dag=False, nocolor=False, quiet=False, keepgoing=False, cluster=None, cluster_config=None, cluster_sync=None, drmaa=None, drmaa_log_dir=None, jobname='snakejob.{rulename}.{jobid}.sh', immediate_submit=False, standalone=False, ignore_ambiguity=False, snakemakepath=None, lock=True, unlock=False, cleanup_metadata=None, cleanup_conda=False, cleanup_shadow=False, force_incomplete=False, ignore_incomplete=False, list_version_changes=False, list_code_changes=False, list_input_changes=False, list_params_changes=False, list_untracked=False, list_resources=False, summary=False, archive=None, delete_all_output=False, delete_temp_output=False, detailed_summary=False, latency_wait=3, wait_for_files=None, print_compilation=False, debug=False, notemp=False, keep_remote_local=False, nodeps=False, keep_target_files=False, allowed_rules=None, jobscript=None, greediness=None, no_hooks=False, overwrite_shellcmd=None, updated_files=None, log_handler=None, keep_logger=False, max_jobs_per_second=None, max_status_checks_per_second=100, restart_times=0, attempt=1, verbose=False, force_use_threads=False, use_conda=False, use_singularity=False, singularity_args='', conda_prefix=None, list_conda_envs=False, singularity_prefix=None, shadow_prefix=None, create_envs_only=False, mode=0, wrapper_prefix=None, kubernetes=None, kubernetes_envvars=None, container_image=None, default_remote_provider=None, default_remote_prefix='', assume_shared_fs=True, cluster_status=None, export_cwl=None)[源代码]

Run snakemake on a given snakefile.

This function provides access to the whole snakemake functionality. It is not thread-safe.

参数:
  • snakefile (str) – the path to the snakefile
  • report (str) – create an HTML report for a previous run at the given path
  • listrules (bool) – list rules (default False)
  • list_target_rules (bool) – list target rules (default False)
  • cores (int) – the number of provided cores (ignored when using cluster support) (default 1)
  • nodes (int) – the number of provided cluster nodes (ignored without cluster support) (default 1)
  • local_cores (int) – the number of provided local cores if in cluster mode (ignored without cluster support) (default 1)
  • resources (dict) – provided resources, a dictionary assigning integers to resource names, e.g. {gpu=1, io=5} (default {})
  • default_resources (dict) – default values for resources not defined in rules (default {})
  • config (dict) – override values for workflow config
  • workdir (str) – path to working directory (default None)
  • targets (list) – list of targets, e.g. rule or file names (default None)
  • dryrun (bool) – only dry-run the workflow (default False)
  • touch (bool) – only touch all output files if present (default False)
  • forcetargets (bool) – force given targets to be re-created (default False)
  • forceall (bool) – force all output files to be re-created (default False)
  • forcerun (list) – list of files and rules that shall be re-created/re-executed (default [])
  • prioritytargets (list) – list of targets that shall be run with maximum priority (default [])
  • stats (str) – path to file that shall contain stats about the workflow execution (default None)
  • printreason (bool) – print the reason for the execution of each job (default false)
  • printshellcmds (bool) – print the shell command of each job (default False)
  • printdag (bool) – print the dag in the graphviz dot language (default False)
  • printrulegraph (bool) – print the graph of rules in the graphviz dot language (default False)
  • printd3dag (bool) – print a D3.js compatible JSON representation of the DAG (default False)
  • nocolor (bool) – do not print colored output (default False)
  • quiet (bool) – do not print any default job information (default False)
  • keepgoing (bool) – keep goind upon errors (default False)
  • cluster (str) – submission command of a cluster or batch system to use, e.g. qsub (default None)
  • cluster_config (str,list) – configuration file for cluster options, or list thereof (default None)
  • cluster_sync (str) – blocking cluster submission command (like SGE ‘qsub -sync y’) (default None)
  • drmaa (str) – if not None use DRMAA for cluster support, str specifies native args passed to the cluster when submitting a job
  • drmaa_log_dir (str) – the path to stdout and stderr output of DRMAA jobs (default None)
  • jobname (str) – naming scheme for cluster job scripts (default “snakejob.{rulename}.{jobid}.sh”)
  • immediate_submit (bool) – immediately submit all cluster jobs, regardless of dependencies (default False)
  • standalone (bool) – kill all processes very rudely in case of failure (do not use this if you use this API) (default False) (deprecated)
  • ignore_ambiguity (bool) – ignore ambiguous rules and always take the first possible one (default False)
  • snakemakepath (str) – deprecated parameter whose value is ignored. Do not use.
  • lock (bool) – lock the working directory when executing the workflow (default True)
  • unlock (bool) – just unlock the working directory (default False)
  • cleanup_metadata (list) – just cleanup metadata of given list of output files (default None)
  • cleanup_conda (bool) – just cleanup unused conda environments (default False)
  • cleanup_shadow (bool) – just cleanup old shadow directories (default False)
  • force_incomplete (bool) – force the re-creation of incomplete files (default False)
  • ignore_incomplete (bool) – ignore incomplete files (default False)
  • list_version_changes (bool) – list output files with changed rule version (default False)
  • list_code_changes (bool) – list output files with changed rule code (default False)
  • list_input_changes (bool) – list output files with changed input files (default False)
  • list_params_changes (bool) – list output files with changed params (default False)
  • list_untracked (bool) – list files in the workdir that are not used in the workflow (default False)
  • summary (bool) – list summary of all output files and their status (default False)
  • archive (str) – archive workflow into the given tarball
  • delete_all_output (bool) remove all files generated by the workflow (default False) –
  • delete_temp_output (bool) remove all temporary files generated by the workflow (default False) –
  • latency_wait (int) – how many seconds to wait for an output file to appear after the execution of a job, e.g. to handle filesystem latency (default 3)
  • wait_for_files (list) – wait for given files to be present before executing the workflow
  • list_resources (bool) – list resources used in the workflow (default False)
  • summary – list summary of all output files and their status (default False). If no option is specified a basic summary will be ouput. If ‘detailed’ is added as an option e.g –summary detailed, extra info about the input and shell commands will be included
  • detailed_summary (bool) – list summary of all input and output files and their status (default False)
  • print_compilation (bool) – print the compilation of the snakefile (default False)
  • debug (bool) – allow to use the debugger within rules
  • notemp (bool) – ignore temp file flags, e.g. do not delete output files marked as temp after use (default False)
  • keep_remote_local (bool) – keep local copies of remote files (default False)
  • nodeps (bool) – ignore dependencies (default False)
  • keep_target_files (bool) – do not adjust the paths of given target files relative to the working directory.
  • allowed_rules (set) – restrict allowed rules to the given set. If None or empty, all rules are used.
  • jobscript (str) – path to a custom shell script template for cluster jobs (default None)
  • greediness (float) – set the greediness of scheduling. This value between 0 and 1 determines how careful jobs are selected for execution. The default value (0.5 if prioritytargets are used, 1.0 else) provides the best speed and still acceptable scheduling quality.
  • overwrite_shellcmd (str) – a shell command that shall be executed instead of those given in the workflow. This is for debugging purposes only.
  • updated_files (list) – a list that will be filled with the files that are updated or created during the workflow execution
  • verbose (bool) – show additional debug output (default False)
  • max_jobs_per_second (int) – maximal number of cluster/drmaa jobs per second, None to impose no limit (default None)
  • restart_times (int) – number of times to restart failing jobs (default 0)
  • attempt (int) – initial value of Job.attempt. This is intended for internal use only (default 1).
  • force_use_threads – whether to force use of threads over processes. helpful if shared memory is full or unavailable (default False)
  • use_conda (bool) – create conda environments for each job (defined with conda directive of rules)
  • use_singularity (bool) – run jobs in singularity containers (if defined with singularity directive)
  • singularity_args (str) – additional arguments to pass to singularity
  • conda_prefix (str) – the directory in which conda environments will be created (default None)
  • singularity_prefix (str) – the directory to which singularity images will be pulled (default None)
  • shadow_prefix (str) – prefix for shadow directories. The job-specific shadow directories will be created in $SHADOW_PREFIX/shadow/ (default None)
  • create_envs_only (bool) – if specified, only builds the conda environments specified for each job, then exits.
  • list_conda_envs (bool) – list conda environments and their location on disk.
  • mode (snakemake.common.Mode) – execution mode
  • wrapper_prefix (str) – prefix for wrapper script URLs (default None)
  • kubernetes (str) – submit jobs to kubernetes, using the given namespace.
  • kubernetes_envvars (list) – environment variables that shall be passed to kubernetes jobs.
  • container_image (str) – Docker image to use, e.g., for kubernetes.
  • default_remote_provider (str) – default remote provider to use instead of local files (e.g. S3, GS)
  • default_remote_prefix (str) – prefix for default remote provider (e.g. name of the bucket).
  • assume_shared_fs (bool) – assume that cluster nodes share a common filesystem (default true).
  • cluster_status (str) – status command for cluster execution. If None, Snakemake will rely on flag files. Otherwise, it expects the command to return “success”, “failure” or “running” when executing with a cluster jobid as single argument.
  • export_cwl (str) – Compile workflow to CWL and save to given file
  • log_handler (function) –

    redirect snakemake output to this custom log handler, a function that takes a log message dictionary (see below) as its only argument (default None). The log message dictionary for the log handler has to following entries:

    level:the log level (“info”, “error”, “debug”, “progress”, “job_info”)
    level=”info”, “error” or “debug”:
     
    msg:the log message
    level=”progress”:
     
    done:number of already executed jobs
    total:number of total jobs
    level=”job_info”:
     
    input:list of input files of a job
    output:list of output files of a job
    log:path to log file of a job
    local:whether a job is executed locally (i.e. ignoring cluster)
    msg:the job message
    reason:the job reason
    priority:the job priority
    threads:the threads of the job
返回:

True if workflow execution was successful.

返回类型:

bool

snakemake.unparse_config(config)[源代码]