Add shared merge_inputs module

[victorlin]

Note

Based on #78

Description of proposed changes

This PR contains 2 prep commits + 1 main commit. Message from main commit:

This should work for most repos. Repos that handle a different schema of inputs in config can continue to use their own copy of these rules.

Related issue(s)

Follow-up to nextstrain/pathogen-repo-guide#108 (comment)

Checklist

• Checks pass
• If adding a script, add an entry for it in the README.

[joverlee521]

non-blocking

It currently does not cause any issues in the downstream zika repo to have config.smk included twice (once here and once in phylo/Snakefile), but if we were to add a rule to config.smk then the double inclusion would raise an unexpected CreateRuleException.

Options to avoid this issue are

1. only include config.smk and call set_search_paths()
2. only include init.smk but not config.smk
3. add tests to guard against this issue in this repo

[jameshadfield]

This (the init.smk) is getting too close to magic for me. It's adding another layer to remember about when authoring / understanding workflows.

I'd prefer it if we made the pathogen repo's entry Snakemake file do the includes & call functions explicitly as I'd find it much easier to understand the moving parts involved when reading an entry Snakefile like so:

# e.g. measles/phylogenetic/Snakefile
include: "../shared/config.smk"
set_search_paths()

[victorlin]

Dropped init.smk and updated the commit message 52907a1:

Downstream workflows should call this function to retain existing behavior.

[victorlin]

add tests to guard against this issue in this repo

I think this can be addressed by adding pathogen-repo-ci on changes applied to downstream pathogen repos, which would be a good thing to have regardless?

[joverlee521]

I was again wary of double inclusion of rules, but it's fine in this case since this is expected to be used as a Snakemake module and module rules are evaluated under a separate namespace.

I do think it'd good to note that these include statements are required since the included variables/functions of the calling workflows are not visible inside the module.

[victorlin]

I was again wary of double inclusion of rules, but it's fine in this case since this is expected to be used as a Snakemake module and module rules are evaluated under a separate namespace.

Thanks for noting, I didn't realize that's how it works!

I do think it'd good to note that these include statements are required since the included variables/functions of the calling workflows are not visible inside the module.

Noted:

shared/snakemake/merge_inputs.smk

Lines 98 to 101 in c9497c4

	# These include statements are required since any included variables/functions
	# from calling workflows are not visible inside this module.
	include: "config.smk"
	include: "remote_files.smk"