The best way to learn is by example, so here’s an extended tutorial to get you started using looper to run pre-made pipelines on a pre-made project.
First, install looper and pypiper. Pypiper is our pipeline development framework; it is not required to use looper, which can work with any command-line pipeline, but this tutorial uses pypiper pipelines so we must install it now:
pip install --user https://github.com/pepkit/looper/zipball/master pip install --user https://github.com/databio/pypiper/zipball/master
Now, you will need to grab a project to run, and some pipelines to run on it. We have a functional working project example and an open source pipeline repository on github.
git clone https://github.com/epigen/microtest.git git clone https://github.com/epigen/open_pipelines.git
Now you can run this project with looper! Just use
looper run microtest/config/microtest_config.tutorial.yaml
If the looper executable isn’t in your path, add it with
Outputs of pipeline runs will be under the directory specified in the
output_dir variable under the
paths section in the project config file (see Configuration files ) this is usually the name of the project being run.
Inside there will be two directories:
results_pipeline- pipeline results with one subdirectory per sample
submissions- yaml representations of the samples and log files of the submitted jobs.
In this example, we just ran one example sample (an amplicon sequencing library) through a pipeline that processes amplicon data (to determine percentage of indels in amplicon).
From here to running hundreds of samples of various sample types is virtually the same effort!
On your own¶
To use looper on your own, you will need to prepare 2 things: your project (what data do you want to process), and your pipelines (what do you want to do with that data). The next sections provide detailed instructions on how to tell looper about these 2 things:
- Project. To link your project to looper, you will need to define your project using looper’s standard format.
- Pipelines. You will want to either use pre-made looper-compatible pipelines, or link your own, custom built pipelines. Either way, the next section includes detailed instructions on how to connect your project to your pipeline.