{"type":"doc","content":[{"type":"paragraph","content":[{"type":"inlineExtension","attrs":{"extensionType":"com.atlassian.confluence.macro.core","extensionKey":"wiki-markup","parameters":{"macroParams":{"__bodyContent":{"value":"{toc}\n \n{anchor:overview}\nh1. Overview\nMany tools now can be installed and registered to the DE as _*Biocontainers*_. These pages specifically define the general process by which tools are integrated into the DE as Biocontainers (that may or may not involve creating Bioconda packages), including guidelines and recommendations.\n \n{note}The previous method of [DE Tool Docker Image registration and Installation|https://wiki.cyverse.org/wiki/display/csmgmt/DE+Tool+Docker+Image+registration+and+installation] will still be used for integrating tools, but this method will be preferred{note}\n \nThe intent of this process is not simply to \"install a tool\", but to satisfy the [#goals] listed in the next section as well.\n \nOne of the goals of the DE has been to provide our users with a platform on which they can perform and publish reproducible research. \nOne of the primary components of this research are the underlying tools which our users execute in their apps!\n \nDocker allows users to ensure that there are no mysteries as to how a tool was installed and/or configured. \nAll of the tools accessed through the DE may eventually be shared, either between users or publicly when a user publishes their research.\n \n{anchor:goals}\nh1. Goals\n# Construction of Tool recipe(s) is repeatable.\n# Each tool's installation process is clearly documented.\n# Tool execution is reliable, and repeatable.\n \n{anchor:roles}\nh1. Roles\nThroughout these instructions, there will be references to _tool-integrator_, _tool-requestor_, _integrator_, _requestor_, etc. These are defined as follows:\n* _*Tool-Integrator*_ or _*Integrator*_\n** The person(s) who will be responsible for reviewing Bioconda recipes prior to committing, registering, and deploying. Usually an employee of CyVerse with access to the underlying cyberinfrastructure.\n* _*Tool-Requestor*_ or _*requestor*_\n** The user who is requesting the tool. Usually a researcher not affiliated with CyVerse.\n\n{anchor:setup}\nh1. 1: One-time Setup\n\nh2. 1.1 Install Bioconda\nBioconda is a channel for the conda package manager specializing in bioinformatics software. It consists of:\n* A [repository of recipes|https://github.com/bioconda/bioconda-recipes] hosted on GitHub\n* A [build system|https://github.com/bioconda/bioconda-utils] that turns these recipes into conda packages\n* A [repository of >1500 bioinformatics packages|https://anaconda.org/bioconda] ready to use with a simple _*conda install*_ command\n* Over 130 contributors that add, modify, update and maintain the recipes\n\n{info}\nRecipe vs package\nA recipe is a directory containing a small set of files that defines name, version, dependencies, and URL for source code. A recipe \ntypically contains a meta.yaml file that defines these settings and a build.sh script that builds the software. A recipe is \nconverted into a package by running \"conda-build\" on the recipe. \nA package is a bgzipped tar file (.tar.bz2) that contains the built software. Packages are uploaded to anaconda.org so that users can install them with \"conda install\" command.\n{info}\n\nBrowse packages in the bioconda channel: [Available packages|https://bioconda.github.io/recipes.html#recipes]\n\nBioconda requires the conda package manager to be installed. If you have an Anaconda Python installation, you already have it. \nOtherwise, the best way to install it is with the Miniconda package. \n{note}Bioconda supports only 64-bit Linux and Mac OSX. The Python 3 version is recommended.{note}\n\n{info}\nInstall [miniconda|http://conda.pydata.org/miniconda.html] specific for your platform. The following code shows the Miniconda installation on MacOSX and Linux\n\n{code}\n# MacOSX\nwget https://repo.continuum.io/miniconda/Miniconda3-latest-MacOSX-x86_64.sh\nbash Miniconda3-latest-MacOSX-x86_64.sh\n\n# Linux\nwget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh\nbash Miniconda3-latest-Linux-x86_64.sh\n \nAccept all the default settings and let conda prepend the PATH in ~/.bashrc\n\nsource ~/.bashrc\n{code}\n \n{note}\nIf you already have miniconda installed on your MacOSX/Linux, you can update that using\n{code}\nconda upgrade conda\nconda upgrade conda-build\n{code}\n{note}\n{info}\n\nh3. 1.1.1 - Setting up of Channels of Bioconda\nAfter installing conda you will need to add the bioconda channel as well as the other channels bioconda depends on. It is important to run the following commands in this order so that the priority is set correctly.\n\n{info}\nThe _*conda-forge*_ channel contains many general-purpose packages not already found in the defaults channel. The _*r*_ channel \ncontains common R packages used as dependencies for bioconda packages.\n{code}\nconda config --add channels conda-forge\nconda config --add channels defaults\nconda config --add channels r\nconda config --add channels bioconda\n{code}\n{info}\n\nh3. 1.1.2 - Test Bioconda installation\nAfter installing Bioconda and setting-up channels, test to see if the installation of Bioconda worked properly by installing a package\n\n{info}\n{code}\nconda install \n\n# Example\nconda install bwa\n# Or a specific version can be installed like this\nconda install bwa=0.7.12\n{code}\n{info}\n \n{info}\nIf there are no errors during installation of bwa, your Bioconda set-up is complete \n{info}\n\nh2. 1.2 - Git and GitHub\n\nh3. 1.2.1 Create a [github|https://github.com/] account if you don't have one already.\n\nh3. 1.2.2 Next, create a fork of [bioconda-recipes repo|https://github.com/bioconda/bioconda-recipes] onto your GitHub account and then clone it locally. \n \n{info}\n{code} \ngit clone https://github.com//bioconda-recipes.git\n\n# Example\ngit clone https://github.com/upendrak/bioconda-recipes.git\n{code}\n \nh3. 1.2.3 Add the main bioconda-recipes repo as an upstream remote to more easily update your branch with the upstream master branch\n\n{code}\ncd bioconda-recipes\ngit remote add upstream https://github.com/bioconda/bioconda-recipes.git\n{code}\n{info}\n\nh3. 1.2.4 Request to be added to the bioconda team\nWhile not required, you can be added to the bioconda by posting in [Issue #1|https://github.com/bioconda/bioconda-recipes/issues/1]. Members of the bioconda team can merge their own recipes once tests pass, though we ask that first-time contributions and anything out of the ordinary be reviewed by the @bioconda/core team.\n\n{note}\nEven if you are a member of the bioconda team with push access, using your own fork will allow testing of your recipes \non travis-ci using your own account's free resources without consuming resources allocated by travis-ci to the bioconda \ngroup. This makes the tests go faster for everyone.\n{note} \n\nh2. 1.3 - Install Docker (optional)\n{note}Skip this step if you already installed Docker on your machine{note}\nInstall [Docker|https://docker.com] specific for your platform (optional, but allows you to simulate most closely the Travis-CI tests). \n\n{anchor: Creating-Biconda-recipes}\nh1. 2: Create the Tool's Required Bioconda recipe (for generating Biocontainers)\n \nh2. 2.1 - Update Bioconda repo and requirements\nBefore starting, it's best to update your fork with any changes made recently to the upstream bioconda repo. Assuming you've set up your fork as above:\n\n{code}\ngit checkout master\ngit pull upstream master\n{code}\n\nh2. 2.2 - Set up the channel order and install the versions of dependencies currently used in the master branch \n{note}This is optional but is recommended{note}\n\n{info}\nThe channel order should generally stay the same and the dependencies are not likely to change much, but this ensures that the local environment most closely resembles the build environment on travis-ci\n{code}\n./simulate-travis.py --set-channel-order\n./simulate-travis.py --install-requirements\n{code}\n{info}\n \nh2. 2.3 - Checkout a new branch\nCheck out a new branch in your fork (here the branch is arbitrarily named *_my-recipe_*):\n\n{code}git checkout -b my-recipe{code}\n\nh2. 2.4 - Create the recipe\n{note}\nBefore you create a recipe, make sure to check that a package doesn't exist for that recipe. If the package is already present, then you don't need to create the recipe.\nThere are a couple of ways to check for the package\n1. Search for the package name in here - http://bioconda.github.io/recipes.html\n2. Search for the package name on the command line\n{code}\nconda search -c bioconda\n \n# Example\nconda search taco -c bioconda\n{code}\n{note}\n\nIf the package of your interest, is not available, you can create the Bioconda recipe for the tool of your interest as below\n\n{code}conda skeleton {code}\n{info}\nSource: The source of the tool can be pypi, cran, bioconductor or cpan. Guidelines for bioconda recipe - https://bioconda.github.io/guidelines.html\nPackage: The name of the package\n{info}\n{note}\nIf the tool is not available from any of the above sources, then you need to generate a bioconda package from [scratch|http://conda.pydata.org/docs/build_tutorials/pkgs2.html].\n{note}\n \nh2. 2.5 - Test it locally\nAfter you create your recipe (using one of the above methods), to make sure your recipe works, you need to test it locally. There are two options. \n\nh3. 2.5.1 - Quick test\n{info} \nThe quickest, but not necessarily most complete, is to run \"conda-build\" command directly\n\n{code}\nconda install conda-build\nconda build ./\n \n# Example\nconda build bowtie2/2.2.4 \n{code}\n{info}\n\nh3. 2.5.2 - Advanced test \n{info}\nTo test the recipe in a way more representative of the travis-ci environment, use the *_simulate-travis.py_* script in the top-level directory of the repo. *_simulate-travis.py_* reads the config files in the repo and sets things up as closely as possible to how the builds will be run on travis-ci. It should be run from the top-level dir of the repo. Any arguments are passed on to the bioconda-utils build command, so check *_bioconda-utils build -h_* for help and more options.\n\n{code}\n# This tests everything, using the installed *_conda-build_*. It will check all recipes to see what needs to be built and so it is the most comprehensive:\n./simulate-travis.py\n\nSame thing but using -docker. If you're on OSX and have docker installed, you can use this to test the recipe under Linux:\n./simulate-travis.py --docker\n\nUse the -quick option which will just check recipes that have changed since the last commit to master branch or that have been newly removed from any configured blacklists. This can help speed up the recipe filter stage which can take 5 mins to thoroughly check 1500+ recipes. Note that this will not find cases where a pinned version (e.g., { CONDA_BOOST }) has been changed:\n./simulate-travis.py --docker --quick\n\nTo specify exactly which packages you want to try building, use the -packages argument. Note that the arguments to -packages can be globs and are of package names rather than paths to recipe directories. For example, to consider all R and Bioconductor packages:\n./simulate-travis.py --docker --package r-* bioconductor-*\n{code}\n{info}\n\nh2. 2.6 - Push your changes to your fork on github repo\n{info}\nOnce your tests are successful, before pushing your changes to your fork on github, it is best to merge any changes that have happened recently on the upstream master branch. See sycncing a fork for details, or run\n\n{code}\ngit fetch upstream\n\n# syncs the fork's master branch with upstream\ngit checkout master\ngit merge upstream/master\n\n# merges those changes into the recipe's branch\ngit checkout my-recipe\ngit merge master\n\n# push your changes to your fork on github\ngit push origin my-recipe\n{code}\n{info}\n\nh2. 2.7 - Open a pull request on the bioconda-recipes repo\nIf it's your first recipe or the recipe is doing something non-standard, please ask @bioconda/core for a review.\n\n# Example:\n!Comparing_bioconda_master.png|thumbnail!\n \nh2. 2.8 - Test the built package using Tool's new recipe and Biocontainer\nAfter the pull request, travis-ci will again do the builds to make sure everthing works. When the pull request is merged with the master branch by Bioconda team, the package will be uploaded to anaconda.org and Biocontainers will be pushed to [quay.io|https://quay.io/organization/biocontainers]. \n \n{info}\n{note}Once the bioconda package is available on Anaconda and biocontainer is available on quay.io, it may be a good idea to test those.{note}\n \n* Install the built bioconda pacakge from Anaconda (optional but recommended)\n{code}\nconda create -n myenvironment my-package # This is optional but it is always good to test this\n{code}\n \n{note}This method will install the package in the /home/username/minconda3/envs/myenvironment/bin{note}\n\n* Pull your Biocontainer from quay.io of your new recipe (Mandatory) \n{code}\ndocker pull quay.io/biocontainers/:--_\n{code}\n\n* Run the tool's Biocontainer using the image:tag name, sample parameters, and inputs with a [docker run|http://docs.docker.com/reference/commandline/run/] command.\n\n* If the tool crashes, or the output does not match the sample output, contact the Bioconda.\n\n* Clean up any data containers and dangling images created in testing with [docker rm -v|http://docs.docker.com/reference/commandline/rm/] and [docker rmi|http://docs.docker.com/reference/commandline/rmi/] commands.\n\n* This command will cleanup any 'dangling' images:\n{code}docker rmi $(docker images -f 'dangling=true' -q){code}\n{info}\n\n{anchor:deploy-register}\nh2. 2.9: Request Integration of the Tool\n!request_tool_DE.png|thumbnail!\n\n\n"},"atlassian-macro-output-type":{"value":"INLINE"}},"macroMetadata":{"schemaVersion":{"value":"1"},"title":"wiki-markup"}}}},{"text":" ","type":"text"}]},{"type":"paragraph","content":[{"text":" ","type":"text"}]}],"version":1}

Biocontainers for the Discovery Environment