iCommands 4.1.910, the latest supported version for CyVerse services, has been released by iRODS but we are currently testing it for compatibility with CyVerse services. Instructions will be available on this page by September 5, 2016. For a list of affected services and testing done, see Upgrading to iRODS 4.1 (work in progress).

Using the Community Contributed Data and CyVerse Curated Data sites

Using the Community Contributed Data and CyVerse Curated Data sites

The easiest way to download files up to 2 GB in size from a web browser (size cutoff due to speed limitations) without a CyVerse account is using the CyVerse Data Commons repositories, Community Contributed Data or CyVerse Curated Data. It is also possible to download without an account using iCommands or Cyberduck. No installation or configuration is needed for the DCR.

For files larger than 2 GB, you will need to use iCommands or Cyberduck (Mac OS and Windows), which require installation and setup.

1. Go to CyVerse Data Commons at http://datacommons.cyverse.org/. No login is required.
2. Click either Community Released to access data that has been provided by community collaborators for public access, or click CyVerse Curated to access data that has been curated and given a permanent identifier by CyVerse curators.

Downloading a file from the Data Commons

Files up to 2 GB can be downloaded directly from Data commons repositories, while files larger than 2 GB must be downloaded using another mode.

1. In either repository, click the folder name with the file to download.

2. Click the filename.

3. Depending on the file size, do one of the following:

   • If the file is smaller than 2 GB:
     
     1. Click Download.
     2. Click the checkbox for I'm not a robot.
     3. Change the path if needed, click Save, and click OK.
   • If the file is larger than 2 GB:
     1. Click Download Options to view the other download options (or read the information below on this page).
     2. Copy the path to the file and note the filename.
     3. Continue to your preferred method, following the instructions in the sections below:
        • iCommands
        • iDrop Desktop
        • Cyberduck
     • To learn more about other methods for managing and using data, see Manage Data on the CyVerse website.
     • To learn more about downloading files from the Discovery Environment as a registered CyVerse user, see Downloading Files and Folders in the Discovery Environment manual.
4. Change the download location if needed and click Save.
5. Repeat for each file to download.

Downloading a public file from a link

1. Click the link you received.
2. Download the file to your system.

Using iCommands

iCommands is a collection of commands for Linux and Mac OS operating systems that are used in the iRODS system to interact with the CyVerse Data Store. Many commands are very similar to Unix utilities. For example, to list files and directories, in Linux you use ls, but in iCommands you use ils. While iCommands are great for all transfers and for automating tasks via scripts, they are the best choice for downloading large files (> 2 -100 GB each) and for bulk file transfers (many small files) without a CyVerse account.

iCommands must be installed and configured for your system. Once set up, you can use iCommands to download files that have been shared by other users. If you have a CyVerse account, you have access to all public files in the DE Community Data folder, but even if you don't have an account, you can download public files using iCommands. For more information, see Using iCommands.

Setting up iCommands to download files from Community Contributed Data and CyVerse Curated Data

iCommands must be set up and configured on each user's local machine or server before you can download data from either of the CyVerse Data Commons repositories using iCommands. Before you begin, you may want to watch a CyVerse video about iCommands.

Step 1: Download iCommands for your operating system

Because CyVerse products are interdependent with iRODS iCommands and iRODS products such as FUSE, the instructions for downloading iCommands vary according to your OS:

• Linux
• Mac OS
• Windows

Linux

If you are using Linux and have the ability to install software through a package manager such as yum or apt, you may install the irods-icommands package obtainable from the iRODS Download page. one of the following links.

• CentOS 6
• CentOS 7
• Ubuntu 12
• Ubuntu 14

If you cannot use one of the packages from iRODS, we provide some installers for CentOS 6 and Ubuntu 14.04 that don't require special privileges to use. To do so, do the following.

1. Download the correct installer: CentOS 6 Installer or Ubuntu 14.04 Installer. 
   

2. From a terminal window, run the installer with sh.
   For example, if you were installing iCommands on an Ubuntu 14.04 system as the user cyverse-user, you would do something like the following example. The iCommands would end up being placed in the directory /home/cyverse-user/icommands.

   Code Block
   ? sh irods-icommands-4.1.910-ubuntu14.installer Where would you like to install it? [/home/cyverse-user] Expanding contents under /home/cyverse-user.

Tip: If using iCommands in an HPC environment, which has many systems with iCommands installed, run the module load irods command to get access to iRODS iCommands.  

    3, Continue to Step 2 to configure iCommands.  

Mac OS

iRODS does not currently offer a 4.1 .9 download for Mac OS. However, CyVerse has created a custom build of the iRODS 4.1.9 iCommands. To obtain these, do the following.

1. Download the Mac OS X iCommands 4.1.9 installer. 
2. Find the downloaded package, hold down the control button, and select the downloaded package.
3. Click Open to give permission for Mac to open the package.
4. Use the installer and follow the instructions.
5. Open a new terminal window, if necessary, and run ienv to check that the new version was installed.
   
6. Continue to Step 2 to configure iCommands.
   

Microsoft Windows

iRODS does not currently offer a 4.1 .9 download for Windows users. If you already had version 3 of iCommands for Windows installed, the main functionality should work, but it has not been tested by CyVerse. If necessary, you can use Cyberduck insteadare running Windows 10,  iCommands can be run on the Linux subsystem.  See https://docs.microsoft.com/en-us/windows/wsl/install-win10 for information on installing the Linux subsystem. Then follow the iCommands for Linux instructions to install iCommands.

Step 2: Initialize/start the iRODS connection and configure the settings (One-time only or after changing your configuration)

The first time you use iCommands, or after you change a configuration detail such as resetting your CyVerse password, you must initiate the connection to iRODS.

1. In a terminal window, enter iinit to initialize iCommands and your Data Store connection. For example, here's what you would do if your user name were cyverse-user:

   Expand
   title Example: iinit
   Code Block title Terminal window displays: bash-3.2$ iinit One or more fields in your iRODS environment file (irods_environment.json) are missing; please enter them. Enter the host name (DNS) of the server to connect to: data.cyverse.org Enter the port number: 1247 Enter your irods user name: cyverse-user Enter your irods zone: iplant Those values will be added to your environment file (for use by other i-commands) if the login succeeds. Enter your current iRODS password: bash-3.2$

2. For both platforms, enter the following:
   

   Include Page
   docs:_ConfigurationSettingsDataStoreConnection docs:_ConfigurationSettingsDataStoreConnection

3. When done, log out of the Data Store (especially important if on a shared or public computer) by entering iexit. (To log back in on the same device, enter iinit.)
   After you are successfully connected, you do not need to reenter your password to use the CyVerse Data Store from that computer unless you log out.

Advanced configuration settings (for power users only)

A full list of iRODS environment variables that may be set can be determined from ~/.irods/irods_environment.json section of the official iRODS documentation. The names environment variables are the all capitals version of the corresponding json field. Only IRODS_HOST, IRODS_PORT, IRODS_USER_NAME, and IRODS_ZONE_NAME are required use the iCommands.

Setting up Bash autocomplete for iCommands

If using the Bash shell, you can enable tab auto-complete when using iCommands:

1. Download i-commands-auto.bash.

2. In your home directory, rename i-commands-auto.bash to .i-commands-auto.bash

3. In your .bashrc or .bash_profile, enter the following:
source .i-commands-auto.bash

Next steps

To learn more about using iCommands and some common commands to use, see Using iCommands.

Cyberduck

Cyberduck is a free cross-platform open source file transfer program that supports multiple transfer protocols (FTP, SFTP, WebDAV, Cloud files, Amazon S3, etc.) for Mac OS (OS X 10.6 on Intel 64-bit) and Windows (XP, Vista, Windows 7, Windows 8). LINUX users should use iCommands or iDrop Desktop. For an overview of using Cyberduck at CyVerse, see Using Cyberduck for Uploading and Downloading to the Data Store. Instructions for installing for anonymous login are below.

Installing and configuring Cyberduck

Step 1: Install or update Cyberduck

If Cyberduck is already installed, check if you need to update:

1. Click the Cyberduck menu.
2. Click Check for Updates.
3. If an update is available (you must be at version 4.7.1 or later), click Install Update.

To install Cyberduck for your operating system for initial installation:

1. Go to the Cyberduck installation page at https://cyberduck.io/.
2. Follow the steps for your OS (not available for Linux users):

• • For Mac OS:
  1. Click Download Cyberduck-4.7.1.zip (or later version than 4.7.1).
  2. Move the downloaded file (either a zip file or the unzipped application file, depending on your browser) to your Applications folder. If the zip file is listed, unzip the file in your Applications folder.
     IMPORTANT: The file must be located in your Applications folder.

• • For Windows:

1. Click Download Cyberduck-Installer-4.7.1.exe (or later version than 4.7.1, if displayed).
2. Locate the downloaded file and double-click to begin installation.
3. Go through the install process.

Step 2: Configure Cyberduck for use with the Data Store (one-time only)

These instructions are for configuring the iRODS Connective Profile for the Data Store, both for CyVerse user account access and for anonymous access, if you do not have a CyVerse user account.

1. Click to open Cyberduck.
   See the Cyberduck Preferences Help page on the Cyberduck website for more information on installation.
2. Click this link to download the Connection Profile, which contains preconfigured settings for using Cyberduck with the CyVerse Data Store.

3. Click to open the downloaded CyVerse Data Store.cyberduckprofile file.

4. Verify CyVerse Data Store is displayed in the first field.
5. In the Nickname field, enter data.cyverse.org – iRODS.
6. Create the connection:
   
   1. In the Server field, enter data.cyverse.org.
   2. In the Port field, enter 1247.
   3. Click the Anonymous Login checkbox.
   4. Verify anonymous is displayed in the Username field.
      (To connect using your CyVerse user account, enter your CyVerse user name in the Username field, and then verify your username is added to the URL field, as shown above. Example: irods://yourusername@data.cyverse.org.
      The remaining fields are populated.

7. Click in the Transfer Files drop-down list and select Open multiple connections.

8. Close the window.

Using Cyberduck

Logging in to Cyberduck with anonymous login (no user account)

1. If necessary, click Image Removed to open Cyberduck.
2. In the Cyberduck window, click the bookmark to use and then click Open Connection.
3. In the drop-down list, click the repository to use.
4. Click the Anonymous Login checkbox.
5. Click Connect.

Downloading a file or folder from the Data Store (CyVerse users and anonymous users)

• To change your default download folder, click Image Removed, click Download Folder and select the new default download folder.