Objectives

You can run RStudio and the all the code in this workshop using University of Michigan’s Great Lakes High Performance Cluster (HPC). By the end of this guide you will be able to:

  • Use Great Lakes and Open OnDemand to start a RStudio session with pre-installed tidyverse libraries.
  • Copy/paste content from your workstation to RStudio and back.
  • Open a new browser window within an Open OnDemand session.
  • Transfer files between Great Lakes and your workstation

Background

You can run RStudio locally on your workstation. However, running RStudio on Great Lakes is a nice approach because:

  • Sometimes data analysis requires a LOT of memory (RAM). It’s easy to ask for more memory on Great Lakes and Great Lakes nodes have more memory available than a typical laptop/workstation.
  • Many software libraries are pre-installed.

That said, using Great Lakes is different than using your laptop/workstation. A few key ideas:

  • Great Lakes is provided by UM Advanced Research Computing. It is only available to folks affiliated with UM.
  • Great Lakes uses Open OnDemand to reserve a part of the cluster to run your analysis in RStudio.
  • Great Lakes requires some setup.
  • Great Lakes is a recharge service, BUT it is heavily subsidized and extremely affordable for most researchers. (Many researchers who are doing a low to moderate amount of data analysis may not have to pay anything.)

How to get help

  • If you have problems/questions, please don’t hesitate to email us at: bioinformatics-workshops@umich.edu

  • When emailing it will speed things along if you could include:

    • Whether you are using Windows or Mac (and optionally which version of the OS you are using).
    • What browser you are using.
    • Whether you are on campus or using the VPN.
    • The specific text of any error messages, if applicable.

What you need

  1. You need a UM core-imaged workstation and a web browser.

  2. You need to be on UM campus or connected to the UM VPN or Michigan Medicine VPN using Cisco Secure Client. (This will require Duo 2-factor authentication.)

  3. You need a user account on Great Lakes. The first time, you need to request this from ARC; it can take 1-2 days to get set up:

  4. You will need data. Below, we’ll show you how to download the input data used in the workshop, but you can adapt that to use your own data.

Using Open OnDemand

Open OnDemand (OOD) connects you to the Great Lakes cluster through your web browser. Once you’ve got everything you need from above, you’re going to use OOD to launch a RStudio session.

  1. If off-campus, connect to the VPN using Cisco Secure Client.

  2. In your browser, go to Great Lakes Open On Demand:
    greatlakes.arc-ts.umich.edu and login with your uniqname and password.

  3. In the menu at the top of the screen, click Interactive Apps and select RStudio.

  4. The previous step will display a launch configuration page with several fields, e.g. partial screen shot …

    Enter values below in the corresponding fields:

    Field Value
    R Version Rtidyverse/4.4.0
    RStudio version RStudio/2024.04.1
    Slurm account (This is in the email from ARC)
    Partition standard
    Number of hours 4
    (Enough to get started, adjust if you need more time)
    Number of cores 1
    Memory (GB) 4
    (Enough to get started, ok to boost this if you ever run out)
    Module commands

  5. After you’ve entered the values above, at the bottom of this page click Launch. (Conveniently, the values you entered above will now be your default for launching an RStudio session.) The screen will update to show Great Lakes is preparing your session.

  6. When the session is ready (usually a few seconds later), the screen will update:

    • Drag the Compression slider all the way to the right
    • Drag Image Quality slider all the way to the right
    • Click Launch RStudio

  7. This will open a new browser tab that contains your RStudio session:

  8. You may see a prompt asking permission to use the clipboard. Click Allow.

Using RStudio

Shortcuts / copy-paste

  • In the Rstudio window, you can Zoom in and out using control + and control -.
  • Running RStudio in the browser inside the Great Lakes HPC complicates copying and pasting from your computer to/from the remote session.
    • Copy-paste functionality doesn’t always work perfectly. Sometimes you may need to copy twice to get the content into the buffer. So check your work.
    • Within the RStudio window you can right-click to get a copy or paste context menu.
    • Keyboard shortcuts are control C and control V. This is familliar to PC users but will take some getting used to for Mac-users.
    • Copy and paste mostly work as expected using Chrome or Edge browsers. Other browsers (e.g. Safari) require extra steps:
      • Open the control panel (left side)
      • Select the clipboard icon.
      • Copy text into the clipboard area; within that area, use the native keyboard shortcuts, i.e. control-C on PC or command-C on Mac). Close the control panel.
      • Right-click (or control V) to paste the contents into the console.
  • If the copy-paste pattern is too flaky or clumsy for your tastes, you can upgrade your experience by switching from using Open OnDemand in the browser to a VNC viewer. (Macs have a built in VNC viewer, but that viewer doesn’t support copy/paste with a Unix system.) Details on how to set this up are outside the scope of this guide, but reach out and we can help.

Open a browser

  • FYI, you can launch the Firefox browser by clicking on Applications (top left) | Web Browser. This will open another application tab (along the top).
  • If you want to do any substantial copy/paste, we recommend you use this browser window because it’s more reliable to copy/paste within the session than from your local computer.

Exiting

  • The RStudio session will last for 4 hours (or however long you specified in launch config above).
  • If you close the browser window, you can click Launch RStudio again from the Open OnDemand session page
  • You can end the session several ways. Make sure your scripts and data are saved because these actions do not prompt to save or confirm; and once the session is closed it is gone for good.
    • In RStudio menu at the top, click File | Quit
    • In RStudio, click Session | Quit Session
    • In RStudio, close the RStudio window (top right X)
    • In Open OnDemand session page, click Delete.

Download workshop inputs

  1. The inputs used in the workshop can be installed locally. In the RStudio window, click on the Terminal tab. The tab will be blank with a prompt that looks something like this: [YOUR_UNIQNAME@glXXXX ~]$

  2. Paste the following block into the Terminal prompt: and hit Enter/Return.

    # download workshop inputs --------------------------------------------------
mkdir -p intro-r-rstudio-workshop
cd intro-r-rstudio-workshop

# Use curl to download
# We'll use evironment variables to avoid extremely long command lines
source_url="https://umich-brcf-bioinf-workshop.s3.us-east-1.amazonaws.com"
source_file="IRR/workshop-inputs-irr.tgz"
curl -o workshop_irr_inputs.tgz ${source_url}/${source_file}

# tar unpacks the tarball into directories
tar xzvf workshop_irr_inputs.tgz

# Since we have unpacked the tarball, we can remove it
rm workshop_irr_inputs.tgz

ls IRR/data

    The last line should show two files:

    gapminder_1997.csv
gapminder_data.csv

  3. Now you can review the workshop lessons.

Transferring files

  • Below are details on several ways to move files to or from Great Lakes.

    1. For small transfers, you can use Open OnDemand file browser.
    2. For large transfers, we recommend Globus
    3. You can also use command line tools

  • Note that to transfer a file to/from Great Lakes, you will need to be on the campus network or on the VPN.

Open OnDemand file browser

Open OnDemand (OOD) lets you browse your files and move small files (e.g. scripts or plots) between your workstation and Great Lakes using your web browser.

1.1 In your workstation’s browser, open the OOD Dashboard. Along the top menu, click on Files. (Note that if you shrink the screen very small, the menu items will be hidden in a “hamburger”.) In the dropdown menu, click Home Directory.

1.2 OOD will display the contents of your home directory. You can click on a directory to see its contents. 1.3 To view a plot graphic, you can click on the hamburger and then select View. This will open the plot in a new browser tab. 1.4 You can download one or more files by selecting their checkboxes and clicking the Download button. (Note: if you select a directory and click Download, OOD will download the contents as a single zipped file.)

Globus

To analyze your own files, we recommend transferring them to Great Lakes using Globus.

Command line tools

For larger files/directories, we strongly recommend you use Globus. That said, if you are more comfortable with command line tools, you can transfer files using the secure copy command scp. scp is a lot like cp but it allows you to copy files across a network.

To transfer from your workstation to Great Lakes:

3.1 From you workstation terminal or command window, cd into the directory that contains your data. 3.2 Adjust the scp command below to match the correct source directory and uniqname and hit Enter/Return to execute.

# Copy the SOURCE_DIR dir contents from your workstation to Great Lakes home dir
# -r copies recursively 
# -p preserves the file modification times
scp -pr SOURCE_DIR YOUR_UNIQNAME@greatlakes-xfer.arc-ts.umich.edu:

3.3 The first time you run this command, you may see a prompt like the following; type yes and hit Enter/Return to continue.

The authenticity of host '...' can't be established.
ECDSA key fingerprint is SHA256:....
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes

The command will print a warning (e.g. Warning: Permanently added ‘SERVER_ADDRESS’ to the list of known hosts). This is fine.

3.4 When prompted, type your UM password followed by Enter/Return.

  • Note that the server will not echo any characters when you are typing your password;* this is ok.
  • Note that the password is case sensitive.

3.5 You can also transfer files from Great Lakes to your workstation. From your workstation terminal or command window, adjust the scp command below to match the correct source file and uniqname and hit Enter/Return to execute.

# Copy the SOURCE_FILE from Great Lakes to your current workstation dir
scp YOUR_UNIQNAME@greatlakes-xfer.arc-ts.umich.edu:PATH/TO/SOURCE_FILE .

3.6 When prompted, type your UM password followed by Enter/Return.

  • Note that the server will not echo any characters when you are typing your password;* this is ok.
  • Note that the password is case sensitive.

Summary

  • You can run RStudio using University of Michigan’s Great Lakes High Performance Cluster (HPC).
  • Running RStudio on Great Lakes is a bit different than running it on your workstation. Using Great Lakes requires some set up.
  • Running on Great Lakes is a nice option because the libraries and software are pre-installed and you can run RStudio with as much memory as you need.
  • Open OnDemand (OOD) connects you to the Great Lakes cluster through your web browser.
  • Depending on your workstation and browser, copy/paste on RStudio can require different keyboard shortcuts or different steps.
  • OOD, Globus, and command line tools are several ways to transfer files between Great Lakes and your workstation