Transfer MultiQC Report With scp Exercise


15 Minutes


We just tried two new variations of our MultiQC command, one pointed at the directory of our trimmed FastQC results, and another at the whole project directory. For convenience, we’ve included these reports in the zipped folder you downloaded at the beginning of the class. However, it’s also useful to know how to transfer data to/from a remote instance. We’ll explore how to use scp in this exercise.


Instructions:


  • Work independently in the main room, posting any questions that arise to slack.
  • Recommendations for writing your own code:
    • Read function documentation
    • Test out ideas - it’s okay to make mistakes and generate errors
    • Use a search engine to look up errors or recommended solutions using keywords
  • We’ll review possible solutions after time is up as a group.


  • Make sure that you are viewing your local terminal. The scp command should be run from your own machine.
  • Use scp to transfer the comprehensive multiQC report from the AWS machine to your personal computer.
  • Confirm that you’ve received the report


Hint: The command format is:

scp source destination

You will need to provide your ssh login information to complete the transfer


Solution - Transfer MultiQC Report With scp Exercise

Helper Hint: By default, any multiQC reports will have identical filenames multiqc_report.html. It is helpful to specify a more descriptive destination filename for the transfer below, so that multiple MultiQC reports can be easily differentiated (and don’t overwrite one another).


Helper Hint: Users must know that they should run scp from their local computer, not from the AWS instance.


# Transfer the full report
scp <username>@bfx-workshop01.med.umich.edu:~/RSD_Shell/analysis/out_multiqc_all/multiqc_report.html ~/rsd-workshop/multiqc_report_all.html


LS0tCnRpdGxlOiAiRXhlcmNpc2UgMDMgU29sdXRpb24iCmF1dGhvcjogIlVNIEJpb2luZm9ybWF0aWNzIENvcmUiCm91dHB1dDoKICAgICAgICBodG1sX2RvY3VtZW50OgogICAgICAgICAgICBpbmNsdWRlczoKICAgICAgICAgICAgICAgIGluX2hlYWRlcjogaGVhZGVyLmh0bWwKICAgICAgICAgICAgdGhlbWU6IHBhcGVyCiAgICAgICAgICAgIGZpZ19jYXB0aW9uOiB0cnVlCiAgICAgICAgICAgIG1hcmtkb3duOiBHRk0KICAgICAgICAgICAgY29kZV9kb3dubG9hZDogdHJ1ZQotLS0KPHN0eWxlIHR5cGU9InRleHQvY3NzIj4KYm9keXsgLyogTm9ybWFsICAqLwogICAgICBmb250LXNpemU6IDE0cHQ7CiAgfQpwcmUgewogIGZvbnQtc2l6ZTogMTJwdAp9Cjwvc3R5bGU+Cgo8YnI+CgojIyBUcmFuc2ZlciBNdWx0aVFDIFJlcG9ydCBXaXRoIGBzY3BgIEV4ZXJjaXNlCgo8YnI+CgoqKjE1IE1pbnV0ZXMqKgoKPGJyPgoKV2UganVzdCB0cmllZCB0d28gbmV3IHZhcmlhdGlvbnMgb2Ygb3VyIE11bHRpUUMgY29tbWFuZCwgb25lIHBvaW50ZWQgYXQgdGhlIGRpcmVjdG9yeSBvZiBvdXIgdHJpbW1lZCBGYXN0UUMgcmVzdWx0cywgYW5kIGFub3RoZXIgYXQgdGhlIHdob2xlIHByb2plY3QgZGlyZWN0b3J5LiBGb3IgY29udmVuaWVuY2UsIHdlJ3ZlIGluY2x1ZGVkIHRoZXNlIHJlcG9ydHMgaW4gdGhlIHppcHBlZCBmb2xkZXIgeW91IGRvd25sb2FkZWQgYXQgdGhlIGJlZ2lubmluZyBvZiB0aGUgY2xhc3MuIEhvd2V2ZXIsIGl0J3MgYWxzbyB1c2VmdWwgdG8ga25vdyBob3cgdG8gdHJhbnNmZXIgZGF0YSB0by9mcm9tIGEgcmVtb3RlIGluc3RhbmNlLiBXZSdsbCBleHBsb3JlIGhvdyB0byB1c2UgYHNjcGAgaW4gdGhpcyBleGVyY2lzZS4KCjxicj4KCiMjIyBJbnN0cnVjdGlvbnM6Cgo8YnI+CgotIFdvcmsgaW5kZXBlbmRlbnRseSBpbiB0aGUgbWFpbiByb29tLCBwb3N0aW5nIGFueSBxdWVzdGlvbnMgdGhhdCBhcmlzZSB0byBzbGFjay4KLSBSZWNvbW1lbmRhdGlvbnMgZm9yIHdyaXRpbmcgeW91ciBvd24gY29kZToKICAtIFJlYWQgZnVuY3Rpb24gZG9jdW1lbnRhdGlvbgogIC0gVGVzdCBvdXQgaWRlYXMgLSBpdCdzIG9rYXkgdG8gbWFrZSBtaXN0YWtlcyBhbmQgZ2VuZXJhdGUgZXJyb3JzCiAgLSBVc2UgYSBzZWFyY2ggZW5naW5lIHRvIGxvb2sgdXAgZXJyb3JzIG9yIHJlY29tbWVuZGVkIHNvbHV0aW9ucyB1c2luZyBrZXl3b3JkcwotIFdlJ2xsIHJldmlldyBwb3NzaWJsZSBzb2x1dGlvbnMgYWZ0ZXIgdGltZSBpcyB1cCBhcyBhIGdyb3VwLgoKPGJyPgoKLSBNYWtlIHN1cmUgdGhhdCB5b3UgYXJlIHZpZXdpbmcgeW91ciBsb2NhbCB0ZXJtaW5hbC4gVGhlIGBzY3BgIGNvbW1hbmQgc2hvdWxkIGJlIHJ1biBmcm9tIHlvdXIgb3duIG1hY2hpbmUuCi0gVXNlIGBzY3BgIHRvIHRyYW5zZmVyIHRoZSBjb21wcmVoZW5zaXZlIG11bHRpUUMgcmVwb3J0IGZyb20gdGhlIEFXUyBtYWNoaW5lIHRvIHlvdXIgcGVyc29uYWwgY29tcHV0ZXIuCi0gQ29uZmlybSB0aGF0IHlvdSd2ZSByZWNlaXZlZCB0aGUgcmVwb3J0Cgo8YnI+Cgo+IEhpbnQ6IFRoZSBjb21tYW5kIGZvcm1hdCBpczoKPgo+IH5+fgo+IHNjcCBzb3VyY2UgZGVzdGluYXRpb24KPiB+fn4KPgo+IFlvdSB3aWxsIG5lZWQgdG8gcHJvdmlkZSB5b3VyIHNzaCBsb2dpbiBpbmZvcm1hdGlvbiB0byBjb21wbGV0ZSB0aGUgdHJhbnNmZXIKCjxicj4KCiMjIyBTb2x1dGlvbiAtIFRyYW5zZmVyIE11bHRpUUMgUmVwb3J0IFdpdGggYHNjcGAgRXhlcmNpc2UKCj4gSGVscGVyIEhpbnQ6IEJ5IGRlZmF1bHQsIGFueSBtdWx0aVFDIHJlcG9ydHMgd2lsbCBoYXZlIGlkZW50aWNhbCBmaWxlbmFtZXMgYG11bHRpcWNfcmVwb3J0Lmh0bWxgLiBJdCBpcyBoZWxwZnVsIHRvIHNwZWNpZnkgYSBtb3JlIGRlc2NyaXB0aXZlIGRlc3RpbmF0aW9uIGZpbGVuYW1lIGZvciB0aGUgdHJhbnNmZXIgYmVsb3csIHNvIHRoYXQgbXVsdGlwbGUgTXVsdGlRQyByZXBvcnRzIGNhbiBiZSBlYXNpbHkgZGlmZmVyZW50aWF0ZWQgKGFuZCBkb24ndCBvdmVyd3JpdGUgb25lIGFub3RoZXIpLgoKPGJyPgoKPiBIZWxwZXIgSGludDogVXNlcnMgbXVzdCBrbm93IHRoYXQgdGhleSBzaG91bGQgcnVuIGBzY3BgIGZyb20gdGhlaXIgKipsb2NhbCoqIGNvbXB1dGVyLCBub3QgZnJvbSB0aGUgQVdTIGluc3RhbmNlLgoKPGJyPgoKYGBgCiMgVHJhbnNmZXIgdGhlIGZ1bGwgcmVwb3J0CnNjcCA8dXNlcm5hbWU+QGJmeC13b3Jrc2hvcDAxLm1lZC51bWljaC5lZHU6fi9SU0RfU2hlbGwvYW5hbHlzaXMvb3V0X211bHRpcWNfYWxsL211bHRpcWNfcmVwb3J0Lmh0bWwgfi9yc2Qtd29ya3Nob3AvbXVsdGlxY19yZXBvcnRfYWxsLmh0bWwKYGBgCgo8YnI+Cg==