User Tools

Site Tools


wiki:2018networking

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
wiki:2018networking [2018/07/10 14:20]
erin created
wiki:2018networking [2018/08/30 06:13] (current)
erin
Line 1: Line 1:
-=====CONNECTING TO REMOTE ​SERVERS=====+{{ :​wiki:​presentation1_computers.jpg?​400 |}} 
 + 
 +=====CONNECTING TO REMOTE ​COMPUTERS ​=====
 ---- ----
-Before we can start to manipulate genomics datasets, we need to know how to obtain the data.  Even if you are generating your own data, you will almost certainly need reference genome sequences and genome annotation information to guide the analyses of your data.  A common challenge for biologists is obtaining reference data and manipulating it so that it can be used in their analyses. ​ Linux tools are incredibly useful in this regard. 
  
-====OUTLINE==== +We have already connected to a remote computer in this class!\\ 
-  ​-What is a server? +  ​* genome repositories 
-  ​-How are servers accessed?+  ​* rsync, wget, ftp, sftp, curl, scp
  
-====1. ​What is a server?====+What other types of computers can we log into?
  
-  ​* Desktop/​laptop ​computers ​have limited ability to store and process data. +  ​1. remote personal ​computers 
-  ​* Computer networks involving servers can have tremendous capacity for storing and processing data. +  ​2. servers ​(computing or file storage) 
-  * Large datasets are often stored, processed, analyzed on remote ​servers. +  ​3supercomputers
-  ​* Desktop and laptop computers are getting faster processors, more cores, more RAM, and more storage but datasets are getting bigger and demand more and more computing power to process and more space for storage.+
  
-**Some Common ​Features of a Computer/​Server**+------ 
 + 
 +==== Features of Remote Servers & Supercomputers ==== 
  
   * **Hard drive/​solid-state drive:** non-volatile data storage and retrieval (space measured in GB or TB).   * **Hard drive/​solid-state drive:** non-volatile data storage and retrieval (space measured in GB or TB).
Line 22: Line 24:
   * **CPU cache:** faster memory for temporary storage of copies of frequently used data (usually measured in MB).   * **CPU cache:** faster memory for temporary storage of copies of frequently used data (usually measured in MB).
  
-====2. How are servers accessed?​==== 
-  * Shell or terminal 
-  * GUI (Cyberduck or Filezilla; or web browser) 
-\\ 
-**Accessing servers via the command line (ssh and sftp)** 
  
-**ssh**….secure shell. +==== Accessing remote computers with ssh - Secure SHell ==== 
- ssh username@server_address + 
-\\ +**ssh usage**\\ 
-**A common method for downloading data from public server (ftp)**+**ssh** <​addressOfRemoteServer>​\\ 
 +**ssh** [-l <​yourloginname>​] <​addressOfRemoteServer>​ #​that'​s ​lower case "​L"​ 
 + 
 + 
 +:!: **Mini demo**. Watch Erin log into her desktop computer: 
 + 
 +{{ :​wiki:​img_20180829_171046_pixlr.jpg?​direct&​400 |}} 
 + 
 +[[wiki:​supercomputing|{{:​wiki:​slide2_arrow.png?​40|}}]] [[wiki:​supercomputing|supercomputing]] 
  
-**ftp**….file transfer protocol. 
- $ ftp ftp.someaddress.org  
-\\ 
-**Transferring data between computers/​servers (scp)** 
  
-**scp**….secure copy protocol. 
- $ scp /​path/​to/​local/​file username@hostname:/​path/​to/​remote/​file 
-\\ 
-**Run multiple sessions in one window or keep processes running on a remote server without a job scheduler (GNU screen)** 
- $ screen 
  
-*See UNIX cheat sheet and exercises for additional information. 
  
wiki/2018networking.1531254045.txt.gz · Last modified: 2018/07/10 14:20 by erin