DigitalBiology 1.02: Using Linux Server for Digital Biology

As we transition from traditional biology to Digital Biology, our primary tool shifts from the workbench to the computer. However, for the massive scale of modern genomics, a personal laptop is often insufficient. This guide introduces you to the backbone of bioinformatics research: the Linux Server. We will explore what a server is, why it is the industry standard, and how you can start using it to unlock the secrets of biological data.

In simple terms, a server is a computer designed to process requests and deliver data to other (client) computers over a local network or the internet. Unlike a personal laptop which shuts down when you close the lid, a server is built for reliability, power, and continuous operation. It is the engine room of the internet and scientific research.

Servers can run different Operating Systems (OS):

• Windows Server: Common in corporate environments for managing users and email. While user-friendly with its graphical interface, it consumes more system resources and is less commonly used for high-performance scientific computing.
• macOS: While excellent for personal bioinformatics (because it is Unix-based!), you rarely see "macOS Servers" in science. Apple hardware is expensive and doesn't scale well for data centers compared to standard rack-mounted Linux servers.
• Unix / BSD: The predecessors and cousins of Linux (e.g., FreeBSD, Solaris). While robust and stable, they lack the broad compatibility and specific software ecosystem that the bioinformatics community has built around Linux.
• Linux: The dominant OS for supercomputers and the cloud. It is open-source, incredibly efficient, and strictly command-line focused in server environments.

Why Do We Choose Linux?

For digital biology, Linux is not just a choice; it's the standard.

1. Software Ecosystem: The vast majority of bioinformatics tools (like bwa, samtools, flye) are developed natively for Linux. Many do not have Windows or macOS versions.
2. Efficiency: Linux has minimal overhead. It devotes almost all its power to your analysis rather than running a pretty user interface.
3. Automation: The Linux command line allows for powerful scripting (Shell, Python) to automate complex pipelines on massive datasets.
4. Cost: It is open-source and free, making it ideal for scaling up to hundreds of CPUs without licensing fees.

You might wonder, "Why can't I just use my laptop?" The answer comes down to Software Availability, Scale, Power, and Reliability.

• Challenge 1: Software is Linux-Native.
• The scientific community overwhelmingly develops software for Linux. Many of the most fundamental tools in bioinformatics, like the aligner bwa or the genome assembler flye, do not have official versions for Windows or macOS. To use the core toolset of the field, you must use Linux.
• Challenge 2: Data is HUGE.
• Bioinformatics deals with massive datasets. A single human genome is over 3 billion DNA base pairs. The raw sequencing data from one experiment can be hundreds of gigabytes (GB) or even terabytes (TB). Your laptop's storage would fill up instantly.
• Challenge 3: Analysis Requires Immense Power.
• Your laptop, with its 8 or 16 GB of RAM, simply can't handle tasks like assembling a new genome, which might require over 100 GB of RAM. The analysis would crash before it even starts.
• Servers are high-performance computers packed with multiple powerful CPUs and vast amounts of memory, designed specifically for heavy computation.
• Challenge 4: Analyses Take a Long Time.
• A complex analysis can run for hours, days, or even weeks. Servers are built for 24/7 reliability. You can start a job, disconnect your laptop, and the server will keep working tirelessly.
• Challenge 5: Science is Collaborative.
• Servers are multi-user environments. This allows entire teams of researchers to log in, share the same data, use the same tools, and work together on large projects seamlessly.

In short, a personal computer is a sedan, but for the industrial-scale work of bioinformatics, you need a freight train. The server is your freight train.

It's hard to state exact requirements, as they vary by project. However, looking at a typical Human Whole-Genome Sequencing (WGS) analysis gives a powerful sense of scale. Let's walk through the storage requirements for analyzing one person's genome.

• Step 1: Raw Data
• To accurately analyze a human genome (~3 billion bases), we need to sequence it about 30 times over (known as 30X coverage). This results in about 90 gigabases of raw sequence data.
• Initial Disk Space: The compressed raw data files (.fastq.gz) for this take up ~86 GB.
• Step 2: Quality Control
• We process the raw data to filter out low-quality reads. This "clean" data is smaller but still substantial. If we keep the original files for safety, our total space grows.
• Disk Space after QC: 48 GB (clean data) + 86 GB (raw data) = 134 GB.
• Step 3: Alignment
• Next, we align our clean data to a reference human genome. This is one of the most resource-intensive steps. The output file, which maps every read to its location, is massive.
• The initial alignment file (.sam) can be 251 GB.
• After sorting, compressing, and removing duplicates, we get a final .bam file of ~51 GB.
• Peak Disk Space during Alignment: If you keep all intermediate files, you'll temporarily need over 370 GB.
• Step 4: Variant Calling & Annotation
• Finally, we identify genetic variations (SNPs, etc.) and compare them against known databases.
• Variant files (.vcf) are much smaller, taking up about 8 GB during processing.
• Annotation databases can add another ~14 GB.

The Bottom Line:

Just to analyze a single human genome, the minimum storage required (keeping only the essential raw and final files) is roughly 193 GB. If you keep all the intermediate steps, you'll need over 530 GB.

And this is for a standard analysis. Advanced methods like Nanopore sequencing can require nearly 3 Terabytes (TB) for one person. Now, imagine a study with hundreds of patients. This is why servers with vast storage are not just a luxury—they are a necessity.

Let's get hands-on. Here is a step-by-step guide to the fundamental skills you'll need.

Step 1: Connecting to the Server (SSH)

You connect to a remote server using a protocol called SSH (Secure Shell). All you need are the server's IP address (its unique address on the internet), your username, and your password.

• On macOS or Linux: Open the Terminal app.
• On Windows: Open PowerShell or use a client like [PuTTY](https://www.putty.org/).

Type the following command, replacing username and serveripaddress with your own credentials, and press Enter.

ssh username@server_ip_address

The server will ask for your password. Note: As you type your password, you will not see anything on the screen. This is a security feature. Just type it carefully and press Enter.

Congratulations, you're in!

Step 2: Finding Your Way Around (Navigation)

Now that you're connected, let's learn the three most important commands for navigation.

• pwd (Print Working Directory): Shows you which directory you are currently in.

pwd

• ls (List): Lists the files and subdirectories in your current directory.

ls -lh
# The "-lh" flags make the output long (detailed) and human-readable (e.g., showing KB, MB, GB).

• cd (Change Directory): Moves you to a different directory.

# Move into a directory named 'project_alpha'
cd project_alpha
# Move back up one level to the parent directory
cd ..
# Go straight to your home directory from anywhere
cd ~

Step 3: Managing Files and Directories

Here's how you create, copy, move, and delete things.

• mkdir (Make Directory): Create a new directory.

mkdir my_first_project

• mv (Move/Rename): Use it to either move a file or rename it.

# Rename a file
mv old_filename.txt new_filename.txt
# Move a file into a directory
mv new_filename.txt my_first_project/

• cp (Copy): Copies a file.

cp source_file.txt copy_of_file.txt

• rm (Remove): Deletes a file. BE CAREFUL! There is no "Recycle Bin" on the command line. Once it's gone, it's gone.

rm copy_of_file.txt
# To delete a directory and everything inside it, use "-r" (recursive)
rm -r directory_to_delete

Step 4: Viewing Files

You'll often need to peek inside files.

• less: The best way to view large files. It lets you scroll up and down. Press q to quit.

less my_large_data_file.txt

• head / tail: Quickly view the first or last 10 lines of a file, respectively.

head my_large_data_file.txt

Step 5: Transferring Data to and from the Server

You'll need to move data from your computer to the server (upload) and results from the server back to your computer (download). The scp (Secure Copy) command is perfect for this.

This command is run from your LOCAL computer's terminal, not on the server.

• To upload a file:

# Syntax: scp <local_file> <username@server_ip:destination_path>
scp my_local_data.fastq username@123.45.67.89:/home/username/my_first_project/

• To download a file:

# Syntax: scp <username@server_ip:file_to_download> <local_destination>
scp username@123.45.67.89:/home/username/results.txt .
# The "." at the end means "my current local directory"

Step 6: Keeping Your Work Running (Tmux)

What if your analysis takes 12 hours? If you close your laptop, your SSH connection will break and the process will be terminated. To prevent this, we use a "terminal multiplexer" like tmux. It creates a persistent session on the server that stays alive even if you disconnect.

• Start a new tmux session:

tmux new -s my_analysis

Your screen will flash, but it looks the same. You are now inside a protected session.

• Run your long command.
• Detach from the session: Press Ctrl+b, then release, and then press d. You'll pop back out to your normal terminal. Your job is still running safely inside the tmux session.
• Re-attach to your session later:

tmux attach -t my_analysis

And you're right back where you left off!

You've just learned the absolute essentials of using a Linux server. Like any new language, fluency comes with practice. Log in every day, navigate around, and manage some test files.

These commands are your foundation. Upon this, you will build the skills to install software, write powerful scripts, and ultimately, make groundbreaking discoveries. Good luck!