Building a Multithreaded Web Server in Rust from Scratch

Introduction

Building a web server from scratch is an excellent way to explore Rust's concurrency features, and its focus on performance and safety makes it a great choice for systems programming. We will build a simple Rust multithreaded web server without using third-party web frameworks. Apart from handling basic HTTP requests and serving static files, we will see the usage of a thread pool for concurrency.

Setting Up the Project

Firstly, let's set up a new Rust project and configure the necessary dependencies. Open your terminal and run the following commands.

cargo new rust_web_server
cd rust_web_server

Writing the Web Server Code

The `main.rs` File

Replace the contents of 'src/main.rs' with the following code

use std::net::TcpListener;
use std::net::TcpStream;
use std::io::BufReader;
use std::io::BufRead;
use std::io::Write;
use std::fs;
use std::thread;
use std::time::Duration;
use rust_web_server::ThreadPool;

fn main() {
    let listener = TcpListener::bind("127.0.0.1:7878").unwrap();
    let pool = ThreadPool::new(4);

    for stream in listener.incoming() {
        let stream = stream.unwrap();

        pool.execute(|| {
            handle_connection(stream);
        });
    }
}

fn handle_connection(mut stream: TcpStream) {
     let buf_reader = BufReader::new(&mut stream);
    let request_line = buf_reader.lines().next().unwrap().unwrap();

    let (status_line, filename) = match &request_line[..] {
        "GET / HTTP/1.1" => ("HTTP/1.1 200 OK", "hello.html"),
        "GET /sleep HTTP/1.1" => {
            thread::sleep(Duration::from_secs(5));
            ("HTTP/1.1 200 OK", "hello.html")
        }
        _ => ("HTTP/1.1 404 NOT FOUND", "404.html"),
    };
    let contents = fs::read_to_string(filename).unwrap();
    let length = contents.len();

    let response = format!(
        "{status_line}\r\nContent-Length: {length}\r\n\r\n{contents}"
    );

    stream.write_all(response.as_bytes()).unwrap();
}

In this code, we have set up a basic web server using the 'std::net' module, handle incoming connections, and use a thread pool 'ThreadPool' to concurrently handle multiple requests.

The 'lib.rs' File

Create a new file called 'lib.rs' in the 'src' directory and add the following code.

use std::thread;
use std::sync::mpsc;
use std::sync::Mutex;
use std::sync::Arc;

pub struct ThreadPool {
    workers: Vec<Worker>,
    sender: mpsc::Sender<Job>,
}

type Job = Box<dyn FnOnce() + Send + 'static>;

impl ThreadPool {
    pub fn new(size: usize) -> ThreadPool {
        assert!(size > 0);

        let (sender, receiver) = mpsc::channel();
        let receiver = Arc::new(Mutex::new(receiver));

        let mut workers = Vec::with_capacity(size);

        for id in 0..size {
            workers.push(Worker::new(id, Arc::clone(&receiver)));
        }

        ThreadPool { workers, sender }
    }

    pub fn execute<F>(&self, f: F)
    where
        F: FnOnce() + Send + 'static,
    {
        let job = Box::new(f);
        self.sender.send(job).unwrap();
    }
}

struct Worker {
    id: usize,
    thread: thread::JoinHandle<()>,
}

impl Worker {
    fn new(id: usize, receiver: Arc<Mutex<mpsc::Receiver<Job>>>) -> Worker {
        let thread = thread::spawn(move || {
            while let Ok(job) = receiver.lock().unwrap().recv() {
                println!("Worker {} got a job; executing.", id);
                job();
            }
        });

        Worker { id, thread }
    }
}

In this code, we define a simple thread pool 'ThreadPool' that uses the 'std::sync::mpsc' channel to communicate with worker threads 'Worker'. Each worker thread runs in a loop, waiting for jobs and executing them.

Creating Static HTML Files

Now, create two static HTML files, 'hello.html' and '404.html', in the 'src' directory.

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>Hello!</title>
  </head>
  <body>
    <h1>Hello World!</h1>
    <p>Hi from a Rustian.</p>
  </body>
</html>

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>404 - Page Not Found</title>
    <style>
      body {
        text-align: center;
        padding: 50px;
      }
      h1 {
        font-size: 2em;
        color: #333;
      }
      p {
        font-size: 1.2em;
        color: #666;
      }
    </style>
  </head>
  <body>
    <h1>Oops! Page Not Found</h1>
    <p>Sorry, the page you are looking for might be in another dimension. Let's get you back to reality!</p>
  </body>
</html>

Running the Web Server

Now, let's run our web server. In the terminal, execute the following command.

cargo run

The server will start and listen on '127.0.0.1:7878'. You can open your web browser and navigate to 'http://127.0.0.1:7878' to see the "Hello!" page or 'http://127.0.0.1:7878/sleep' to simulate a delayed response. For any other path, you will get a "404 Not Found" page.

Output

home-page

404 page.

404-output

Code Explanation

Now, let's delve into the code to understand how our web server works.

Main Function

In the `main` function, we create a 'TcpListener' to bind to the address '127.0.0.1:7878'. We also create a 'ThreadPool' with a size of 4 threads.

let listener = TcpListener::bind("127.0.0.1:7878").unwrap();
let pool = ThreadPool::new(4);

We then enter a loop to accept incoming connections. For each connection, we use the thread pool to handle it concurrently.

for stream in listener.incoming() {
    let stream = stream.unwrap();

    pool.execute(|| {
        handle_connection(stream);
    });
}

Handling Connections

The 'handle_connection' function reads the HTTP request from the client and determines the appropriate response based on the requested path.

fn handle_connection(mut stream: TcpStream) {
    let buf_reader = BufReader::new(&mut stream);
    let request_line = buf_reader.lines().next().unwrap().unwrap();

    let (status_line, filename) = match &request_line[..] {
        "GET / HTTP/1.1" => ("HTTP/1.1 200 OK", "hello.html"),
        "GET /sleep HTTP/1.1" => {
            thread::sleep(Duration::from_secs(5));
            ("HTTP/1.1 200 OK", "hello.html")
        }
        _ => ("HTTP/1.1 404 NOT FOUND", "404.html"),
    };
    let contents = fs::read_to_string(filename).unwrap();
    let length = contents.len();

    let response = format!(
        "{status_line}\r\nContent-Length: {length}\r\n\r\n{contents}",
        status_line = status_line,
        length = length,
        contents = contents
    );

    stream.write_all(response.as_bytes()).unwrap();
}

The server responds with either the contents of 'hello.html', a delayed response for 'GET /sleep', or a "404 Not Found" page.

Thread Pool Implementation

The 'ThreadPool' struct manages a collection of worker threads. Each worker runs in a loop, waiting for jobs to execute.

pub struct ThreadPool {
    workers: Vec<Worker>,
    sender: mpsc::Sender<Job>,
}

The 'execute' method of the 'ThreadPool' sends a job to the worker threads through a channel.

pub fn execute<F>(&self, f: F)
where
    F: FnOnce() + Send + 'static,
{
    let job = Box::new(f);
    self.sender.send(job).unwrap();
}

Worker Implementation

The 'Worker' struct represents an individual worker thread. It runs in a loop, continuously receiving and executing jobs.

struct Worker {
    id: usize,
    thread: thread::JoinHandle<()>,
}

The 'new' function creates a new worker, and the spawned thread executes jobs from the channel.

impl Worker {
    fn new(id: usize, receiver: Arc<Mutex<mpsc::Receiver<Job>>>) -> Worker {
        let thread = thread::spawn(move || {
            while let Ok(job) = receiver.lock().unwrap().recv() {
                println!("Worker {} got a job; executing.", id);
                job();
            }
        });

        Worker { id, thread }
    }
}

Conclusion

In a nutshell, we've built a basic web server in Rust without using fancy tools. This server can handle multiple tasks at once and serves web pages. It's like a foundation for creating more advanced websites in Rust. So, we've taken a step into the world of web development with Rust, keeping things simple and hands-on.