HTTP API Design

What is Representational State Transfer (REST)?

REST, which stands for Representational State Transfer, is an architectural style for building distributed systems over the web. It is a set of principles that define how web-based systems should be designed and built. RESTful systems typically communicate over the HTTP protocol, using standard HTTP verbs like GET, POST, PUT, and DELETE to perform CRUD (create, read, update, delete) operations on resources.

There are six architectural properties of REST, as defined by Dr. Roy Fielding in his PhD thesis. These properties are:

Client-Server: The client and the server should be separated, allowing them to evolve independently. This separation enables the client and server to be developed and deployed independently, which can lead to better scalability, reliability, and maintainability of the system.

Example: In a RESTful web application, the client sends requests to the server to retrieve or manipulate resources. The server responds with the appropriate response, which the client can interpret and act upon.

Stateless: The server should not maintain any client state. Each request from the client should contain all the information necessary to perform the request. This means that the server does not need to store any client session state, which can improve scalability and reliability.

Example: In a RESTful system, each request from the client contains all the necessary information, such as authentication credentials, headers, and query parameters, to perform the request. The server does not maintain any session state between requests.

Cacheable: The server should indicate whether a resource can be cached or not. This can improve performance and scalability by reducing the number of requests that need to be made to the server.

Example: The HTTP protocol includes cache-control headers that indicate whether a resource can be cached or not. If a resource is cacheable, the client can cache it and reuse it for subsequent requests, reducing the need to make requests to the server.

Uniform Interface: The interface between the client and the server should be uniform, using standard HTTP methods and media types. This allows different clients to interact with the same server in a consistent way.

Example: In a RESTful system, resources are identified by URIs, and operations on those resources are performed using standard HTTP methods like GET, POST, PUT, and DELETE. The response to a request should be in a standard media type like JSON or XML.

Layered System: The system should be designed in layers, allowing each layer to be developed and deployed independently. This can improve scalability, reliability, and security.

Example: In a RESTful system, there may be multiple layers, such as a load balancer, a caching layer, an API gateway, and a data storage layer. Each layer can be developed and deployed independently, allowing for better scalability and reliability.

Code on Demand (Optional): The server can optionally send executable code to the client to be executed in the client's context. This can be used to implement complex functionality on the client-side.

Example: A RESTful system may include JavaScript code that is sent to the client to be executed in the client's web browser. This code can be used to implement complex functionality on the client-side, reducing the need for server-side processing.

In summary, the architectural properties of REST are client-server, stateless, cacheable, uniform interface, layered system, and optional code on demand. These properties are designed to enable the development of scalable, reliable, and maintainable web-based systems.

Client Server Architecture

A client server design enforces separation of concerns. The user agent (client) is not concerned with the storage of the resources it is interested in. This loose coupling also simplifies the implementation of server components, improving scalability and allowing server components to evolve independently of clients (anarchic scalability)

Statelessness

A stateless protocol is one where the entity receiving a request (typically a server) retains no session information. If session information is required, then the client must send the relevant data to the server with every request. This is to ensure that each packet of data received by a server can be understood in isolation from previous packets in a session. Stateless protocols reduce the server overhead required to track and maintain session data for each client it is in communication with.

Cacheability

User agents (clients) may cache the responses they receive from a server. A server may identify a request and cacheable or non-cacheable. This prevents clients from sending stale information about resource(s) when sending requests to a server. If cacheing is applied correctly, it can reduce or eliminate some client-server interactions. This has the effect of improving scalability on performance by reducing the server's workload. A client may cache response data in memory or on disk (e.g. a browser's local storage). Cacheing can also be handled by content delivery networks (CDNs).

Layered System

Typically, a client has no way of knowing if it is connected directly with the target server. Placing a proxy or a load balancer between the client and the server does not affect their communication and therefore does not effect a change in the implementation of the user agent or the origin server. Intermediary servers can improve the scalability of the system by responding with cached responses and load balancing across multiple servers. Intermediaries can also be used to secure web services, separating security from business domain concerns. When required, intermediate servers can also orchestrate requests to mulitple other servers in order to respond to a single client request.

Code on demand

A server can modify a client's functionality by serving executable scripts (e.g. javascript) in response to a client's request.

Uniform Interface

The "Uniform Interface" is one of the key constraints in the REST (Representational State Transfer) architectural style that defines a set of guidelines for building web services. The Uniform Interface constraint helps to achieve loose coupling between clients and servers, enabling the components to evolve independently. It also enables the creation of generic clients that can interact with any server that follows the same uniform interface.

The Uniform Interface constraint requires that the interface between clients and servers should be consistent and well-defined. This constraint is further broken down into four sub-constraints:

Resource-based

Resources should be identified by unique URIs (Uniform Resource Identifiers) and manipulated using a limited set of methods, such as GET, POST, PUT, DELETE.

fetch('https://example.com/api/users')
  .then(response => response.json())
  .then(data => {
    // Handle the retrieved list of users
  })

Manipulation of resources through representations

Clients should manipulate resources by sending representations of the resources to the server. The server should respond with a representation of the updated resource or a status code indicating the result of the operation.

fetch('https://example.com/api/users', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    name: 'John Doe',
    email: 'john@example.com'
  })
})
.then(response => response.json())
.then(data => {
  // Handle the newly created user resource
})

Self-descriptive messages

Each message should include enough information to describe how to process the message. This can include information about the media type of the representation, cacheability, and language of the representation.

fetch('https://example.com/api/users/1')
  .then(response => {
    const contentType = response.headers.get('Content-Type')
    const cacheControl = response.headers.get('Cache-Control')
    
    return response.json()
  })
  .then(data => {
    // Handle the retrieved user resource
  })

Hypermedia as the engine of application state (HATEOAS):

The server should include links in its responses that enable clients to discover and navigate to related resources. This enables the server to drive the application state and provide a more flexible and dynamic architecture and enables clients to discover available actions without needing prior knowledge of the API structure.

fetch('https://example.com/api/users/1')
  .then(response => response.json())
  .then(data => {
    const ordersLink = data._links.orders.href
    
    // Use the ordersLink to fetch the user's orders
    fetch(ordersLink)
      .then(response => response.json())
      .then(data => {
        // Handle the retrieved list of orders
      })
  })

The HATEOAS specification outlines several types of links that can be used to describe the relationship between resources and guide clients on how to interact with the API. Some common types of HATEOAS links are:

Self: A link pointing to the current resource. This link provides a way for clients to reference the resource's own URI. For example, a user resource might have a self link:

{
  "id": 1,
  "name": "John Doe",
  "_links": {
    "self": {
      "href": "https://api.example.com/users/1"
    }
  }
}

Collection: A link pointing to the collection of resources. This link can be used to access the list of resources that the current resource is a part of. For example, a book resource might have a link to its collection of books:

{
 "id": 42,
 "title": "The Hitchhiker's Guide to the Galaxy",
 "_links": {
   "collection": {
     "href": "https://api.example.com/books"
   }
 }
}

Related: A link pointing to a resource that is related to the current resource. For example, an invoice resource might have a link to the related customer resource:

{
  "id": 123,
  "amount": 100.0,
  "_links": {
    "related_customer": {
      "href": "https://api.example.com/customers/456"
    }
  }
}

Navigation: A link that aids clients in navigating through a collection of resources, like pagination links. For example, a collection of blog posts might have "next" and "prev" links for navigating through pages:

{
  "page": 2,
  "items": [...],
  "_links": {
    "next": {
      "href": "https://api.example.com/blog-posts?page=3"
    },
    "prev": {
      "href": "https://api.example.com/blog-posts?page=1"
    }
  }
}

Action: A link representing an action that can be performed on the current resource. For example, a task resource might have an "update" link to indicate that the task can be updated:

{
  "id": 789,
  "description": "Finish project",
  "completed": false,
  "_links": {
    "update": {
      "href": "https://api.example.com/tasks/789",
      "method": "PUT"
    }
  }
}

Alternate: A link pointing to an alternate representation of the current resource, such as a different format or language. For example, a news article might have a link to an alternate version in a different language:

{
  "id": 1001,
  "title": "Breaking News",
  "content": "This is an English article...",
  "_links": {
    "alternate_es": {
      "href": "https://api.example.com/articles/1001?lang=es",
      "type": "text/html"
    }
  }
}

Canonical: A link pointing to the canonical, or primary, version of a resource when multiple versions or aliases exist. For example, a product with multiple URLs might have a canonical link:

{
  "id": 2001,
  "name": "Widget",
  "_links": {
    "canonical": {
      "href": "https://api.example.com/products/2001"
    }
  }
}

Search: A link representing a search mechanism for finding resources. For example, a collection of movies might have a search link for clients to discover movies:

{
  "movies": [...],
  "_links": {
    "search": {
      "href": "https://api.example.com/movies/search{?query}",
      "templated": true
    }
  }
}

Create-form: A link pointing to a resource that provides a form or template for creating new resources. For example, a collection of tickets might have a create-form link to guide clients in creating new tickets:

{
  "tickets": [...],
  "_links": {
    "create-form": {
      "href": "https://api.example.com/tickets/create-form"
    }
  }
}

First, Last: Links for navigating to the first and last pages of a paginated collection. For example, a paginated collection of blog posts might have "first" and "last" links:

{
  "page": 3,
  "items": [...],
  "_links": {
    "first": {
      "href": "https://api.example.com/blog-posts?page=1"
    },
    "last": {
      "href": "https://api.example.com/blog-posts?page=10"
    }
  }
}

Let's look at how we might incorporate HATEOS into a RESTful Todo API. First, let's break down the different technologies and concepts involved in building this API:

REST (Representational State Transfer): This is an architectural style for building web services, where resources are represented as URIs (Uniform Resource Identifiers) and accessed via HTTP verbs (GET, POST, PUT, DELETE). RESTful APIs are designed to be stateless and have a clear separation of concerns between the client and server.
HATEOAS (Hypermedia as the Engine of Application State): This is a constraint of the REST architecture, which requires that a client be able to navigate the API entirely through hypermedia links that are returned with each response. In other words, the API should provide a set of links that allow the client to discover the available resources and actions that can be taken on those resources.

Now, let's dive into the steps for building the API (In this example we are using Node.js and the Express framework):

Define the data model for the todo items. In this example, we'll use a simple model with the following properties: id (unique identifier), title (string), completed (boolean), and created_at (timestamp).
Define the routes for the API. In this example, we'll create routes for getting all todo items, creating a new todo item, getting a single todo item, updating a todo item, and deleting a todo item. Here's what the routes might look like:

// GET all todo items
app.get('/todos', (req, res) => {
  // TODO: Implement logic to fetch all todo items
  const todos = []; // Placeholder for now
  res.json({ todos });
});

// POST a new todo item
app.post('/todos', (req, res) => {
  const { title } = req.body;
  // TODO: Implement logic to create a new todo item 
  const todo = { id: 1, title, completed: false, created_at: new Date() }; // Placeholder for now
  res.status(201).json({ todo });
});

// GET a single todo item by ID
app.get('/todos/:id', (req, res) => {
  const { id } = req.params;
  // TODO: Implement logic to fetch a single todo item by ID
  const todo = { id: parseInt(id), title: 'Example Todo', completed: false, created_at: new Date() }; // Placeholder for now
  res.json({ todo });
});

// PUT (update) a todo item by ID
app.put('/todos/:id', (req, res) => {
  const { id } = req.params;
  const { title, completed } = req.body;
  // TODO: Implement logic to update a todo item by ID 
  const todo = { id: parseInt(id), title, completed, created_at: new Date() }; // Placeholder for now
  res.json({ todo });
});

// DELETE a todo item by ID
app.delete('/todos/:id', (req, res) => {
  const { id } = req.params;
  // TODO: Implement logic to delete a todo item by ID
  res.sendStatus(204);
});

Add HATEOAS links to the API responses. In each response, we'll include a set of links that allow the client to navigate to other resources or perform actions on the current resource. For example, the response for getting a single todo item might include links to update or delete the todo item:

{
  "todo": {
    "id": 1,
    "title": "Example Todo",
    "completed": false,
    "created_at": "2023-04-05T15:30:00.000Z",
  },
  "_links": {
    "self": { "href": "/todos/1" },
    "update": { "href": "/todos/1", "method": "PUT" },
    "delete": { "href": "/todos/1", "method": "DELETE" }
  }
}

To add these links, we'll define a helper function that takes a resource object and returns a new object with the _links property added. Here's what the helper function might look like:

function addLinks(resource, req) {
  const base = `${req.protocol}://${req.get('host')}`;
  const selfLink = { href: `${base}${req.originalUrl}` };
  const links = { self: selfLink };
  if (resource.id) {
    links.update = { href: `${base}${req.baseUrl}/${resource.id}`, method: 'PUT' };
    links.delete = { href: `${base}${req.baseUrl}/${resource.id}`, method: 'DELETE' };
  }
  return { ...resource, _links: links };
}

We'll call this function in each route handler to add the links to the API responses:

app.get('/todos/:id', (req, res) => {
  const { id } = req.params;
  const todo = { id: parseInt(id), title: 'Example Todo', completed: false, created_at: new Date() };
  const resource = addLinks(todo, req);
  res.json({ todo: resource });
});

app.post('/todos', (req, res) => {
  const { title } = req.body;
  const todo = { id: 1, title, completed: false, created_at: new Date() };
  const resource = addLinks(todo, req);
  res.status(201).json({ todo: resource });
});

// ...and so on for the other routes

Additional Resources

Homework

Make your Todo API RESTful. Your API should be HATEOS compliant.
Add the ability to persist changes in your Todo API. Your Todo domain should be decoupled from persistence concerns.
Write a outline describing how you would go about deploying your Todo API to the cloud. Some considerations are:
- What services would you use?
- What are the pros and cons of the various services?
- What changes would you have to make to your API (if any)