Understanding HTTP

The web is filled with resources explaining what HTTP is, but most of the explanations are from a high level perspective, keeping things abstract and generalized, which does not suit my style since I prefer to take a look at things on the low level, see how they are implemented and why do we even need them in the first place. If you're like me and still wonder what HTTP truly is, then, welcome to this article! :]

Requirements: For this article, a basic understanding of Networking, HTTP and Web development experience would be helpful but not mandatory.

Communication over the internet

First, let's make sure we have the essential building blocks of how computers communicate. Everything we do on the internet is based on sending and receiving electrical or radio signals. For the sake of simplicity, let's assume two wired computers, A and B, are communicating:

When A sends a message to B, it transfers electricity in a specific manner, which B receives and interprets, thereby conveying the original message.

In the world of computers, when talking about these electrical signals, we refer to them as 0s and 1s, or bits, so when we send a message, we're effectively sending a series of bits. For example, to send the word "hello", the computer converts each character to its ASCII binary representation and sends it to the other computer:

01101000 01100101 01101100 01101100 01101111

The next question is: How do we know that we are sending the message to the right computer? In this example it's easy since we have only two computers, but in real life we probably have thousands or more. That's when new concepts such as IP addresses and TCP are introduced. Let's say that these protocols, among others, are the ones responsible for making sure that our message reaches its intended final destination successfully.

So the message arrived! Now what? we said that B receives these bits and interprets them correctly conveying the original message. But how does B know how to interpret those bits?

Sending messages on the web

When you open your browser and go to www.elieasmar.dev, what's happening is that your browser is sending a message over the internet, just like we discussed above, and this message is in fact a request sent to some other computer that hosts the code for my home page. Back to our example, assume computer A is a regular user, and B is the other computer hosting the code for elieasmar.dev home page. Let's try to construct the message in simple english terms, instead of having to write 0s and 1s:

two computers sending messages to each other

Once B receives the message, it looks up the file containing the data for the home page and sends a message back containing this data, and this is what we refer to as a response. So, someone requests something, and someone responds back. Simple huh?

Well, the story doesn't stop here. In fact, it's just the beginning. Let's explore a more complicated example:

Assume now you are on your bank's website: mybank.com, and you need to login to your account and transfer $100. The message your computer will send to your bank's computer is going to be a bit more complex than our previous example, since we need to specify more details:

- Specify the page we want, which is the bank's page that corrensponds to our account.
- Identify ourselves so that the bank can authorize this transfer.
- State the amount we want to transfer.
- State the currency
- State the recipient
- … And a long list of other technical details that might be needed as well.

Now that we know the information we need to send in our message, or request, let's try to construct this message in english, like we did in our previous example:

...⏳

......⏳

.........⌛

Hmm… Still thinking about how to do that? Are you wondering how to start? what information does the bank computer expect first? how to structure the information? Is it ok If we send it in any format? What if your main language is not english, is it ok to send the message in Spanish? or French? or Chinese?

If you're thinking about all these things and wondering what is the right way to structure this complicated request message, then: “Welcome to the world of protocols”

What is a Protocol ?

According to wikipedia:

“A communication protocol is a system of rules that allows two or more entities of a communications system to transmit information via any variation of a physical quantity "Physical quantity". The protocol defines the rules, syntax, semantics, and synchronization of communication and possible error recovery methods.”

In simpler terms, a protocol is a set of rules for how two entities, two computers in our case, can communicate. It's exactly what solves our problem above, by defining a specific set of rules that dictate how the request and response messages should be formatted: What language should be used, what piece of information should be sent first, etc. Remember, for computer B to respond to the request sent by computer A, it needs to understand the message in the first place, just like you understand what i'm writing here because you understand the English language. So using the same communication tool and rules is key, otherwise, if I start writing chinese and you don't know how to speak Chinese, then nothing I say will make sense to you, right?

What does understanding the language mean? Let's take the following example: If I say: "are ? how you", would that mean anything in English? No! Even if those words do exist, that's not how we ask a question. The question word “how” should come first and be capitalized, the verb second, the “you” third, and the question mark last. If we follow these rules we get: How are you?

Now, do you see those rules for how to properly construct a question in English? Well, that's HTTP's role in the Web!

Welcome to HTTP

As we saw above, the same applies to computers communication. In the world of computers, the language they speak is 0s and 1s, but we also need to agree on how the messages should be structured so other computers can understand bits when received and interpret them correctly, and that's what HTTP does; It defines the structure of how a request message should be, so the receiver can understand it properly, and vice versa, since the client (the one who sends the request), should also be able to understand the message sent by the server (the one who serves the response back to the client).

Enough English examples and mental models, let's take a look at a real request message. If you open your browser, open the network tab in dev tools, go to www.elieasmar.dev and inspect the raw request message, this is what you see:

Note: If you uncheck the raw option, the browser will beautify the message to make it more user friendly, but we want the raw textual representations to see what's really being sent.

Let's try sending a new message requesting one of my other blog posts. This will be the request message:

Did you notice that both requests are very similar? This is the textual representation of bits sent from your browser and computer (or a part of it). They both have the same structure, but some values are a bit different. For instance, let's compare the first line of both requests:

GET / HTTP/2
======
GET /blog/sourcing-vs-executing-a-script HTTP/2

HTTP enforces that the first line of a request consists of the following 3 main components:

- The verb: GET in both requests, which states that we are trying to retrieve a page, instead of deleting or updating or saving something. For those other actions you would use different verbs like PUT, PATCH, DELETE, or POST.
- The path: / in the first request, and /blog/sourcing-vs-executing-a-script in the second one, which specifies the path of the resource we are trying to retrieve. In the first request, we are asking for the home page, defined by /, while in the second one, we are asking for a specific blog post that the server should be aware of: blog/sourcing-vs-executing-a-script. If we were to ask for another blog post, then we would get another path, something like: blog/some-other-blog-post.
- The HTTP version: 2 in this case. There are different versions and each one has some specific details which change from one version to another, so specifiying which version of http we're using is essential for the communication to happen successfully.

The second line specifies the destination of our request message. We want to send it to the computer that stores the data for elieasmar.dev since that's the site we want to visit.

And so on so forth!

Now let's look at how the responses for both requests look like:

Response 1:

Response 2:

Again, as you can see, the response structure is almost the same! For instance, the first line is exactly the same:

HTTP/2 200 OK

HTTP enforces that in a response, the message transferred from the server should include on the first line the following:

- The HTTP version: 2 in this case, for the same reasons explained above.
- Status code: 200 OK in this case. And that's the HTTP way of stating whether the requested message was successfully fulfilled by the server or not. In this case, the server is saying: “Yes, I got your message, I found what you're looking for and I sent it to you!”. There are other status codes that signify something else. One example would be: 404 NOT FOUND which can be used when the client requests a resource/page that the server cannot find (maybe a page that was deleted or something), so instead of saying “OK i found it, here you go”, it says: “Sorry I cannot find what you requested”, and the response's first line would become:

HTTP/2 404 NOT FOUND

There are a bunch of other fields included such as the type of data sent, (html page in this case, specified by content-type), and the date when the server responded, etc.

As you can see, the structure of the request/response messages is unified! It's the same every single time, the payload is what changes, but the structure is the same, and there are plenty of fields to specify the smallest detail you can think of. This standardization of how messages look like, is what allows your app server, like Express, Flask or whatever technology you're using, to properly parse the request. Remember it's a text being received by the server, and when you access all the details on the request object, your server application manages to give you this information because it was able to successfully parse the request message in the first place thanks to this standardization. How does your server know if the request is a GET or a POST request? It's exactly by extracting this information from the first line of this universally formatted request message. So if you decide to be creative and send the verb GET on the 5th line instead of the first, then hopefully by now you know that things will not work as intended because programs expect this information to be on line 1.

Summary

This is the essence of HTTP! This is the key fundamental understanding that hopefully people will be able to get after this article. HTTP is simply an agreement of how the message transmitted via other protocols like TCP, IP, etc., should be structured and how the 1s and 0s should be organized in order to have a uniform way of interpreting messages sent from clients and servers. Whether you send the request from Chrome, Firefox, Safari, curl, or any other tool, and whether your server is responding from the US, Australia, Africa or Honolulu, the request/response message will always have the same structure so that all computers can understand it and interpret it in the same exact way. That's the power of protocols, and that's the key responsibility of HTTP on the web!

This was a long article, It's very far from being a complete coverage of HTTP, but it's hopefully the building block you need to research and learn more about what HTTP is, how it works, and understand everything they're saying when explaining things like http verbs, status codes, headers, the body, the need for HTTPS (now that you see how messages are textual representations of transferred bytes), and even deep concepts like:

What does it mean when they say HTTP is a stateless protocol?

This one might require some thinking, so I'll leave you with this thought for now and see you again in the future with a part 2 maybe? Who knows :]