What is HTTP Part IV - Clients, Servers, and Proxies. Oh My!

So far in this series, we have covered the basics of HTTP messaging, the underlying protocols, and last week, we focused on digging a little deeper into the terminology. This week we will focus on the HTTP speakers: clients, servers, and proxies. Borrowing from the underlying protocols article, we’ll just place a happy little accidental blur over the devices in the hop path that aren’t actively engaged at the HTTP protocol layer. Note that the devices speaking HTTP also perform these lower layers, but that is not the focus here.

Clients and Servers

Every journey has a beginning and an end, a source and a destination. So it is with HTTP. I’m lumping these two together because they are the consumers and suppliers in this story. An application is an HTTP client if it speaks HTTP. Now, the client might do a lot of other things unrelated to HTTP itself, but the core client functionality is this: send a request message and receive a response message. With that in mind, you can use all sorts of tools to be HTTP clients. Like telnet:

MACDADDY:~ jrahm$ telnet
telnet> open 10.10.10.10 5000
Trying 10.10.10.10...
Connected to vmlab.
Escape character is '^]'.
GET /rants.html HTTP/1.1

HTTP/1.0 200 OK
Content-Type: text/html; charset=utf-8
Content-Length: 1126
Server: Werkzeug/0.11.15 Python/2.7.12
Date: Thu, 28 Sep 2017 18:06:33 GMT
...

Or build your own socket client with python!

>>> import socket
>>> s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
>>> s.connect(('10.10.10.10', 5000))
>>> s.send('GET /rants.html HTTP/1.1\n\n')
26
>>> while True:
...     response = s.recv(1024)
...     if response == '': break
...     print response,
...
HTTP/1.0 200 OK
Content-Type: text/html; charset=utf-8
Content-Length: 1126
Server: Werkzeug/0.11.15 Python/2.7.12
Date: Thu, 28 Sep 2017 18:10:41 GMT
...
>>> s.close()

In many client-server systems, most if not all intelligence resides on the server side of the relationship. This is not the case with web traffic. There is shared intelligence between web clients and servers, with more intelligence seemingly accelerating toward the big daddy of all HTTP clients—the browser. I don't want to dive too far down the rabbit hole, but the browser evolution over the years is nothing short of astounding. Do you recall the crazy number of plugins or off-browser technical gymnastics you had to do to play a video? Now we can’t get them NOT to play! Browsers manage name resolution, compression, caching, cookies, request routing, rendering, security, etc. The browser is an incredibly powerful and rich application platform that far exceeds the scope of this article or series, but an HTTP client it is nonetheless.

Servers serve content. It’s what they do (thank you Capt. Obvious!) It can be static content like scripts, stylesheets, images, etc. But usually most modern sites are incredibly dynamic.

Quick aside - In some architectures, you have the web server, the application server, and then the database server. Sometimes, these are collapses in front/back pairs or combined altogether. The added layers and functionality, however, don't change the underlying HTTP messaging required for a server of that web content to return a response back to the client.

Servers can be just as surprisingly simple or mind-numbingly complex as the client. For example, an iRule running on your BIG-IP can be a “server” and serve up HTTP responses with a single command in the HTTP_RESPONSE event, or even just respond to a certain url:

# respond to every URI with simple "ok"
when HTTP_RESPONSE {
HTTP::respond 200 ok

# respond to /service-check.html
when HTTP_RESPONSE {
  if { [HTTP::path] eq "/service-check.html" } {
    HTTP::respond 200 ok
  }
}

Also, with micro-frameworks like python’s Flask package, you can serve up HTTP response messages with a few lines of code:

from flask import Flask
app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'Hello, World!'

In addition to content management, like clients, there are additional services that can be offered by servers. Some of the same services as the client with compression and caching, and some additional services like authentication and authorization, session handling, tenant isolation, etc.

Lightweight command line HTTP clients (ab, netcat, curl) and server applications (like twisted) are fantastic for doing functional testing of your BIG-IP deployments, and many of them can be scaled with help to do some pretty helpful performance testing as well. And whereas BIG-IP (via iRules, policies, or HTP profiles) can be clients and/or servers in the messaging game, it is not usually considered as such. BIG-IP is considered a proxy.

Proxies

If clients and servers are the consumers and suppliers in our story, then proxies are the slick used car salesmen. The high volume operators. The border agents inspecting the precious cargo. The custom fabricators. They are the middle men! And just like in society, there are positive use cases for middle men, and some not so positive ones (man-in-the-middle attacks.)

Earlier this year as part of our DevCentral Basics series wrote an excellent article covering the basic concepts of proxies, so I won’t belabor that here but I do want to address some proxy functionality:

Caching - en vogue on client-side networks when bandwidth was a premium back in the day, a caching proxy would retrieve and store locally so multiple requests for the same content didn’t need to rob bandwidth for repeat functions. In this use case it would be a forward proxy, but you can cache at the reverse proxy as well. This is available in the Application Acceleration Manager module. We will discuss caching more in-depth later in this series.
Non-HTTP services gateway - as an intermediary, a proxy manages client and server traffic, and can implement other services that may or may not be HTTP related at all. ICAP is a good example of this.
Transformations - Managing client and server available functionalities separately. For example, maintaining HTTP/1.0 client side and HTTP/1.1 server side.
Client Anonymity - On the server side, you might want to track the history of where your clients are coming from for various purposes, but if a large percentage of them are behind a shared proxy, this could impact not only your statistics, but also your load distribution in your infrastructure, depending on how your architect your solution. Some companies like Akamai will insert in TCP options the (as far as they can tell) source IP so that information is available. The more common way proxies deal with this (where it is activated) is by inserting the HTTP X-Forwarded-For header.
Request/Response Inspection/Filtering - This happens with forward or reverse proxy implementations. You want a border control agent making sure the tourists are who they say they are and aren’t trying to cross the border with unauthorized goods.

The BIG-IP platform supports forward proxy through the Secure Web Gateway Services and the reverse proxy functionality through the core Local Traffic Manager module and with additional access and security services with Access Policy Manager and Application Security Manager respectively.

Moving forward in this series, we will focus primarily on the reverse proxy functionality within BIG-IP. Join us next week when we start to talk about the BIG-IP HTTP profile.

Published Sep 29, 2017

Version 1.0