I’ve (finally) decided to start writing regularly about the interesting things I’m learning every week. Previously, I would just include them in the wiki, but then I would have to think of a category for each one, and some of them belong to multiple categories, and some of them belong to no category, and it’s such a massive headache. Argh. But I digress.

Here’s some interesting things I’ve learned this week:

When applied to class instance methods, aiocache will use self as part of the cache key

Which was not what I had expected. I hadn’t paid attention to how this class was setup in my dependency injection, and apparently it was one instance per request (this is all in a FastAPI context). Meaning that each instance would have its very own cache key, which in turn meant that my cache was pretty much useless across requests.

I ended up setting its noself argument to True.

But this wasn’t enough because I actually wanted to ..

Configure aiocache to cache stuff per IP

To simplify things, I’ve extracted the functionality in a main module method, using client_host as a parameter, which in turn is used by my mighty key_builder. Something like this:

@cached(  
    ttl=4200,
    cache=Cache.MEMORY,  
    key_builder=lambda *args, **kw: f"cache_key:{args[1]}",  
)  
async def cached_wrapper_async(  
    client_host: str, other_args: Any
):
# ...

YARP versus Ocelot

I’ve been looking for some kind of an application gateway/reverse proxy to put in front of a bunch of services, hosted both in the cloud and on-prem.

My ideal solution was defined as being open-source, highly-performant, and should allow for a significant amount of customization.

After evaluating cloud options like Azure API Management (too expensive & too cloudy), reverse proxies (Caddy! Traefik! heck, even NGINX!), and custom .NET services (I did mention high-performance), in the end I was left with two options: YARP and Ocelot.

Both seemed to cover my needs, with Ocelot going above and beyond to cover “API Gateway” features, while YARP is more focused on high performance and “Reverse Proxy” features.

In the end, I decided to go with YARP. It’s created, maintained, and used internally by Microsoft, it’s fast, extensible, and doesn’t try to do too much.

Another reason is that, after looking at Ocelot’s releases I’ve found them to be in the fail fast and break things category – i.e. they have 23.3.4, a minor release, delivering “a number of bug fixes for the predecessor’s 23.3.0 release, which is full of new features but was not tested well”, and also adding about 3 breaking changes. Three. In a minor version update 😕. No thanks.

I especially enjoyed the comment below on this 4-year old Reddit thread:

Using a reverse proxy as the public endpoint for your apps has a number of advantages:

  • The urlspace exposed publicly can be different from what is used by the backend servers. You can have a heterogenous mess of different servers and technologies, and map them into something more coherent by the proxy. http://example.com/foo and example.com/foo/bar can be handled by different clusters of servers.
  • You can spread load across multiple back ends (destinations), using a selection of different load balancing algorithms. Active and passive health checks monitor the state of the destinations and they will be taken out of the load balancing if they are having problems.
  • Routing can be based on almost any part of the url/headers for each request, including authentication, client IP (such as geo location), device type etc.
  • Work can be offloaded from the backend servers, including Authentication & Authorization, TLS, Caching, static file handling, telemetry & logging, rate limiting etc.

For YARP we are seeing the ability to easily customize these aspects as its compelling feature compared to NGINX, Envoy etc.

The biggest difference between YARP and Ocelot is that YARP is optimized around being agnostic to the content of requests & responses - it just passes them on as-is. Cracking the body open and understanding the content to be able to make changes - such as merging results from multiple back ends - requires buffering and logic which will affect throughput performance.

Further reading: