Envoy is an L7 proxy and communication bus designed for large modern service oriented architectures.The project was born out of the belief that:
The network should be transparent to applications. When network and application problems do occurit should be easy to determine the source of the problem.
In practice, achieving the previously stated goal is incredibly difficult. Envoy attempts to do soby providing the following high level features:
Out of process architecture: Envoy is a self contained process that is designed to runalongside every application server. All of the Envoys form a transparent communication mesh in whicheach application sends and receives messages to and from localhost and is unaware of the networktopology. The out of process architecture has two substantial benefits over the traditional libraryapproach to service to service communication:
Envoy works with any application language. A single Envoy deployment can form a mesh betweenJava, C++, Go, PHP, Python, etc. It is becoming increasingly common for service orientedarchitectures to use multiple application frameworks and languages. Envoy transparently bridgesthe gap.
As anyone that has worked with a large service oriented architecture knows, deploying libraryupgrades can be incredibly painful. Envoy can be deployed and upgraded quickly across anentire infrastructure transparently.
L3/L4 filter architecture: At its core, Envoy is an L3/L4 network proxy. A pluggablefilter chain mechanism allows filters to be written toperform different TCP/UDP proxy tasks and inserted into the main server. Filters have already beenwritten to support various tasks such as raw TCP proxy, UDPproxy, HTTP proxy, TLS clientcertificate authentication, Redis,MongoDB, Postgres, etc.
HTTP L7 filter architecture: HTTP is such a critical component of modern applicationarchitectures that Envoy supports an additional HTTP L7 filterlayer. HTTP filters can be plugged into the HTTP connection management subsystem that performdifferent tasks such as buffering, rate limiting, routing/forwarding, sniffingAmazon’s DynamoDB, etc.
First class HTTP/2 support: When operating in HTTP mode, Envoy supports both HTTP/1.1 and HTTP/2. Envoy can operate as a transparentHTTP/1.1 to HTTP/2 proxy in both directions. This means that any combination of HTTP/1.1 and HTTP/2clients and target servers can be bridged. The recommended service to service configuration usesHTTP/2 between all Envoys to create a mesh of persistent connections that requests and responses canbe multiplexed over.
HTTP/3 support (currently in alpha): As of 1.19.0, Envoy now supports HTTP/3 upstream and downstream,and translating between any combination of HTTP/1.1, HTTP/2 and HTTP/3 in either direction.
HTTP L7 routing: When operating in HTTP mode, Envoy supports arouting subsystem that is capable of routing and redirectingrequests based on path, authority, content type, runtime values, etc.This functionality is most useful when using Envoy as a front/edge proxy but is also leveraged whenbuilding a service to service mesh.
gRPC support: gRPC is an RPC framework from Google that uses HTTP/2 or aboveas the underlying multiplexed transport. Envoy supports all of theHTTP/2 features required to be used as the routing and load balancing substrate for gRPC requestsand responses. The two systems are very complementary.
Service discovery and dynamic configuration: Envoy optionally consumes a layered set ofdynamic configuration APIs for centralized management.The layers provide an Envoy with dynamic updates about: hosts within a backend cluster, thebackend clusters themselves, HTTP routing, listening sockets, and cryptographic material.For a simpler deployment, backend host discovery can bedone through DNS resolution(or evenskipped entirely),with the further layers replaced by static config files.
Health checking: The recommendedway of building an Envoy mesh is to treat service discovery as an eventually consistent process.Envoy includes a health checking subsystem which canoptionally perform active health checking of upstream service clusters. Envoy then uses the union ofservice discovery and health checking information to determine healthy load balancing targets. Envoyalso supports passive health checking via an outlier detection subsystem.
Advanced load balancing: Load balancing among differentcomponents in a distributed system is a complex problem. Because Envoy is a self contained proxyinstead of a library, it is able to implement advanced load balancing techniques in a single placeand have them be accessible to any application. Currently Envoy includes support for automaticretries, circuit breaking,global rate limiting via an external rate limiting service,request shadowing, andoutlier detection. Future support is planned for requestracing.
Front/edge proxy support: There is substantial benefit in using the same software at the edge(observability, management, identical service discovery and load balancing algorithms, etc.). Envoyhas a feature set that makes it well suited as an edge proxy for most modern web application usecases. This includes TLS termination, HTTP/1.1 HTTP/2 and HTTP/3 support, as well as HTTP L7 routing.
Best in class observability: As stated above, the primary goal of Envoy is to make the networktransparent. However, problems occur both at the network level and at the application level. Envoyincludes robust statistics support for all subsystems. statsd (and compatible providers) is the currently supported statisticssink, though plugging in a different one would not be difficult. Statistics are also viewable viathe administration port. Envoy also supports distributedtracing via thirdparty providers.
