Thanks to its efficiency and support for numerous programming languages, gRPC is a popular choice for microservice integrations and client-server communications. gRPC is a high performance remote procedure call (RPC) framework using HTTP/2 for transport and Protocol Buffers to describe the interface.
To make it easier to use gRPC with your applications, Application Load Balancer (ALB) now supports HTTP/2 end-to-end, enabling you to publish gRPC services alongside non-gRPC services via a single load balancer. You can use Amazon Elastic Compute Cloud (EC2) instances or IP addresses (for example with AWS Fargate) as gRPC targets, with support for gRPC health checks for the target groups. In this way, you can use ALBs to terminate, route and load balance the gRPC traffic between your microservices or between gRPC-enabled clients and services.
ALB provides rich content based routing features to inspect the gRPC calls and route them to appropriate services. More specifically, the ALB provides health checks that can examine the gRPC status code, metrics for gRPC request count, access logs that differentiate gRPC requests, and gRPC specific response headers. Additionally, you can benefit from native features like stickiness, different load balancing algorithms, and TLS termination.
How to Use gRPC with Application Load Balancer
To test this new feature, I start by preparing the gRPC server application. I am using the Python programming language and the
route_guide demo included in the
grpc repo. I use this application because it quickly introduces some of the many ways a client and a server can interact via gRPC, such as:
- A simple unary remote procedure call (RPC) – the client sends a request to the server and waits for a response, similar to how a function call works.
- Server-side streaming RPC – the client sends a request to the server, gets back a stream of messages, and reads from the stream until there are no more messages.
- Client-side streaming RPC – the client writes a sequence of messages, sends them to the server, and then waits for the server to read them all and return its response.
- Bidirectional streaming RPC – client and server both send messages independently, preserving the order.
First, I prepare a Dockerfile to have the
route_guide application run in a container. Technically, this isn’t necessary because I could use a plain EC2 instance, but using containers with gRPC is so common than this makes the example more relevant.
FROM python:3.7 RUN pip install protobuf grpcio COPY ./grpc/examples/python/route_guide . CMD python route_guide_server.py EXPOSE 50051
I build the container image and upload it to Amazon Elastic Container Registry.
aws ecr get-login-password --region eu-north-1 | docker login --username AWS --password-stdin 123412341234.dkr.ecr.eu-north-1.amazonaws.com docker build -t route-guide . docker tag route-guide:latest 123412341234.dkr.ecr.eu-north-1.amazonaws.com/route-guide:latest docker push 123412341234.dkr.ecr.eu-north-1.amazonaws.com/route-guide:latest
In the Amazon ECS console, I create a new cluster using the Networking only template. I name the cluster
demo and create a new VPC for this cluster, leaving all other values to their default. The ECS console is using AWS CloudFormation to set up the resources for the cluster. After a few minutes, all resources have been successfully created and the cluster is ready.
To give access to the gRPC traffic, I create a security group to allow inbound TCP traffic on port
50051 from my client laptop. I name this security group
Now, I create the load balancer. In the EC2 console, I select Load Balancers on the left, and then Create Load Balancer. In the next step, I create an Application Load Balancer and name it
route-guide. I leave the default internet-facing Scheme, because I want to connect to the gRPC service over public internet. In the Listeners section, I select HTTPS as protocol and
50051 for the port. Just below, I select the newly created VPC and the two Availability Zones created by the ECS console.
Next, in the security settings, I choose a certificate I created before, managed by AWS Certificate Manager (ACM).
Then, I select the
default security group and the
gRPC security group I created above. Each VPC automatically comes with a default security group that gives network access to other resources using the same security group.
In the routing section, I create a new target group named
route-guide. Since I am planning to use AWS Fargate to run the gRPC server, I select the IP address target type. The ALB supports both secure and insecure connections for target groups using the gRPC protocol. Here, I use HTTP (because I am using an insecure connection between the load balancer and the containers running the gRPC server) with port 50051. Then, I select the VPC used by the ECS cluster. For Protocol Version, I use gRPC.
In the Advanced health check settings, I can specify which gRPC Success codes to use when checking for a correct response. I leave the default of
12 which means unimplemented.
12 is returned by a gRPC server if a method is not found. It works in this case because I’m using a path that is not implemented by the
route_guide application. More generally, checking for code
12 is a quick way to verify that your gRPC server is running correctly. To be more specific in your health checks, you can use a single code, a list, or a range, depending on what you expect from your implementation.
In the next step, I don’t register any target and complete the creation of the target group.
Back in the ECS console, I create a new task definition compatible with the Fargate launch type. I name it
route-guideand give it a minimum amount of resources: 0.5 GB of memory and 0.25 CPU units. I add a container definition using the image URI of the container image I uploaded to ECR, and the container port
50051/tcp, the network port exposed in the Dockerfile above.
route-guidetask definition selected, I select Create Service in the Actions menu. I use the Fargate launch type,
route-guide as service name, and 2 for the number of tasks. In the next step, I add the two subnets in the VPC, and select the
default security group to allow the load balancer to reach the tasks. In the Load balancing section, I select Application Load Balancer and the
route-guide load balancer.
For the Container to balance, I select
route-guide:50051:50051 and Add to load balancer.
Then, I choose the
route-guide target group.
At the next step, I select Do not adjust the service’s desired count to not use auto scaling for this demo.
I complete the creation of the ECS service. After a few minutes the two tasks are
RUNNING. Looking at the Targets tab of the
route-guide target group, I see that the two targets are healthy. The load balancer is now ready to accept traffic.
# create credentials credentials = grpc.ssl_channel_credentials() # create secure channel using credentials with grpc.secure_channel('
:50051', credentials) as channel: stub = route_guide_pb2_grpc.RouteGuideStub(channel) print("-------------- GetFeature --------------") guide_get_feature(stub) print("-------------- ListFeatures --------------") guide_list_features(stub) print("-------------- RecordRoute --------------") guide_record_route(stub) print("-------------- RouteChat --------------") guide_route_chat(stub)
Now, I start the client to test the gRPC channel with some workload. In the output, I see that the
route_guide application is using unary, server-side streaming, client-side streaming, and bi-directional streaming in the request and the response.
$ python3 route_guide_client.py -------------- GetFeature -------------- Feature called Berkshire Valley Management Area Trail, Jefferson, NJ, USA at latitude: 409146138 longitude: -746188906 Found no feature at -------------- ListFeatures -------------- Looking for features between 40, -75 and 42, -73 Feature called Patriots Path, Mendham, NJ 07945, USA at latitude: 407838351 longitude: -746143763 Feature called 101 New Jersey 10, Whippany, NJ 07981, USA at latitude: 408122808 longitude: -743999179 Feature called U.S. 6, Shohola, PA 18458, USA at latitude: 413628156 longitude: -749015468 Feature called 5 Conners Road, Kingston, NY 12401, USA at latitude: 419999544 longitude: -740371136 Feature called Mid Hudson Psychiatric Center, New Hampton, NY 10958, USA at latitude: 414008389 longitude: -743951297 Feature called 287 Flugertown Road, Livingston Manor, NY 12758, USA at latitude: 419611318 longitude: -746524769 Feature called 4001 Tremley Point Road, Linden, NJ 07036, USA at latitude: 406109563 longitude: -742186778 Feature called 352 South Mountain Road, Wallkill, NY 12589, USA at latitude: 416802456 longitude: -742370183 Feature called Bailey Turn Road, Harriman, NY 10926, USA at latitude: 412950425 longitude: -741077389 Feature called 193-199 Wawayanda Road, Hewitt, NJ 07421, USA at latitude: 412144655 longitude: -743949739 Feature called 406-496 Ward Avenue, Pine Bush, NY 12566, USA at latitude: 415736605 longitude: -742847522 Feature called 162 Merrill Road, Highland Mills, NY 10930, USA at latitude: 413843930 longitude: -740501726 Feature called Clinton Road, West Milford, NJ 07480, USA at latitude: 410873075 longitude: -744459023 Feature called 16 Old Brook Lane, Warwick, NY 10990, USA at latitude: 412346009 longitude: -744026814 Feature called 3 Drake Lane, Pennington, NJ 08534, USA at latitude: 402948455 longitude: -747903913 Feature called 6324 8th Avenue, Brooklyn, NY 11220, USA at latitude: 406337092 longitude: -740122226 Feature called 1 Merck Access Road, Whitehouse Station, NJ 08889, USA at latitude: 406421967 longitude: -747727624 Feature called 78-98 Schalck Road, Narrowsburg, NY 12764, USA at latitude: 416318082 longitude: -749677716 Feature called 282 Lakeview Drive Road, Highland Lake, NY 12743, USA at latitude: 415301720 longitude: -748416257 Feature called 330 Evelyn Avenue, Hamilton Township, NJ 08619, USA at latitude: 402647019 longitude: -747071791 Feature called New York State Reference Route 987E, Southfields, NY 10975, USA at latitude: 412567807 longitude: -741058078 Feature called 103-271 Tempaloni Road, Ellenville, NY 12428, USA at latitude: 416855156 longitude: -744420597 Feature called 1300 Airport Road, North Brunswick Township, NJ 08902, USA at latitude: 404663628 longitude: -744820157 Feature called at latitude: 407113723 longitude: -749746483 Feature called at latitude: 402133926 longitude: -743613249 Feature called at latitude: 400273442 longitude: -741220915 Feature called at latitude: 411236786 longitude: -744070769 Feature called 211-225 Plains Road, Augusta, NJ 07822, USA at latitude: 411633782 longitude: -746784970 Feature called at latitude: 415830701 longitude: -742952812 Feature called 165 Pedersen Ridge Road, Milford, PA 18337, USA at latitude: 413447164 longitude: -748712898 Feature called 100-122 Locktown Road, Frenchtown, NJ 08825, USA at latitude: 405047245 longitude: -749800722 Feature called at latitude: 418858923 longitude: -746156790 Feature called 650-652 Willi Hill Road, Swan Lake, NY 12783, USA at latitude: 417951888 longitude: -748484944 Feature called 26 East 3rd Street, New Providence, NJ 07974, USA at latitude: 407033786 longitude: -743977337 Feature called at latitude: 417548014 longitude: -740075041 Feature called at latitude: 410395868 longitude: -744972325 Feature called at latitude: 404615353 longitude: -745129803 Feature called 611 Lawrence Avenue, Westfield, NJ 07090, USA at latitude: 406589790 longitude: -743560121 Feature called 18 Lannis Avenue, New Windsor, NY 12553, USA at latitude: 414653148 longitude: -740477477 Feature called 82-104 Amherst Avenue, Colonia, NJ 07067, USA at latitude: 405957808 longitude: -743255336 Feature called 170 Seven Lakes Drive, Sloatsburg, NY 10974, USA at latitude: 411733589 longitude: -741648093 Feature called 1270 Lakes Road, Monroe, NY 10950, USA at latitude: 412676291 longitude: -742606606 Feature called 509-535 Alphano Road, Great Meadows, NJ 07838, USA at latitude: 409224445 longitude: -748286738 Feature called 652 Garden Street, Elizabeth, NJ 07202, USA at latitude: 406523420 longitude: -742135517 Feature called 349 Sea Spray Court, Neptune City, NJ 07753, USA at latitude: 401827388 longitude: -740294537 Feature called 13-17 Stanley Street, West Milford, NJ 07480, USA at latitude: 410564152 longitude: -743685054 Feature called 47 Industrial Avenue, Teterboro, NJ 07608, USA at latitude: 408472324 longitude: -740726046 Feature called 5 White Oak Lane, Stony Point, NY 10980, USA at latitude: 412452168 longitude: -740214052 Feature called Berkshire Valley Management Area Trail, Jefferson, NJ, USA at latitude: 409146138 longitude: -746188906 Feature called 1007 Jersey Avenue, New Brunswick, NJ 08901, USA at latitude: 404701380 longitude: -744781745 Feature called 6 East Emerald Isle Drive, Lake Hopatcong, NJ 07849, USA at latitude: 409642566 longitude: -746017679 Feature called 1358-1474 New Jersey 57, Port Murray, NJ 07865, USA at latitude: 408031728 longitude: -748645385 Feature called 367 Prospect Road, Chester, NY 10918, USA at latitude: 413700272 longitude: -742135189 Feature called 10 Simon Lake Drive, Atlantic Highlands, NJ 07716, USA at latitude: 404310607 longitude: -740282632 Feature called 11 Ward Street, Mount Arlington, NJ 07856, USA at latitude: 409319800 longitude: -746201391 Feature called 300-398 Jefferson Avenue, Elizabeth, NJ 07201, USA at latitude: 406685311 longitude: -742108603 Feature called 43 Dreher Road, Roscoe, NY 12776, USA at latitude: 419018117 longitude: -749142781 Feature called Swan Street, Pine Island, NY 10969, USA at latitude: 412856162 longitude: -745148837 Feature called 66 Pleasantview Avenue, Monticello, NY 12701, USA at latitude: 416560744 longitude: -746721964 Feature called at latitude: 405314270 longitude: -749836354 Feature called at latitude: 414219548 longitude: -743327440 Feature called 565 Winding Hills Road, Montgomery, NY 12549, USA at latitude: 415534177 longitude: -742900616 Feature called 231 Rocky Run Road, Glen Gardner, NJ 08826, USA at latitude: 406898530 longitude: -749127080 Feature called 100 Mount Pleasant Avenue, Newark, NJ 07104, USA at latitude: 407586880 longitude: -741670168 Feature called 517-521 Huntington Drive, Manchester Township, NJ 08759, USA at latitude: 400106455 longitude: -742870190 Feature called at latitude: 400066188 longitude: -746793294 Feature called 40 Mountain Road, Napanoch, NY 12458, USA at latitude: 418803880 longitude: -744102673 Feature called at latitude: 414204288 longitude: -747895140 Feature called at latitude: 414777405 longitude: -740615601 Feature called 48 North Road, Forestburgh, NY 12777, USA at latitude: 415464475 longitude: -747175374 Feature called at latitude: 404062378 longitude: -746376177 Feature called at latitude: 405688272 longitude: -749285130 Feature called at latitude: 400342070 longitude: -748788996 Feature called at latitude: 401809022 longitude: -744157964 Feature called 9 Thompson Avenue, Leonardo, NJ 07737, USA at latitude: 404226644 longitude: -740517141 Feature called at latitude: 410322033 longitude: -747871659 Feature called at latitude: 407100674 longitude: -747742727 Feature called 213 Bush Road, Stone Ridge, NY 12484, USA at latitude: 418811433 longitude: -741718005 Feature called at latitude: 415034302 longitude: -743850945 Feature called at latitude: 411349992 longitude: -743694161 Feature called 1-17 Bergen Court, New Brunswick, NJ 08901, USA at latitude: 404839914 longitude: -744759616 Feature called 35 Oakland Valley Road, Cuddebackville, NY 12729, USA at latitude: 414638017 longitude: -745957854 Feature called at latitude: 412127800 longitude: -740173578 Feature called at latitude: 401263460 longitude: -747964303 Feature called at latitude: 412843391 longitude: -749086026 Feature called at latitude: 418512773 longitude: -743067823 Feature called 42-102 Main Street, Belford, NJ 07718, USA at latitude: 404318328 longitude: -740835638 Feature called at latitude: 419020746 longitude: -741172328 Feature called at latitude: 404080723 longitude: -746119569 Feature called at latitude: 401012643 longitude: -744035134 Feature called at latitude: 404306372 longitude: -741079661 Feature called at latitude: 403966326 longitude: -748519297 Feature called at latitude: 405002031 longitude: -748407866 Feature called at latitude: 409532885 longitude: -742200683 Feature called at latitude: 416851321 longitude: -742674555 Feature called 3387 Richmond Terrace, Staten Island, NY 10303, USA at latitude: 406411633 longitude: -741722051 Feature called 261 Van Sickle Road, Goshen, NY 10924, USA at latitude: 413069058 longitude: -744597778 Feature called at latitude: 418465462 longitude: -746859398 Feature called at latitude: 411733222 longitude: -744228360 Feature called 3 Hasta Way, Newton, NJ 07860, USA at latitude: 410248224 longitude: -747127767 -------------- RecordRoute -------------- Visiting point latitude: 400066188 longitude: -746793294 Visiting point latitude: 412452168 longitude: -740214052 Visiting point latitude: 401827388 longitude: -740294537 Visiting point latitude: 414777405 longitude: -740615601 Visiting point latitude: 409642566 longitude: -746017679 Visiting point latitude: 406685311 longitude: -742108603 Visiting point latitude: 406523420 longitude: -742135517 Visiting point latitude: 405047245 longitude: -749800722 Visiting point latitude: 418858923 longitude: -746156790 Visiting point latitude: 409532885 longitude: -742200683 Finished trip with 10 points Passed 10 features Travelled 863981 meters It took 0 seconds -------------- RouteChat -------------- Sending First message at Sending Second message at longitude: 1 Sending Third message at latitude: 1 Sending Fourth message at Sending Fifth message at latitude: 1 Received message First message at Received message Third message at latitude: 1
Well, the output is pretty long, but as I said at the beginning of this post, the
route-guide demo is showing many of the different ways a client and a server can interact using gRPC, beyond basic RPC invocations.
End-to-end HTTP/2 and gRPC Support is available today for new and existing Application Load Balancers in all regions. You can use this feature via the console, AWS Command Line Interface (CLI), AWS SDKs. We are working to add AWS CloudFormation support soon.
There is no additional cost for using the gRPC protocol with an ALB. For more information, you can see the Elastic Load Balancing pricing page.
With these new features, it’s much easier to use gRPC to integrate your applications, or to improve client/server communication. To learn more, please see the documentation.