Transient fault handling in gRPC with retries in C#

gRPC calls can be interrupted by transient faults. Transient faults include:

  • Momentary loss of network connectivity.
  • Temporary unavailability of a service.
  • Timeouts due to server load.

When a gRPC call is interrupted, the client throws an RpcException with details about the error. The client app must catch the exception and choose how to handle the error.

var client = new Greeter.GreeterClient(channel);
try
{
    var response = await client.SayHelloAsync(
        new HelloRequest { Name = ".NET" });

    Console.WriteLine("From server: " + response.Message);
}
catch (RpcException ex)
{
    // Write logic to inspect the error and retry
    // if the error is from a transient fault.
}

Duplicating retry logic throughout an app is verbose and error-prone. Fortunately, the .NET gRPC client now has built-in support for automatic retries. Retries are centrally configured on a channel, and there are many options for customizing retry behavior using a RetryPolicy.

var defaultMethodConfig = new MethodConfig
{
    Names = { MethodName.Default },
    RetryPolicy = new RetryPolicy
    {
        MaxAttempts = 5,
        InitialBackoff = TimeSpan.FromSeconds(1),
        MaxBackoff = TimeSpan.FromSeconds(5),
        BackoffMultiplier = 1.5,
        RetryableStatusCodes = { StatusCode.Unavailable }
    }
};

// Clients created with this channel will automatically retry failed calls.
var channel = GrpcChannel.ForAddress("https://localhost:5001", new GrpcChannelOptions
{
    ServiceConfig = new ServiceConfig { MethodConfigs = { defaultMethodConfig } }
});

References
https://devblogs.microsoft.com/dotnet/grpc-in-dotnet-6/
https://docs.microsoft.com/en-us/aspnet/core/grpc/retries?view=aspnetcore-6.0

gRPC client-side load balancing in C#

The following code example configures a channel to use DNS service discovery with round-robin load balancing:

var channel = GrpcChannel.ForAddress(
    "dns:///my-example-host",
    new GrpcChannelOptions
    {
        Credentials = ChannelCredentials.Insecure,
        ServiceConfig = new ServiceConfig { LoadBalancingConfigs = { new RoundRobinConfig() } }
    });
var client = new Greet.GreeterClient(channel);

var response = await client.SayHelloAsync(new HelloRequest { Name = "world" });

References
https://devblogs.microsoft.com/dotnet/grpc-in-dotnet-6/
https://docs.microsoft.com/en-us/aspnet/core/grpc/loadbalancing?view=aspnetcore-6.0

Bidirectional Streaming with gRPC in C#

.proto

syntax = "proto3";

import "google/protobuf/timestamp.proto";

option csharp_namespace = "GrpcBidiStearming";

package greet;

service Greeter {
  rpc SayHello (stream HelloRequest) returns (stream HelloReply);
}

message HelloRequest {
  string name = 1;
  google.protobuf.Timestamp timestamp = 2;
}

message HelloReply {
  string message = 1;
  google.protobuf.Timestamp timestamp = 2;
}

Server

public override async Task SayHello(IAsyncStreamReader<HelloRequest> requestStream,
    IServerStreamWriter<HelloReply> responseStream, ServerCallContext context)
{
    while (await requestStream.MoveNext() && !context.CancellationToken.IsCancellationRequested)
    {
        Console.WriteLine("Request : {0}, Timestamp : {1}", requestStream.Current.Name,
            requestStream.Current.Timestamp);

        await responseStream.WriteAsync(new HelloReply
        {
            Message = $"Hello {requestStream.Current.Name}",
            Timestamp = Timestamp.FromDateTime(DateTime.UtcNow)
        });
        
        Console.WriteLine("Response : {0}, Timestamp : {1}", requestStream.Current.Name,
            requestStream.Current.Timestamp);
    }
}

Client

static async Task Main(string[] args)
{
    using var channel = GrpcChannel.ForAddress("http://localhost:5000");
    var client = new Greeter.GreeterClient(channel);

    CancellationToken cancellationToken = new CancellationToken(false);
    var response = client.SayHello(new CallOptions(null, null, cancellationToken));


    for (int i = 0; i < 100; i++)
    {
        var timestamp = Timestamp.FromDateTime(DateTime.UtcNow);
        await response.RequestStream.WriteAsync(new HelloRequest
        {
            Name = "Mahmood",
            Timestamp = timestamp
        });

        Console.WriteLine(String.Format("Request : {0}", timestamp.ToString()));

        await response.ResponseStream.MoveNext();

        Console.WriteLine(String.Format("Response : {0}, Timestamp : {1}",
            response.ResponseStream.Current.Message,
            response.ResponseStream.Current.Timestamp));

        await Task.Delay(1000);
    }
}

References
https://www.youtube.com/watch?v=wY4nMSUF9e0&list=PLUOequmGnXxPOlhyA57ijmEyOeVmYQt32&index=4

Client Streaming with gRPC in C#

,proto

syntax = "proto3";

option csharp_namespace = "GrpcClientStreaming";

import "google/protobuf/timestamp.proto";

package greet;

service Greeter {
  rpc SayHello (stream HelloRequest) returns (HelloReply);
}

message HelloRequest {
  string name = 1;
  google.protobuf.Timestamp timestamp = 2;
}

message HelloReply {
  repeated string messages = 1;
}

Server

public override async Task<HelloReply> SayHello(IAsyncStreamReader<HelloRequest> requestStream,
    ServerCallContext context)
{
    HelloReply response = new HelloReply();

    while (await requestStream.MoveNext() && !context.CancellationToken.IsCancellationRequested)
    {
        response.Messages.Add(requestStream.Current.Timestamp.ToString());
        Console.WriteLine(requestStream.Current.Timestamp.ToString());
    }

    return response;
}

Client

static async Task Main(string[] args)
{
    using var channel = GrpcChannel.ForAddress("http://localhost:5000");
    var client = new Greeter.GreeterClient(channel);

    CancellationToken cancellationToken = new CancellationToken(false);
    var response = client.SayHello(new CallOptions(null, null, cancellationToken));

    for (int i = 0; i < 5; i++)
    {
        var timestamp = Timestamp.FromDateTime(DateTime.UtcNow);
        await response.RequestStream.WriteAsync(new HelloRequest
        {
            Name = "Mahmood",
            Timestamp = timestamp
        });

        Console.WriteLine(String.Format("Request : {0}", timestamp.ToString()));

        await Task.Delay(1000);
    }

    await response.RequestStream.CompleteAsync();

    Console.WriteLine("--------------------------------------");

    var result = response.ResponseAsync.Result;

    foreach (string message in result.Messages)
    {
        Console.WriteLine("Response : {0}", message);
    }
}

References
https://www.youtube.com/watch?v=DNxdvRQ4qRQ&list=PLUOequmGnXxPOlhyA57ijmEyOeVmYQt32&index=3

Server Streaming with gRPC in C#

.proto

syntax = "proto3";

import "google/protobuf/timestamp.proto";

option csharp_namespace = "GrpcServerStreaming";

package greet;

service Greeter {
  rpc SayHello (HelloRequest) returns (stream HelloReply);
}

message HelloRequest {
  string name = 1;
}

message HelloReply {
  string message = 1;
  google.protobuf.Timestamp timestamp = 2;
}

Server

public override async Task SayHello(HelloRequest request, IServerStreamWriter<HelloReply> responseStream,
    ServerCallContext context)
{
    while (!context.CancellationToken.IsCancellationRequested)
    {
        await responseStream.WriteAsync(new HelloReply
        {
            Message = String.Format("Hello {0}", request.Name),
            Timestamp = Timestamp.FromDateTime(DateTime.UtcNow)
        });
        
        Console.WriteLine(Timestamp.FromDateTime(DateTime.UtcNow));
        await Task.Delay(1000);
    }
}

Client

static async Task Main(string[] args)
{
    // The port number(5001) must match the port of the gRPC server.
    using var channel = GrpcChannel.ForAddress("http://localhost:5000");
    var client = new Greeter.GreeterClient(channel);
    var response = client.SayHello(new HelloRequest {Name = "Mahmood"});
    
    while (await response.ResponseStream.MoveNext(new CancellationToken(false)))
    {
        Console.WriteLine(response.ResponseStream.Current.Message);
        Console.WriteLine(response.ResponseStream.Current.Timestamp);
        Console.WriteLine("----------------------------------------------------------------");
    }

    Console.ReadKey();
}

References
https://www.youtube.com/watch?v=F2T6xNRoa1E&list=PLUOequmGnXxPOlhyA57ijmEyOeVmYQt32&index=2

Host ASP.NET Core on Linux with Apache

Configure a proxy server

Invoke the UseForwardedHeaders method at the top of Startup.Configure before calling other middleware. Configure the middleware to forward the X-Forwarded-For and X-Forwarded-Proto headers:

// using Microsoft.AspNetCore.HttpOverrides;

app.UseForwardedHeaders(new ForwardedHeadersOptions
{
    ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto
});

app.UseAuthentication();
// using System.Net;

services.Configure<ForwardedHeadersOptions>(options =>
{
    options.KnownProxies.Add(IPAddress.Parse("10.0.0.100"));
});

References
https://docs.microsoft.com/en-us/aspnet/core/host-and-deploy/linux-apache?view=aspnetcore-5.0

Self-hosted gRPC applications

Run your app as a Linux service with systemd

To configure your ASP.NET Core application to run as a Linux service (or daemon in Linux parlance), install the Microsoft.Extensions.Hosting.Systemd package from NuGet. Then add a call to UseSystemd to the CreateHostBuilder method in Program.cs

public static IHostBuilder CreateHostBuilder(string[] args) =>
    Host.CreateDefaultBuilder(args)
        .UseSystemd() // Enable running as a Systemd service
        .ConfigureWebHostDefaults(webBuilder =>
        {
            webBuilder.UseStartup<Startup>();
        });
dotnet publish -c Release -r linux-x64 -o ./publish

/etc/systemd/system/myapp.service

[Unit]
Description=My gRPC Application

[Service]
Type=notify
ExecStart=/usr/sbin/myapp

[Install]
WantedBy=multi-user.target
sudo systemctl daemon-reload
sudo systemctl status myapp
sudo systemctl start myapp.service
sudo systemctl enable myapp

References
https://docs.microsoft.com/en-us/dotnet/architecture/grpc-for-wcf-developers/self-hosted

Bind gRPC Service to specific port in ASP.NET Core

This works (server side) with Kestrel:

public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.ConfigureWebHostDefaults(webBuilder =>
{
    webBuilder.ConfigureKestrel(options =>
    {
       options.Listen(IPAddress.Loopback, 5000);
       options.Listen(IPAddress.Loopback, 5005, configure => configure.UseHttps());
    });
    webBuilder.UseStartup<Startup>();
});

client side:

var httpHandler = new HttpClientHandler
 {
     ServerCertificateCustomValidationCallback =
     HttpClientHandler.DangerousAcceptAnyServerCertificateValidator
 };  

AppContext.SetSwitch("System.Net.Http.SocketsHttpHandler.Http2UnencryptedSupport", true);
                
using var channel = GrpcChannel.ForAddress("https://localhost:5005", new GrpcChannelOptions { HttpHandler = httpHandler } );
            
var client = new Greeter.GreeterClient(channel);

References
https://stackoverflow.com/questions/63827667/bind-grpc-services-to-specific-port-in-aspnetcore
https://docs.microsoft.com/en-us/aspnet/core/fundamentals/servers/kestrel/options?view=aspnetcore-5.0
https://andrewlock.net/5-ways-to-set-the-urls-for-an-aspnetcore-app/