# Faster integration tests with ASP.Net Core Test Server?

# Introduction

I have been working recently on integration tests, an important topic for building sustainable software. I then discovered [ASP.Net Core's TestServer](https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.testhost.testserver?view=aspnetcore-7.0) to perform fast integration testing. But is it really faster than a classic ASP.Net Core server? Let's see that.

# The test project

The project we are going to work on is a simple online sales website. It contains some microservices, all written using a [functional architecture](https://grenat.hashnode.dev/functional-ddd-with-c-part-2-functional-architecture). We will focus on the Shopping Cart microservice. It contains an endpoint to add a product to a cart: we will use it to perform some benchmarking.

```csharp
[HttpPost]
public async Task<ActionResult> AddProductToCart([FromBody] AddProductToCartDto addProductToCartDto)
{
    /*...*/
}
```

# ASP.Net Core TestServer

ASP.Net Core TestServer is a web server you can find in [ASPNetCore.TestHost](https://www.nuget.org/packages/Microsoft.AspNetCore.TestHost) package. It makes the handling of HTTP requests fast because they don't go through a normal network stack.

# BenchmarkDotNet

We will use [BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet) to compare the performance between the two approaches. It is a small and extremely well-made library to carry out performance measurements. The purpose of this article is not to show how it works, for that you need to refer to its documentation.

# The benchmark classes

## ServerBase.cs

To make a fair comparison, the context must be the same between the two servers. So we will pool as much code as possible between the two approaches, i.e.:

* Creation of an in-memory SQLite database.
    
* Initialization of the test data.
    
* Initialization of DI services.
    
* Initialization of the web server.
    

These initializations are done in the constructor of an abstract `ServerBase.cs` class:

```csharp
public abstract class ServerBase
{
	protected WebApplicationBuilder WebAppBuilder { get; set; }
	protected WebApplication WebApp { get; set; }
	protected CallistoDbContext DbContext { get; set; }
	protected SqliteConnection Connection { get; set; }
	protected HttpClient? Sut { get; set; }

	private readonly string _baseUrl;

	protected ServerBase(string baseUrl)
	{
		WebAppBuilder = WebApplication.CreateBuilder();
		WebAppBuilder.Services.AddControllers().PartManager.ApplicationParts.Add(new AssemblyPart(typeof(CartController).Assembly)); // Necessary to add controllers from other projects
	
		Connection = new SqliteConnection("DataSource=file::memory:"); // We will use an in-memory Database
		Connection.Open();
	
		// Database Initialization
		var options = new DbContextOptionsBuilder<CallistoDbContext>().UseSqlite(Connection);
		DbContext = new CallistoDbContext(options.Options);
		DbContext.Database.Migrate();
		DbContext.Product.Add(new ProductModel("2EJX-6HCC", "PS5", "Playstation 5", 500, DateTime.UtcNow));
		DbContext.SaveChanges();
	
		// Services configuration
		WebAppBuilder.Services.AddSingleton<CallistoDbContext>(_ => DbContext);
		WebAppBuilder.Services.AddTransient<ICartRepository, CartRepository>();
		WebAppBuilder.Services.AddTransient<IProductRepository, ProductRepository>();
	
		WebAppBuilder.WebHost.UseUrls(baseUrl);
		_baseUrl = baseUrl;
	}
}
```

Finally, we will add a test method to call the endpoint for adding a product to a cart. The first call adds a product to a new cart (because no `CartId` is given), then we add 10 times the same product in this cart:

```csharp
public async Task RunTest()
{
	var addProductToCart = new
	{
		CartId = "",
		ProductId = "2EJX-6HCC",
		Quantity = 1
	};

	var response = await Sut!.PostUtf8Async($"{_baseUrl}/carts", addProductToCart);
	var cartId = (await response!.Content.ReadFromJsonAsync<List<Read.Entities.Cart>>())!.ElementAt(0).CartId;


	// Adding 10 times the same product in the cart
	for (int i = 0; i < 10; i++)
	{
		addProductToCart = new
		{
			CartId = cartId,
			ProductId = "2EJX-6HCC",
			Quantity = 1
		};
		await Sut.PostUtf8Async($"{_baseUrl}/carts", addProductToCart);
	}
}
```

## ClassicServer.cs

This class contains initializations specific to the test with the classic server. It inherits from `ServerBase.cs`:

```csharp
using Microsoft.AspNetCore.Builder;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;

namespace Callisto.Sales.Benchmark
{
	internal class ClassicServer : ServerBase
	{
		public ClassicServer(string baseUrl) : base(baseUrl)
		{
			WebAppBuilder.Services.AddHttpClient();
			var serviceProvider = WebAppBuilder.Services.BuildServiceProvider();
			Sut = serviceProvider.GetService<HttpClient>()!;

			WebApp = WebAppBuilder.Build();
			WebApp.MapControllers();
			WebApp.Start();
		}
	}
}
```

## TestServer.cs

This class contains initializations specific to testing with the ASP.Net Core TestServer; it also inherits from the ServerBase.cs class:

```csharp
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.TestHost;
using Microsoft.Extensions.Hosting;

namespace Callisto.Sales.Benchmark 
{
	internal class TestServer : ServerBase
	{
		public TestServer(string baseUrl) : base(baseUrl)
		{
			WebAppBuilder.WebHost.UseTestServer(); // Using TestServer

			WebApp = WebAppBuilder.Build();
			WebApp.MapControllers();
			WebApp.Start();

			Sut = WebApp.GetTestClient(); // Getting an HttpClient directly from the app.
		}
	}
}
```

# The benchmark class

The benchmark class is as follows:

```csharp
using BenchmarkDotNet.Attributes;

namespace Callisto.Sales.Benchmark
{
	[SimpleJob(launchCount: 5, warmupCount: 2, iterationCount: 10)]
	public class ClassicServerVSTestServerBenchmark
	{
		private readonly ClassicServer _classicServer;
		private readonly TestServer _testServer;

		public ClassicServerVSTestServerBenchmark()
		{
			_classicServer = new ClassicServer("http://localhost:5000");
			_testServer = new TestServer("http://localhost:5001");
		}

		[Benchmark]
		public async Task Classic_Server_Benchmark()
		{
			await _classicServer.RunTest();
		}

		[Benchmark]
		public async Task Test_Server_Benchmark()
		{
			await _testServer.RunTest();
		}
	}
}
```

And finally, to run the benchmark, we create a simple console project:

```csharp
using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Running;

namespace Callisto.Sales.Benchmark
{
    internal class Program
    {
        static void Main(string[] args)
        {
            BenchmarkRunner.Run<ClassicServerVSTestServerBenchmark>();
        }
    }
}
```

# First results

Now, let's run the benchmark:

```powershell
dotnet run -c Release
```

After a few minutes, I get these results:

```powershell
|                      Method |         Mean |        Error |       StdDev |
|---------------------------- |-------------:|-------------:|-------------:|
|    Classic_Server_Benchmark | 455,511.2 us | 24,192.14 us | 48,314.32 us |
|       Test_Server_Benchmark | 454,201.1 us |  7,812.13 us | 15,780.88 us |
```

Intermediate conclusion: the gain with the test server is not obvious (~1.5 ms/test). I was a bit disappointed... So I wondered what the result would be without database interaction, only focusing on the network part. Let's see that.

# A benchmark without database interaction

In the Shopping Cart microservice, I added a stub endpoint that does strictly nothing.

Then, in `ServerBase.cs`, I added a test for this endpoint :

```csharp
public async Task RunTestWithoutDatabaseInteraction()
{
	await Sut.GetAsync($"{_baseUrl}/carts/stub");
}
```

I then added the following two new benchmarks in `ClassicServerVSTestServerBenchmark.cs` :

```csharp
[Benchmark]
public async Task Classic_Server_Without_Database_Interaction_Benchmark()
{
	await _classicServer.RunTestWithoutDatabaseInteraction();
}

[Benchmark]
public async Task Test_Server_Without_Database_Interaction_Benchmark()
{
	await _testServer.RunTestWithoutDatabaseInteraction();
}
```

Here are the results :

```powershell
|                                                Method |         Mean |        Error |       StdDev |
|------------------------------------------------------ |-------------:|-------------:|-------------:|
| Classic_Server_Without_Database_Interaction_Benchmark |   2,137.9 us |     32.98 us |     61.94 us |
|    Test_Server_Without_Database_Interaction_Benchmark |     177.6 us |      6.99 us |     13.48 us |
```

This time, the gain brought by the TestServer is clear: the calls to an endpoint are 12x faster! We gain ~2 ms per endpoint call, which is consistent with the results from the previous benchmark.

# Conclusion

ASP.Net Core's TestServer only saves time on network calls by a few milliseconds. Perhaps the gains could be significant on a broad base of integration tests.

In any case, your database will remain the bottleneck for integration testing.

# About the cover image

The Source of the Universe. In the desert of Atacama in Chile, the Milky Way seems to spring from a small and perfectly calm water hole...

[View it in large size on my website](https://www.bastienfoucher.com/en/galleries/milky-way/voieLacteeSourceUnivers).