Container host network access with .NET Aspire

1 month ago
5 min read

Working on a project for work, I set out with what I expected to be a simple enough task: using .NET Aspire, I wanted to run an emulator for Azure EventGrid. The emulator came in the form of an OCI container to run under Docker or Podman, and needs two things:

  • It must be possible to send HTTP requests to the emulator.
  • The emulator must be able to send HTTP requests to an API running on the host machine.

This proved much more difficult than expected, and after many hours of trial and error, I have managed to figure out a solution. Docs for this are all over the place, so hopefully this serves as an up to date set of steps for anybody else who hits this. I can't say it's the best solution but it does work.

I should note before anybody gets too far, this does require Windows 11 22H2 or later. Earlier versions of Windows do not support the required configuration and you'll probably have to get dirty with iptables or other equally nasty virtual networking options.

The problem was threefold:

  1. WSL machines run under their own network which cannot access the host.
  2. Containers also run under their own network. These can access their host (aka the WSL machine) but not the Windows host. Additionally, Aspire hard-codes containers to use network=bridge.
  3. Aspire binds all ASP.NET projects to localhost.

For a complete picture, here is the emulator orchestration code from my app host:

// https://github.com/workleap/wl-eventgrid-emulator
builder
    .AddContainer("EventGrid", "workleap/eventgridemulator")
    .WithEndpoint(6500, 6500)
    .WithBindMount("infra/EventGridEmulator.json", "/app/appsettings.json")
    .WithReference(api);

WSL & Container Networking

Issues 1 and 2 can be resolved in one go, using two separate settings.

The first, networkingMode=mirrored effectively tells WSL 2 machines to run using the host network, and not using their own virtual adapter. This trick alone allows anything running inside the WSL machine to access localhost on the Windows host machine.

Unfortunately this does not get us all of the way - as containers run under their own network, they can talk to the WSL machine (using the special host.containers.internal or host.docker.internal hostnames, and NOT localhost) but still cannot talk to host machine. This can be resolved by using the experiemental hostAddressLoopback option, designed specifically for this purpose. At least at the time of writing...

To configure WSL globally, add the below to $HOME/.wslconfig. Create the file if it does not already exist.

[wsl2]
# Run WSL machines on the host network
# This allows them to access anything running on the host as `localhost`.
networkingMode=mirrored

[experimental]
# Enable loopback to allow containers running under the WSL machine to access the host.
# Note containers must use `host.containers.internal` or `host.docker.internal` in place of `localhost`.
# This is resolved as the WSL machine's network, which loopback then forwards onto the Windows host.
hostAddressLoopback=true

Bind to an address other than localhost

Since even with both options above, containers still run under their own network, any services running that bind to localhost or 127.0.0.1 will not allow traffic from the container. Instead, we need to open these up to bind to a wider address. You can probably be specific with an exact IP (range) if needed, but in almost any reasonable case 0.0.0.0 will work nicely. Just don't port forward to your local dev machine.

Aspire makes this harder than it should be, as it automatically creates proxy endpoints for an ASP.NET project that bind to localhost, and you cannot control it via environment variables. Supposedly you should be able to, but I had no luck at least.

Given a simple project:

var api = builder.AddProject<Projects.Api>("API");

You can grab the EndpointAnnotation instances and modify these to bind to the right host:

// By default, the API binds to `localhost`,
// which is not accessible from the container.
// As the EventGrid emulator is in a container,
// we need to open up to the virtual network.
var apiEndpoints = api.Resource.Annotations.OfType<EndpointAnnotation>();
foreach (var endpoint in apiEndpoints)
{
    endpoint.TargetHost = "0.0.0.0";
}

Closing thoughts

Is this a nasty hack and completely overkill? Probably. But it works. Networking sucks at the best of times. When it's 3 levels of abstraction deep, within Linux in Linux in Windows orchestrated by some C#, anything to make it go away is a good thing.

Hopefully with time, this sort of thing will improve. Aspire is still in its infancy, but there does seem to be a big push from Microsoft to move towards containers for local development (I guess they've finally realised Windows sucks as a dev platform) so I can only imagine things will improve over the next few years.

I hope this helps somebody. Alternatively if this sucks and somebody has a better way, please let me know.