How to Test Kubernetes Applications in Go with EnvTest: A Practical Guide

Introduction

When building applications that interact with a Kubernetes cluster (be it operators, controllers, or custom tools) robust testing is a must. Traditional approaches might involve spinning up a full cluster with Minikube, using TestContainers, or even relying on a dedicated CI/CD testing cluster. However, these solutions can be resource-intensive or hard to integrate in every pipeline.

Enter EnvTest: a lightweight, fast alternative provided by the controller-runtime project. EnvTest spins up a minimal Kubernetes API server and an etcd instance, enabling you to perform integration tests that cover the core declarative interactions with the API server without the overhead of a full cluster.

In this post, I’ll walk through how to set up and use EnvTest in your Go projects, especially when testing operators or controllers. I'll even revisit my kubectl kill namespace plugin example to illustrate practical testing scenarios.

What is EnvTest?

EnvTest is a powerful package within the controller-runtime project designed for integration testing of Kubernetes applications.

Here are a few key points:

• Lightweight and Fast: It runs a local API server and etcd, giving you a near-cluster environment without the need for full Kubernetes infrastructure.
• Resource Efficient: Perfect for local development and CI/CD pipelines where spinning up an entire cluster is impractical.
• Focused on Declarative Operations: Ideal when your tests involve creating, updating, or deleting Kubernetes resources, rather than simulating full Pod scheduling or node operations.

Setting up EnvTest

Most of the guides rely on the EnvTest binaries to be already installed in the system. Some of them show you how to download them using a Makefile, while others don't even mention how to get them.

In this section, I will show you how to use the EnvTest package directly from your Go code, without relying on any external tools.

Creating the setupEnvTest function

Let's begin by creating a setupEnvTest function responsible for setting up the EnvTest environment. The following snippet shows some pseudocode describing the function:

func setupEnvTest() *envtest.Environment {
	// Download EnvTest binaries
	// Create a new instance of the EnvTest environment
}

The function is responsible for downloading if necessary the EnvTest binaries, and creating a new instance of the EnvTest environment for those binaries.

Let's focus on the first part, downloading the binaries.

func setupEnvTest() *envtest.Environment {
	envTestDir, err := store.DefaultStoreDir()
	if err != nil {
		panic(err)
	}
	envTest := &env.Env{
		FS:  afero.Afero{Fs: afero.NewOsFs()},
		Out: os.Stdout,
		Client: &remote.HTTPClient{
			IndexURL: remote.DefaultIndexURL,
		},
		Platform: versions.PlatformItem{
			Platform: versions.Platform{
				OS:   runtime.GOOS,
				Arch: runtime.GOARCH,
			},
		},
		Version: versions.AnyVersion,
		Store:   store.NewAt(envTestDir),
	}
	envTest.CheckCoherence()
	workflows.Use{}.Do(envTest)
}

Let's dig through the code step by step:

• First, we get the default EnvTest directory using the store.DefaultStoreDir() function. This way, we can reuse any binary that might have already been downloaded by any other tests or external tools.
• Then, we create a new instance of the env.Env struct. This struct provides all the necessary information such as OS, architecture, version, and so on to download the binaries.
• Next, we validate and initialize the environment by using the envTest.CheckCoherence() function.
• Finally, we use the workflows.Use{}.Do(envTest) function to download the necessary binaries.

The next part of the function logic deals with setting up the EnvTest environment:

func setupEnvTest() *envtest.Environment {
	// Create a new instance of the EnvTest environment
	versionDir := envTest.Platform.Platform.BaseName(*envTest.Version.AsConcrete())
	return &envtest.Environment{
		BinaryAssetsDirectory: filepath.Join(envTestDir, "k8s", versionDir),
	}
}

We use the envTest.Platform.Platform.BaseName(*envTest.Version.AsConcrete()) function to get the directory containing the binaries for the EnvTest version we downloaded. Then, we create a new instance of the envtest.Environment struct, passing the directory where the binaries are located.

Now that we have the setupEnvTest function ready, let's continue by seeing how to use it in our tests.

Using EnvTest for Integration Testing

In this section, I'll show you how to use the setupEnvTest function we created in the previous section to test the kubectl kill namespace plugin I analyzed in a previous post. You can find the complete code for the tests in the killns_test.go file of the project.

Since I want to test the Namespace deletion logic, I'll leverage subtests to spin up a common EnvTest environment and then run each scenario.

The following code snippet shows the setup and tear down sections, and one of the subtests of the TestKillNamespace function:

func TestKillNamespace(t *testing.T) {
	envTest := setupEnvTest()
	envTestConfig, err := envTest.Start()
	if err != nil {
		t.Errorf("Error starting test environment: %s", err)
		return
	}
	defer func() {
		if stopErr := envTest.Stop(); stopErr != nil {
			panic(stopErr)
		}
	}()
	// Test with no namespace
	// Test with existent namespace
	t.Run("With existent namespace with finalizer", func(t *testing.T) {
		// Given
		client, _ := kubernetes.NewForConfig(envTestConfig)
		client.CoreV1().Namespaces().Create(context.TODO(), &corev1.Namespace{
			ObjectMeta: metav1.ObjectMeta{
				Name: "finalizer",
			},
			Spec: corev1.NamespaceSpec{Finalizers: []corev1.FinalizerName{"kubernetes"}},
		}, metav1.CreateOptions{})
		// When
		KillNamespace(envTestConfig, "finalizer")
		// Then
		_, err := client.CoreV1().Namespaces().Get(context.TODO(), "finalizer", metav1.GetOptions{})
		if err.Error() != "namespaces \"finalizer\" not found" {
			t.Errorf("Namespace should have been deleted, but it wasn't")
		}
	})
}

Test Breakdown:

• Environment Start:
  I start the test by calling the setupEnvTest function to initialize the EnvTest environment.
  Then, I start the environment by calling the envTest.Start() function.
  If there is any error while starting the environment, I fail the test and return.
• Environment Stop:
  I use a defer statement to stop and tear down the environment once the test finishes.
  This way, I make sure that the environment is always stopped, even if the test fails.
• Client Initialization:
  I initialize a client-go Kubernetes Client instance by passing the rest.Config struct I get from the envTest.Start() function.
• Subtests for Clarity:
  I use Go's subtest functionality (t.Run),to group the different scenarios we want to test. This way, I can keep the test code organized and easy to read.
• Test Scenario:
  
  • I create a new Namespace with a finalizer using the client (assemble/given).
  • Then, I call the KillNamespace function we want to test (act/when).
  • Finally, I check that the Namespace was deleted by trying to get it using the client (assert/then).

Conclusion

In this post, I showed you how to leverage the controller-runtime EnvTest package to perform lightweight yet effective integration tests for Kubernetes applications written in Go. I walked through setting up the testing environment, downloading necessary binaries programmatically, and using EnvTest in practical scenarios like testing namespace deletion.

By integrating these practices into your development workflow, you can achieve faster feedback cycles, more reliable tests, and a more efficient CI/CD pipeline, all without the overhead of managing a full Kubernetes cluster.

For the complete source code and further examples, visit the GitHub.