Testing Roslyn Incremental Source Generators

Roslyn Source Generators produce code based on the current project's code and additional files. Because each keystroke in the editor can trigger source generators, performance matters. Incremental generation addresses this by re-running the generator only when significant changes occur, and each generator defines what counts as a significant change. This post describes how to write a test to verify that incremental generation works as expected.

Let's create a solution with a class library containing the source generator and a test project. The source generator will produce a file for each struct in the project. The test project will verify that the source generator is only invoked when a struct is added or removed.

dotnet new classlib --output SampleGenerator
dotnet add SampleGenerator package Microsoft.CodeAnalysis

dotnet new xunit --output SampleGenerator.Tests
dotnet add SampleGenerator.Tests reference SampleGenerator
dotnet add SampleGenerator.Tests package Basic.Reference.Assemblies.Net70

dotnet new sln --name SampleGenerator
dotnet sln add SampleGenerator
dotnet sln add SampleGenerator.Tests

using System.Text;
using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.CSharp;
using Microsoft.CodeAnalysis.CSharp.Syntax;
using Microsoft.CodeAnalysis.Text;

[Generator]
public sealed partial class SampleSourceGenerator : IIncrementalGenerator
{
    public void Initialize(IncrementalGeneratorInitializationContext context)
    {
        var structPovider = context.SyntaxProvider
            .CreateSyntaxProvider(
                predicate: static (syntax, cancellationToken) => syntax.IsKind(SyntaxKind.StructDeclaration),
                transform: static (ctx, cancellationToken) => (TypeDeclarationSyntax)ctx.Node)
            .WithTrackingName("Syntax"); // WithTrackingName allow to record data about the step and access them from the tests

        var assemblyNameProvider = context.CompilationProvider
            .Select((compilation, cancellationToken) => compilation.AssemblyName)
            .WithTrackingName("AssemblyName");

        var valueProvider = structPovider.Combine(assemblyNameProvider);

        context.RegisterSourceOutput(valueProvider, (spc, valueProvider) =>
        {
            (var node, var assemblyName) = (valueProvider.Left, valueProvider.Right);
            spc.AddSource(node.Identifier.ValueText + ".cs", SourceText.From($"// {node.Identifier.Text} - {assemblyName}", Encoding.UTF8));
        });
    }
}

The WithTrackingName call is essential for verifying incremental generation. It configures Roslyn to track diagnostic information about each step in the pipeline, which is then accessible via TrackedSteps and TrackedOutputSteps on the GeneratorRunResult. The following code shows how to use this information in a test.

The test creates a compilation containing a single struct, then sets up a GeneratorDriver and runs the generator. Next, it adds a new file to the compilation and runs the generator again. The test then asserts that the GeneratorDriver did not recompute the output, and that the cached results from the AssemblyName and Syntax steps were used.

public sealed class SampleSourceGeneratorTests
{
    [Fact]
    public void Test()
    {
        var compilation = CSharpCompilation.Create("TestProject",
            new[] { CSharpSyntaxTree.ParseText("struct Test { }") },
            Basic.Reference.Assemblies.Net70.References.All,
            new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary));

        var generator = new SampleSourceGenerator();
        var sourceGenerator = generator.AsSourceGenerator();

        // trackIncrementalGeneratorSteps allows to report info about each step of the generator
        GeneratorDriver driver = CSharpGeneratorDriver.Create(
            generators: new ISourceGenerator[] { sourceGenerator },
            driverOptions: new GeneratorDriverOptions(default, trackIncrementalGeneratorSteps: true));

        // Run the generator
        driver = driver.RunGenerators(compilation);

        // Update the compilation and rerun the generator
        compilation = compilation.AddSyntaxTrees(CSharpSyntaxTree.ParseText("// dummy"));
        driver = driver.RunGenerators(compilation);

        // Assert the driver doesn't recompute the output
        var result = driver.GetRunResult().Results.Single();
        var allOutputs = result.TrackedOutputSteps.SelectMany(outputStep => outputStep.Value).SelectMany(output => output.Outputs);
        Assert.Collection(allOutputs, output => Assert.Equal(IncrementalStepRunReason.Cached, output.Reason));

        // Assert the driver use the cached result from AssemblyName and Syntax
        var assemblyNameOutputs = result.TrackedSteps["AssemblyName"].Single().Outputs;
        Assert.Collection(assemblyNameOutputs, output => Assert.Equal(IncrementalStepRunReason.Unchanged, output.Reason));

        var syntaxOutputs = result.TrackedSteps["Syntax"].Single().Outputs;
        Assert.Collection(syntaxOutputs, output => Assert.Equal(IncrementalStepRunReason.Unchanged, output.Reason));
    }
}

#Additional resources

• Incremental Generators

Do you have a question or a suggestion about this post? Contact me!