checkAllAllocationFailures

Exhaustively check that allocation failures within test_fn are handled without introducing memory leaks. If used with the testing.allocator as the backing_allocator, it will also be able to detect double frees, etc (when runtime safety is enabled).

The provided test_fn must have a std.mem.Allocator as its first argument, and must have a return type of !void. Any extra arguments of test_fn can be provided via the extra_args tuple.

Any relevant state shared between runs of test_fn must be reset within test_fn.

The strategy employed is to:

Run the test function once to get the total number of allocations.
Then, iterate and run the function X more times, incrementing the failing index each iteration (where X is the total number of allocations determined previously)

Expects that test_fn has a deterministic number of memory allocations:

If an allocation was made to fail during a run of test_fn, but test_fn didn't return error.OutOfMemory, then error.SwallowedOutOfMemoryError is returned from checkAllAllocationFailures. You may want to ignore this depending on whether or not the code you're testing includes some strategies for recovering from error.OutOfMemory.
If a run of test_fn with an expected allocation failure executes without an allocation failure being induced, then error.NondeterministicMemoryUsage is returned. This error means that there are allocation points that won't be tested by the strategy this function employs (that is, there are sometimes more points of allocation than the initial run of test_fn detects).

Here's an example using a simple test case that will cause a leak when the allocation of bar fails (but will pass normally):

test {
    const length: usize = 10;
    const allocator = std.testing.allocator;
    var foo = try allocator.alloc(u8, length);
    var bar = try allocator.alloc(u8, length);

    allocator.free(foo);
    allocator.free(bar);
}

The test case can be converted to something that this function can use by doing:

fn testImpl(allocator: std.mem.Allocator, length: usize) !void {
    var foo = try allocator.alloc(u8, length);
    var bar = try allocator.alloc(u8, length);

    allocator.free(foo);
    allocator.free(bar);
}

test {
    const length: usize = 10;
    const allocator = std.testing.allocator;
    try std.testing.checkAllAllocationFailures(allocator, testImpl, .{length});
}

Running this test will show that foo is leaked when the allocation of bar fails. The simplest fix, in this case, would be to use defer like so:

fn testImpl(allocator: std.mem.Allocator, length: usize) !void {
    var foo = try allocator.alloc(u8, length);
    defer allocator.free(foo);
    var bar = try allocator.alloc(u8, length);
    defer allocator.free(bar);
}

Function parameters

Parameters

backing_allocator:std.mem.Allocator
test_fn:anytype
extra_args:anytype

Type definitions in this namespace

Types

FailingAllocator
TmpDir
FuzzInputOptions
Reader: A `std.Io.Reader` that writes a predetermined list of buffers during `stream`.
ReaderIndirect: A `std.Io.Reader` that gets its data from another `std.Io.Reader`, and always

This function is intended to be used only in tests.

Functions

expectError: This function is intended to be used only in tests.
expectEqual: This function is intended to be used only in tests.
expectFmt: This function is intended to be used only in tests.
expectApproxEqAbs: This function is intended to be used only in tests.
expectApproxEqRel: This function is intended to be used only in tests.
expectEqualSlices: This function is intended to be used only in tests.
expectEqualSentinel: This function is intended to be used only in tests.
expect: This function is intended to be used only in tests.
tmpDir
expectEqualStrings
expectStringStartsWith
expectStringEndsWith
expectEqualDeep: This function is intended to be used only in tests.
checkAllAllocationFailures: Exhaustively check that allocation failures within `test_fn` are handled without
refAllDecls: Given a type, references all the declarations inside, so that the semantic analyzer sees them.
refAllDeclsRecursive: Given a type, recursively references all the declarations inside, so that the semantic analyzer sees them.
fuzz: Inline to avoid coverage instrumentation.

Provides deterministic randomness in unit tests.

Values

random_seed: Provides deterministic randomness in unit tests.
failing_allocator: = failing_allocator_instance.allocator()
allocator: This should only be used in temporary test programs.
allocator_instance
log_level: TODO https://github.com/ziglang/zig/issues/5738
backend_can_print: = switch (builtin.zig_backend) { .stage2_aarch64, .stage2_powerpc, .stage2_riscv64, .stage2_spirv, => false, else => true, }

Source

Implementation

pub fn checkAllAllocationFailures(backing_allocator: std.mem.Allocator, comptime test_fn: anytype, extra_args: anytype) !void {
    switch (@typeInfo(@typeInfo(@TypeOf(test_fn)).@"fn".return_type.?)) {
        .error_union => |info| {
            if (info.payload != void) {
                @compileError("Return type must be !void");
            }
        },
        else => @compileError("Return type must be !void"),
    }
    if (@typeInfo(@TypeOf(extra_args)) != .@"struct") {
        @compileError("Expected tuple or struct argument, found " ++ @typeName(@TypeOf(extra_args)));
    }

    const ArgsTuple = std.meta.ArgsTuple(@TypeOf(test_fn));
    const fn_args_fields = @typeInfo(ArgsTuple).@"struct".fields;
    if (fn_args_fields.len == 0 or fn_args_fields[0].type != std.mem.Allocator) {
        @compileError("The provided function must have an " ++ @typeName(std.mem.Allocator) ++ " as its first argument");
    }
    const expected_args_tuple_len = fn_args_fields.len - 1;
    if (extra_args.len != expected_args_tuple_len) {
        @compileError("The provided function expects " ++ std.fmt.comptimePrint("{d}", .{expected_args_tuple_len}) ++ " extra arguments, but the provided tuple contains " ++ std.fmt.comptimePrint("{d}", .{extra_args.len}));
    }

    // Setup the tuple that will actually be used with @call (we'll need to insert
    // the failing allocator in field @"0" before each @call)
    var args: ArgsTuple = undefined;
    inline for (@typeInfo(@TypeOf(extra_args)).@"struct".fields, 0..) |field, i| {
        const arg_i_str = comptime str: {
            var str_buf: [100]u8 = undefined;
            const args_i = i + 1;
            const str_len = std.fmt.printInt(&str_buf, args_i, 10, .lower, .{});
            break :str str_buf[0..str_len];
        };
        @field(args, arg_i_str) = @field(extra_args, field.name);
    }

    // Try it once with unlimited memory, make sure it works
    const needed_alloc_count = x: {
        var failing_allocator_inst = std.testing.FailingAllocator.init(backing_allocator, .{});
        args.@"0" = failing_allocator_inst.allocator();

        try @call(.auto, test_fn, args);
        break :x failing_allocator_inst.alloc_index;
    };

    var fail_index: usize = 0;
    while (fail_index < needed_alloc_count) : (fail_index += 1) {
        var failing_allocator_inst = std.testing.FailingAllocator.init(backing_allocator, .{ .fail_index = fail_index });
        args.@"0" = failing_allocator_inst.allocator();

        if (@call(.auto, test_fn, args)) |_| {
            if (failing_allocator_inst.has_induced_failure) {
                return error.SwallowedOutOfMemoryError;
            } else {
                return error.NondeterministicMemoryUsage;
            }
        } else |err| switch (err) {
            error.OutOfMemory => {
                if (failing_allocator_inst.allocated_bytes != failing_allocator_inst.freed_bytes) {
                    print(
                        "\nfail_index: {d}/{d}\nallocated bytes: {d}\nfreed bytes: {d}\nallocations: {d}\ndeallocations: {d}\nallocation that was made to fail: {f}",
                        .{
                            fail_index,
                            needed_alloc_count,
                            failing_allocator_inst.allocated_bytes,
                            failing_allocator_inst.freed_bytes,
                            failing_allocator_inst.allocations,
                            failing_allocator_inst.deallocations,
                            failing_allocator_inst.getStackTrace(),
                        },
                    );
                    return error.MemoryLeakDetected;
                }
            },
            else => return err,
        }
    }
}