A while back, in 2016 - before the release of .NET Core 1.0 and Microsofts strategy for a modernized .NET platform - I wrote an article on CodeProject on how to improve stream reading performance. I recently reviewed the old code, so this article will compare the old results to the performance of modern .NET and discuss, whether any of the performance tips still make sense.

Spoiler: in case you are targeting legacy .NET framework versions (for example net48), all tips are still relevant.

The CodeProject article

Motivation

Let’s assume we want to read properties of an MP3 file, for example get a bitrate histogram of a file that is encoded with a variable bitrate. MP3 files consist of a bunch of frames (in the magnitude of 10k for a 4 or 5 minute song). Each frame has a header of 4 bytes (see Wikipedia article), which contains information like the bitrate and the length of the data block that follows the header. To build the histogram, we need a buffer of 4 bytes, read the header and then skip to the next header.

Performance tests

Implementing that read some bytes, then skip a few behavior, you might find yourself writing code like

private static long Test1(Stream stream, byte[] buffer)
{
    int count;
    long hash = 0;

    // Read bytes from the stream.
    while ((count = stream.Read(buffer, 0, N)) > 0)
    {
        hash += Checksum(buffer, 0, count);

        // Advance position, skipping some bytes.
        stream.Position += SKIP;
    }

    return hash;
}

The above code seems straightforward, but running the test on a 7 MB file (targeting .NET framework 4.8) we get

$ dotnet run -c Release -f net48
Warm up ...
Test 1: 1994 ms

Nearly two seconds of execution time seems quite a lot. With just one small change we can significantly improve performance:

private static long Test2(Stream stream, byte[] buffer)
{
    int count;
    long hash = 0;

    // Read bytes from the stream.
    while ((count = stream.Read(buffer, 0, N)) > 0)
    {
        hash += Checksum(buffer, 0, count);

        // Advance position, skipping some bytes.
        stream.Seek(SKIP, SeekOrigin.Current);
    }

    return hash;
}

By using stream.Seek instead of stream.Position performance improves by a factor of 2.4:

$ dotnet run -c Release -f net48
Warm up ...
Test 1: 1994 ms
Test 2:  815 ms

Now let’s see what happens if we don’t seek from the current position, but from the beginning of the stream:

private static long Test3(Stream stream, byte[] buffer)
{
    int count;
    long hash = 0;
    long position = 0;

    // Read a couple of bytes from the stream.
    while ((count = stream.Read(buffer, 0, N)) > 0)
    {
        hash += Checksum(buffer, 0, count);

        // Advance position, skipping some bytes.
        position += (N + SKIP);
        stream.Seek(position, SeekOrigin.Begin);
    }

    return hash;
}

We get

$ dotnet run -c Release -f net48
Warm up ...
Test 1: 1994 ms
Test 2:  815 ms
Test 3:  656 ms

Again, we see a small speedup and get an overall improvement by a factor of 3!

Intermediate Conclusion

Though results may vary on different hardware, here are some rules you should follow when reading from .NET streams:

• Avoid setting the Position property. Always prefer the Seek method
• Avoid reading and writing properties in loops (like Position or Length)
• Prefer using SeekOrigin.Begin

Reading without seeking

Say you want to seek to a particular time offset in an audio file. That’s obviously a valid use-case for seeking in a stream, but then it won’t make a difference if you are using stream.Position or stream.Seek since it is just a single call. On the other hand, using seeking the way it is implemented above will always have an impact on performance.

If you are dealing with performance critical code, writing your own buffer logic will likely outperform any buffering the .NET Stream implementation does internally. The idea is to read larger chunks of the file at once, thus reducing cycles spent on actual file operations done by your OS. The following solution doesn’t use seeking at all, but it’s more involved, because you need to synchronize two successive buffer reads:

private static long Test4(Stream stream, byte[] buffer)
{
    const int SIZE = buffer.Length;
    
    long hash = 0;
    int position = 0;
    
    int count, end;
    
    // Fill the buffer.
    while ((count = stream.Read(buffer, 0, SIZE)) > 0)
    {
        if (position > SKIP)
        {
            // The previous frame overlapped with the current.
            hash += Checksum(buffer, 0, position - SKIP);
        }

        // Process the buffer.
        while (position < count)
        {
            end = position + N;
            if (end > count) end = count;
            hash += Checksum(buffer, position, end);
            position += (N + SKIP);
        }

        // Set the correct offset.
        position = position % SIZE;
    }

    return hash;
}

Running the above code with a buffer size of 4096, the final result is

$ dotnet run -c Release -f net48
Warm up ...
Test 1: 1994 ms
Test 2:  815 ms
Test 3:  656 ms
Test 4:    7 ms

Boilerplate Code

To run the benchmark, the following boilerplate code was used:

// Number of bytes to read.
private const int N = 4;

// Number of bytes to skip.
private const int SKIP = 3;

public static void Run()
{
    string file = "~/Music/some-audio-file.mp3";

    if (!File.Exists(file))
    {
        Console.WriteLine($"Error: file '{file}' doesn't exist");
        return;
    }

    // Open file for reading.
    using var stream = File.OpenRead(file);

    var s = new Stopwatch();

    byte[] buffer = new byte[N];

    Console.WriteLine("Warm up ...");

    long hash = Test1(stream, buffer);

    stream.Position = 0;

    Run("Test 1", stream, s, buffer, hash, Test1);
    Run("Test 2", stream, s, buffer, hash, Test2);
    Run("Test 3", stream, s, buffer, hash, Test3);

    buffer = new byte[4096];

    Run("Test 4", stream, s, buffer, hash, Test4);
}

private static void Run(string name, Stream stream, Stopwatch s, byte[] buffer,
    long hash, Func<Stream, byte[], long> func)
{
    Console.Write($"{name}: ");

    s.Restart();
    long h = func(stream, buffer);
    s.Stop();

    // Reset stream position.
    stream.Position = 0;

    Console.Write($"{s.ElapsedMilliseconds,4} ms");

    if (h != hash)
    {
        Console.Write(" -- error: unexpected hash");
    }

    Console.WriteLine();
}

private static int Checksum(byte[] buffer, int offsetStart, int offsetEnd)
{
    int sum = 0;
    for (int i = offsetStart; i < offsetEnd; i++)
    {
        sum += buffer[i];
    }
    return sum;
}

The csproj project file:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFrameworks>net48;net9.0</TargetFrameworks>
    <LangVersion>latest</LangVersion>
  </PropertyGroup>
</Project>

Performance on modern .NET

To cut a long story short, all our efforts to improve stream reading performance have mostly become irrelevant on modern .NET platforms. Running the tests on .NET 9 we get:

$ dotnet run -c Release -f net9.0
Warm up ...
Test 1:   34 ms
Test 2:   31 ms
Test 3:   34 ms
Test 4:    7 ms

The .NET developers have clearly done a great job optimizing many parts of the platform. Though the custom code reading the stream without seeking outperforms the other methods, this kind of optimization probably only makes sense in performance critical scenarios. Such code often tends to get ugly and hard to maintain, so for a use case as described in the motivation section, it’s better to keep the code simple and clean.

Please use the contact form for comments.