Serilog: Structured Logging Explained

This post is a continuation of the Serilog series (previous part: Serilog: Configuration via appsettings.json).

In this part, we will:

• Learn what structured logging is and why it matters
• Configure a file sink with JSON-formatted structured logging
• Explore the benefits of structured logging for querying and analysis

Let's get started! 🔥

Why Structured Logging?

Picture this: you're debugging a production issue, sifting through thousands of log entries trying to find a specific event. Sounds familiar?

If you check the logs generated in the previous post, you'll have no problem finding relevant information in a small sample. However, real-world applications generate massive streams of logs with varying levels of importance and contextual data. Finding the right needle in that haystack? That's where things get tricky.

I once needed to find all logs containing two specific keywords. Simple enough, right? Wrong. One keyword was in uppercase, the other in lowercase, and my case-sensitive search came up empty. After some troubleshooting and making future searches case-insensitive, I learned a valuable lesson: unstructured logs are hard to query.

Here's the fundamental problem: from a machine's perspective, a plain text log is just a string of characters with no semantic meaning. There's no structure, no queryable fields, no easy way to filter or aggregate data.

Structured logging solves this by capturing not just the message string, but also additional data as key-value pairs in a standardized format. This transforms logs from human-readable text into machine-queryable data.

Configuring Structured Logging with Serilog

This episode starts from branch 003-structured-logging. You will find it here.

Let's configure two Serilog sinks side-by-side to see the difference. Both will log to files, but one uses plain text format while the other uses JSON (structured logging). Update your appsettings.json:

Now, run the app and check the structured-log-.json file in the Logs folder. Here's the C# code that generates the log:

The structured output looks like this:

Notice how the {Time} placeholder in the message template becomes a structured property in the JSON output. This is the foundation of structured logging.

Message Templates vs String Interpolation

Here's a critical mistake many developers make. Let's run an experiment. Replace the log line with this version using string interpolation:

Run the app and check the structured log:

See the problem? The Properties object is now empty, and the timestamp is baked directly into the MessageTemplate as plain text.

Understanding Structured Log Shape

Each structured log entry is a JSON object with multiple properties:

• Timestamp - when the log was created
• Level - log level (Information, Warning, Error, etc.). Read more about log levels in the previous part
• MessageTemplate - the message template with placeholders
• TraceId and SpanId - distributed tracing identifiers (appear automatically in ASP.NET Core when Activity.Current is set within the request pipeline)
• Properties - additional data logged as key-value pairs

Focus on MessageTemplate and Properties. The message template contains placeholders (e.g., {Time}) that get replaced with actual values stored in the Properties object. This separation allows you to query logs based on specific property values without worrying about message formatting or string parsing.

Why are TraceId and SpanId outside the Properties object?

They're part of the log context, not the message itself. Serilog supports enriching logs with additional context data, which is invaluable for correlating logs across different components or services. This topic was mentioned in the previous part and will be covered in depth in a future post.

Benefits of Structured Logging

Why should you care about structured logging? Here are the game-changers:

• Easier querying - Query logs based on specific properties (e.g., "show all logs where UserId=123") without regex gymnastics or text parsing
• Better analysis - Structured data integrates seamlessly with tools like Elasticsearch, Splunk, or Grafana Loki for aggregation and analytics
• Improved correlation - Correlate logs across different components or services using context data like TraceId and SpanId
• Enhanced visualization - Build dashboards and alerts based on log properties to monitor application health and performance in real-time

Summary

Key takeaways from this post:

• Serilog supports structured logging out of the box with built-in formatters for JSON and other formats
• Structured logging captures additional data as key-value pairs, transforming logs from text to queryable data
• Always prefer message templates with placeholders over string interpolation to preserve structured logging benefits

In the Next Part

Now that you understand what structured logging is, it's time to explore its benefits in practice. In the next part of the Serilog series, we'll introduce Grafana Loki sink that will allow us to query and analyze logs using Grafana dashboards (What is Loki and Grafana? Stay tuned!).

With these in place, we'll benchmark querying structured vs. unstructured logs in the environment that imitate real-world scenario to show you the tangible benefits.

Stay tuned! 🚀