uCalc API Version: 2.1.3-preview.2 Released: 6/16/2026

Warning

uCalc API Preview Release Notice:The documentation describes the intended behavior of the API. The current preview build contains incomplete features, unoptimized performance, and is subject to breaking changes.

Project: Extracting and Aggregating Metrics from Server Logs

Product: 

Class: 

A step-by-step guide to building a server log parser that extracts and aggregates key performance metrics using the uCalc Transformer.

Remarks

📊 Project: Extracting and Aggregating Metrics from Server Logs

This project will guide you through building a powerful and efficient server log parser. It's a classic real-world problem that perfectly showcases the advantages of uCalc's token-aware Transformer over traditional tools like Regular Expressions or manual string splitting.

The Goal: From Raw Logs to Actionable Insights

Our objective is to take a raw, unstructured log file and transform it into a structured summary of key performance indicators (KPIs). We want to answer questions like:

  • How many total requests were processed?
  • What was the error rate?
  • What was the average response time?
  • What was the peak response time?

The Input Data

Here is a sample of the access.log content we will be working with. It contains a mix of successful requests and errors, with inconsistent spacing.

2024-10-26 10:00:05 INFO    192.168.1.10   "GET /api/users HTTP/1.1" 200 15ms2024-10-26 10:00:06 INFO    192.168.1.15   "GET /api/products HTTP/1.1" 200 22msThis is a malformed line and should be ignored.2024-10-26 10:00:07 ERROR   192.168.1.22   "POST /api/login HTTP/1.1" 500 120ms2024-10-26 10:00:08 INFO    192.168.1.10   "GET /api/users/1 HTTP/1.1" 200 8ms

The Strategy: A Single, Powerful Rule

We will use a single Transformer rule to process the entire log file. This rule will be designed to:

  1. Match a Valid Log Line: The pattern will capture all the relevant fields: timestamp, log level, IP address, request details, status code, and response time.
  2. Aggregate Metrics with {@Exec}: The replacement part of the rule will not produce any text. Instead, it will use a series of {@Exec} directives to update a set of counter variables for each line it processes. This is a highly efficient way to perform calculations as a side effect of a transformation.
  3. Ignore Invalid Lines: The pattern will be specific enough that it won't match malformed lines, which will be gracefully ignored.

Step 1: Setting Up the Metric Variables

First, we need to define the variables in our uCalc instance that will store the aggregated metrics. We initialize them all to zero.

uc.DefineVariable("request_count = 0");uc.DefineVariable("error_count = 0");uc.DefineVariable("total_response_time = 0.0");uc.DefineVariable("max_response_time = 0.0");

Step 2: Defining the Log Parsing Rule

Next, we create our Transformer and define the main rule. The pattern is designed to be robust against variable whitespace. The key is using {@String:request} to capture the entire HTTP request, including the spaces inside it, as a single token.

var t = new uCalc.Transformer();// The pattern captures all fields from a valid log line.var pattern = "{@date} {@time} {@level} {@ip} {@String:request} {@Number:status} {@Number:time}ms";

Step 3: The Aggregation Logic

The replacement string is where the magic happens. It contains no visible text, only a sequence of {@Exec} calls that update our metric variables for each match. Note that we must use Double(time) to convert the captured time string into a number for calculations.

var replacement = ""    + "{@Exec: request_count++}"    + "{@Exec: total_response_time = total_response_time + Double(time)}"    + "{@Exec: max_response_time = Max(max_response_time, Double(time))}"    + "{@Exec: if(status >= 400, error_count++)}";t.FromTo(pattern, replacement);

Step 4: Running the Analysis and Displaying Results

Finally, we pass the entire multi-line log string to the Transform method. After it completes, our metric variables will hold the final aggregated values, which we can retrieve with EvalStr.

// After running t.Transform(logText)...Console.WriteLine("--- Log Analysis Summary ---");Console.WriteLine($"Total Requests: {uc.EvalStr("request_count")}");Console.WriteLine($"Total Errors: {uc.EvalStr("error_count")}");Console.WriteLine($"Average Response Time: {uc.EvalStr("total_response_time / request_count")}ms");Console.WriteLine($"Maximum Response Time: {uc.EvalStr("max_response_time")}ms");

⚖️ Why uCalc? (Comparative Analysis)

  • vs. Regex: Crafting a single, robust regex to capture all these fields while handling variable whitespace would be extremely complex and difficult to maintain. Furthermore, regex has no built-in mechanism for performing calculations during the match.
  • vs. Manual Split(): A simple Split(' ') would fail because of the spaces inside the quoted request string (e.g., "GET /api/users HTTP/1.1"). uCalc's tokenizer correctly identifies this as a single string token, making the pattern robust and simple.

Examples

A succinct example that extracts the status code and response time from a single log line using a simple pattern.
				
					using uCalcSoftware;

var uc = new uCalc();
using (var t = new uCalc.Transformer()) {
   string logLine = """
2024-10-26 10:00:05 INFO 192.168.1.10 "GET /api/users HTTP/1.1" 200 15ms
""";

   // Define a pattern to capture the status and time
   string pattern = "{@String:request} {@Number:status} {@Number:time}ms";

   // The replacement string formats the captured data for display
   // {request} or {request(1)} would return the string without surrounding quotes
   // {request(0)} returns the string with surrounding quotes
   string replacement = "Request: {request(0)}, Status: {status}, Time: {time}ms";

   t.FromTo(pattern, replacement);

   // Use Filter() to extract only the transformed match
   Console.WriteLine(t.Transform(logLine).Matches);
}
				
			
Request: "GET /api/users HTTP/1.1", Status: 200, Time: 15ms
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;
   {
      uCalc::Transformer t;
      t.Owned(); // Causes t to be released when it goes out of scope
      string logLine = R"(2024-10-26 10:00:05 INFO 192.168.1.10 "GET /api/users HTTP/1.1" 200 15ms)";

      // Define a pattern to capture the status and time
      string pattern = "{@String:request} {@Number:status} {@Number:time}ms";

      // The replacement string formats the captured data for display
      // {request} or {request(1)} would return the string without surrounding quotes
      // {request(0)} returns the string with surrounding quotes
      string replacement = "Request: {request(0)}, Status: {status}, Time: {time}ms";

      t.FromTo(pattern, replacement);

      // Use Filter() to extract only the transformed match
      cout << t.Transform(logLine).Matches() << endl;
   }
}
				
			
Request: "GET /api/users HTTP/1.1", Status: 200, Time: 15ms
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      Using t As New uCalc.Transformer()
         Dim logLine As String = "2024-10-26 10:00:05 INFO 192.168.1.10 ""GET /api/users HTTP/1.1"" 200 15ms"
         
         '// Define a pattern to capture the status and time
         Dim pattern As String = "{@String:request} {@Number:status} {@Number:time}ms"
         
         '// The replacement string formats the captured data for display
         '// {request} or {request(1)} would return the string without surrounding quotes
         '// {request(0)} returns the string with surrounding quotes
         Dim replacement As String = "Request: {request(0)}, Status: {status}, Time: {time}ms"
         
         t.FromTo(pattern, replacement)
         
         '// Use Filter() to extract only the transformed match
         Console.WriteLine(t.Transform(logLine).Matches)
      End Using
   End Sub
End Module
				
			
Request: "GET /api/users HTTP/1.1", Status: 200, Time: 15ms
A complete log processing pipeline that parses multiple lines and calculates aggregate metrics like total requests, error count, and average response time.
				
					using uCalcSoftware;

var uc = new uCalc();
// 1. Define variables to hold the metrics
uc.DefineVariable("request_count = 0");
uc.DefineVariable("error_count = 0");
uc.DefineVariable("total_response_time = 0.0");
uc.DefineVariable("max_response_time = 0.0");

// 2. Create the transformer and define the rule
using (var t = new uCalc.Transformer(uc)) {
   var pattern = "{@String:request} {@Number:status} {@Number:time}ms";

   // 3. The replacement string uses @Exec for side-effects (updating variables)
   var replacement = """

{@Exec: request_count++}
{@Exec: total_response_time = total_response_time + Double(time)}
{@Exec: max_response_time = Max(max_response_time, Double(time))}
{@Exec: iif(Double(status) >= 400, error_count++, 0)}

""";

   t.FromTo(pattern, replacement);

   // 4. Define the multi-line log data
   var logText = """

2024-10-26 10:00:05 INFO    192.168.1.10   "GET /api/users HTTP/1.1" 200 15ms
2024-10-26 10:00:06 INFO    192.168.1.15   "GET /api/products HTTP/1.1" 200 22ms
2024-10-26 10:00:07 ERROR   192.168.1.22   "POST /api/login HTTP/1.1" 500 120ms
2024-10-26 10:00:08 INFO    192.168.1.10   "GET /api/users/1 HTTP/1.1" 200 8ms

""";

   // 5. Run the transformation (the output will be empty as we only use @Exec)
   t.Transform(logText);
}

// 6. Display the final aggregated metrics
Console.WriteLine("--- Log Analysis Summary ---");
Console.WriteLine($"Total Requests: {uc.EvalStr("request_count")}");
Console.WriteLine($"Total Errors: {uc.EvalStr("error_count")}");
Console.WriteLine($"Average Response Time: {uc.EvalStr("total_response_time / request_count")}ms");
Console.WriteLine($"Maximum Response Time: {uc.EvalStr("max_response_time")}ms");
				
			
--- Log Analysis Summary ---
Total Requests: 4
Total Errors: 1
Average Response Time: 41.25ms
Maximum Response Time: 120ms
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;
   // 1. Define variables to hold the metrics
   uc.DefineVariable("request_count = 0");
   uc.DefineVariable("error_count = 0");
   uc.DefineVariable("total_response_time = 0.0");
   uc.DefineVariable("max_response_time = 0.0");

   // 2. Create the transformer and define the rule
   {
      uCalc::Transformer t(uc);
      t.Owned(); // Causes t to be released when it goes out of scope
      auto pattern = "{@String:request} {@Number:status} {@Number:time}ms";

      // 3. The replacement string uses @Exec for side-effects (updating variables)
      auto replacement = R"(
{@Exec: request_count++}
{@Exec: total_response_time = total_response_time + Double(time)}
{@Exec: max_response_time = Max(max_response_time, Double(time))}
{@Exec: iif(Double(status) >= 400, error_count++, 0)}
)";

      t.FromTo(pattern, replacement);

      // 4. Define the multi-line log data
      auto logText = R"(
2024-10-26 10:00:05 INFO    192.168.1.10   "GET /api/users HTTP/1.1" 200 15ms
2024-10-26 10:00:06 INFO    192.168.1.15   "GET /api/products HTTP/1.1" 200 22ms
2024-10-26 10:00:07 ERROR   192.168.1.22   "POST /api/login HTTP/1.1" 500 120ms
2024-10-26 10:00:08 INFO    192.168.1.10   "GET /api/users/1 HTTP/1.1" 200 8ms
)";

      // 5. Run the transformation (the output will be empty as we only use @Exec)
      t.Transform(logText);
   }

   // 6. Display the final aggregated metrics
   cout << "--- Log Analysis Summary ---" << endl;
   cout << "Total Requests: " << uc.EvalStr("request_count") << endl;
   cout << "Total Errors: " << uc.EvalStr("error_count") << endl;
   cout << "Average Response Time: " << uc.EvalStr("total_response_time / request_count") << "ms" << endl;
   cout << "Maximum Response Time: " << uc.EvalStr("max_response_time") << "ms" << endl;
}
				
			
--- Log Analysis Summary ---
Total Requests: 4
Total Errors: 1
Average Response Time: 41.25ms
Maximum Response Time: 120ms
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      '// 1. Define variables to hold the metrics
      uc.DefineVariable("request_count = 0")
      uc.DefineVariable("error_count = 0")
      uc.DefineVariable("total_response_time = 0.0")
      uc.DefineVariable("max_response_time = 0.0")
      
      '// 2. Create the transformer and define the rule
      Using t As New uCalc.Transformer(uc)
         Dim pattern = "{@String:request} {@Number:status} {@Number:time}ms"
         
         '// 3. The replacement string uses @Exec for side-effects (updating variables)
         Dim replacement = "
{@Exec: request_count++}
{@Exec: total_response_time = total_response_time + Double(time)}
{@Exec: max_response_time = Max(max_response_time, Double(time))}
{@Exec: iif(Double(status) >= 400, error_count++, 0)}
"
         
         t.FromTo(pattern, replacement)
         
         '// 4. Define the multi-line log data
         Dim logText = "
2024-10-26 10:00:05 INFO    192.168.1.10   ""GET /api/users HTTP/1.1"" 200 15ms
2024-10-26 10:00:06 INFO    192.168.1.15   ""GET /api/products HTTP/1.1"" 200 22ms
2024-10-26 10:00:07 ERROR   192.168.1.22   ""POST /api/login HTTP/1.1"" 500 120ms
2024-10-26 10:00:08 INFO    192.168.1.10   ""GET /api/users/1 HTTP/1.1"" 200 8ms
"
         
         '// 5. Run the transformation (the output will be empty as we only use @Exec)
         t.Transform(logText)
      End Using
      
      '// 6. Display the final aggregated metrics
      Console.WriteLine("--- Log Analysis Summary ---")
      Console.WriteLine($"Total Requests: {uc.EvalStr("request_count")}")
      Console.WriteLine($"Total Errors: {uc.EvalStr("error_count")}")
      Console.WriteLine($"Average Response Time: {uc.EvalStr("total_response_time / request_count")}ms")
      Console.WriteLine($"Maximum Response Time: {uc.EvalStr("max_response_time")}ms")
   End Sub
End Module
				
			
--- Log Analysis Summary ---
Total Requests: 4
Total Errors: 1
Average Response Time: 41.25ms
Maximum Response Time: 120ms
Previous
Next