uCalc API Version: 2.1.3-preview.2 Released: 6/16/2026

Warning

uCalc API Preview Release Notice:The documentation describes the intended behavior of the API. The current preview build contains incomplete features, unoptimized performance, and is subject to breaking changes.

Clear

Method

Product: 

Transformer Library

Class: 

Tokens

Removes all token definitions from the collection, restoring it to an empty state.

Syntax

Clear()

Parameters

[None]

Return

void

This method does not return a value.

Remarks

The Clear() method erases all token definitions from a Tokens collection. This is a powerful tool for completely resetting the lexical rules of a Transformer before defining a new, custom set of tokens from scratch.

🎯 Primary Use Case: Custom Language Definition

This method is the first step in creating a parser for a completely new syntax. By default, a Transformer inherits a rich set of tokens for parsing general-purpose expressions. Clear() allows you to wipe this default set and build a new tokenizer tailored to a specific format, such as a simplified configuration language, a data format, or a Domain-Specific Language (DSL).

⚠️ Warning: The Need for a Catch-All Token

A Transformer cannot function with a completely empty token list. If no patterns match a character, the tokenizer will not advance, leading to an infinite loop or failure. After clearing the tokens, you must add at least one "catch-all" token that can match any single character to ensure the tokenizer can always make progress.

// After clearing, always add a fallback token.t.Tokens().Clear();t.Tokens().Add("."); // Matches any single character.

⚖️ Comparative Analysis

  • vs. Manual Remove(): While you could iterate through a Tokens collection and call Remove() on each item, Clear() is a single, optimized operation that is both faster and more direct.

  • vs. New Transformer Instance: Creating a New(uCalc::Transformer, t) also gives you a fresh start, but it begins with uCalc's default token set. Clear() is used when you want to modify the lexical rules of an existing transformer instance in-place, rather than creating a new one.

This method is the key to unlocking uCalc's full potential for dynamic language creation, allowing you to move beyond expression evaluation and into full-custom parsing.

Examples

Succinct: Demonstrates the basic functionality of clearing default tokens and adding a single new one.
				
					using uCalcSoftware;

var uc = new uCalc();
var t = uc.NewTransformer();
t.FromTo("is", "<IS>");

Console.WriteLine("--- With Default Tokens ---");
// By default, 'is' is a whole word (token)
Console.WriteLine(t.Transform("This is a test"));

// Clear all default token definitions
t.Tokens.Clear();

// Add a new, simple token that matches any single character
t.Tokens.Add(".");

Console.WriteLine("");
Console.WriteLine("--- After Clearing and Adding '.' Token ---");
// Now, 'i' and 's' are matched as separate characters
t.FromTo("is", "<IS>"); // The rule must be redefined
Console.WriteLine(t.Transform("This is a test"));
				
			
--- With Default Tokens ---
This <IS> a test

--- After Clearing and Adding '.' Token ---
Th<IS> <IS> a test
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;
   auto t = uc.NewTransformer();
   t.FromTo("is", "<IS>");

   cout << "--- With Default Tokens ---" << endl;
   // By default, 'is' is a whole word (token)
   cout << t.Transform("This is a test") << endl;

   // Clear all default token definitions
   t.Tokens().Clear();

   // Add a new, simple token that matches any single character
   t.Tokens().Add(".");

   cout << "" << endl;
   cout << "--- After Clearing and Adding '.' Token ---" << endl;
   // Now, 'i' and 's' are matched as separate characters
   t.FromTo("is", "<IS>"); // The rule must be redefined
   cout << t.Transform("This is a test") << endl;
}
				
			
--- With Default Tokens ---
This <IS> a test

--- After Clearing and Adding '.' Token ---
Th<IS> <IS> a test
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      Dim t = uc.NewTransformer()
      t.FromTo("is", "<IS>")
      
      Console.WriteLine("--- With Default Tokens ---")
      '// By default, 'is' is a whole word (token)
      Console.WriteLine(t.Transform("This is a test"))
      
      '// Clear all default token definitions
      t.Tokens.Clear()
      
      '// Add a new, simple token that matches any single character
      t.Tokens.Add(".")
      
      Console.WriteLine("")
      Console.WriteLine("--- After Clearing and Adding '.' Token ---")
      '// Now, 'i' and 's' are matched as separate characters
      t.FromTo("is", "<IS>") '// The rule must be redefined
      Console.WriteLine(t.Transform("This is a test"))
   End Sub
End Module
				
			
--- With Default Tokens ---
This <IS> a test

--- After Clearing and Adding '.' Token ---
Th<IS> <IS> a test
Importing a list of tokens from one transformer to another
				
					using uCalcSoftware;

var uc = new uCalc();
var t = uc.NewTransformer();

// This creates a simpler (not very useful) set of tokens for sake of example
var MyTokens = t.Tokens;
MyTokens.Clear(); // removes the default list of tokens
MyTokens.Add("."); // Should always have a fallback token like this one
MyTokens.Add(";", TokenType.StatementSep);
MyTokens.Add("<", TokenType.Generic, ">");
MyTokens.Add("[0-9]+", TokenType.Literal);
MyTokens.Add(" +", TokenType.Whitespace);
MyTokens.Add("[a-z]+", TokenType.AlphaNumeric);

t.FromTo("{ This | That} {etc}", "[{@Self}]"); // {etc} stops at the semicolon ";" TokenType.StatementSep

Console.WriteLine(t.Transform("This is a test; That < 123.456; abc >; ABC"));
Console.WriteLine("");

var OtherTransform = new uCalc.Transformer(uc); // This is an alternative way of constructing a new transformer
OtherTransform.Tokens.Add(MyTokens); // Imports the entire token list from the other transformer
OtherTransform.Pattern("{word:1}");

Console.WriteLine(OtherTransform.Filter("This is a test; That < 123.456; abc >; ABC").Matches.Text);

				
			
[This is a test]; [That < 123.456; abc >]; ABC

This
is
a
test
;
That
<
123
.
456
;
abc
>
;
ABC
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;
   auto t = uc.NewTransformer();

   // This creates a simpler (not very useful) set of tokens for sake of example
   auto MyTokens = t.Tokens();
   MyTokens.Clear(); // removes the default list of tokens
   MyTokens.Add("."); // Should always have a fallback token like this one
   MyTokens.Add(";", TokenType::StatementSep);
   MyTokens.Add("<", TokenType::Generic, ">");
   MyTokens.Add("[0-9]+", TokenType::Literal);
   MyTokens.Add(" +", TokenType::Whitespace);
   MyTokens.Add("[a-z]+", TokenType::AlphaNumeric);

   t.FromTo("{ This | That} {etc}", "[{@Self}]"); // {etc} stops at the semicolon ";" TokenType.StatementSep

   cout << t.Transform("This is a test; That < 123.456; abc >; ABC") << endl;
   cout << "" << endl;

   uCalc::Transformer OtherTransform(uc); // This is an alternative way of constructing a new transformer
   OtherTransform.Tokens().Add(MyTokens); // Imports the entire token list from the other transformer
   OtherTransform.Pattern("{word:1}");

   cout << OtherTransform.Filter("This is a test; That < 123.456; abc >; ABC").Matches().Text() << endl;

}
				
			
[This is a test]; [That < 123.456; abc >]; ABC

This
is
a
test
;
That
<
123
.
456
;
abc
>
;
ABC
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      Dim t = uc.NewTransformer()
      
      '// This creates a simpler (not very useful) set of tokens for sake of example
      Dim MyTokens = t.Tokens
      MyTokens.Clear() '// removes the default list of tokens
      MyTokens.Add(".") '// Should always have a fallback token like this one
      MyTokens.Add(";", TokenType.StatementSep)
      MyTokens.Add("<", TokenType.Generic, ">")
      MyTokens.Add("[0-9]+", TokenType.Literal)
      MyTokens.Add(" +", TokenType.Whitespace)
      MyTokens.Add("[a-z]+", TokenType.AlphaNumeric)
      
      t.FromTo("{ This | That} {etc}", "[{@Self}]") '// {etc} stops at the semicolon ";" TokenType.StatementSep
      
      Console.WriteLine(t.Transform("This is a test; That < 123.456; abc >; ABC"))
      Console.WriteLine("")
      
      Dim OtherTransform As New uCalc.Transformer(uc) '// This is an alternative way of constructing a new transformer
      OtherTransform.Tokens.Add(MyTokens) '// Imports the entire token list from the other transformer
      OtherTransform.Pattern("{word:1}")
      
      Console.WriteLine(OtherTransform.Filter("This is a test; That < 123.456; abc >; ABC").Matches.Text)
      
   End Sub
End Module
				
			
[This is a test]; [That < 123.456; abc >]; ABC

This
is
a
test
;
That
<
123
.
456
;
abc
>
;
ABC
Previous
Next