uCalc API Version: 2.1.3-preview.2 Released: 6/16/2026

Warning

uCalc API Preview Release Notice:The documentation describes the intended behavior of the API. The current preview build contains incomplete features, unoptimized performance, and is subject to breaking changes.

Add(Item)

Method

Product: 

Transformer Library

Class: 

Tokens

Imports a pre-existing token definition from an Item object into the current token collection.

Syntax

Add(Item)

Parameters

tokenItem
Item
The `Item` object representing the token definition to be imported. This item must have been created by a previous call to an `Add` method.

Return

Tokens

Returns the current Tokens object to allow for a fluent, chainable interface.

Remarks

♻️ Reusing Token Definitions

The Add(Item) method imports a pre-existing, fully configured token definition into the current token collection. This is a cornerstone of reusability, allowing you to define a complex token once and share it across multiple Transformer or String objects.

This promotes the DRY (Don't Repeat Yourself) principle for lexical rules. Instead of re-creating a custom comment style or a specific literal format for every new parser, you can define it once, get its Item handle, and import it wherever needed.

⚙️ How It Works

This method takes an Item object that represents a token and adds a reference to that token's definition to the current Tokens list. The imported token inherits the precedence of a newly added token (i.e., it is checked first) unless a different insertion point is specified in another overload.

Prerequisites

  • The tokenItem must be a valid token definition, typically created by a previous call to another Add overload (e.g., uc.Tokens().Add(regex, type)).
  • The token being imported must belong to the same parent uCalc instance. You cannot share token definitions between different, isolated uCalc engine instances.

💡 Comparative Analysis

Without this method, sharing a token definition would require you to manually extract the regex pattern, token type, and other properties from the original token and pass them to the regex-based Add method again.

Manual (Verbose) Approach:

// Assume 'originalToken' is an Item from another Tokens collection.var regex = originalToken.Regex();var type = originalToken.TypeOfToken();// ... get other properties ...newTokens.Add(regex, type, ...);

Using Add(Item) (Clean & Efficient):

newTokens.Add(originalToken);

This approach is not only cleaner but also ensures that all of the token's properties (like its data type or QuotedText flag) are correctly carried over without risk of error.

Examples

A simple demonstration of defining a token in one transformer and reusing it in another.
				
					using uCalcSoftware;

var uc = new uCalc();
var t1 = new uCalc.Transformer();
var t2 = new uCalc.Transformer();

// Define a custom token in the first transformer
var customToken = t1.Tokens.Add("###", TokenType.Generic);

// Import the token definition into the second transformer
t2.Tokens.Add(customToken);
t2.FromTo("###", "MATCH");

Console.WriteLine($"t2 can now find '###': {t2.Transform("Test ### Test")}");
				
			
t2 can now find '###': Test MATCH Test
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;
   uCalc::Transformer t1;
   uCalc::Transformer t2;

   // Define a custom token in the first transformer
   auto customToken = t1.Tokens().Add("###", TokenType::Generic);

   // Import the token definition into the second transformer
   t2.Tokens().Add(customToken);
   t2.FromTo("###", "MATCH");

   cout << "t2 can now find '###': " << t2.Transform("Test ### Test") << endl;
}
				
			
t2 can now find '###': Test MATCH Test
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      Dim t1 As New uCalc.Transformer()
      Dim t2 As New uCalc.Transformer()
      
      '// Define a custom token in the first transformer
      Dim customToken = t1.Tokens.Add("###", TokenType.Generic)
      
      '// Import the token definition into the second transformer
      t2.Tokens.Add(customToken)
      t2.FromTo("###", "MATCH")
      
      Console.WriteLine($"t2 can now find '###': {t2.Transform("Test ### Test")}")
   End Sub
End Module
				
			
t2 can now find '###': Test MATCH Test
Shows how importing a comment token definition can prevent find-and-replace rules from incorrectly modifying text inside comments.
				
					using uCalcSoftware;

var uc = new uCalc();
// Create a transformer with a custom comment token definition
var t_source = new uCalc.Transformer();
var commentToken = t_source.Tokens.Add("""
/\*([\s\S]*?)\*/
""", TokenType.Whitespace);
commentToken.Description = "C-style block comment";

// Create a second transformer that will do replacements
var t_replacer = new uCalc.Transformer();
t_replacer.FromTo("secret", "REDACTED");

var sourceText = "a secret /* contains another secret */ value";

Console.WriteLine("--- Before Importing Comment Token ---");
// Without knowing about comments, t_replacer incorrectly modifies the text inside the comment.
Console.WriteLine(t_replacer.Transform(sourceText));
Console.WriteLine("");

Console.WriteLine("--- After Importing Comment Token ---");
// Import the comment token definition. Now, the comment block is treated as a single whitespace token.
t_replacer.Tokens.Add(commentToken);
// The transformer must be reset with the original text for the new token to apply.
t_replacer.Text = sourceText;
Console.WriteLine(t_replacer.Transform());
				
			
--- Before Importing Comment Token ---
a REDACTED /* contains another REDACTED */ value

--- After Importing Comment Token ---
a REDACTED /* contains another secret */ value
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;
   // Create a transformer with a custom comment token definition
   uCalc::Transformer t_source;
   auto commentToken = t_source.Tokens().Add(R"(/\*([\s\S]*?)\*/)", TokenType::Whitespace);
   commentToken.Description("C-style block comment");

   // Create a second transformer that will do replacements
   uCalc::Transformer t_replacer;
   t_replacer.FromTo("secret", "REDACTED");

   auto sourceText = "a secret /* contains another secret */ value";

   cout << "--- Before Importing Comment Token ---" << endl;
   // Without knowing about comments, t_replacer incorrectly modifies the text inside the comment.
   cout << t_replacer.Transform(sourceText) << endl;
   cout << "" << endl;

   cout << "--- After Importing Comment Token ---" << endl;
   // Import the comment token definition. Now, the comment block is treated as a single whitespace token.
   t_replacer.Tokens().Add(commentToken);
   // The transformer must be reset with the original text for the new token to apply.
   t_replacer.Text(sourceText);
   cout << t_replacer.Transform() << endl;
}
				
			
--- Before Importing Comment Token ---
a REDACTED /* contains another REDACTED */ value

--- After Importing Comment Token ---
a REDACTED /* contains another secret */ value
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      '// Create a transformer with a custom comment token definition
      Dim t_source As New uCalc.Transformer()
      Dim commentToken = t_source.Tokens.Add("/\*([\s\S]*?)\*/", TokenType.Whitespace)
      commentToken.Description = "C-style block comment"
      
      '// Create a second transformer that will do replacements
      Dim t_replacer As New uCalc.Transformer()
      t_replacer.FromTo("secret", "REDACTED")
      
      Dim sourceText = "a secret /* contains another secret */ value"
      
      Console.WriteLine("--- Before Importing Comment Token ---")
      '// Without knowing about comments, t_replacer incorrectly modifies the text inside the comment.
      Console.WriteLine(t_replacer.Transform(sourceText))
      Console.WriteLine("")
      
      Console.WriteLine("--- After Importing Comment Token ---")
      '// Import the comment token definition. Now, the comment block is treated as a single whitespace token.
      t_replacer.Tokens.Add(commentToken)
      '// The transformer must be reset with the original text for the new token to apply.
      t_replacer.Text = sourceText
      Console.WriteLine(t_replacer.Transform())
   End Sub
End Module
				
			
--- Before Importing Comment Token ---
a REDACTED /* contains another REDACTED */ value

--- After Importing Comment Token ---
a REDACTED /* contains another secret */ value
Token Add(ExistingToken)
				
					using uCalcSoftware;

var uc = new uCalc();

var t1 = uc.NewTransformer();
var CommentRegex = """
/\*([\s\S]*?)\*/
""";
var WhitespaceToken = t1.Tokens.Add(CommentRegex, TokenType.Whitespace);
WhitespaceToken.Description = "Treats text between /* and */ as whitespace";

var txt = "a b, a /* comment a b */ b, a x b, ab, a   b, a, b";

var t2 = uc.NewTransformer();
t2.FromTo("a b", "<{@Self}>");
t2.Transform(txt);
Console.WriteLine(t2);

// The token defined in the other transformer (WhitespaceToken) is imported into this one.
// Now, everything between /* and */ will be treated as whitespace
t2.Tokens.Add(WhitespaceToken);
Console.WriteLine(t2.Transform(txt));
				
			
<a b>, a /* comment <a b> */ b, a x b, ab, <a   b>, a, b
<a b>, <a /* comment a b */ b>, a x b, ab, <a   b>, a, b
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;

   auto t1 = uc.NewTransformer();
   auto CommentRegex = R"(/\*([\s\S]*?)\*/)";
   auto WhitespaceToken = t1.Tokens().Add(CommentRegex, TokenType::Whitespace);
   WhitespaceToken.Description("Treats text between /* and */ as whitespace");

   auto txt = "a b, a /* comment a b */ b, a x b, ab, a   b, a, b";

   auto t2 = uc.NewTransformer();
   t2.FromTo("a b", "<{@Self}>");
   t2.Transform(txt);
   cout << t2 << endl;

   // The token defined in the other transformer (WhitespaceToken) is imported into this one.
   // Now, everything between /* and */ will be treated as whitespace
   t2.Tokens().Add(WhitespaceToken);
   cout << t2.Transform(txt) << endl;
}
				
			
<a b>, a /* comment <a b> */ b, a x b, ab, <a   b>, a, b
<a b>, <a /* comment a b */ b>, a x b, ab, <a   b>, a, b
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      
      Dim t1 = uc.NewTransformer()
      Dim CommentRegex = "/\*([\s\S]*?)\*/"
      Dim WhitespaceToken = t1.Tokens.Add(CommentRegex, TokenType.Whitespace)
      WhitespaceToken.Description = "Treats text between /* and */ as whitespace"
      
      Dim txt = "a b, a /* comment a b */ b, a x b, ab, a   b, a, b"
      
      Dim t2 = uc.NewTransformer()
      t2.FromTo("a b", "<{@Self}>")
      t2.Transform(txt)
      Console.WriteLine(t2)
      
      '// The token defined in the other transformer (WhitespaceToken) is imported into this one.
      '// Now, everything between /* and */ will be treated as whitespace
      t2.Tokens.Add(WhitespaceToken)
      Console.WriteLine(t2.Transform(txt))
   End Sub
End Module
				
			
<a b>, a /* comment <a b> */ b, a x b, ab, <a   b>, a, b
<a b>, <a /* comment a b */ b>, a x b, ab, <a   b>, a, b
Defining quoted text 
				
					using uCalcSoftware;

var uc = new uCalc();

// In example we'll define quoted text using < and > as
// surrounding quotes.  Singe and double quotes ' and "
// are already defined by default.  This is just an example.

// Quoted text must be defined as TokenType.Literal
// The last argument, 1, means that the literal part of the match will
// be the part of the regex found int the first parenthesis (here's
// there's only one set of parenthesis).

var t = uc.NewTransformer();
var SpecialQuotes = t.Tokens.Add("<([^>]*)>", TokenType.Literal, "", 1);
SpecialQuotes.SetDataType(BuiltInType.String);
SpecialQuotes.IsProperty(ItemIs.QuotedText, true);

t.Pattern("{token:1}");
Console.WriteLine(t.Filter("abc <some quoted text> xyz 123.456 + 25e2").Matches.Text);
Console.WriteLine("");

// Based on the definition, the part within < and > is the literal part
// passed to the string + operator used by EvalStr

uc.ExpressionTokens.Add(SpecialQuotes);
Console.WriteLine(uc.EvalStr("<some quoted text> + < plus more>"));
				
			
abc
<some quoted text>
xyz
123.456
+
25e2

some quoted text plus more
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;

   // In example we'll define quoted text using < and > as
   // surrounding quotes.  Singe and double quotes ' and "
   // are already defined by default.  This is just an example.

   // Quoted text must be defined as TokenType.Literal
   // The last argument, 1, means that the literal part of the match will
   // be the part of the regex found int the first parenthesis (here's
   // there's only one set of parenthesis).

   auto t = uc.NewTransformer();
   auto SpecialQuotes = t.Tokens().Add("<([^>]*)>", TokenType::Literal, "", 1);
   SpecialQuotes.SetDataType(BuiltInType::String);
   SpecialQuotes.IsProperty(ItemIs::QuotedText, true);

   t.Pattern("{token:1}");
   cout << t.Filter("abc <some quoted text> xyz 123.456 + 25e2").Matches().Text() << endl;
   cout << "" << endl;

   // Based on the definition, the part within < and > is the literal part
   // passed to the string + operator used by EvalStr

   uc.ExpressionTokens().Add(SpecialQuotes);
   cout << uc.EvalStr("<some quoted text> + < plus more>") << endl;
}
				
			
abc
<some quoted text>
xyz
123.456
+
25e2

some quoted text plus more
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      
      '// In example we'll define quoted text using < and > as
      '// surrounding quotes.  Singe and double quotes ' and " 
      '// are already defined by default.  This is just an example.
      
      '// Quoted text must be defined as TokenType.Literal
      '// The last argument, 1, means that the literal part of the match will
      '// be the part of the regex found int the first parenthesis (here's
      '// there's only one set of parenthesis).
      
      Dim t = uc.NewTransformer()
      Dim SpecialQuotes = t.Tokens.Add("<([^>]*)>", TokenType.Literal, "", 1)
      SpecialQuotes.SetDataType(BuiltInType.String)
      SpecialQuotes.IsProperty(ItemIs.QuotedText, true)
      
      t.Pattern("{token:1}")
      Console.WriteLine(t.Filter("abc <some quoted text> xyz 123.456 + 25e2").Matches.Text)
      Console.WriteLine("")
      
      '// Based on the definition, the part within < and > is the literal part
      '// passed to the string + operator used by EvalStr
      
      uc.ExpressionTokens.Add(SpecialQuotes)
      Console.WriteLine(uc.EvalStr("<some quoted text> + < plus more>"))
   End Sub
End Module
				
			
abc
<some quoted text>
xyz
123.456
+
25e2

some quoted text plus more
Previous
Next