uCalc API Version: 2.1.3-preview.2 Released: 6/17/2026

Warning

uCalc API Preview Release Notice:The documentation describes the intended behavior of the API. The current preview build contains incomplete features, unoptimized performance, and is subject to breaking changes.

Add(string, TokenType, string, int, RegExGrammar, int)

Method

Product: 

Transformer Library

Class: 

Tokens

Defines a new lexical token using a regular expression and adds it to the collection.

Syntax

Add(string, TokenType, string, int, RegExGrammar, int)

Parameters

regexPattern
string
The regular expression pattern that defines the token.
tokenType
TokenType
(Default = TokenType::Generic)
The lexical category of the token, which determines its parsing behavior.
closingBracketRegex
string
(Default = "")
An optional regex pattern for the corresponding closing bracket if this token is an opening bracket.
subMatchGroup
int
(Default = 0)
The 1-based index of a capture group within the regex. If greater than 0, the content of this group is used as the token's value instead of the entire match.
regexGrammar
RegExGrammar
(Default = RegExGrammar::Default)
The regular expression grammar to use for parsing the pattern.
insertionIndex
int
(Default = -1)
The position in the token list to insert the new definition. The default (-1) adds it to the end, giving it the highest precedence. Use 0 to give it the lowest precedence.

Return

Item

Returns the newly created token as an Item object, which can be used to further configure its properties (e.g., setting its DataType).

Remarks

The Add method is the core of uCalc's dynamic lexical analysis engine. It allows you to define new tokens—the fundamental building blocks of a language—at runtime using regular expressions.

⚙️ Core Concepts

Token Precedence & Insertion

By default, tokens are evaluated in a Last-In, First-Out (LIFO) order. The most recently added token is checked first, giving it the highest precedence. You can override this behavior using the insertionIndex parameter:

  • -1 (Default): Appends the token, giving it the highest precedence.
  • 0: Inserts the token at the beginning, giving it the lowest precedence.

Token Types (tokenType)

A token's tokenType dictates its role in the parser. For a full list, see the TokenType enum.

TypeBehaviorExample
GenericA standard symbol with no special behavior..
LiteralRepresents a data value like a number or string.The regex [0-9]+ to match numbers.
WhitespaceIgnored by the parser.The regex [ \t]+ to skip spaces.
BracketMarks the start of a nested group.The regex \( for an opening parenthesis.

Specialized Token Definitions

Defining Bracket Pairs

To define a bracket pair (e.g., < and >), provide the opening bracket's regex as regexPattern and the closing bracket's regex in the closingBracketRegex parameter. This allows patterns with BracketSensitive enabled to correctly handle nested structures.

Defining Literals (Numbers, Strings, etc.)

Creating a token that can be used in an expression (like a number or string) is a multi-step process:

  1. Call Add() with tokenType set to TokenType::Literal.
  2. On the returned Item object, call the .DataType() method to assign a uCalc DataType (e.g., uc.DataTypeOf("Double")).
  3. For quoted string literals, you must also set the ItemIs::QuotedText property to true on the Item.

Capturing Sub-Matches (subMatchGroup)

If your regex contains capture groups (parentheses), you can use the subMatchGroup parameter to make the token's value equal to a specific group's content instead of the entire match. For example, to create a string literal token that automatically strips the surrounding quotes, you could use the pattern "([^"]*)" and set subMatchGroup to 1.

💡 Why uCalc? (Comparative Analysis)

  • vs. Static Lexer Generators (ANTLR, Flex/Bison): Traditional compiler tools require you to define token rules in separate grammar files, generate source code, and recompile your application to see changes. This process is static and happens at compile-time. uCalc's approach is fully programmatic and dynamic. You can add, remove, or modify token definitions at runtime, from within your code, without any external tools or recompilation. This provides unparalleled flexibility for creating adaptable and user-extendable Domain-Specific Languages (DSLs).

  • vs. Ad-hoc Regex & String Splitting: Manually tokenizing an input string is brittle and often fails to handle language context correctly (e.g., distinguishing an operator from a character inside a string). uCalc provides a robust, structured framework for tokenization, and the Add method is your entry point for configuring that framework.

Examples

Succinct: Defines a C-style line comment token (`//...`) and categorizes it as whitespace so it is ignored by the parser.
				
					using uCalcSoftware;

var uc = new uCalc();
var t = uc.NewTransformer();
// By default, a comment would cause a syntax error.
Console.Write("Before: ");
Console.WriteLine(uc.EvalStr("10 + 5 // Add 5"));

// Add a new token definition for C-style comments.
// The regex `//.*` matches from '//' to the end of the line.
// We classify it as Whitespace so the parser skips it.
uc.ExpressionTokens.Add("//.*", TokenType.Whitespace);

Console.Write("After:  ");
Console.WriteLine(uc.EvalStr("10 + 5 // Add 5"));
				
			
Before: Undefined identifier
After:  15
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;
   auto t = uc.NewTransformer();
   // By default, a comment would cause a syntax error.
   cout << "Before: ";
   cout << uc.EvalStr("10 + 5 // Add 5") << endl;

   // Add a new token definition for C-style comments.
   // The regex `//.*` matches from '//' to the end of the line.
   // We classify it as Whitespace so the parser skips it.
   uc.ExpressionTokens().Add("//.*", TokenType::Whitespace);

   cout << "After:  ";
   cout << uc.EvalStr("10 + 5 // Add 5") << endl;
}
				
			
Before: Undefined identifier
After:  15
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      Dim t = uc.NewTransformer()
      '// By default, a comment would cause a syntax error.
      Console.Write("Before: ")
      Console.WriteLine(uc.EvalStr("10 + 5 // Add 5"))
      
      '// Add a new token definition for C-style comments.
      '// The regex `//.*` matches from '//' to the end of the line.
      '// We classify it as Whitespace so the parser skips it.
      uc.ExpressionTokens.Add("//.*", TokenType.Whitespace)
      
      Console.Write("After:  ")
      Console.WriteLine(uc.EvalStr("10 + 5 // Add 5"))
   End Sub
End Module
				
			
Before: Undefined identifier
After:  15
Extends the parser to support C-style `0x` hex and `0b` binary notations using a token transformer.
				
					using uCalcSoftware;

var uc = new uCalc();
// Define a token for C-like 0x hex notation
uc.ExpressionTokens.Add("0x[0-9A-Fa-f]+", TokenType.TokenTransform);
uc.TokenTransformer.FromTo("{'0x'}{Num:'[0-9A-Fa-f]+'}", "BaseConvert('{Num}', 16)");
Console.WriteLine($"0xFF is evaluated as: {uc.EvalStr("0xFF")}");

// Define a token for C++-style 0b binary notation
uc.ExpressionTokens.Add("0b[01]+", TokenType.TokenTransform);

// Using {@Eval} is more efficient as the conversion happens once during the token transform pass.
uc.TokenTransformer.FromTo("{'0b'}{Num:'[01]+'}", "{@Eval: BaseConvert(Num, 2)}");
Console.WriteLine($"0b1011 is evaluated as: {uc.EvalStr("0b1011")}");

// Note: uCalc has built-in support for hex/binary using # notation (e.g., #hFF, #b1011)
Console.WriteLine($"uCalc's built-in #hFF is: {uc.EvalStr("#hFF")}");
				
			
0xFF is evaluated as: 255
0b1011 is evaluated as: 11
uCalc's built-in #hFF is: 255
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;
   // Define a token for C-like 0x hex notation
   uc.ExpressionTokens().Add("0x[0-9A-Fa-f]+", TokenType::TokenTransform);
   uc.TokenTransformer().FromTo("{'0x'}{Num:'[0-9A-Fa-f]+'}", "BaseConvert('{Num}', 16)");
   cout << "0xFF is evaluated as: " << uc.EvalStr("0xFF") << endl;

   // Define a token for C++-style 0b binary notation
   uc.ExpressionTokens().Add("0b[01]+", TokenType::TokenTransform);

   // Using {@Eval} is more efficient as the conversion happens once during the token transform pass.
   uc.TokenTransformer().FromTo("{'0b'}{Num:'[01]+'}", "{@Eval: BaseConvert(Num, 2)}");
   cout << "0b1011 is evaluated as: " << uc.EvalStr("0b1011") << endl;

   // Note: uCalc has built-in support for hex/binary using # notation (e.g., #hFF, #b1011)
   cout << "uCalc's built-in #hFF is: " << uc.EvalStr("#hFF") << endl;
}
				
			
0xFF is evaluated as: 255
0b1011 is evaluated as: 11
uCalc's built-in #hFF is: 255
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      '// Define a token for C-like 0x hex notation
      uc.ExpressionTokens.Add("0x[0-9A-Fa-f]+", TokenType.TokenTransform)
      uc.TokenTransformer.FromTo("{'0x'}{Num:'[0-9A-Fa-f]+'}", "BaseConvert('{Num}', 16)")
      Console.WriteLine($"0xFF is evaluated as: {uc.EvalStr("0xFF")}")
      
      '// Define a token for C++-style 0b binary notation
      uc.ExpressionTokens.Add("0b[01]+", TokenType.TokenTransform)
      
      '// Using {@Eval} is more efficient as the conversion happens once during the token transform pass.
      uc.TokenTransformer.FromTo("{'0b'}{Num:'[01]+'}", "{@Eval: BaseConvert(Num, 2)}")
      Console.WriteLine($"0b1011 is evaluated as: {uc.EvalStr("0b1011")}")
      
      '// Note: uCalc has built-in support for hex/binary using # notation (e.g., #hFF, #b1011)
      Console.WriteLine($"uCalc's built-in #hFF is: {uc.EvalStr("#hFF")}")
   End Sub
End Module
				
			
0xFF is evaluated as: 255
0b1011 is evaluated as: 11
uCalc's built-in #hFF is: 255
Internal Test: Verifies that the `subMatchGroup` parameter correctly extracts a capture group's content and that `ItemIs::QuotedText` works as expected.
				
					using uCalcSoftware;

var uc = new uCalc();
var t = uc.NewTransformer();

// Define a custom string literal using pipe characters `|...|`.
// The regex `\|([^\|]*)\|` captures the inner content in group 1.
// We set `subMatchGroup` to 1 to use this group as the token's value.
var pipeStringToken = t.Tokens.Add("""
\|([^\|]*)\|
""", TokenType.Literal, "", 1);

// Complete the definition by setting the data type and QuotedText property.
pipeStringToken.DataType = uc.DataTypeOf("String");
pipeStringToken.IsProperty(ItemIs.QuotedText, true);

// Define a simple rule to prove the token works.
t.FromTo("Test: {@String:s}", "Found: {s(0)} | Content: {s(1)}");

// The transformer now recognizes |...| as a string literal.
Console.WriteLine(t.Transform("Test: |hello|"));
				
			
Found: |hello| | Content: hello
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;
   auto t = uc.NewTransformer();

   // Define a custom string literal using pipe characters `|...|`.
   // The regex `\|([^\|]*)\|` captures the inner content in group 1.
   // We set `subMatchGroup` to 1 to use this group as the token's value.
   auto pipeStringToken = t.Tokens().Add(R"(\|([^\|]*)\|)", TokenType::Literal, "", 1);

   // Complete the definition by setting the data type and QuotedText property.
   pipeStringToken.DataType(uc.DataTypeOf("String"));
   pipeStringToken.IsProperty(ItemIs::QuotedText, true);

   // Define a simple rule to prove the token works.
   t.FromTo("Test: {@String:s}", "Found: {s(0)} | Content: {s(1)}");

   // The transformer now recognizes |...| as a string literal.
   cout << t.Transform("Test: |hello|") << endl;
}
				
			
Found: |hello| | Content: hello
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      Dim t = uc.NewTransformer()
      
      '// Define a custom string literal using pipe characters `|...|`.
      '// The regex `\|([^\|]*)\|` captures the inner content in group 1.
      '// We set `subMatchGroup` to 1 to use this group as the token's value.
      Dim pipeStringToken = t.Tokens.Add("\|([^\|]*)\|", TokenType.Literal, "", 1)
      
      '// Complete the definition by setting the data type and QuotedText property.
      pipeStringToken.DataType = uc.DataTypeOf("String")
      pipeStringToken.IsProperty(ItemIs.QuotedText, true)
      
      '// Define a simple rule to prove the token works.
      t.FromTo("Test: {@String:s}", "Found: {s(0)} | Content: {s(1)}")
      
      '// The transformer now recognizes |...| as a string literal.
      Console.WriteLine(t.Transform("Test: |hello|"))
   End Sub
End Module
				
			
Found: |hello| | Content: hello
Transformer: Matching by tokens vs match by character; also whitespace sensitivity
				
					using uCalcSoftware;

var uc = new uCalc();
// This examples shows the default match by
// token mode, as well as how to reconfigure
// it in order to do match by character
// along with a whitespace variation

var t = uc.NewTransformer();
var txt = "This is an island test, I said.";
t.FromTo("is", "<is>");

Console.WriteLine(t.Transform(txt));
Console.WriteLine("");

t.Tokens.Description = "Match by character";
t.Tokens.Add("."); // This overrides existing tokens
t.FromTo("is", "<is>");
Console.WriteLine(t.Tokens.Description);
Console.WriteLine(t.Transform(txt));
Console.WriteLine("");

// Note: whitespace sensitivity is off by default
// Whitespace token is re-introduced
// (after being overridden in the previous Token Add())
t.Tokens.Description = "By char + whitespace ignored";
t.Tokens.Add("[\\t\\v ]+", TokenType.Whitespace);
t.FromTo("is", "<{@Self}>");
Console.WriteLine(t.Tokens.Description);
Console.WriteLine(t.Transform(txt));
				
			
This <is> an island test, I said.

Match by character
Th<is> <is> an <is>land test, I said.

By char + whitespace ignored
Th<is> <is> an <is>land test, <I s>aid.
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;
   // This examples shows the default match by
   // token mode, as well as how to reconfigure
   // it in order to do match by character
   // along with a whitespace variation

   auto t = uc.NewTransformer();
   auto txt = "This is an island test, I said.";
   t.FromTo("is", "<is>");

   cout << t.Transform(txt) << endl;
   cout << "" << endl;

   t.Tokens().Description("Match by character");
   t.Tokens().Add("."); // This overrides existing tokens
   t.FromTo("is", "<is>");
   cout << t.Tokens().Description() << endl;
   cout << t.Transform(txt) << endl;
   cout << "" << endl;

   // Note: whitespace sensitivity is off by default
   // Whitespace token is re-introduced
   // (after being overridden in the previous Token Add())
   t.Tokens().Description("By char + whitespace ignored");
   t.Tokens().Add("[\\t\\v ]+", TokenType::Whitespace);
   t.FromTo("is", "<{@Self}>");
   cout << t.Tokens().Description() << endl;
   cout << t.Transform(txt) << endl;
}
				
			
This <is> an island test, I said.

Match by character
Th<is> <is> an <is>land test, I said.

By char + whitespace ignored
Th<is> <is> an <is>land test, <I s>aid.
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      '// This examples shows the default match by
      '// token mode, as well as how to reconfigure
      '// it in order to do match by character
      '// along with a whitespace variation
      
      Dim t = uc.NewTransformer()
      Dim txt = "This is an island test, I said."
      t.FromTo("is", "<is>")
      
      Console.WriteLine(t.Transform(txt))
      Console.WriteLine("")
      
      t.Tokens.Description = "Match by character"
      t.Tokens.Add(".") '// This overrides existing tokens
      t.FromTo("is", "<is>")
      Console.WriteLine(t.Tokens.Description)
      Console.WriteLine(t.Transform(txt))
      Console.WriteLine("")
      
      '// Note: whitespace sensitivity is off by default
      '// Whitespace token is re-introduced
      '// (after being overridden in the previous Token Add())
      t.Tokens.Description = "By char + whitespace ignored"
      t.Tokens.Add("[\\t\\v ]+", TokenType.Whitespace)
      t.FromTo("is", "<{@Self}>")
      Console.WriteLine(t.Tokens.Description)
      Console.WriteLine(t.Transform(txt))
   End Sub
End Module
				
			
This <is> an island test, I said.

Match by character
Th<is> <is> an <is>land test, I said.

By char + whitespace ignored
Th<is> <is> an <is>land test, <I s>aid.
Defining a token bracket pair
				
					using uCalcSoftware;

var uc = new uCalc();

// Here we define < and > as a bracket pair.
// Such pairs can be part of a pattern match
// and a match can be found within a bracket pair
// but a match will not cross boundaries with one
// part of the match out and another part inside

var t = uc.NewTransformer();
var txt = "a < b c > c, < a > b c, < a b c >";

t.Tokens.Add("<", TokenType.Generic, ">");
t.FromTo("a {etc} c", "((a {etc} c))");

Console.WriteLine(txt);
Console.WriteLine(t.Transform(txt));
				
			
a < b c > c, < a > b c, < a b c >
((a < b c > c)), < a > b c, < ((a b c)) >
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;

   // Here we define < and > as a bracket pair.
   // Such pairs can be part of a pattern match
   // and a match can be found within a bracket pair
   // but a match will not cross boundaries with one
   // part of the match out and another part inside

   auto t = uc.NewTransformer();
   auto txt = "a < b c > c, < a > b c, < a b c >";

   t.Tokens().Add("<", TokenType::Generic, ">");
   t.FromTo("a {etc} c", "((a {etc} c))");

   cout << txt << endl;
   cout << t.Transform(txt) << endl;
}
				
			
a < b c > c, < a > b c, < a b c >
((a < b c > c)), < a > b c, < ((a b c)) >
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      
      '// Here we define < and > as a bracket pair.
      '// Such pairs can be part of a pattern match
      '// and a match can be found within a bracket pair
      '// but a match will not cross boundaries with one
      '// part of the match out and another part inside
      
      Dim t = uc.NewTransformer()
      Dim txt = "a < b c > c, < a > b c, < a b c >"
      
      t.Tokens.Add("<", TokenType.Generic, ">")
      t.FromTo("a {etc} c", "((a {etc} c))")
      
      Console.WriteLine(txt)
      Console.WriteLine(t.Transform(txt))
   End Sub
End Module
				
			
a < b c > c, < a > b c, < a b c >
((a < b c > c)), < a > b c, < ((a b c)) >
Defining quoted text 
				
					using uCalcSoftware;

var uc = new uCalc();

// In example we'll define quoted text using < and > as
// surrounding quotes.  Singe and double quotes ' and "
// are already defined by default.  This is just an example.

// Quoted text must be defined as TokenType.Literal
// The last argument, 1, means that the literal part of the match will
// be the part of the regex found int the first parenthesis (here's
// there's only one set of parenthesis).

var t = uc.NewTransformer();
var SpecialQuotes = t.Tokens.Add("<([^>]*)>", TokenType.Literal, "", 1);
SpecialQuotes.SetDataType(BuiltInType.String);
SpecialQuotes.IsProperty(ItemIs.QuotedText, true);

t.Pattern("{token:1}");
Console.WriteLine(t.Filter("abc <some quoted text> xyz 123.456 + 25e2").Matches.Text);
Console.WriteLine("");

// Based on the definition, the part within < and > is the literal part
// passed to the string + operator used by EvalStr

uc.ExpressionTokens.Add(SpecialQuotes);
Console.WriteLine(uc.EvalStr("<some quoted text> + < plus more>"));
				
			
abc
<some quoted text>
xyz
123.456
+
25e2

some quoted text plus more
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;

   // In example we'll define quoted text using < and > as
   // surrounding quotes.  Singe and double quotes ' and "
   // are already defined by default.  This is just an example.

   // Quoted text must be defined as TokenType.Literal
   // The last argument, 1, means that the literal part of the match will
   // be the part of the regex found int the first parenthesis (here's
   // there's only one set of parenthesis).

   auto t = uc.NewTransformer();
   auto SpecialQuotes = t.Tokens().Add("<([^>]*)>", TokenType::Literal, "", 1);
   SpecialQuotes.SetDataType(BuiltInType::String);
   SpecialQuotes.IsProperty(ItemIs::QuotedText, true);

   t.Pattern("{token:1}");
   cout << t.Filter("abc <some quoted text> xyz 123.456 + 25e2").Matches().Text() << endl;
   cout << "" << endl;

   // Based on the definition, the part within < and > is the literal part
   // passed to the string + operator used by EvalStr

   uc.ExpressionTokens().Add(SpecialQuotes);
   cout << uc.EvalStr("<some quoted text> + < plus more>") << endl;
}
				
			
abc
<some quoted text>
xyz
123.456
+
25e2

some quoted text plus more
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      
      '// In example we'll define quoted text using < and > as
      '// surrounding quotes.  Singe and double quotes ' and " 
      '// are already defined by default.  This is just an example.
      
      '// Quoted text must be defined as TokenType.Literal
      '// The last argument, 1, means that the literal part of the match will
      '// be the part of the regex found int the first parenthesis (here's
      '// there's only one set of parenthesis).
      
      Dim t = uc.NewTransformer()
      Dim SpecialQuotes = t.Tokens.Add("<([^>]*)>", TokenType.Literal, "", 1)
      SpecialQuotes.SetDataType(BuiltInType.String)
      SpecialQuotes.IsProperty(ItemIs.QuotedText, true)
      
      t.Pattern("{token:1}")
      Console.WriteLine(t.Filter("abc <some quoted text> xyz 123.456 + 25e2").Matches.Text)
      Console.WriteLine("")
      
      '// Based on the definition, the part within < and > is the literal part
      '// passed to the string + operator used by EvalStr
      
      uc.ExpressionTokens.Add(SpecialQuotes)
      Console.WriteLine(uc.EvalStr("<some quoted text> + < plus more>"))
   End Sub
End Module
				
			
abc
<some quoted text>
xyz
123.456
+
25e2

some quoted text plus more
Token Add(ExistingToken)
				
					using uCalcSoftware;

var uc = new uCalc();

var t1 = uc.NewTransformer();
var CommentRegex = """
/\*([\s\S]*?)\*/
""";
var WhitespaceToken = t1.Tokens.Add(CommentRegex, TokenType.Whitespace);
WhitespaceToken.Description = "Treats text between /* and */ as whitespace";

var txt = "a b, a /* comment a b */ b, a x b, ab, a   b, a, b";

var t2 = uc.NewTransformer();
t2.FromTo("a b", "<{@Self}>");
t2.Transform(txt);
Console.WriteLine(t2);

// The token defined in the other transformer (WhitespaceToken) is imported into this one.
// Now, everything between /* and */ will be treated as whitespace
t2.Tokens.Add(WhitespaceToken);
Console.WriteLine(t2.Transform(txt));
				
			
<a b>, a /* comment <a b> */ b, a x b, ab, <a   b>, a, b
<a b>, <a /* comment a b */ b>, a x b, ab, <a   b>, a, b
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;

   auto t1 = uc.NewTransformer();
   auto CommentRegex = R"(/\*([\s\S]*?)\*/)";
   auto WhitespaceToken = t1.Tokens().Add(CommentRegex, TokenType::Whitespace);
   WhitespaceToken.Description("Treats text between /* and */ as whitespace");

   auto txt = "a b, a /* comment a b */ b, a x b, ab, a   b, a, b";

   auto t2 = uc.NewTransformer();
   t2.FromTo("a b", "<{@Self}>");
   t2.Transform(txt);
   cout << t2 << endl;

   // The token defined in the other transformer (WhitespaceToken) is imported into this one.
   // Now, everything between /* and */ will be treated as whitespace
   t2.Tokens().Add(WhitespaceToken);
   cout << t2.Transform(txt) << endl;
}
				
			
<a b>, a /* comment <a b> */ b, a x b, ab, <a   b>, a, b
<a b>, <a /* comment a b */ b>, a x b, ab, <a   b>, a, b
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      
      Dim t1 = uc.NewTransformer()
      Dim CommentRegex = "/\*([\s\S]*?)\*/"
      Dim WhitespaceToken = t1.Tokens.Add(CommentRegex, TokenType.Whitespace)
      WhitespaceToken.Description = "Treats text between /* and */ as whitespace"
      
      Dim txt = "a b, a /* comment a b */ b, a x b, ab, a   b, a, b"
      
      Dim t2 = uc.NewTransformer()
      t2.FromTo("a b", "<{@Self}>")
      t2.Transform(txt)
      Console.WriteLine(t2)
      
      '// The token defined in the other transformer (WhitespaceToken) is imported into this one.
      '// Now, everything between /* and */ will be treated as whitespace
      t2.Tokens.Add(WhitespaceToken)
      Console.WriteLine(t2.Transform(txt))
   End Sub
End Module
				
			
<a b>, a /* comment <a b> */ b, a x b, ab, <a   b>, a, b
<a b>, <a /* comment a b */ b>, a x b, ab, <a   b>, a, b
Token Context switch
				
					using uCalcSoftware;

var uc = new uCalc();
var CommentTransform = uc.NewTransformer();
var CommentTokens = CommentTransform.Tokens;
CommentTokens.Add(".");
CommentTokens.Add("[a-z]+");

var txt = "'This is it' /* 'This is it' This is it */ This is it";

var t = uc.NewTransformer();
t.FromTo("is", "<is>");
Console.WriteLine(t.Transform(txt).Text);

// Now the context will switch between /* and */
// In that context there's no quoted text token,
t.Tokens.ContextSwitch(CommentTokens, "/\\*", "\\*/");
t.FromTo("is", "<is>");
Console.WriteLine(t.Transform(txt).Text);

				
			
'This is it' /* 'This is it' This <is> it */ This <is> it
'This is it' /* 'This <is> it' This <is> it */ This <is> it
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;
   auto CommentTransform = uc.NewTransformer();
   auto CommentTokens = CommentTransform.Tokens();
   CommentTokens.Add(".");
   CommentTokens.Add("[a-z]+");

   auto txt = "'This is it' /* 'This is it' This is it */ This is it";

   auto t = uc.NewTransformer();
   t.FromTo("is", "<is>");
   cout << t.Transform(txt).Text() << endl;

   // Now the context will switch between /* and */
   // In that context there's no quoted text token,
   t.Tokens().ContextSwitch(CommentTokens, "/\\*", "\\*/");
   t.FromTo("is", "<is>");
   cout << t.Transform(txt).Text() << endl;

}
				
			
'This is it' /* 'This is it' This <is> it */ This <is> it
'This is it' /* 'This <is> it' This <is> it */ This <is> it
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      Dim CommentTransform = uc.NewTransformer()
      Dim CommentTokens = CommentTransform.Tokens
      CommentTokens.Add(".")
      CommentTokens.Add("[a-z]+")
      
      Dim txt = "'This is it' /* 'This is it' This is it */ This is it"
      
      Dim t = uc.NewTransformer()
      t.FromTo("is", "<is>")
      Console.WriteLine(t.Transform(txt).Text)
      
      '// Now the context will switch between /* and */
      '// In that context there's no quoted text token,
      t.Tokens.ContextSwitch(CommentTokens, "/\*", "\*/")
      t.FromTo("is", "<is>")
      Console.WriteLine(t.Transform(txt).Text)
      
   End Sub
End Module
				
			
'This is it' /* 'This is it' This <is> it */ This <is> it
'This is it' /* 'This <is> it' This <is> it */ This <is> it
Turn new line into whitespace with Tokens.Add
				
					using uCalcSoftware;

var uc = new uCalc();
var t = uc.NewTransformer();
t.FromTo("<{tag}>{code}</{tag}>", "[{tag}]{code}[/{tag}]");
var MyStr = """
<div>Single line</div>
<div>Line 1
Line 2
Line 3
Line 4</div>
""";

Console.WriteLine("New line as statement separator (default)");
Console.WriteLine("");
Console.WriteLine(t.Transform(MyStr).Text);
Console.WriteLine("");

Console.WriteLine("New line as whitespace");
Console.WriteLine("");
t.Tokens.Add("[\\r\\n]+", TokenType.Whitespace);
Console.WriteLine(t.Transform(MyStr).Text);



				
			
New line as statement separator (default)

[div]Single line[/div]
<div>Line 1
Line 2
Line 3
Line 4</div>

New line as whitespace

[div]Single line[/div]
[div]Line 1
Line 2
Line 3
Line 4[/div]
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;
   auto t = uc.NewTransformer();
   t.FromTo("<{tag}>{code}</{tag}>", "[{tag}]{code}[/{tag}]");
   auto MyStr = R"(<div>Single line</div>
<div>Line 1
Line 2
Line 3
Line 4</div>)";

   cout << "New line as statement separator (default)" << endl;
   cout << "" << endl;
   cout << t.Transform(MyStr).Text() << endl;
   cout << "" << endl;

   cout << "New line as whitespace" << endl;
   cout << "" << endl;
   t.Tokens().Add("[\\r\\n]+", TokenType::Whitespace);
   cout << t.Transform(MyStr).Text() << endl;



}
				
			
New line as statement separator (default)

[div]Single line[/div]
<div>Line 1
Line 2
Line 3
Line 4</div>

New line as whitespace

[div]Single line[/div]
[div]Line 1
Line 2
Line 3
Line 4[/div]
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      Dim t = uc.NewTransformer()
      t.FromTo("<{tag}>{code}</{tag}>", "[{tag}]{code}[/{tag}]")
      Dim MyStr = "<div>Single line</div>
<div>Line 1
Line 2
Line 3
Line 4</div>"
      
      Console.WriteLine("New line as statement separator (default)")
      Console.WriteLine("")
      Console.WriteLine(t.Transform(MyStr).Text)
      Console.WriteLine("")
      
      Console.WriteLine("New line as whitespace")
      Console.WriteLine("")
      t.Tokens.Add("[\r\n]+", TokenType.Whitespace)
      Console.WriteLine(t.Transform(MyStr).Text)
      
      
      
   End Sub
End Module
				
			
New line as statement separator (default)

[div]Single line[/div]
<div>Line 1
Line 2
Line 3
Line 4</div>

New line as whitespace

[div]Single line[/div]
[div]Line 1
Line 2
Line 3
Line 4[/div]
Previous
Next