uCalc API Version: 2.1.3-preview.2 Released: 6/16/2026

Warning

uCalc API Preview Release Notice:The documentation describes the intended behavior of the API. The current preview build contains incomplete features, unoptimized performance, and is subject to breaking changes.

ByName

Method

Product: 

Transformer Library

Class: 

Tokens

Retrieves a token definition by its name and can optionally re-categorize it in a single operation.

Syntax

ByName(string, TokenType)

Parameters

tokenName
string
The name of the token to retrieve (e.g., `_token_alphanumeric`).
newType
TokenType
(Default = TokenType::None)
Optional. The new lexical category to assign to the token. If `TokenType::None` (the default), the token's type is not changed.

Return

Item

Returns the Item object for the token. If the token is not found, an empty item is returned (for which IsEmpty() is true).

Remarks

The ByName method is a versatile tool for interacting with a Tokens collection. It serves a dual purpose: it retrieves a specific token definition by its unique name and can optionally re-categorize it in the same operation.

This method is the primary way to access built-in or user-defined tokens programmatically when you know their name (e.g., _token_alphanumeric). It is often used as a precursor to modifying a token's properties, such as its regex pattern or its lexical type.

⚙️ Dual Getter/Setter Behavior

The method's behavior depends on the newType parameter:

  • Getter Mode (Default): If newType is omitted or set to TokenType::None, ByName acts purely as a getter. It finds the token with the specified name and returns its corresponding Item object.

  • Setter Mode: If you provide a specific TokenType for newType, the method first finds the token and then immediately changes its lexical category to the new type. This is a convenient shortcut for tokens.ByName("...").TypeOfToken(newType).

🎯 Primary Use Cases

  1. Introspection: Retrieve a token to inspect its properties, such as its regex pattern via item.Regex().
  2. Dynamic Syntax Modification: Change the parser's behavior at runtime. A common example is re-categorizing the newline token (_token_newline) from a StatementSep to Whitespace to allow patterns to match across multiple lines.

💡 Why uCalc? (Comparative Analysis)

In traditional compiler tools like ANTLR or Flex/Bison, token definitions are static. They are defined in a grammar file, and changing a token's type requires modifying that file, regenerating parser code, and recompiling the application. This is a compile-time, static process.

uCalc's token system is entirely dynamic and programmatic. The ByName method is a key part of this dynamism. It allows your application to:

  • Modify Syntax at Runtime: Your code can re-categorize tokens on the fly, effectively switching between different language "dialects" without a restart.
  • Create Adaptable Parsers: Build tools that can be configured by the end-user to support different comment styles or literal formats.

This runtime flexibility provides a significant advantage over static parsing systems, enabling the creation of highly adaptable and user-configurable applications.

Examples

Retrieves a built-in token by its name and inspects its regex pattern.
				
					using uCalcSoftware;

var uc = new uCalc();
var tokens = uc.ExpressionTokens;
var alphaToken = tokens.ByName("_token_alphanumeric");
Console.WriteLine($"Alphanumeric Regex: {alphaToken.Regex}");
				
			
Alphanumeric Regex: [a-zA-Z_][a-zA-Z0-9_]*
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;
   auto tokens = uc.ExpressionTokens();
   auto alphaToken = tokens.ByName("_token_alphanumeric");
   cout << "Alphanumeric Regex: " << alphaToken.Regex() << endl;
}
				
			
Alphanumeric Regex: [a-zA-Z_][a-zA-Z0-9_]*
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      Dim tokens = uc.ExpressionTokens
      Dim alphaToken = tokens.ByName("_token_alphanumeric")
      Console.WriteLine($"Alphanumeric Regex: {alphaToken.Regex}")
   End Sub
End Module
				
			
Alphanumeric Regex: [a-zA-Z_][a-zA-Z0-9_]*
Dynamically re-categorizes the newline token to treat it as whitespace, allowing a pattern to match across multiple lines.
				
					using uCalcSoftware;

var uc = new uCalc();
var t = new uCalc.Transformer();
var source = """
<data>
  content spans
  multiple lines
</data>
""";
t.FromTo("<data>{body}</data>", "Body: [{body}]");

Console.WriteLine("--- Before: Newline is a Separator ---");
Console.WriteLine(t.Transform(source));

// Use ByName to find the newline token and change its type.
t.Tokens.ByName("_token_newline", TokenType.Whitespace);

Console.WriteLine("");
Console.WriteLine("--- After: Newline is Whitespace ---");
Console.WriteLine(t.Transform(source));
				
			
--- Before: Newline is a Separator ---
<data>
  content spans
  multiple lines
</data>

--- After: Newline is Whitespace ---
Body: [content spans
  multiple lines]
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;
   uCalc::Transformer t;
   auto source = R"(<data>
  content spans
  multiple lines
</data>)";
   t.FromTo("<data>{body}</data>", "Body: [{body}]");

   cout << "--- Before: Newline is a Separator ---" << endl;
   cout << t.Transform(source) << endl;

   // Use ByName to find the newline token and change its type.
   t.Tokens().ByName("_token_newline", TokenType::Whitespace);

   cout << "" << endl;
   cout << "--- After: Newline is Whitespace ---" << endl;
   cout << t.Transform(source) << endl;
}
				
			
--- Before: Newline is a Separator ---
<data>
  content spans
  multiple lines
</data>

--- After: Newline is Whitespace ---
Body: [content spans
  multiple lines]
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      Dim t As New uCalc.Transformer()
      Dim source = "<data>
  content spans
  multiple lines
</data>"
      t.FromTo("<data>{body}</data>", "Body: [{body}]")
      
      Console.WriteLine("--- Before: Newline is a Separator ---")
      Console.WriteLine(t.Transform(source))
      
      '// Use ByName to find the newline token and change its type.
      t.Tokens.ByName("_token_newline", TokenType.Whitespace)
      
      Console.WriteLine("")
      Console.WriteLine("--- After: Newline is Whitespace ---")
      Console.WriteLine(t.Transform(source))
   End Sub
End Module
				
			
--- Before: Newline is a Separator ---
<data>
  content spans
  multiple lines
</data>

--- After: Newline is Whitespace ---
Body: [content spans
  multiple lines]
Token Remove
				
					using uCalcSoftware;

var uc = new uCalc();
// This example removes the single quote pattern from the list

var t = uc.NewTransformer();
var txt = "This is a test, 'This is a test'";
t.FromTo("{token:1}", "<{@Self}>");
Console.WriteLine(t.Transform(txt).Text);

// Now the ' character will no longer be special
// (see the Name() example for list of token names
t.Tokens.Remove(t.Tokens["_token_string_singlequoted"]);
Console.WriteLine(t.Transform(txt).Text);
				
			
<This> <is> <a> <test><,> <'This is a test'>
<This> <is> <a> <test><,> <'><This> <is> <a> <test><'>
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;
   // This example removes the single quote pattern from the list

   auto t = uc.NewTransformer();
   auto txt = "This is a test, 'This is a test'";
   t.FromTo("{token:1}", "<{@Self}>");
   cout << t.Transform(txt).Text() << endl;

   // Now the ' character will no longer be special
   // (see the Name() example for list of token names
   t.Tokens().Remove(t.Tokens()["_token_string_singlequoted"]);
   cout << t.Transform(txt).Text() << endl;
}
				
			
<This> <is> <a> <test><,> <'This is a test'>
<This> <is> <a> <test><,> <'><This> <is> <a> <test><'>
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      '// This example removes the single quote pattern from the list
      
      Dim t = uc.NewTransformer()
      Dim txt = "This is a test, 'This is a test'"
      t.FromTo("{token:1}", "<{@Self}>")
      Console.WriteLine(t.Transform(txt).Text)
      
      '// Now the ' character will no longer be special
      '// (see the Name() example for list of token names
      t.Tokens.Remove(t.Tokens("_token_string_singlequoted"))
      Console.WriteLine(t.Transform(txt).Text)
   End Sub
End Module
				
			
<This> <is> <a> <test><,> <'This is a test'>
<This> <is> <a> <test><,> <'><This> <is> <a> <test><'>
Change newline from statement separator to whitespace
				
					using uCalcSoftware;

var uc = new uCalc();
var t = uc.NewTransformer();

t.Str("""

<div>a b c</div>
<div>
x
y
z
</div>
<div>1 2 3</div>

""");

t.Pattern("<div>{body}</div>");

Console.WriteLine("Newline as statement separator (default)");
Console.WriteLine("----------------------------------------");
Console.WriteLine(t.Find().Matches.Text);
Console.WriteLine("");

Console.WriteLine("Newline as whitespace");
Console.WriteLine("---------------------");
t.Tokens["_token_newline"].TypeOfToken = TokenType.Whitespace;
Console.WriteLine(t.Find().Matches.Text);
				
			
Newline as statement separator (default)
----------------------------------------
<div>a b c</div>
<div>1 2 3</div>

Newline as whitespace
---------------------
<div>a b c</div>
<div>
x
y
z
</div>
<div>1 2 3</div>
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;
   auto t = uc.NewTransformer();

   t.Str(R"(
<div>a b c</div>
<div>
x
y
z
</div>
<div>1 2 3</div>
)");

   t.Pattern("<div>{body}</div>");

   cout << "Newline as statement separator (default)" << endl;
   cout << "----------------------------------------" << endl;
   cout << t.Find().Matches().Text() << endl;
   cout << "" << endl;

   cout << "Newline as whitespace" << endl;
   cout << "---------------------" << endl;
   t.Tokens()["_token_newline"].TypeOfToken(TokenType::Whitespace);
   cout << t.Find().Matches().Text() << endl;
}
				
			
Newline as statement separator (default)
----------------------------------------
<div>a b c</div>
<div>1 2 3</div>

Newline as whitespace
---------------------
<div>a b c</div>
<div>
x
y
z
</div>
<div>1 2 3</div>
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      Dim t = uc.NewTransformer()
      
      t.Str("
<div>a b c</div>
<div>
x
y
z
</div>
<div>1 2 3</div>
")
      
      t.Pattern("<div>{body}</div>")
      
      Console.WriteLine("Newline as statement separator (default)")
      Console.WriteLine("----------------------------------------")
      Console.WriteLine(t.Find().Matches.Text)
      Console.WriteLine("")
      
      Console.WriteLine("Newline as whitespace")
      Console.WriteLine("---------------------")
      t.Tokens("_token_newline").TypeOfToken = TokenType.Whitespace
      Console.WriteLine(t.Find().Matches.Text)
   End Sub
End Module
				
			
Newline as statement separator (default)
----------------------------------------
<div>a b c</div>
<div>1 2 3</div>

Newline as whitespace
---------------------
<div>a b c</div>
<div>
x
y
z
</div>
<div>1 2 3</div>
Token(TokenType)
				
					using uCalcSoftware;

var uc = new uCalc();
Console.WriteLine(uc.ExpressionTokens[TokenType.AlphaNumeric].Name);
Console.WriteLine(uc.ExpressionTokens[TokenType.AlphaNumeric].Regex);
Console.WriteLine("");

// Note: In C# or VB you can simply use [TokenType::Literal, n]
// instead of ByType(TokenType::Literal, n)
Console.WriteLine(uc.ExpressionTokens.ByType(TokenType.Literal, 0).Name);
Console.WriteLine(uc.ExpressionTokens.ByType(TokenType.Literal, 1).Name);
Console.WriteLine(uc.ExpressionTokens.ByType(TokenType.Literal, 2).Name);
				
			
_token_alphanumeric
[a-zA-Z_][a-zA-Z0-9_]*

_token_string_singlequoted
_token_string_doublequoted
_token_string_tripledoublequoted
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;
   cout << uc.ExpressionTokens()[TokenType::AlphaNumeric].Name() << endl;
   cout << uc.ExpressionTokens()[TokenType::AlphaNumeric].Regex() << endl;
   cout << "" << endl;

   // Note: In C# or VB you can simply use [TokenType::Literal, n]
   // instead of ByType(TokenType::Literal, n)
   cout << uc.ExpressionTokens().ByType(TokenType::Literal, 0).Name() << endl;
   cout << uc.ExpressionTokens().ByType(TokenType::Literal, 1).Name() << endl;
   cout << uc.ExpressionTokens().ByType(TokenType::Literal, 2).Name() << endl;
}
				
			
_token_alphanumeric
[a-zA-Z_][a-zA-Z0-9_]*

_token_string_singlequoted
_token_string_doublequoted
_token_string_tripledoublequoted
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      Console.WriteLine(uc.ExpressionTokens(TokenType.AlphaNumeric).Name)
      Console.WriteLine(uc.ExpressionTokens(TokenType.AlphaNumeric).Regex)
      Console.WriteLine("")
      
      '// Note: In C# or VB you can simply use [TokenType::Literal, n]
      '// instead of ByType(TokenType::Literal, n)
      Console.WriteLine(uc.ExpressionTokens.ByType(TokenType.Literal, 0).Name)
      Console.WriteLine(uc.ExpressionTokens.ByType(TokenType.Literal, 1).Name)
      Console.WriteLine(uc.ExpressionTokens.ByType(TokenType.Literal, 2).Name)
   End Sub
End Module
				
			
_token_alphanumeric
[a-zA-Z_][a-zA-Z0-9_]*

_token_string_singlequoted
_token_string_doublequoted
_token_string_tripledoublequoted
Previous
Next