uCalc API Version: 2.1.3-preview.2 Released: 6/16/2026

Warning

uCalc API Preview Release Notice:The documentation describes the intended behavior of the API. The current preview build contains incomplete features, unoptimized performance, and is subject to breaking changes.

IndexOf

Method

Product: 

Transformer Library

Class: 

Tokens

Gets the zero-based index of a token definition, which determines its precedence in the matching order.

Syntax

IndexOf(Item)

Parameters

tokenItem
Item
The `Item` object representing the token definition whose index is to be found.

Return

int32

The zero-based index of the token definition in the collection, representing its precedence rank. Returns -1 if the token is not found.

Remarks

The IndexOf method finds the zero-based position of a specific token definition within the collection. This index is not just an arbitrary position; it directly corresponds to the token's precedence.

⚙️ Precedence: Higher Index Wins

uCalc's tokenizer evaluates token patterns in a Last-In, First-Out (LIFO) order. This means:

  • The most recently added token has the highest index (Count() - 1) and is checked first.
  • The first token added (or the oldest built-in token) has the lowest index (0) and is checked last.

A higher index means higher precedence. This method is the primary tool for programmatically querying the precedence rank of any token.

🎯 Use Cases

  • Debugging: Verify the order in which tokens are being evaluated to resolve matching conflicts.
  • Dynamic Analysis: Write code that can inspect a Tokens collection and understand its precedence hierarchy at runtime.
  • Advanced Configuration: Programmatically insert tokens at specific indices (using other overloads of Add or Insert) and then use IndexOf to confirm their final precedence.

If the specified token is not found in the collection, the method returns -1.

💡 Why uCalc? (Comparative Analysis)

  • vs. Static Lexers (ANTLR, Flex/Bison): In traditional parser generators, token precedence is often implicit, determined by the order of rules in a static grammar file. There is no simple, programmatic way to query a token's rank at runtime. uCalc's IndexOf makes precedence a first-class, inspectable property of its dynamic lexical model.

  • vs. Standard IndexOf: Unlike a typical IndexOf method that just finds an element's position in a list, this method's return value has direct semantic meaning for the parser's behavior. This makes it a powerful introspection tool rather than a simple collection utility.

Examples

A succinct demonstration of how adding tokens affects their index and, therefore, their precedence.
				
					using uCalcSoftware;

var uc = new uCalc();
var t = new uCalc.Transformer();
var tokens = t.Tokens;
tokens.Clear();
tokens.Add("."); // Add a fallback token at index 0

// The order of definition determines the index (precedence)
var tokenA = tokens.Add("A");
var tokenB = tokens.Add("B");

Console.WriteLine($"Index of A: {tokens.IndexOf(tokenA)}"); // Will have a lower index
Console.WriteLine($"Index of B: {tokens.IndexOf(tokenB)}"); // Will have a higher index, thus higher precedence
				
			
Index of A: 1
Index of B: 2
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;
   uCalc::Transformer t;
   auto tokens = t.Tokens();
   tokens.Clear();
   tokens.Add("."); // Add a fallback token at index 0

   // The order of definition determines the index (precedence)
   auto tokenA = tokens.Add("A");
   auto tokenB = tokens.Add("B");

   cout << "Index of A: " << tokens.IndexOf(tokenA) << endl; // Will have a lower index
   cout << "Index of B: " << tokens.IndexOf(tokenB) << endl; // Will have a higher index, thus higher precedence
}
				
			
Index of A: 1
Index of B: 2
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      Dim t As New uCalc.Transformer()
      Dim tokens = t.Tokens
      tokens.Clear()
      tokens.Add(".") '// Add a fallback token at index 0
      
      '// The order of definition determines the index (precedence)
      Dim tokenA = tokens.Add("A")
      Dim tokenB = tokens.Add("B")
      
      Console.WriteLine($"Index of A: {tokens.IndexOf(tokenA)}") '// Will have a lower index
      Console.WriteLine($"Index of B: {tokens.IndexOf(tokenB)}") '// Will have a higher index, thus higher precedence
   End Sub
End Module
				
			
Index of A: 1
Index of B: 2
Internal Test: Verifies the LIFO precedence order and the return value for a token not present in the collection.
				
					using uCalcSoftware;

var uc = new uCalc();
var t = new uCalc.Transformer();
var tokens = t.Tokens;
tokens.Clear();
tokens.Add("."); // Fallback

var tokenPlus = tokens.Add("[+]");
var tokenStar = tokens.Add("[*]");
var tokenCaret = tokens.Add("^");

// LIFO order means precedence is: '^' > '*' > '+'
Console.WriteLine("Precedence Check:");
Console.WriteLine($"Caret (^) > Star (*): {tokens.IndexOf(tokenCaret) > tokens.IndexOf(tokenStar)}");
Console.WriteLine($"Star (*) > Plus (+): {tokens.IndexOf(tokenStar) > tokens.IndexOf(tokenPlus)}");

// Test for a token not in this collection
var t2 = new uCalc.Transformer();
var unaddedToken = t2.Tokens.Add("unrelated");
Console.WriteLine($"Index of un-added token: {tokens.IndexOf(unaddedToken)}");
				
			
Precedence Check:
Caret (^) > Star (*): True
Star (*) > Plus (+): True
Index of un-added token: -1
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

#define tf(IsTrue) ((IsTrue) ? "True" : "False")

int main() {
   uCalc uc;
   uCalc::Transformer t;
   auto tokens = t.Tokens();
   tokens.Clear();
   tokens.Add("."); // Fallback

   auto tokenPlus = tokens.Add("[+]");
   auto tokenStar = tokens.Add("[*]");
   auto tokenCaret = tokens.Add("^");

   // LIFO order means precedence is: '^' > '*' > '+'
   cout << "Precedence Check:" << endl;
   cout << "Caret (^) > Star (*): " << tf(tokens.IndexOf(tokenCaret) > tokens.IndexOf(tokenStar)) << endl;
   cout << "Star (*) > Plus (+): " << tf(tokens.IndexOf(tokenStar) > tokens.IndexOf(tokenPlus)) << endl;

   // Test for a token not in this collection
   uCalc::Transformer t2;
   auto unaddedToken = t2.Tokens().Add("unrelated");
   cout << "Index of un-added token: " << tokens.IndexOf(unaddedToken) << endl;
}
				
			
Precedence Check:
Caret (^) > Star (*): True
Star (*) > Plus (+): True
Index of un-added token: -1
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      Dim t As New uCalc.Transformer()
      Dim tokens = t.Tokens
      tokens.Clear()
      tokens.Add(".") '// Fallback
      
      Dim tokenPlus = tokens.Add("[+]")
      Dim tokenStar = tokens.Add("[*]")
      Dim tokenCaret = tokens.Add("^")
      
      '// LIFO order means precedence is: '^' > '*' > '+'
      Console.WriteLine("Precedence Check:")
      Console.WriteLine($"Caret (^) > Star (*): {tokens.IndexOf(tokenCaret) > tokens.IndexOf(tokenStar)}")
      Console.WriteLine($"Star (*) > Plus (+): {tokens.IndexOf(tokenStar) > tokens.IndexOf(tokenPlus)}")
      
      '// Test for a token not in this collection
      Dim t2 As New uCalc.Transformer()
      Dim unaddedToken = t2.Tokens.Add("unrelated")
      Console.WriteLine($"Index of un-added token: {tokens.IndexOf(unaddedToken)}")
   End Sub
End Module
				
			
Precedence Check:
Caret (^) > Star (*): True
Star (*) > Plus (+): True
Index of un-added token: -1
Displays the complete list of default token definitions, showing their type, internal name, and regex pattern.
				
					using uCalcSoftware;

var uc = new uCalc();
// Lists all tokens currently defined for the expression evaluator.

Console.WriteLine($"Token Count: {uc.ExpressionTokens.Count}");
Console.WriteLine("");
Console.WriteLine("Index  Type  Name: regex");
Console.WriteLine("========================");
var Tokens = uc.ExpressionTokens;
foreach(var token in Tokens) {
   Console.Write(Tokens.IndexOf(token));
   Console.WriteLine($"  {token.Description}  {token.Name}: {token.Regex}");
}

// Note that the expression evaluator token list has a few extra tokens,
// related to hex/bin/oct notation, string interpolation, and imaginary number
// notation, which are not found in the default Transformer token list.
				
			
Token Count: 30

Index  Type  Name: regex
========================
0  generic  _token_line: .*
1  generic  _token_catchall: .
2  generic  _token_catchall_utf8_other: [\xf0-\xf7][\x80-\xbf][\x80-\xbf][\x80-\xbf]|[\xe0-\xef][\x80-\xbf][\x80-\xbf]|[\xc0-\xdf][\x80-\xbf]
3  generic  _token_punctuation: (--|\.{3}|\xE2\x80\xA6|[!"#$%&'()*+,\-./:;<=>?@\[\\\]^_`{|}~]|\xE2\x80[\x90-\x95])
4  generic  _token_quotechar: ("){3}|"|'
5  generic  _token_quotechar_single: '
6  generic  _token_quotechar_double: "
7  generic  _token_quotechar_tripledouble: """
8  memberaccess  _token_memberaccess: \.
9  generic  _token_variableargs: \.\.\.
10  reducible  _token_reducible2: [-:|+/*^&=%@!`\\<>?#$~]+
11  bracket  _token_parenthesis: \(
12  bracketclose  _token_parenthesis_close: \)
13  bracket  _token_curlybrace: \{
14  bracketclose  _token_curlybrace_close: \}
15  bracket  _token_squarebracket: \[
16  bracketclose  _token_squarebracket_close: \]
17  argseparator  _token_argseparator: ,
18  statementseparator  _token_newline: (?:\r?\n)|\r
19  statementseparator  _token_semicolon: ;
20  literal  _token_string_singlequoted: '([^']*(?:''[^']*)*)'
21  literal  _token_string_doublequoted: "([^"]*(?:""[^"]*)*)"
22  literal  _token_string_tripledoublequoted: """([\s\S]*?)"""
23  whitespace  _token_whitespace: [\t\v ]+
24  reducible  _token_reducible: [-:|+/*^&=%@!`\\<>?]+
25  literal  _token_floatnumber: [0-9]*\.?[0-9]+([eE][+-]?[0-9]+)?
26  alphanumeric  _token_alphanumeric: [a-zA-Z_][a-zA-Z0-9_]*
27  literal  _token_imaginaryunit: #i
28  tokentransform  _token_binaryhexoctalnotation: #[bho][0-9A-F]+
29  tokentransform  _token_stringinterpolationquote: \$['"]
				
					#include <iostream>
#include "uCalc.h"

using namespace std;
using namespace uCalcSoftware;

int main() {
   uCalc uc;
   // Lists all tokens currently defined for the expression evaluator.

   cout << "Token Count: " << uc.ExpressionTokens().Count() << endl;
   cout << "" << endl;
   cout << "Index  Type  Name: regex" << endl;
   cout << "========================" << endl;
   auto Tokens = uc.ExpressionTokens();
   for(auto token : Tokens) {
      cout << Tokens.IndexOf(token);
      cout << "  " << token.Description() << "  " << token.Name() << ": " << token.Regex() << endl;
   }

   // Note that the expression evaluator token list has a few extra tokens,
   // related to hex/bin/oct notation, string interpolation, and imaginary number
   // notation, which are not found in the default Transformer token list.
}
				
			
Token Count: 30

Index  Type  Name: regex
========================
0  generic  _token_line: .*
1  generic  _token_catchall: .
2  generic  _token_catchall_utf8_other: [\xf0-\xf7][\x80-\xbf][\x80-\xbf][\x80-\xbf]|[\xe0-\xef][\x80-\xbf][\x80-\xbf]|[\xc0-\xdf][\x80-\xbf]
3  generic  _token_punctuation: (--|\.{3}|\xE2\x80\xA6|[!"#$%&'()*+,\-./:;<=>?@\[\\\]^_`{|}~]|\xE2\x80[\x90-\x95])
4  generic  _token_quotechar: ("){3}|"|'
5  generic  _token_quotechar_single: '
6  generic  _token_quotechar_double: "
7  generic  _token_quotechar_tripledouble: """
8  memberaccess  _token_memberaccess: \.
9  generic  _token_variableargs: \.\.\.
10  reducible  _token_reducible2: [-:|+/*^&=%@!`\\<>?#$~]+
11  bracket  _token_parenthesis: \(
12  bracketclose  _token_parenthesis_close: \)
13  bracket  _token_curlybrace: \{
14  bracketclose  _token_curlybrace_close: \}
15  bracket  _token_squarebracket: \[
16  bracketclose  _token_squarebracket_close: \]
17  argseparator  _token_argseparator: ,
18  statementseparator  _token_newline: (?:\r?\n)|\r
19  statementseparator  _token_semicolon: ;
20  literal  _token_string_singlequoted: '([^']*(?:''[^']*)*)'
21  literal  _token_string_doublequoted: "([^"]*(?:""[^"]*)*)"
22  literal  _token_string_tripledoublequoted: """([\s\S]*?)"""
23  whitespace  _token_whitespace: [\t\v ]+
24  reducible  _token_reducible: [-:|+/*^&=%@!`\\<>?]+
25  literal  _token_floatnumber: [0-9]*\.?[0-9]+([eE][+-]?[0-9]+)?
26  alphanumeric  _token_alphanumeric: [a-zA-Z_][a-zA-Z0-9_]*
27  literal  _token_imaginaryunit: #i
28  tokentransform  _token_binaryhexoctalnotation: #[bho][0-9A-F]+
29  tokentransform  _token_stringinterpolationquote: \$['"]
				
					Imports System
Imports uCalcSoftware
Public Module Program
   Public Sub Main()
      Dim uc As New uCalc()
      '// Lists all tokens currently defined for the expression evaluator.
      
      Console.WriteLine($"Token Count: {uc.ExpressionTokens.Count}")
      Console.WriteLine("")
      Console.WriteLine("Index  Type  Name: regex")
      Console.WriteLine("========================")
      Dim Tokens = uc.ExpressionTokens
      For Each token In Tokens
         Console.Write(Tokens.IndexOf(token))
         Console.WriteLine($"  {token.Description}  {token.Name}: {token.Regex}")
      Next
      
      '// Note that the expression evaluator token list has a few extra tokens,
      '// related to hex/bin/oct notation, string interpolation, and imaginary number
      '// notation, which are not found in the default Transformer token list.
   End Sub
End Module
				
			
Token Count: 30

Index  Type  Name: regex
========================
0  generic  _token_line: .*
1  generic  _token_catchall: .
2  generic  _token_catchall_utf8_other: [\xf0-\xf7][\x80-\xbf][\x80-\xbf][\x80-\xbf]|[\xe0-\xef][\x80-\xbf][\x80-\xbf]|[\xc0-\xdf][\x80-\xbf]
3  generic  _token_punctuation: (--|\.{3}|\xE2\x80\xA6|[!"#$%&'()*+,\-./:;<=>?@\[\\\]^_`{|}~]|\xE2\x80[\x90-\x95])
4  generic  _token_quotechar: ("){3}|"|'
5  generic  _token_quotechar_single: '
6  generic  _token_quotechar_double: "
7  generic  _token_quotechar_tripledouble: """
8  memberaccess  _token_memberaccess: \.
9  generic  _token_variableargs: \.\.\.
10  reducible  _token_reducible2: [-:|+/*^&=%@!`\\<>?#$~]+
11  bracket  _token_parenthesis: \(
12  bracketclose  _token_parenthesis_close: \)
13  bracket  _token_curlybrace: \{
14  bracketclose  _token_curlybrace_close: \}
15  bracket  _token_squarebracket: \[
16  bracketclose  _token_squarebracket_close: \]
17  argseparator  _token_argseparator: ,
18  statementseparator  _token_newline: (?:\r?\n)|\r
19  statementseparator  _token_semicolon: ;
20  literal  _token_string_singlequoted: '([^']*(?:''[^']*)*)'
21  literal  _token_string_doublequoted: "([^"]*(?:""[^"]*)*)"
22  literal  _token_string_tripledoublequoted: """([\s\S]*?)"""
23  whitespace  _token_whitespace: [\t\v ]+
24  reducible  _token_reducible: [-:|+/*^&=%@!`\\<>?]+
25  literal  _token_floatnumber: [0-9]*\.?[0-9]+([eE][+-]?[0-9]+)?
26  alphanumeric  _token_alphanumeric: [a-zA-Z_][a-zA-Z0-9_]*
27  literal  _token_imaginaryunit: #i
28  tokentransform  _token_binaryhexoctalnotation: #[bho][0-9A-F]+
29  tokentransform  _token_stringinterpolationquote: \$['"]
Previous
Next