Skip to content

Make AST & Symbol immutable to allow for multi-file support#842

Open
huydo862003 wants to merge 8 commits intomasterfrom
refactor/ast-immutable
Open

Make AST & Symbol immutable to allow for multi-file support#842
huydo862003 wants to merge 8 commits intomasterfrom
refactor/ast-immutable

Conversation

@huydo862003
Copy link
Contributor

@huydo862003 huydo862003 commented Mar 17, 2026

Doc: Link

Problem

Context: Currently, the AST & Symbol is mutable:

  • The Validator sets SyntaxNode.parent and SyntaxNode.symbol.
  • The Binder sets SyntaxNode.referees and NodeSymbol.references.

Problems:

  • Parsing is entirely local: That is, parsing only requires 1 file at a time & theoretically, the parsed AST can be reused if the file does not change.
  • Validation is entirely local: It just validates the structure of the AST and registers symbols into the symbol table, it doesn’t require information from other files.
  • Binding are global: It requires many files to resolve symbols, etc.

=> When a file changes, its binding phase reruns. Binding writes two things that span across files:

  1. Symbols onto AST nodes — including nodes belonging to other files
  2. References onto NodeSymbols — the symbol objects themselves are mutated

This means both ASTs and NodeSymbols become stale across the entire project, so everything must be discarded and re-parsed & re-validated from scratch — even files that didn't change.

The fix: make both ASTs and NodeSymbols immutable after parsing and validation, so a re-bind doesn't corrupt other files' data and unchanged files can be safely reused.

Solution:

  • Move symbols outside the AST.
  • The AST is immutable after parsing.

⇒ Allow us to reuse most ASTs if only 1 file changes.

Solution: Move symbol/referee mappings out of the AST so it becomes immutable after parsing -> ASTs can be reused across files when only one file changes.

Changes:

  • Remove symbol and referee from SyntaxNode; introduce NodeToSymbolMap and NodeToRefereeMap on Analyzer
  • Remove references from NodeSymbol; introduce NodeToReferencesMap on Analyzer
  • Add bind method to Analyzer; move parent-node assignment to the parser (after gatherInvalid)
  • Update binder, validator, interpreter, and LSP services to use the new maps
@huydo862003 huydo862003 marked this pull request as draft March 17, 2026 03:49
@huydo862003 huydo862003 force-pushed the refactor/ast-immutable branch from ff5272c to 1627dcb Compare March 17, 2026 03:53
@huydo862003 huydo862003 reopened this Mar 17, 2026
@huydo862003 huydo862003 force-pushed the refactor/ast-immutable branch 5 times, most recently from 42b4ee9 to 7b72e6e Compare March 17, 2026 07:45
@huydo862003 huydo862003 force-pushed the refactor/ast-immutable branch from 7b72e6e to 3e3e473 Compare March 17, 2026 08:05
@huydo862003 huydo862003 marked this pull request as ready for review March 17, 2026 10:16
@huydo862003 huydo862003 force-pushed the refactor/ast-immutable branch from b4e7f6a to 632bc32 Compare March 17, 2026 10:24
@huydo862003 huydo862003 changed the title Make AST immutable to allow for multi-file support Make AST & Symbol immutable to allow for multi-file support Mar 20, 2026
@github-actions
Copy link

Coverage Report

Commit: 8986732

Overall Coverage

Metric Coverage
Lines ✅ 84.18% (5450/6474)
Statements ✅ 83.49% (5815/6965)
Functions ✅ 89.44% (1271/1421)
Branches ⚠️ 75.21% (3010/4002)

Package Coverage

Package Lines Statements Functions Branches
@dbml/dbml-cli ✅ 100.00% ✅ 100.00% ✅ 100.00% ✅ 100.00%
@dbml/dbml-connector ⚠️ 64.62% ⚠️ 64.36% ⚠️ 59.53% ⚠️ 59.09%
@dbml/dbml-core N/A N/A N/A N/A
@dbml/dbml-parse ✅ 87.52% ✅ 86.73% ✅ 93.58% ⚠️ 77.68%

⚠️ Coverage Warnings

The following packages have coverage below 80%:

  • @dbml/dbml-connector: 64.62% line coverage

Files with Coverage Below 80%

@dbml/dbml-connector

9 file(s) below 80% coverage
File Lines Statements Functions Branches
src/connectors/bigquery/index.ts 0.00% 0.00% 0.00% 0.00%
src/utils/credential-loader.ts 0.00% 0.00% 0.00% 0.00%
src/utils/helpers.ts 0.00% 0.00% 0.00% 0.00%
src/connectors/snowflake/index.ts 10.56% 10.31% 0.00% 0.00%
src/utils/parseSchema.ts 46.15% 42.85% 28.57% 27.27%
src/connectors/connector.ts 66.66% 66.66% 100.00% 57.14%
src/connectors/oracle/tables.ts 71.25% 67.39% 100.00% 56.96%
src/connectors/oracle/index.ts 80.00% 80.76% 100.00% 25.00%
src/connectors/oracle/utils.ts 85.71% 85.71% 100.00% 71.42%

@dbml/dbml-parse

40 file(s) below 80% coverage
File Lines Statements Functions Branches
src/compiler/queries/container/scope.ts 0.00% 0.00% 0.00% 0.00%
src/services/diagnostics/provider.ts 0.00% 0.00% 0.00% 0.00%
src/core/interpreter/elementInterpreter/project.ts 51.42% 51.42% 100.00% 36.36%
src/core/interpreter/records/utils/data/sqlTypes.ts 56.25% 58.82% 75.00% 46.55%
src/core/analyzer/symbol/utils.ts 56.52% 56.52% 100.00% 50.00%
src/core/analyzer/analyzer.ts 61.53% 60.00% 57.14% 0.00%
src/core/analyzer/binder/elementBinder/note.ts 62.50% 64.70% 83.33% 50.00%
src/compiler/queries/utils.ts 64.83% 65.95% 88.88% 47.43%
src/core/interpreter/records/utils/data/values.ts 65.13% 57.14% 72.72% 50.37%
src/compiler/queries/token.ts 66.66% 66.66% 66.66% 100.00%
src/compiler/queries/parse.ts 70.00% 70.00% 66.66% 100.00%
src/core/parser/utils.ts 71.71% 72.43% 100.00% 78.98%
src/core/analyzer/binder/elementBinder/enum.ts 72.22% 73.68% 100.00% 62.50%
src/core/analyzer/validator/elementValidators/indexes.ts 73.33% 74.07% 90.90% 57.69%
src/core/analyzer/validator/elementValidators/note.ts 74.50% 72.72% 76.92% 68.75%
src/core/analyzer/binder/elementBinder/ref.ts 75.75% 77.14% 90.90% 70.00%
src/core/analyzer/symbol/symbolIndex.ts 76.92% 77.50% 92.85% 50.00%
src/core/analyzer/binder/elementBinder/project.ts 77.77% 78.94% 100.00% 50.00%
src/core/analyzer/validator/elementValidators/project.ts 78.12% 78.78% 100.00% 56.25%
src/core/analyzer/utils.ts 78.50% 77.06% 88.88% 77.55%
src/core/utils.ts 78.57% 77.41% 80.00% 60.71%
src/core/analyzer/validator/elementValidators/records.ts 79.45% 80.00% 93.75% 74.19%
src/core/analyzer/binder/elementBinder/indexes.ts 79.54% 78.26% 90.90% 68.18%
src/core/analyzer/validator/elementValidators/checks.ts 79.62% 81.03% 93.75% 71.87%
src/core/analyzer/binder/elementBinder/tableGroup.ts 80.00% 80.64% 100.00% 50.00%
src/core/interpreter/records/utils/constraints/pk.ts 82.00% 80.00% 92.30% 54.54%
src/services/suggestions/utils.ts 82.35% 76.78% 92.85% 70.37%
src/core/analyzer/binder/elementBinder/records.ts 82.97% 83.50% 93.75% 71.42%
src/compiler/queries/container/token.ts 83.33% 85.71% 100.00% 75.00%
src/core/analyzer/validator/elementValidators/tablePartial.ts 83.52% 80.82% 87.23% 64.28%
src/core/analyzer/binder/elementBinder/tablePartial.ts 86.00% 86.53% 100.00% 69.23%
src/core/parser/parser.ts 87.08% 87.23% 100.00% 79.25%
src/services/suggestions/recordRowSnippet.ts 89.28% 85.29% 100.00% 78.00%
src/compiler/queries/symbol.ts 92.00% 92.59% 100.00% 78.57%
__tests__/utils/compiler.ts 92.92% 92.75% 100.00% 70.32%
__tests__/utils/testHelpers.ts 93.54% 94.28% 75.00% 88.09%
src/core/interpreter/elementInterpreter/sticky_note.ts 95.00% 95.23% 100.00% 66.66%
src/core/interpreter/records/utils/constraints/unique.ts 96.15% 93.93% 100.00% 66.66%
src/core/analyzer/symbol/symbolTable.ts 100.00% 100.00% 100.00% 75.00%
src/services/definition/provider.ts 100.00% 94.44% 100.00% 70.00%
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant