Abstract Syntax Tree (AST)

Also Known As

TL;DR

Explanation

An Abstract Syntax Tree (AST) represents source code as a hierarchical tree of syntactic constructs such as expressions, statements, and declarations. The 'abstract' aspect means non-essential syntax details (whitespace, comments, formatting, and often redundant parentheses) are removed, focusing only on semantic structure. ASTs are produced by parsers after tokenisation and are the foundation for compilers, static analysers, linters, and code transformation tools. In PHP, nikic/php-parser generates ASTs that power tools like PHPStan, Psalm, Rector, and PHP CS Fixer. Understanding ASTs enables building custom static analysis rules, automated refactoring (codemods), and code generation tools.

Diagram

flowchart TD
    SRC[SourceCode] --> LEXER[Lexer]
    LEXER --> TOKENS[TokenStream]
    TOKENS --> PARSER[Parser]
    PARSER --> AST[AbstractSyntaxTree]

    AST --> FUNC[FunctionDeclaration<br/>name:add]
    FUNC --> PARAMS[Parameters<br/>a:int,b:int]
    FUNC --> BODY[ReturnStatement]
    BODY --> BINOP[BinaryExpr:+]
    BINOP --> LEFT[Identifier:a]
    BINOP --> RIGHT[Identifier:b]

    AST --> PHPSTAN[PHPStan type check]
    AST --> RECTOR[Rector refactor]
    AST --> COMPILER[Compiler bytecode]

Common Misconception

Why It Matters

Common Mistakes

• Parsing PHP with regex — fails on real syntax like nested expressions, strings, and comments.
• Confusing AST with CST (Concrete Syntax Tree) — AST omits formatting details.
• Handling only a subset of node types — missing StaticCall or NullsafeMethodCall leads to incomplete analysis.
• Mutating nodes incorrectly during traversal instead of using proper NodeVisitor patterns.

Avoid When

• You only need simple text search without understanding syntax.
• Working with non-code structured data like JSON or XML (use native parsers instead).

When To Use

• Building static analysis tools or linters.
• Automating refactoring or codemods.
• Generating or transforming code programmatically.
• Understanding compiler or interpreter internals.

Code Examples

// ❌ Regex-based parsing — unreliable
preg_match_all('/function\s+(\w+)\s*\(/', $code, $matches);
// Breaks on closures, arrow functions, comments, strings, complex syntax

// ✅ Proper AST parsing using nikic/php-parser
use PhpParser\ParserFactory;
use PhpParser\Node;
use PhpParser\NodeTraverser;
use PhpParser\NodeVisitorAbstract;

$parser = (new ParserFactory())->createForNewestSupportedVersion();
$ast = $parser->parse($code);

$traverser = new NodeTraverser();

$traverser->addVisitor(new class extends NodeVisitorAbstract {
    public function enterNode(Node $node): void {
        if ($node instanceof Node\Stmt\Function_) {
            echo 'Function: ' . $node->name->toString() . PHP_EOL;
        }
    }
});

$traverser->traverse($ast);

References

• ↗ https://github.com/nikic/PHP-Parser

Tags

Related Terms