Skip to content

About

Educational C compiler built from scratch with Flex, Bison and LLVM. Walks through every core compiler construction phase: lexical analysis, parsing, semantic analysis with a scoped symbol table and type checking, and LLVM IR code generation you can run with lli. A small, readable, hackable reference for students learning how compilers work.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

C Compiler with Flex, Bison and LLVM

An educational C compiler built from scratch with Flex (lexer), Bison (parser), and the LLVM C++ API (code generation). It compiles a teaching-sized subset of C to LLVM IR that runs directly with lli. Lexical analysis, parsing, semantic analysis with a scoped symbol table and type checking, and IR generation are all in a small codebase you can read in one sitting, which makes it a good companion for a compiler construction course.

CI C++ LLVM Flex Bison Release Tests License PRs Welcome

Demo: compiling a small C program, showing each token and grammar rule, the generated LLVM IR, running it with lli (exit code 42), and the test suite passing


Table of contents


Why this project

Most compiler tutorials stop at parsing, or they jump straight to a full production compiler. This project sits in between: a complete front end plus real LLVM code generation, small enough to understand end to end. It's useful for:

  • Students in a compiler construction or compiler design course
  • Developers learning how Flex and Bison work together in practice
  • Anyone who wants a hands-on introduction to generating LLVM IR with IRBuilder

Features

Language

  • Types: int, float, bool, void. As in C, an int is converted to float automatically where a float is needed (float f = 3;, x + 0.5, x > 1.5); other mismatches are type errors
  • Global and local variables, one-dimensional arrays with compile-time bounds checks for constant indices
  • Functions with parameters and return values; prototypes, which may leave out parameter names (int add(int, int);)
  • Arithmetic + - * / % and unary minus (-x), relational < <= > >= == !=, logical && || !, increment and decrement ++ --
  • Control flow: if/else and else if chains (conditions may be any int, float or bool, like C), while, for, switch/case/default (fallthrough, grouped and negative labels), break, continue, return
  • I/O: printf, scanf. printf accepts a real C format string (printf("x=%d\n", x)) or a simple label style (printf("Sum: ", sum) prints Sum: 15 and a newline)

Compiler phases

  • Lexical analysis with Flex: keywords, identifiers, literals, operators, comments
  • Syntax analysis with Bison: LALR(1) grammar with operator precedence and associativity
  • Semantic analysis: scoped symbol table, undeclared and redeclared identifier detection, type checking, function call argument checking
  • Code generation: LLVM IR through the LLVM C++ API, verified and written to output.ll

Quick start

Prerequisites

Tool Purpose
flex Lexer generator
bison Parser generator
g++ C++ compiler
LLVM 14 (llvm-config, lli) IR library and interpreter

On Ubuntu / Debian / WSL:

sudo apt install flex bison g++ llvm-14 llvm-14-dev
export PATH=/usr/lib/llvm-14/bin:$PATH   # puts llvm-config and lli on the PATH

Build and run

git clone https://github.com/CodeRafay/c-compiler-flex-bison-llvm.git
cd c-compiler-flex-bison-llvm
make                                   # builds ./compiler
./compiler testCaseExamples/test.c     # writes output.ll
./compiler -v testCaseExamples/test.c  # same, but shows every step: tokens, grammar rules, IR generation
lli output.ll; echo $?                 # runs the program, prints main's return value

If the program has any syntax, semantic, or type errors, ./compiler prints them, exits with status 1, and writes no output.ll.

Run make clean to remove all build artifacts.

Run the tests

make test        # builds the compiler, then runs every test
./tests/run_tests.sh -v   # same, but also lists the passing tests

The tests live in tests/. There are 166 of them, and all pass:

  • tests/cases/*.txt (125 tests): small programs, one per line, grouped by feature (arithmetic, arrays, functions, globals, if/else, loops, printf, switch). Each is compiled, run with lli, and checked against its expected exit code and output.
  • tests/errors.txt (35 tests): incorrect programs. The compiler must report the expected error, exit with 1, and write no output.ll.
  • tests/examples.txt (6 tests): the sample programs in testCaseExamples/.

To add a test, add a line to the matching file. The format is explained at the top of each file.

Example

Input (example.c):

int main() {
    int a = 6;
    int b = 7;
    return a * b;
}

Generated LLVM IR (output.ll):

define i32 @main() {
entry:
  %b = alloca i32, align 4
  %a = alloca i32, align 4
  store i32 6, i32* %a, align 4
  store i32 7, i32* %b, align 4
  %a1 = load i32, i32* %a, align 4
  %b2 = load i32, i32* %b, align 4
  %multmp = mul i32 %a1, %b2
  ret i32 %multmp

after.jump:                                       ; No predecessors!
  ret i32 0
}

Every local's alloca goes at the top of the entry block, which is the form LLVM's mem2reg pass expects. After a return, the compiler opens a fresh after.jump block so that any dead code that follows still has somewhere to go. Nothing jumps to that block, and LLVM accepts it.

Run it:

$ ./compiler example.c && lli output.ll; echo $?
Compiled example.c to output.ll
42

Semantic errors are reported with line numbers:

int main() {
    int x = y + 2;   // y was never declared
    return x;
}
Semantic Error: Variable 'y' not declared (line 2)
Error: Expression for assignment to 'x' is invalid (line 2)
Compilation failed with 2 error(s).

How it works

flowchart LR
    A[Source code] --> B["Lexer<br/>lexical.l (Flex)"]
    B -- tokens --> C["Parser<br/>parser.y (Bison)"]
    C <--> D["Symbol table<br/>symbol_table.c"]
    C -- semantic actions --> E["IR generator<br/>llvm_ir.c (LLVM API)"]
    E --> F[output.ll]
    F --> G[lli / llc]
Loading
  1. Lexical analysis. lexical.l defines token patterns. Flex generates lex.yy.c, which turns raw characters into tokens like INT, IDENTIFIER, NUMBER, and PLUS.
  2. Syntax analysis. parser.y defines the grammar. Bison generates an LALR(1) parser. Precedence declarations (%left, %right, %nonassoc) resolve operator ambiguity and the dangling else.
  3. Semantic analysis. As grammar rules are reduced, semantic actions look up and insert identifiers in a scoped symbol table, check types, and validate function calls.
  4. Code generation. The same actions call llvm_ir.c, which uses LLVM's IRBuilder. Each local variable becomes an alloca; reads and writes become load and store; expressions map to LLVM instructions. The module is verified and written to output.ll.

The pipeline, step by step

The compiler has no separate syntax tree (AST). It does single-pass syntax-directed translation: each time Bison reduces a grammar rule, that rule's action checks the semantics and emits LLVM IR on the spot. You can watch this happen with -v. For this program:

int main() {
    int a = 6;
    return a * 7;
}

./compiler -v prints each token as Flex produces it and each grammar rule as Bison reduces it. Lines 2 and 3, filtered to tokens and rules:

Processing token at line 2: INT, Lexeme: int
Expanding rule: type_specifier → INT (line 2)
Processing token at line 2: IDENTIFIER, Lexeme: a
Processing token at line 2: ASSIGN, Lexeme: =
Processing token at line 2: NUMBER, Lexeme: 6
Expanding rule: expr → NUMBER (line 2)
Processing token at line 2: SEMI, Lexeme: ;
Expanding rule: var_decl → IDENTIFIER ASSIGN expr (line 2)
Expanding rule: var_list → var_decl (line 2)
Expanding rule: declaration → type_specifier var_list (line 2)
Expanding rule: stmt → declaration SEMI (line 2)
Expanding rule: stmt_list → stmt (line 2)
Processing token at line 3: RETURN, Lexeme: return
Processing token at line 3: IDENTIFIER, Lexeme: a
Processing token at line 3: MULT, Lexeme: *
Expanding rule: expr → IDENTIFIER (line 3)
Processing token at line 3: NUMBER, Lexeme: 7
Expanding rule: expr → NUMBER (line 3)
Expanding rule: expr → expr MULT expr (line 3)
Processing token at line 3: SEMI, Lexeme: ;
Expanding rule: stmt → RETURN expr SEMI (line 3)

The rules reduce bottom-up. expr → IDENTIFIER emits the load of a, expr → NUMBER produces the constant 7, and expr → expr MULT expr emits the mul. Then stmt → RETURN expr SEMI emits the ret:

  %a1 = load i32, i32* %a, align 4
  %multmp = mul i32 %a1, 7
  ret i32 %multmp

Bottom-up order makes control flow the tricky part. A loop's condition is reduced before the while rule itself, so the parser uses mid-rule actions and a stack of basic blocks to put each condition in the right block. See parser.y and llvm_ir.c.

Project structure

.
├── lexical.l            # Flex lexer specification
├── parser.y             # Bison grammar, semantic actions, main()
├── symbol_table.c/.h    # Scoped symbol table
├── llvm_ir.c/.h         # LLVM IR code generation
├── Makefile             # Build rules (make, make test, make clean)
├── tests/               # Automated tests (make test)
└── testCaseExamples/    # Sample input programs

Project status

Every feature listed under Features generates working LLVM IR, and each one is covered by the automated tests (make test). The language is still a teaching-sized subset of C; see Limitations for what is not supported yet.

Area Status
Lexer ✅ Working
Parser ✅ Working
Semantic analysis ✅ Working
IR: expressions, variables, assignment ✅ Working and verified with lli
IR: functions (params, calls, prototypes) ✅ Working and verified with lli
IR: global variables ✅ Working and verified with lli
IR: printf (format strings, labels, int/float/bool) ✅ Working and verified with lli
IR: if / else / else if (nested, early return) ✅ Working and verified with lli
IR: while / for (nested, for (int i ...) scoped to the loop) ✅ Working and verified with lli
IR: switch (fallthrough, grouped labels, case -1:, default anywhere, nested) ✅ Working and verified with lli
Arrays (int a[10], a[i], local and global) ✅ Working and verified with lli

Limitations

These are valid C but not supported yet. Unless noted, the compiler reports a syntax error.

Statements

  • if/else, while and for bodies need braces: write if (x) { return 1; }, not if (x) return 1;
  • No do … while, no ternary ?:, and no bare { … } blocks inside a function
  • for needs all three parts; for (;;) is not accepted

Operators

  • No compound assignment (+=, -=, …)
  • ++ and -- work only as statements (i++;), not inside expressions (x = i++;)
  • No casts such as (int)f, so there is no way to turn a float into an int yet
  • && and || always evaluate both sides. This compiles, but behaves differently from C, which skips the right side once the result is known

Types and data

  • Only int, float, bool and void: no char, double, strings, pointers or structs
  • Arrays are one-dimensional, can't have initializer lists ({1, 2, 3}), and can't be passed to functions
  • printf arguments must be numbers (no %s with a string argument), and scanf reads only into plain variables

Preprocessor

  • Lines starting with # are skipped, so #include <stdio.h> is harmless but #define has no effect

Roadmap

  • Fix control flow code generation (if/else, while, for, switch)
  • Accept negative case labels (case -1:) in the switch rewrite
  • break/continue inside loops (with the switch rewrite, which makes break a normal statement)
  • Automated test suite (make test) with expected outputs
  • GitHub Actions CI: builds and runs make test on every push and pull request
  • Hide debug output behind a -v flag
  • Stop writing IR when semantic errors occur
  • Optimization passes (mem2reg, constant folding)
  • Native executables via llc and the system linker
  • String literals and format strings in printf (string arguments such as %s are not supported yet)

FAQ

What is this project? An educational compiler for a subset of C. It's written in C++ with Flex, Bison, and LLVM, and it compiles source code to LLVM IR.

How do Flex and Bison work together? Flex produces yylex(), which returns one token at a time. Bison's generated yyparse() calls yylex() repeatedly and matches the tokens against grammar rules. Token values are passed through yylval, and token IDs are shared through parser.tab.h.

Why generate LLVM IR instead of assembly? LLVM IR is portable and strongly typed, and LLVM can optimize it. The same IR can be interpreted with lli, optimized with opt, or compiled to native code for any target with llc. That lets the project focus on front-end concepts.

Is this a full C compiler? No. It supports a teaching-sized subset of C; see Limitations for what is missing. Use GCC or Clang for real C code.

Which LLVM version is required? LLVM 14. Other versions may need small API changes.

Can I use this for my compiler construction course? Yes. It's meant as a learning reference. Please follow your institution's academic integrity rules.

Contributing

Contributions are welcome, especially fixes on the roadmap.

  1. Fork the repo and create a branch: git checkout -b fix/while-loop-codegen
  2. Make a focused change, add a test for it in tests/, and check that make test passes
  3. Open a pull request that describes what you changed and how you tested it.

Bug reports with a minimal input program are very helpful.

Learn more

License

Released under the MIT License.


If this project helped you learn about compilers, consider giving it a ⭐. It helps others find it.

About

Educational C compiler built from scratch with Flex, Bison and LLVM. Walks through every core compiler construction phase: lexical analysis, parsing, semantic analysis with a scoped symbol table and type checking, and LLVM IR code generation you can run with lli. A small, readable, hackable reference for students learning how compilers work.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages