Ascend 1.0: package manager, C-style syntax, structs, lambdas and a full standard library

CHANGELOG.md

@@ -1,5 +1,113 @@
 # Changelog
 
+## 1.0.0
+
+### New
+
+- **Inheritance**: `struct Dog : Animal { ... }` gets Animal's fields and
+  methods; methods can be replaced and call the replaced version with
+  `super.name()`. A Dog can be used wherever an Animal is expected, and
+  `is_a(value, Animal)` checks it.
+- **Operator overloading** with methods named `operator+`, `operator-`,
+  `operator*`, `operator/`, `operator%`, `operator**`, `operator==`,
+  `operator<` (which also gives `>`, `<=`, `>=`, `sort`, `min` and `max`) and
+  `operator[]`.
+- **Special methods**: `to_string()` (used by `print`, `str`, f-strings and
+  string concatenation), `iter()` (for loops and comprehensions), `len()` and
+  `contains()` (`in`).
+- **Typed lists and maps**: `list<i32>`, `map<string, f64>`,
+  `list<list<string>>`. Elements are checked and converted, and the list or
+  map keeps checking what's added to it later.
+- **Tracebacks**: uncaught errors list the calls they passed through.
+- **Deep recursion**: programs run on a thread with a large stack, so
+  recursion can go 100,000 calls deep (it was about 2,000).
+- **`ascend test`** runs every `*_test.asc` / `test_*.asc` file and reports
+  which passed.
+- **Running programs**: `run(["git", "status"])` and `run("ls | wc -l")`
+  capture the output and exit code.
+- **HTTP**: `http_get`, `http_post` and `http_request` (using curl).
+- **Files and folders**: `walk_dir`, `remove_dir`, `copy_file`, `move_file`,
+  `file_size`, `path_join`, `basename`, `dirname`, `extension`, `abs_path`,
+  `cwd`, `chdir`, `temp_dir`.
+- **Data**: `csv_parse`, `csv_stringify`, `base64_encode`, `base64_decode`,
+  `sha256`, `uuid`.
+- **More builtins**: `now`, `format_time`, `set_env`, `platform`,
+  `capitalize`, `title`, `trim_start`, `trim_end`, `last_index_of`, `lcm`,
+  `factorial`, `trunc`, `degrees`, `radians`, `union`, `intersect`,
+  `difference`, `is_a`.
+
+### Changed
+
+- Comparing values that can't be compared says what they are: "can't compare
+  a string with an int".
+
+## 0.6.0
+
+### New
+
+- **Struct methods**: functions written inside a struct, called as
+  `value.method()`, get the value as `this`. An `init()` method runs whenever
+  a value is made.
+- **Lambdas**: `x => x * 2`, `(a, b) => a + b`, `(i32 n) => { ... }`.
+- **Comprehensions**: `[x * x for x in xs if x > 0]`, with several `for` and
+  `if` clauses, and map comprehensions `{k: v for k, v in m}`.
+- **Destructuring**: `a, b := 1, 2`, `a, b = b, a`, `low, high := minmax(xs)`
+  (with `return a, b`), and `x, y := point` for structs.
+- **Ranges**: `0..10` and `1..=10` (also for chars: `'a'..='z'`), usable in
+  `for` loops and `in` tests without building a list, and as `switch` cases
+  and `match` patterns: `case 90..=100:`.
+- **Null-safe operators**: `a ?? b` and `a?.b` / `a?.method()`.
+- **Named arguments**: `greet("Ada", mark: "?")`, `Box(height: 2)`.
+- **Variadic functions**: `i32 sum(i32 ...numbers)`, and `...list` to spread
+  a list into a call's arguments or another list, or a map into a map.
+- **`finally`** blocks, which run however a `try` ends; `try` can also be used
+  with only `finally`.
+- **Raw strings** `r"\d+"` and **multi-line strings** `"""..."""`.
+- New builtins: `group_by`, `count_by`, `chunk`, `flat_map`, `min_by`,
+  `max_by`, `mean`, `median`.
+
+## 0.5.0
+
+### New
+
+- **C-style syntax**:
+  - `for (init; condition; step)` loops, with several comma-separated
+    statements in the first and last parts (`for (i32 i = 0, j = n; i < j;
+    i++, j--)`), empty parts (`for (;;)`), and `for { }` to loop until
+    `break`. Closures created in the body see that iteration's variables.
+  - `++` and `--`, before or after a variable, element or field.
+  - `switch` / `case` / `default` with C's fall-through and `break`; a case
+    can list several values (`case 1, 2:`).
+  - `do { ... } while (condition)`.
+  - Bodies without braces for `if`, `else`, `while`, `for` and `do`:
+    `if (x < 0) x = -x`, on the same line or the next.
+  - Casts: `(f64) total / count`, `(i32) 3.9`, `(char) 65`.
+  - Several declarations in one statement: `i32 a = 1, b = 2`.
+  - `double`, `long` and `short` as names for `f64`, `i64` and `i16`.
+  - `auto x = value` fixes the variable's type to its first value's type.
+  - A top-level `main()` runs automatically after the rest of the program
+    (unless the program calls it), with `args` or `(argc, argv)` if it takes
+    them; an integer result is the exit code.
+  - `printf` and `sprintf` with C format strings.
+- Typed loop variables, and `:` instead of `in`: `for (string s : names)`.
+- **Import statements**: `import geometry`, `import lib.shapes as shapes`,
+  `import "path/file.asc" as name`, `from geometry import area, Point as P`
+  and `from geometry import *`. The `import(...)` function still works.
+- **Structs**: `struct Point { f64 x; f64 y = 0 }` declares a type with typed
+  fields and a constructor, `Point(1, 2)`. Struct names work as types in
+  declarations, parameters and return values; fields keep their types, and
+  unknown fields are errors. Struct values print as `Point{x: 1.0, y: 2.0}`.
+- **Enums**: `enum Color { Red, Green = 5, Blue }` makes a constant map of
+  names to ints.
+- Regular expressions: `regex_match`, `regex_search`, `regex_find_all`,
+  `regex_replace`, `regex_split`.
+- Folders: `is_dir`, `make_dir`, `list_dir`.
+
+### Fixed
+
+- Floats with few significant digits printed in exponent form: `20.0`
+  printed as `2e+01`.
+
 ## 0.4.0
 
 ### New

CMakeLists.txt

@@ -1,6 +1,6 @@
 cmake_minimum_required(VERSION 3.10)
 
-project(ascend VERSION 0.4.0 LANGUAGES CXX)
+project(ascend VERSION 1.0.0 LANGUAGES CXX)
 
 set(CMAKE_CXX_STANDARD 17)
 set(CMAKE_CXX_STANDARD_REQUIRED ON)
@@ -23,12 +23,14 @@ add_library(ascend_core STATIC
     src/numeric.cpp
     src/typecheck.cpp
     src/stack_limit.cpp
+    src/big_stack.cpp
     src/lexer.cpp
     src/parser.cpp
     src/semantic_analyzer.cpp
     src/value.cpp
     src/interpreter.cpp
     src/stdlib.cpp
+    src/stdlib_extra.cpp
     src/format.cpp
     src/json.cpp
     src/runner.cpp
@@ -39,6 +41,10 @@ add_library(ascend_core STATIC
 )
 target_include_directories(ascend_core PUBLIC src/include)
 
+# The interpreter runs programs on a thread with a large stack.
+find_package(Threads REQUIRED)
+target_link_libraries(ascend_core PUBLIC Threads::Threads)
+
 # GCC 8 keeps std::filesystem in a separate library.
 if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU" AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 9)
     target_link_libraries(ascend_core PUBLIC stdc++fs)
@@ -78,6 +84,7 @@ if(ASCEND_BUILD_TESTS)
         tests/lexer_tests.cpp
         tests/parser_tests.cpp
         tests/interpreter_tests.cpp
+        tests/language_tests.cpp
         tests/package_tests.cpp
     )
     target_link_libraries(ascend_tests PRIVATE ascend_core)
@@ -92,4 +99,7 @@ if(ASCEND_BUILD_TESTS)
         get_filename_component(example_name ${example} NAME_WE)
         add_test(NAME example_${example_name} COMMAND ascend ${example})
     endforeach()
+
+    # `ascend test` finds and runs the test files among the examples.
+    add_test(NAME ascend_test_command COMMAND ascend test ${CMAKE_CURRENT_SOURCE_DIR}/examples)
 endif()

README.md

@@ -2,75 +2,96 @@
 
 [![Build](https://github.com/csubbdev/Ascend/actions/workflows/build.yml/badge.svg)](https://github.com/csubbdev/Ascend/actions/workflows/build.yml)
 
-Ascend is a small, friendly scripting language with optional types, written in
-C++17.
+Ascend is a friendly, general-purpose scripting language with optional C-style
+types. Write quick scripts without types, or add them where you want the
+checks: typed variables, functions, structs and collections are enforced, and
+many mistakes are reported before the program even runs. It comes with a
+package manager, a test runner, a REPL and a large standard library, and runs
+on Linux, macOS and Windows.
 
 ```
-string fizzbuzz(i32 n) {
-    if n % 15 == 0 { return "FizzBuzz" }
-    if n % 3 == 0 { return "Fizz" }
-    if n % 5 == 0 { return "Buzz" }
-    return str(n)
+struct Animal {
+    string name
+    string sound() { return "..." }
+    string to_string() { return f"{this.name} says {this.sound()}" }
 }
 
-for i in range(1, 16) {
-    print(fizzbuzz(i))
+struct Dog : Animal {
+    string sound() { return "Woof" }
 }
 
-u8 level = 2
-match level {
-    1 => print("easy")
-    2, 3 => print("normal")
-    else => print("hard")
-}
+list<Animal> pets = [Dog("Rex"), Animal("Generic")]
+for (Animal pet : pets) print(pet)
 
-scores := {ada: 95, linus: 88}
-for name, score in scores {
-    print(f"{name:<6} {score:>3}")
-}
+scores := {ada: 95, linus: 88, grace: 91}
+best := max_by(keys(scores), name => scores[name])
+printf("best: %s with %d\n", best, scores[best])
+
+squares := [n * n for n in 1..=10 if n % 2 == 0]
+response := http_get("https://api.github.com/repos/csubbdev/Ascend")
+if (response.ok) print(json_parse(response.body).full_name)
 ```
 
 ## Features
 
-- **Optional C-style types**: `u8 age = 30`, `i32 add(i32 a, i32 b) { }` with
-  `i8`–`i64`, `u8`–`u64`, `byte`, `f32`, `f64`, `bool`, `char`, `string`,
-  `list`, `map`, `func`, `void` and nullable types like `string?`. Typed
-  variables, parameters and return values are checked, and integer overflow is
-  an error instead of a silent wrap.
-- Constants, `match`, `if` / `else if`, `while`, `for ... in` over ranges,
-  lists, strings and maps, `break` / `continue`
-- Functions with default parameters, recursion, closures and anonymous functions
-- f-strings with format specs: `f"{price:>8.2f}"`
-- Hex, binary and octal literals, type suffixes (`255u8`) and bitwise operators
-- Unicode strings and a `char` type
-- Modules with `import("file")`, and **`apkg`, a package manager** like pip:
-  `apkg install colors`, then `import("colors")`
-- `try` / `catch` / `throw` error handling
-- 100+ builtins for strings, lists, maps, math, files, JSON and the system
-- Clear error messages that point at the problem, many caught before running:
+**The language**
+- **Optional C-style types**: `u8 age = 30`, `f64 area(f64 w, f64 h) { }`, with
+  `i8`–`i64`, `u8`–`u64`, `f32`, `f64`, `bool`, `char`, `string`, nullable
+  types like `string?`, `auto`, and typed collections like `list<i32>` and
+  `map<string, f64>`. Integer overflow is an error instead of a silent wrap.
+- **Structs** with typed fields, methods, inheritance (`struct Dog : Animal`),
+  operator overloading (`operator+`, `operator==`, `operator<`, ...) and
+  special methods (`to_string`, `iter`, `len`); **enums**.
+- **Functions**: default and named arguments, `...rest` parameters, closures,
+  lambdas (`x => x * 2`), multiple return values (`return lo, hi`).
+- **Expressive code**: comprehensions (`[x * x for x in xs if x > 0]`), ranges
+  (`0..n`), destructuring (`a, b = b, a`), `??` and `?.`, `match`, f-strings
+  with format specs, raw and multi-line strings.
+- **C-style code works too**: `for (i32 i = 0; i < n; i++)`, `i++`,
+  `switch`/`case` with fall-through, `do`/`while`, bodies without braces,
+  casts, `printf`, and a `main()` that runs automatically.
+- **Errors**: `try`/`catch`/`finally`/`throw`, clear messages that point at
+  the problem, and tracebacks through function calls:
 
   ```
   script.asc:3:10: error: 'age' is declared as u8, so it can't hold 300 (u8 is 0 to 255)
    3 | u8 age = 300
      |          ^
   ```
 
-- An interactive REPL, a `--check` mode, and syntax highlighting for VS Code
+**The tools**
+- **`apkg`**, a package manager like pip: `apkg install colors`, then
+  `import colors`. Packages come from the registry, GitHub, any git
+  repository or a folder.
+- **`ascend test`** runs your `*_test.asc` files.
+- An interactive REPL, `ascend --check` to find errors without running, and
+  syntax highlighting for VS Code.
+
+**The standard library** (170+ builtin functions)
+- Strings, lists, maps and sets, math, regular expressions, JSON and CSV
+- Files, folders and paths; running programs and capturing their output
+- HTTP requests, dates and times, base64, SHA-256, UUIDs
 
 ## Quick start
 
 ```sh
 cmake -S . -B build
 cmake --build build
-./build/ascend examples/types.asc
-./build/ascend          # interactive REPL
+./build/ascend examples/shapes.asc     # run a program
+./build/ascend                          # interactive REPL
+./build/ascend test examples            # run the test files in a folder
 ```
 
+See [examples/](examples/) for more: [functional.asc](examples/functional.asc),
+[methods.asc](examples/methods.asc), [scripting.asc](examples/scripting.asc),
+[c_style.asc](examples/c_style.asc) and others.
+
 Install a package and use it:
 
 ```sh
 ./build/apkg install colors
-./build/ascend -e 'print(import("colors").green("Hello!"))'
+./build/ascend -e 'import colors
+print(colors.green("Hello!"))'
 ```
 
 ## Documentation
@@ -90,8 +111,9 @@ Install a package and use it:
 | Checks before running (types of literals, `break` outside loops, ...) | `src/semantic_analyzer.cpp`, `src/typecheck.cpp` |
 | Tree-walking interpreter | `src/interpreter.cpp` |
 | Values, numeric types and checked arithmetic | `src/value.cpp`, `src/numeric.cpp`, `src/types.cpp` |
-| Builtin functions, formatting and JSON | `src/stdlib.cpp`, `src/format.cpp`, `src/json.cpp` |
-| Command line and REPL | `src/main.cpp`, `src/repl.cpp`, `src/runner.cpp` |
+| Builtin functions, formatting and JSON | `src/stdlib.cpp`, `src/stdlib_extra.cpp`, `src/format.cpp`, `src/json.cpp` |
+| Running other programs | `src/process.cpp` |
+| Command line, REPL, `ascend test` | `src/main.cpp`, `src/repl.cpp`, `src/runner.cpp`, `src/big_stack.cpp` |
 | Package manager (`apkg`) and finding packages for `import` | `src/apkg_main.cpp`, `src/package_manager.cpp`, `src/packages.cpp` |
 
 ## License

docs/installation.md

@@ -5,9 +5,10 @@
 - A C++17 compiler: GCC 8+, Clang 7+ or Visual Studio 2019+
 - CMake 3.10 or newer
 
-Ascend has no other dependencies. The package manager, `apkg`, uses
-[git](https://git-scm.com) to download packages, so install git too if you
-want to use packages from the internet.
+Ascend has no other dependencies. A few things use programs you probably
+have already: the package manager, `apkg`, uses [git](https://git-scm.com) to
+download packages, and `http_get` and the other HTTP functions use
+[curl](https://curl.se) (included with macOS and Windows 10 and later).
 
 ## Build
 
@@ -27,6 +28,7 @@ Visual Studio). Builds default to the optimized `Release` configuration.
 ./build/ascend                          # start the interactive REPL
 ./build/ascend -e 'print(2 ** 10)'      # run a one-liner
 ./build/ascend --check my_program.asc   # find errors without running
+./build/ascend test                     # run the *_test.asc files in this folder
 ```
 
 To run `ascend` and `apkg` from anywhere, copy them from `build/` into a
@@ -35,7 +37,8 @@ directory on your `PATH`, or run `cmake --install build` (add
 
 ```sh
 apkg install colors                     # see docs/packages.md
-ascend -e 'print(import("colors").bold("ready"))'
+ascend -e 'import colors
+print(colors.bold("ready"))'
 ```
 
 ## Test

docs/packages.md

@@ -12,7 +12,7 @@ apkg uninstall colors      # remove it again
 Programs use installed packages with `import`:
 
 ```
-colors := import("colors")
+import colors
 print(colors.green("it works!"))
 ```
 
@@ -94,7 +94,7 @@ somewhere else.
 
 ## How import finds packages
 
-`import("name")` first looks for a file `name.asc` next to the importing file,
+`import name` first looks for a file `name.asc` next to the importing file,
 as always. If there isn't one, it looks for an installed package called
 `name`:
 
@@ -103,7 +103,8 @@ as always. If there isn't one, it looks for an installed package called
 2. in your own packages (`~/.ascend/packages/name`).
 
 It runs the package's main file (`main` in its `ascend.json`). To import a
-different file from a package, add its path: `import("strings/strings")`.
+different file from a package, add its path: `import strings.strings` (or
+`import("strings/strings")`).
 
 A package that depends on another one imports it the same way, so packages
 can be used together without knowing where they're installed.
@@ -131,7 +132,7 @@ A package is a folder with an `ascend.json`:
 | `name` | Required. Lowercase letters, digits, `-` and `_`. This is the name used to `import` it. |
 | `version` | Required. Use `major.minor.patch` numbers. |
 | `description` | A one-line summary, shown by `apkg show` and `apkg search`. |
-| `main` | The file `import("name")` runs. Defaults to `main.asc`. |
+| `main` | The file `import name` runs. Defaults to `main.asc`. |
 | `dependencies` | Other packages it needs: `"*"` for any version from the registry, a version number like `"1.2.0"`, or any source `apkg install` accepts, like `"github:user/repo@v1.0"` or `"file:../other-folder"`. |
 
 Everything at the top level of the main file that doesn't start with `_`