Skip to content

Commit 0643a87

Browse files
Moonbot-TechMoonbot-Tech
committed
docs: publish MoonCompiler 1.0.0
1 parent ad10e40 commit 0643a87

12 files changed

Lines changed: 3473 additions & 0 deletions

‎README.md‎

Lines changed: 336 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,336 @@
1+
<p align="center">
2+
<a href="https://moonbot.pro">
3+
<img src="assets/moonbot-logo-full.svg" alt="Moonbot" width="199">
4+
</a>
5+
</p>
6+
7+
<h1 align="center">MoonCompiler</h1>
8+
9+
<p align="center">
10+
<b>Delphi-compatible Pascal toolchain for Win64 and Linux x86-64</b><br>
11+
compiler, Unicode RTL, memory manager, and qualification as one integrated whole
12+
</p>
13+
14+
<p align="center">
15+
<a href="doc/LICENSING.md"><img src="https://img.shields.io/badge/license-GPLv2%2B%20%C2%B7%20modified%20LGPL-4C6EF5" alt="Licenses: GPLv2 or later and modified LGPL"></a>
16+
<img src="https://img.shields.io/badge/targets-Win64%20%C2%B7%20Linux%20x86--64-8B5CF6" alt="Targets: Win64 and Linux x86-64">
17+
<a href="https://github.com/Moonbot-Tech/MoonCompiler/releases/latest"><img src="https://img.shields.io/github/v/release/Moonbot-Tech/MoonCompiler?label=release" alt="Latest release"></a>
18+
<a href="https://github.com/Moonbot-Tech/MoonCompiler/actions/workflows/qualification.yml"><img src="https://github.com/Moonbot-Tech/MoonCompiler/actions/workflows/qualification.yml/badge.svg" alt="Qualification"></a>
19+
</p>
20+
21+
<p align="center">
22+
<a href="#what-mooncompiler-is-for">What it is for</a> ·
23+
<a href="#quick-start">Quick start</a> ·
24+
<a href="#the-build-profile">Build profile</a> ·
25+
<a href="#changes-from-unleashed">What changed</a> ·
26+
<a href="#performance">Performance</a> ·
27+
<a href="#how-it-is-validated">Qualification</a> ·
28+
<a href="#lazarus">Lazarus</a> ·
29+
<a href="#documentation">Documentation</a>
30+
</p>
31+
32+
MoonCompiler is a self-contained environment for building modern Delphi code on
33+
Win64 and Linux x86-64. It grew out of a specific need: moving the core of a
34+
high-load cryptocurrency-scalping trading terminal, written in and still being
35+
developed with Delphi 12.2, to Linux.
36+
37+
The starting point was Unleashed, an FPC fork with support for modern Delphi
38+
syntax. Working against a complete production application required substantial
39+
work on its compiler, RTL, and memory manager.
40+
41+
MoonCompiler brings the modified compiler, Unicode RTL, packages, memory
42+
manager, and build driver together as one supported x86-64 configuration. We
43+
validate and optimize that configuration as a whole.
44+
45+
The supported surface and intentionally retained boundaries are listed in
46+
[Known Issues](doc/KNOWN_ISSUES.md); future improvements are in the
47+
[Backlog](doc/BACKLOG.md). Supported targets are Win64 and Linux x86-64.
48+
32-bit targets, other CPU architectures, and macOS are neither supported nor
49+
planned.
50+
51+
## What MoonCompiler Is For
52+
53+
MoonCompiler lets Delphi 12.2 code run on Linux without a rewrite—and, in our
54+
measurements, it will usually run faster than under Delphi. The same toolchain
55+
also builds native Win64 applications.
56+
57+
In practice, this means:
58+
59+
- the same source continues to build with Delphi 12.2 and MoonCompiler;
60+
- the Linux version uses the same types, algorithms, and application
61+
architecture rather than a port with different semantics;
62+
- Unicode, threading, RTTI, exceptions, and the memory manager come from one
63+
ready-made profile instead of being configured anew for each project;
64+
- the result is validated beyond “it compiled”: tests compare calculations,
65+
lifetime, ABI, and behaviour at different optimization levels;
66+
- compiler, RTL, and MM performance is measured together on application hot
67+
paths, and bottlenecks are fixed where they originate—in the compiler, RTL,
68+
or MM.
69+
70+
## Quick Start
71+
72+
Clone the repository first. It contains the build driver, project profile, and
73+
the pinned MM source used by the installed toolchain:
74+
75+
```bash
76+
git clone https://github.com/Moonbot-Tech/MoonCompiler.git
77+
cd MoonCompiler
78+
```
79+
80+
The shortest path is to download the archive for your platform from
81+
[GitHub Releases](https://github.com/Moonbot-Tech/MoonCompiler/releases) and
82+
install it into the clone. No bootstrap compiler is required.
83+
84+
Linux:
85+
86+
```bash
87+
./build toolchain ~/Downloads/mooncompiler-toolchain-v1.0.0-linux-x86-64.tar.gz
88+
./build examples/hello.dpr debug
89+
./build examples/hello.dpr release
90+
```
91+
92+
Win64 PowerShell:
93+
94+
```powershell
95+
.\build.ps1 toolchain $HOME\Downloads\mooncompiler-toolchain-v1.0.0-win64.zip
96+
.\build.ps1 examples\hello.dpr debug
97+
.\build.ps1 examples\hello.dpr release
98+
```
99+
100+
Alternatively, build the same toolchain from source with the FPC 3.2.2
101+
bootstrap compiler:
102+
103+
```bash
104+
./build compiler
105+
```
106+
107+
```powershell
108+
.\build.ps1 compiler
109+
```
110+
111+
The source-build dependencies and exact bootstrap commands are listed in
112+
[Setup](doc/SETUP.md). Afterwards, application projects are always compiled
113+
with the pinned compiler in `.moonbot/toolchain`, regardless of how it was
114+
installed.
115+
116+
That is enough for a normal project. The driver adds the Unicode RTL, Delphi
117+
namespaces, required runtime units, and bundled MM itself. For a larger project,
118+
add a `<project>.mooncompiler` file next to the `.dpr` once, containing source
119+
trees, aliases, and pinned Git dependencies; the daily command stays the same.
120+
The format is described in [Project Build](doc/PROJECT_BUILD.md).
121+
122+
Heavy MM diagnostics can be enabled separately without changing Debug/Release
123+
semantics:
124+
125+
```bash
126+
./build examples/hello.dpr debug --diagnostic-mm
127+
```
128+
129+
```powershell
130+
.\build.ps1 examples\hello.dpr debug -DiagnosticMM
131+
```
132+
133+
## The Build Profile
134+
135+
Users should not have to enumerate internal units or remember their ordering.
136+
The product compiler automatically adds the required prefix before the user's
137+
`uses` clause:
138+
139+
- Win64: bundled MM → `fpwinmonitor`;
140+
- Linux x86-64: bundled MM → `cthreads` → `cwstring` → `fpmonitor`.
141+
142+
Plain `String` always means `UnicodeString`. The byte domain is declared
143+
explicitly with `AnsiString`, `RawByteString`, or `TBytes`. Debug and Release
144+
use one validated runtime-check profile: I/O checking is enabled, while
145+
overflow, range, and stack checking are disabled. Release uses `-O3` and
146+
AUTOINLINE; the presence of line information does not change program semantics.
147+
148+
The product runtime can be explicitly disabled with
149+
`-dMOONCOMPILER_VANILLA_RUNTIME`; Valgrind and ASan profiles automatically use
150+
`cmem` instead of the bundled MM.
151+
152+
For the full layout of profiles and dependencies, see [Setup](doc/SETUP.md)
153+
and [Project Build](doc/PROJECT_BUILD.md).
154+
155+
## Changes from Unleashed
156+
157+
### Compiler and Language
158+
159+
The repository contains more than 120 individual correctness, compatibility,
160+
and performance fixes across the compiler and RTL. The supported Delphi surface
161+
includes inline variables, anonymous methods and `reference to`, generics,
162+
advanced and managed records, attributes, extended RTTI, and namespaces. For
163+
example:
164+
165+
- after loop unrolling, the optimizer reused a stale table address, causing AES
166+
to diverge from FIPS-197 starting with the second round;
167+
- Win64 code generation for `Currency * Currency` truncated an intermediate
168+
result to 64 bits and corrupted exact financial arithmetic;
169+
- an exception from `Initialize` or `Assign` on a managed record or array left
170+
leaks or caused repeated finalization of a partially constructed value;
171+
- Linux exception unwinding could restore the wrong nonvolatile register and
172+
corrupt a live exception object inside `finally`.
173+
174+
For the full catalogue of symptoms, causes, and regression tests, see
175+
[Compiler Fixes](doc/COMPILER_FIXES.md).
176+
177+
### RTL and API
178+
179+
The product RTL uses `UnicodeString` as the normal `String` and provides the
180+
Delphi surface applications need on both platforms. We fixed managed-value
181+
lifetime, strings and encodings, collections, streams, tasks and threads, RTTI
182+
invocation, file and network helpers, and the platform ABI. Win64 and Linux
183+
build from one source contract; platform differences remain inside the RTL.
184+
185+
### Optimizer
186+
187+
The accepted branch includes more than local peephole fixes; it also contains a
188+
dedicated optimization block:
189+
190+
- safe LICM with an explicit effects model;
191+
- ADDRESSGVN and reuse of proven-stable addresses;
192+
- precise register allocation and liveness around Windows SEH and Linux EH;
193+
- shorter FP live ranges and register preservation through exception paths;
194+
- CODEALIGN and x86-64 machine facts checked by dedicated gates.
195+
196+
The architecture, safety boundaries, and measured results are described in
197+
[Optimizer](doc/OPTIMIZER.md).
198+
199+
### Memory Manager
200+
201+
The bundled MM is based on the mORMot FPC x86-64 MM and is part of the product
202+
profile. It is included before any user unit and has multithreaded arenas,
203+
validated small and medium size classes, and a separate diagnostic mode with an
204+
allocation registry, poison, structural checks, and a leak report. For details
205+
and licensing, see [Memory Manager](doc/MEMORY_MANAGER.md).
206+
207+
## Performance
208+
209+
Performance is measured by Pulse, the benchmark system included in the
210+
repository. It combines compiler, RTL, and MM microbenchmarks with compact
211+
models of server and trading hot paths. Each case is built with both compilers
212+
from the same Pascal source; speed is considered only after their calculation
213+
results agree.
214+
215+
Across the 243 cases shared by the original baseline and final snapshot,
216+
Moon/Unleashed was at parity with Delphi (`0.9978×`), while current Moon reached
217+
`0.7625×`. Our compiler/RTL/MM optimization series accounts for the entire gain
218+
on this set: the current version is `1.31×` faster than the original. The final
219+
extended matrix contains 744 cases, where Moon is `1.20×` faster than Delphi
220+
12.2. Across twenty Heartbeat application hot paths, the advantage is `1.22×`.
221+
Separately, the bundled MM is `1.65×` faster than the standard FPC MM on
222+
allocator workloads with the same compiler and source. Stock FPC is absent from
223+
the table because it cannot compile the Delphi code under test.
224+
225+
| Workload | Comparison | Cases | Moon result |
226+
|---|---|---:|---:|
227+
| Compiler/RTL/MM optimization series | Moon now / Moon before work began | 243 shared | `1.31×` faster |
228+
| Full matrix: ABI, code generation, RTL, MM, and application workloads | Delphi 12.2 + FastMM4 | 744 | `1.20×` faster |
229+
| Heartbeat: server and trading hot paths | Delphi 12.2 + FastMM4 | 20 | `1.22×` faster |
230+
| RTL: strings, numbers, streams, and helpers | Delphi 12.2 + FastMM4 | 77 | `1.45×` faster |
231+
| Collections | Delphi 12.2 + FastMM4 | 48 | `1.37×` faster |
232+
| JSON through the mORMot API | Delphi 12.2 + FastMM4 | 18 | `1.16×` faster |
233+
| Memory allocation | Standard FPC MM with the same MoonCompiler | 15 | Bundled MM `1.65×` faster |
234+
235+
The complete report, including all cases and source numbers, is
236+
[release-final-20260830](qualification/performance/evidence/release-final-20260830/REPORT.md).
237+
Result changes after each optimization stage are retained in the
238+
[Pulse history](qualification/performance/PULSE_HISTORY.html). For methodology
239+
and calculation rules, see [Performance Qualification](doc/PERFORMANCE_QUALIFICATION.md).
240+
For the known slow tail, see [Backlog](doc/BACKLOG.md).
241+
242+
To reproduce the medium snapshot on Win64 from a RAD Studio command environment:
243+
244+
```powershell
245+
python qualification\performance\tools\pulse.py run `
246+
--mode medium --systems delphi,moon,moon-default `
247+
--tag local-medium
248+
```
249+
250+
## How It Is Validated
251+
252+
A green build of one project is not considered evidence of compatibility.
253+
Qualification is split into independent layers:
254+
255+
| System | What it validates |
256+
|---|---|
257+
| Focused regressions | The exact defect and the neighbouring boundaries of each fix |
258+
| Mega and Omni | Broad language forms, their combinations, and optimization modes |
259+
| Devil | Generated expression classes, ABI, managed lifetime, and a differential oracle |
260+
| Chimera | Whole and split compositions transferred from MoonBot, Arbitrage, mORMot, and other Pascal projects |
261+
| Resident | A multithreaded mix of runtime, collections, crypto, hashing, compression, numerical algorithms, and long-lived managed values |
262+
| Project checks | Both mORMot lines, Lazarus, upstream regressions, and complete forms from other Pascal projects |
263+
| RTL-test | RTL API, boundaries, ownership, exception cleanup, and multithreading |
264+
| Pulse and Heartbeat | A semantic digest plus comparative performance |
265+
266+
Win64 and Linux run Debug/O2/O3 wherever the optimization level is part of the
267+
risk being checked. Light and impact-scoped gates are for the short fix cycle;
268+
full qualification is for a release exact HEAD. See [Testing](doc/TESTING.md)
269+
for commands and a map of the layers.
270+
271+
## Lazarus
272+
273+
The pinned Lazarus version builds and launches with one command:
274+
275+
```bash
276+
./lazarus
277+
```
278+
279+
```powershell
280+
.\lazarus.ps1
281+
```
282+
283+
On its first run, the driver builds the compiler, checks out the supported
284+
Lazarus commit, and creates an isolated IDE configuration. Lazarus sources are
285+
not patched. The toolchain contains two non-overlapping profiles: a normal FPC
286+
ABI for the IDE/LCL itself, and a Unicode product profile for Delphi-compatible
287+
applications.
288+
289+
Lazarus provides the editor, navigation, debugger, and designer. The same
290+
`build`/`build.ps1` performs the final product build of a larger `.dpr`, so the
291+
IDE does not create a second set of hidden settings. For details, see [Lazarus
292+
setup](doc/SETUP.md#lazarus).
293+
294+
## Repository
295+
296+
- `compiler`, `rtl`, `packages`, `utils` — toolchain;
297+
- `runtime/mm` — the sole product memory manager;
298+
- `examples` — minimal Delphi-compatible projects for a quick start;
299+
- `tests` — upstream tests and minimal compiler regressions;
300+
- `RTL-test` — a separate matrix for RTL semantics and lifetime;
301+
- `qualification/suite` — Mega, Omni, Devil, Chimera, corpora, and integration;
302+
- `qualification/performance` — Pulse, Heartbeat, and versioned evidence;
303+
- `qualification/vendor/mormot-product` — the pinned mORMot 2.3.8832 product
304+
corpus without its own MM;
305+
- `doc` — public documentation.
306+
307+
A clone and normal build require only the repository contents and the bootstrap
308+
tools from [Setup](doc/SETUP.md).
309+
310+
## Documentation
311+
312+
- [Setup](doc/SETUP.md) — a clean Linux and Win64 installation;
313+
- [Project Build](doc/PROJECT_BUILD.md) — simple and multi-repository projects;
314+
- [Testing](doc/TESTING.md) — Light/full qualification and the role of each layer;
315+
- [Compiler Fixes](doc/COMPILER_FIXES.md) — catalogue of correctness, API, and ABI fixes;
316+
- [Performance Qualification](doc/PERFORMANCE_QUALIFICATION.md) — Pulse methodology;
317+
- [Optimizer](doc/OPTIMIZER.md) — LICM, ADDRESSGVN, RA, and CODEALIGN;
318+
- [Memory Manager](doc/MEMORY_MANAGER.md) — MM, diagnostic mode, and limitations;
319+
- [Known Issues](doc/KNOWN_ISSUES.md) — accepted observable boundaries;
320+
- [Backlog](doc/BACKLOG.md) — intentionally deferred improvements;
321+
- [Development](doc/DEVELOPMENT.md) — rules for the next fix;
322+
- [Licensing](doc/LICENSING.md) — licenses, notices, and the linking exception.
323+
324+
## Licensing
325+
326+
The compiler is distributed under GPLv2 or later. The RTL and packages retain
327+
the modified LGPL and FPC static-link exception. The bundled MM retains its
328+
original disjunctive MPL-1.1/GPL/LGPL header and FPC linking exception. For the
329+
complete component map, see [Licensing](doc/LICENSING.md).
330+
331+
The memory manager is based on open-source [Synopse mORMot](https://synopse.info/);
332+
the modified source and original notices are in `runtime/mm`.
333+
334+
---
335+
336+
Moonbot · MoonCompiler — Delphi-compatible Pascal toolchain · [moonbot.pro](https://moonbot.pro)

0 commit comments

Comments
 (0)