Skip to content

Commit

Permalink
Refresh installation process (#204)
Browse files Browse the repository at this point in the history
denismerigoux authored Dec 8, 2022
2 parents a38c654 + 65d738e commit 9c7fc6a
Showing 6 changed files with 117 additions and 28 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/check_correctness.yml
Original file line number Diff line number Diff line change
@@ -46,7 +46,7 @@ jobs:
run: |
sudo apt install m4 perl python3 clang git build-essential lzip libgmp-dev libmpfr-dev
opam update
make deps-without-ocaml
make init-without-switch
- name: Formatting
run : |
2 changes: 1 addition & 1 deletion .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[submodule "ir-calcul"]
path = ir-calcul
url = https://gitlab.adullact.net/dgfip/ir-calcul
url = https://gitlab.adullact.net/dgfip/ir-calcul.git
16 changes: 12 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
@@ -32,13 +32,20 @@ default: build
# Building the compiler
##################################################

deps-without-ocaml:
# Workaround for Opam 2.0 bug. Empty switch creation then installation could be a one line
# "opam switch create . --deps-only" otherwise
create-switch:
opam switch create . --empty

init-without-switch:
opam install . --deps-only
git submodule update --init --recursive
git submodule update --init

init: create-switch init-without-switch

deps:
opam switch create . --deps-only
git submodule update --init --recursive
opam switch reinstall --deps-only
git submodule update

format:
dune build @fmt --auto-promote | true
@@ -90,6 +97,7 @@ doc: FORCE
clean:
$(MAKE) -C examples/dgfip_c clean
$(MAKE) -C examples/java clean
rm -f doc/doc.html
dune clean

FORCE:
70 changes: 55 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
@@ -8,52 +8,92 @@ The M language has been invented by the French Direction Générale des Finances
Publiques (DGFiP), equivalent to the IRS, to transcribe the tax code into machine-readable
instructions. It is a small Domain Specific Language based on variable
declarations and arithmetic operations. This work is based on a retro-engineering
of the syntax and the semantics of M, from the [codebase](https://framagit.org/dgfip/ir-calcul)
released by the DGFiP.
of the syntax and the semantics of M, from the codebase previously released by the DGFiP on
[Framagit forge](https://framagit.org) and now regularly published on
[Adullact forge](https://gitlab.adullact.net/dgfip/ir-calcul).

## Disclaimer

There is currently no legal guarantee of any kind about the correctness of the code
produced by the Mlang compiler, or by the results produced by the interpreter of
Mlang. However, authors have been in contact with DGFiP to validate Mlang, and
the system passes all the private DGFiP tests as of Sept. 2021 for the version
of the source files responsible for computing the tax on the 2018, 2019 and 2020 income.
the system passes all the private DGFiP tests as of Sept. 2022 for the version
of the source files responsible for computing the tax on the 2018, 2019, 2020 and 2021 income.

## Installation

Mlang is implemented in OCaml. To manage dependencies,
[install opam](https://opam.ocaml.org/doc/Install.html) and switch to a version
of OCaml that is at least 4.0.9. In order to support multi-precision floating-point
computation, you will need also need to install the MPFR library. For Debian-based
distributions, simply invoke
computation, you will need also need to install the MPFR library.

sudo apt install libgmp-dev libmpfr-dev m4 perl python3 clang git opam
For Debian-based distributions, simply invoke

sudo apt install libgmp-dev libmpfr-dev git opam

For Red Hat-based distributions, first invoke

sudo yum install gmp-devel mpfr-devel git

Opam is only packaged for Fedora. For other distributions using RPM, please refer to
the [official documentation](https://opam.ocaml.org/doc/Install.html). Note that to use
the Opam binary release and install Mlang dependencies, you will need a C compiler and
the following softwares as Opam dependencies: `patch`,`unzip`, `bubblewrap` and `bzip2`.

If you would like to generate tests using the fuzzer, you will need to install AFL:

sudo apt install afl++ afl++-clang

We require OCaml >= 4.0.9. If you have not used opam before, run:
If you have not used opam before, run:

opam init
opam update -y
opam switch create 4.11.1 -y


Then, you can install Mlang's Ocaml dependencies using
Then, you can initialize your Mlang projet using

make deps
make init

This command also fetchs the M source code released by the DGFiP with
This command creates a local Opam switch (analogous to a virtual environment), installs
Mlang's OCaml dependencies and clones the M source code repository released by the DGFiP with

git submodule update --init --recursive
git submodule update --init

You can then use

make build

to build the compiler.

If needed,

make deps

will reinstall OCaml dependencies and refetch the M source code.

The interpreter and the C backend in `examples/dgfip_c/` should be usable straight away,
as the C compiler was installed for Opam. Mlang results are tested on GCC and Clang, the latter
being preferred if available.

The Java backend in `examples/java/` requires Java development environment.
The generated code targets Java 7, and could be used with OpenJDK 1.7 or more.
However, the test harness code requires Java 8, so to use the automated backend tests, we ask for
OpenJDK 1.8 or more.

For Debian-based distributions, you can try:

sudo apt install default-jdk

For Red Hat-based distributions, depending on your version:

sudo yum install java-1.8.0-openjdk-devel

or

sudo yum install java-11-openjdk-devel

NB : if you are using JDK 1.8, in order to cross-compile the generated code to 1.7, you would also need JDK 1.7
installed in order to provide the correct version of the base classes.

## Usage

Please read the `m_specs/complex_case_with_ins_outs_2018.m_spec` for a walk-through
@@ -106,7 +146,7 @@ the testing process of the interpreter (with or without optimizations) and
report test errors in a convenient format.

Mlang backends are also tested using the same `FIP` format, see for instance
`examples/python/backend_test`.
`examples/java/backend_test`.

When running `--run_all_tests`, you can enable code coverage instrumentation
with the `--code_coverage` option. Another interesting option is `--precision`,
14 changes: 12 additions & 2 deletions examples/dgfip_c/ml_primitif/build.sh
Original file line number Diff line number Diff line change
@@ -68,12 +68,22 @@ if [ "$1" == 'nomc' ]; then
echo 'Compilation des fichiers C issus des fichiers M ignorée'

else
if command -v clang; then
echo "Compilation avec Clang."
CCOMPILER="clang"
CCOPTIONS="-fbracket-depth=2048"
else
echo "Clang est absent. Compilation avec GCC."
CCOMPILER="gcc"
CCOPTIONS=""
fi

echo "Compilation des fichiers C issus des fichiers M"


cd ./calc

clang -std=c89 -pedantic -fbracket-depth=2048 -O2 -c irdata.c enchain.c var.c contexte.c famille.c revenu.c revcor.c penalite.c variatio.c tableg01.c restitue.c chap-*.c res-ser*.c coc*.c coi*.c horiz*.c
$CCOMPILER -std=c89 -pedantic $CCOPTIONS -O2 -c irdata.c enchain.c var.c contexte.c famille.c revenu.c revcor.c penalite.c variatio.c tableg01.c restitue.c chap-*.c res-ser*.c coc*.c coi*.c horiz*.c

if [ $? -ne 0 ]; then
echo 'La compilation des fichiers C issus des fichiers M a échoué'
@@ -86,7 +96,7 @@ fi

echo 'Compilation de la calculette primitive'

ocamlopt -cc clang -ccopt -fno-common -ccopt -fbracket-depth=2048 unix.cmxa ./calc/*.o stubs.c common.ml m.ml read_test.ml main.ml -o prim
ocamlopt -cc $CCOMPILER -ccopt $CCOPTIONS -ccopt -std=c99 -ccopt -fno-common unix.cmxa ./calc/*.o stubs.c common.ml m.ml read_test.ml main.ml -o prim

if [ $? -ne 0 ]; then
echo 'La compilation de la calculette primitive a échoué'
41 changes: 36 additions & 5 deletions examples/java/Makefile
Original file line number Diff line number Diff line change
@@ -9,6 +9,28 @@ MLANG_DEFAULT_OPTS=\

MLANG=$(MLANG_BIN) $(MLANG_DEFAULT_OPTS) $(OPTIMIZE_FLAG)

# Check Java compiler version : 
#  - if < 8 : backend won't compile due to TestHarness, period.
#  - if = 8 : calculator code prefers JRE 7 base class rt.jar to be compiled targetting Java 7, so we try to find it.
# - if > 8 : calculator code is cross-compiled to Java 7, test code is cross-compiled to Java 8.
JAVAC8=$(shell javac -version 2>&1|grep -q "javac 1.8"; printf $$?)
ifeq ($(JAVAC8),0)
# Default path is debian-based OpenJDK 7 package path, overridable by environment variables.
JAVA7_PATH?=/usr/lib/jvm/java-7-openjdk-amd64
# if path exists, use it as bootclasspath
ifeq (test -d $(JAVA7_PATH),0)
BOOTCLASSPATH=-bootclasspath $(JAVA7_PATH)/jre/lib/rt.jar
# Otherwise we default to empty bootclasspath, meaning javac will use its own version of base class (JRE 8).
endif
# JDK 8 specific cross-compilation option: -source and optional -bootclasspath. -target defaults to the value of -source
JAVACOPT_CALC=-source 7 $(BOOTCLASSPATH)
JAVACOPT_TEST=
else
# JDK9+ cross-compilation options. From version 9, reference base classes for older JRE are provided.
JAVACOPT_CALC=--release 7
JAVACOPT_TEST=--release 8
endif

all: backend_tests $(shell find . -name "run_*.py")

clean:
@@ -25,11 +47,20 @@ src/com/mlang/Ir_%.java: ../../m_specs/%.m_spec
--function_spec $^ \
$(SOURCE_FILES)

target/com/mlang/Ir_%.class: src/com/mlang/Ir_%.java
javac -J-Xss10m -J-Xmx4096m -target 1.7 -source 1.7 -d target -cp src src/com/mlang/*.java
target:
mkdir $@

backend_tests/target:
mkdir $@

target/com/mlang/Ir_%.class: src/com/mlang/Ir_%.java | target
ifeq ($(JAVAC8),0)
@echo "Cross-compiling Java 7 sources with JDK 8, using JAVA7_PATH=$(JAVA7_PATH) if present. Feel free to override it."
endif
javac -J-Xss10m -J-Xmx4096m $(JAVACOPT_CALC) -d target -cp src src/com/mlang/*.java

backend_tests/target/TestHarness.class: target/com/mlang/Ir_tests_2020.class | backend_tests/target
javac $(JAVACOPT_TEST) -cp "backend_tests/src/:target" -d backend_tests/target backend_tests/src/com/mlang/TestHarness.java

backend_tests/target/TestHarness.class: target/com/mlang/Ir_tests_2020.class
javac -cp "backend_tests/src/:target" -d backend_tests/target backend_tests/src/com/mlang/TestHarness.java

run_tests: backend_tests/target/TestHarness.class
java -cp "target:backend_tests/target" com.mlang.TestHarness $(TESTS_DIR)

0 comments on commit 9c7fc6a

Please sign in to comment.