Introduce a dynamic axi generator (#2132)

* Being work on dynamic AXI wrapper.

Currently, have reduced complexity of channels by not tracking transfer
counts and transaction counts and similar. Changes are a bit haphazard,
so be careful about assuming aything is "done". Perhapes the AR and AW
channels are indeed finished.

TODO: Create address-translate module. In particular look at interface
for mulitpliers as we might only have pipelined version

* add new const_mult to py-builder

* AW AR R and W channels might be done.

Need to make sure write channel interface is correct.
Then need to tie everything together according to diagram.

* Finish(?) read and write controller in builder

TODO: Wire up translator, read controller, write controller.
Then test if this works with current test bench/executor

* begin axi dynamic read controller

* refactor reg calls in dynamix yxi generator

* add import for const-mult in py-builder

* WIP: need to work on axi-seq-mem-comp. Namley, need to be able to add attributes to ports

* add support for port attributes with  method

* refactor to also have outputs iwth attributes

* remove extra files added somehow

* Add attributes to (and rename) add_comp_ports

* Fix some formatting issues, should pass runt tests

* find and replace add_comp_params

* Work on tying together axi_seq_mem comps

* WIP: have control program for axi_seq_mem maybe

* Fix some misnamed signals.

TODO: AFter getting subtyping working want to
instantiate an `axi-seq-mem` in the top level wrapper,
pass that in as a seq-mem to the main kernel

* WIP: attempt at implementing subtyping. TBD if this works

* typo in structure

* change error for wellformedness subtyping runt test

* compile_invoke now compiles, tbd if actually works

* some clippy fixes

* wip compile invoke changes

* added debugging stuff to invoke, still a WIP

* WIP: Need to do clean up but also this might allow subtyping???? Hope so

* add comments to well-formed

* maybe works?

* seems to be working!

* update runt tests

* make clippy happy

* formatting

* update comments

* more runt tests

* replace hashed map with linkedhashmap for determinism

* fix bug where iterating over too many ports in comp_ports in compile_invoke

* more runt tests for subtyping

* might be done with generator? need to fix compiler errors

* add hardcoded vec_add for convenience: todo: not recognizing seq_mem_d1

* get rid of static interval annotation in axi_seq_mem

* first pass at adding dynamic memories

* add dyn-mems to primitve library

* add yxi support for dyn-mems in calyx-ir-utils

* make clippy happy

* add yxi test for dyn-mems

* formatting

* add all mems to generated dynamic axi interface

* add input to padding. Execution completes but incorrect writing to sum0

* change test yxi to dynamic from seq mems

* drive read_data out properly in read_channel

* derive partialeq and eq for InLineAttributes

* extract `get_concrete_port` as helper function

* compile_invoke formatting

* WIP: extract require_subtype. TODO: allow it to take in StaticInvoke as well

* factor out subtyping check

* elaborate on some comments in well_formed

* improve comments

* working dynamic generator???????

* rename axi_seq_mem to axi_dyn_mem

* [fud2] Add dynamic generation option to `axi-wrapped` state (#2136)

* add method to get a confg key witha constrained set of values

* Add dynamic generation for `axi-wrapped` state

Use `--set dynamic=true` targeting 1--to axi-wrapped`

* add method to get a confg key witha constrained set of values

* Make InvalidValue a c-like struct and fix err msg

* formatting

* change debug representation to a join

* formatting

* trigger ci

* Introduce dynamic AXI runt tests (#2139)

* wip move axi tests around

* remove hardcoded dyn-mem-vec-add from dynamic axi generator

* add raw dynamic axi generator tests

* add dyn_mem import into dynamic axi generator

* more dynamic axi runt tests

* fud2 cocotb tests for dynamic axi

* remove hardcoded compute

* remove stderr ipe to dev/null in fud2 cocotb axi tests

* return 2> dev/null in runt cocotb axi tests

* more runt fixes

* more runt fixes, cocotb dynamic test still fails on CI

* upgrade pip to get latest stable version of cocotb

Changes dockerfile

* hook up xPROT signals correctly

* add ARPROT in places as well

* update runt tests

* fix xPROT connections, revert cocotb in dockerfile to 1.6.2

---------

Co-authored-by: eys29 <[email protected]>

Dockerfile

@@ -15,7 +15,7 @@ RUN echo "deb https://repo.scala-sbt.org/scalasbt/debian all main" | tee /etc/ap
 RUN python3 -m venv /opt/venv
 ENV PATH="/opt/venv/bin:$PATH"
 
-# Install python dependencies
+# Install python dependencies cocotb==1.6.2 seems to be for Xilinx cocotb tests
 RUN python3 -m pip install numpy flit prettytable wheel hypothesis pytest simplejson cocotb==1.6.2
 # Current cocotb-bus has a bug that is fixed in more up to date repo
 RUN python3 -m pip install git+https://github.com/cocotb/cocotb-bus.git cocotbext-axi

calyx-py/calyx/builder.py

@@ -132,6 +132,13 @@ def output(
         """
         return self._port_with_attributes(name, size, False, attribute_literals)
 
+    def output_with_attributes(self, name: str, size: int, attribute_literals: List[Union[str, Tuple[str, int]]]) -> ExprBuilder:
+        """Declare an output port on the component with attributes.
+
+        Returns an expression builder for the port.
+        """
+        return self._port_with_attributes(name, size, False, attribute_literals)
+
     def attribute(self, name: str, value: int) -> None:
         """Declare an attribute on the component."""
         self.component.attributes.append(ast.CompAttribute(name, value))
@@ -148,7 +155,6 @@ def _port_with_attributes(
 
         Returns an expression builder for the port.
         """
-
         attributes = []
         for attr in attribute_literals:
             if isinstance(attr, str):
@@ -549,7 +555,7 @@ def const_mult(
         name = name or self.generate_name("const_mult")
         self.prog.import_("primitives/binary_operators.futil")
         return self.cell(name, ast.Stdlib.const_mult(size, const))
-
+        
     def pad(self, in_width: int, out_width: int, name: str = None) -> CellBuilder:
         """Generate a StdPad cell."""
         name = name or self.generate_name("pad")

calyx-py/calyx/py_ast.py

@@ -86,6 +86,9 @@ def get_cell(self, name: str) -> Cell:
         )
 
     def doc(self) -> str:
+        # if(self.name == "axi_seq_mem_A0"):
+        #     for s in self.inputs:
+        #         print(s.doc())
         ins = ", ".join([s.doc() for s in self.inputs])
         outs = ", ".join([s.doc() for s in self.outputs])
         latency_annotation = (
@@ -173,6 +176,10 @@ class CompAttribute(Attribute):
     def doc(self) -> str:
         return f'"{self.name}"={self.value}'
 
+@dataclass
+class PortAttribute(Attribute):
+    name: str
+    value: Optional[int] = None
 
 @dataclass
 class PortAttribute(Attribute):

fud2/src/lib.rs

@@ -1,3 +1,5 @@
+use std::str::FromStr;
+
 use fud_core::{
     exec::{SetupRef, StateRef},
     run::{EmitResult, StreamEmitter},
@@ -792,11 +794,16 @@ pub fn build_driver(bld: &mut DriverBuilder) {
     let wrapper_setup = bld.setup("YXI and AXI generation", |e| {
         // Define a `gen-axi` rule that invokes our Python code generator program.
         // For now point to standalone axi-generator.py. Can maybe turn this into a rsrc file?
-        e.config_var_or(
-            "axi-generator",
-            "axi.generator",
-            "$calyx-base/yxi/axi-calyx/axi-generator.py",
-        )?;
+        let dynamic =
+            e.config_constrained_or("dynamic", vec!["true", "false"], "false")?;
+        let generator_path = if FromStr::from_str(&dynamic)
+            .expect("The dynamic flag should be either 'true' or 'false'.")
+        {
+            "$calyx-base/yxi/axi-calyx/dynamic-axi-generator.py"
+        } else {
+            "$calyx-base/yxi/axi-calyx/axi-generator.py"
+        };
+        e.config_var_or("axi-generator", "axi.generator", generator_path)?;
         e.config_var_or("python", "python", "python3")?;
 
         e.rule("gen-axi", "$python $axi-generator $in > $out")?;

runt.toml

@@ -66,7 +66,7 @@ cmd = """
 """
 
 [[tests]]
-name = "fud2 yxi"
+name = "fud2 yxi invocation"
 paths = ["yxi/tests/ref-mems-vec-add.futil"]
 cmd = """
 fud2 {} --from calyx --to yxi
@@ -552,34 +552,43 @@ fud e {} -s verilog.cycle_limit 500 \
          -s verilog.data {}.data \
          --to dat -q
 """
+
 ##### Calyx AXI tests #####
 [[tests]]
-name = "calyx-py AXI wrapper generation"
-paths = ["yxi/tests/axi/*.yxi"]
+name = "calyx-py read-write-compute AXI wrapper generation"
+paths = ["yxi/tests/axi/read-compute-write/*.yxi"]
 cmd = """
 python3 yxi/axi-calyx/axi-generator.py {}
 """
 
+[[tests]]
+name = "calyx-py dynamic AXI wrapper generation"
+paths = ["yxi/tests/axi/dynamic/*.yxi"]
+cmd = """
+python3 yxi/axi-calyx/dynamic-axi-generator.py {}
+"""
+
 #Ignore fud2 stderr for now due to ninja nondeterminism
 [[tests]]
 name = "fud2 AXI wrapper invocation"
-paths = ["yxi/tests/axi/seq-mem-vec-add.futil"]
+paths = ["yxi/tests/axi/*/*-mem-vec-add.futil"]
 cmd = """
-fud2 {} --to calyx --through axi-wrapped 2> /dev/null
+dynamic=$(head -n 2 {} | tail -n 1 | cut -c 4-)
+fud2 {} --to calyx --through axi-wrapped --set $dynamic 2> /dev/null
 """
 
 [[tests]]
 name = "fud2 AXI-wrapped to verilog"
-paths = ["yxi/tests/axi/seq-mem-vec-add-axi-wrapped.futil"]
+paths = ["yxi/tests/axi/*/*-mem-vec-add-axi-wrapped.futil"]
 cmd = """
 fud2 {} --from calyx --to verilog-noverify 2> /dev/null
 """
 
 [[tests]]
-name = "fud2 cocotb execution"
-paths = ["yxi/tests/axi/seq-mem-vec-add-verilog.v"]
+name = "fud2 cocotb AXI-wrapped xecution"
+paths = ["yxi/tests/axi/*/*-mem-vec-add-verilog.v"]
 cmd = """
-fud2 {} --from verilog-noverify --to cocotb-axi --set sim.data=yxi/tests/axi/vectorized-add.data 2> /dev/null
+fud2 {} --from verilog-noverify --to cocotb-axi --set sim.data=yxi/tests/axi/vectorized-add.data 2> /dev/null 
 """
 
 ##### Xilinx Tests ######