Code_Gen questions

[n0thingNoob]

%6 = "neura.constant"() <{predicate = true, value = 1 : i64}> {mapping_locs = [{id = 2 : i32, resource = "tile", time_step = 0 : i32, x = 2 : i32, y = 0 : i32}]} : () -> !neura.data<i64, i1>
%7 = "neura.data_mov"(%6) {mapping_locs = []} : (!neura.data<i64, i1>) -> !neura.data<i64, i1>
%8 = "neura.grant_once"(%7) {mapping_locs = [{id = 2 : i32, resource = "tile", time_step = 1 : i32, x = 2 : i32, y = 0 : i32}]} : (!neura.data<i64, i1>) -> !neura.data<i64, i1>

In the branch_for mapping at tile (2, 0), %6 seems to be used locally, but the corresponding data_mov doesn’t reference any register resource. Because of this, I can’t determine either the register or the direction of the connection between %6 and %8. I am not sure what's going on with this.

[tancheng]

Nice catch, yeah, we didn't model such case carefully, CCing @ShangkunLi for fix.

For now, can you just give a random reg to it? say reg3? and hope this reg won't be over-write by others? you can use the last register available (reg15 or reg31) in your model architecture.

(if we do see overwriting happen after you generate the instructions, or during simulation, we can manually change it. how does this sound?) Meanwhile, @ShangkunLi can help with a proper fix these days?

[n0thingNoob]

Sure, I can give it a try. I might also have some additional questions. Would it be okay if I use this issue to ask them, instead of opening new ones each time?

[tancheng]

Sure. Just reuse this thread.

[n0thingNoob]

In the instruction

%41 = neura.grant_predicate %39, %40 {mapping_locs = [{id = 6 : i32, resource = "tile", time_step = 5, x = 2, y = 1}]}
neura.ctrl_mov %41 -> %27 {mapping_locs = [{id = 17, resource = "link", time_step = 5}, {id = 20, resource = "register", time_step = 6}, {id = 20, resource = "register", time_step = 7}]}

It shows that %41 is moved via a link at timestep 5, and also written to the same register at timestep 6 and 7.

Also same issue with this one:
%42 = "neura.data_mov"(%32) {mapping_locs = [{id = 33 : i32, resource = "link", time_step = 5 : i32}, {id = 17 : i32, resource = "link", time_step = 6 : i32}, {id = 21 : i32, resource = "register", time_step = 7 : i32}, {id = 21 : i32, resource = "register", time_step = 8 : i32}]} : (!neura.data<f32, i1>) -> !neura.data<f32, i1>

In our YAML, each instruction currently only has a single time_step and src/dst operand list. How should we represent the fact that a single value has multiple destinations at different timesteps (multiple links + repeated register writes)?

[tancheng]

Can u also provide %27 %28 %29 and the one who uses %42?

[n0thingNoob]

%27 = neura.reserve : !neura.data<i64, i1>
%28 = "neura.data_mov"(%5) {mapping_locs = [{id = 4 : i32, resource = "link", time_step = 1 : i32}]} : (!neura.data<i64, i1>) -> !neura.data<i64, i1>
%29 = "neura.phi"(%27, %28) {mapping_locs = [{id = 5 : i32, resource = "tile", time_step = 2 : i32, x = 1 : i32, y = 1 : i32}]} : (!neura.data<i64, i1>, !neura.data<i64, i1>) -> !neura.data<i64, i1>

%44 = neura.grant_predicate %42, %43 {mapping_locs = [{id = 5 : i32, resource = "tile", time_step = 9 : i32, x = 1 : i32, y = 1 : i32}]} : !neura.data<f32, i1>, !neura.data<i1, i1> -> !neura.data<f32, i1>
neura.ctrl_mov %44 -> %24 {mapping_locs = [{id = 14 : i32, resource = "link", time_step = 9 : i32}]} : !neura.data<f32, i1> !neura.data<f32, i1>

[n0thingNoob]

I can post the whole mapping here.

module {
func.func @loop_test() -> f32 attributes {accelerator = "neura", mapping_info = {compiled_ii = 6 : i32, mapping_mode = "spatial-temporal", mapping_strategy = "heuristic", rec_mii = 4 : i32, res_mii = 2 : i32, x_tiles = 4 : i32, y_tiles = 4 : i32}} {
%0 = "neura.constant"() <{predicate = true, value = 10 : i64}> {mapping_locs = [{id = 0 : i32, resource = "tile", time_step = 1 : i32, x = 0 : i32, y = 0 : i32}]} : () -> !neura.data<i64, i1>
%1 = "neura.data_mov"(%0) {mapping_locs = [{id = 0 : i32, resource = "link", time_step = 1 : i32}]} : (!neura.data<i64, i1>) -> !neura.data<i64, i1>
%2 = "neura.grant_once"(%1) {mapping_locs = [{id = 1 : i32, resource = "tile", time_step = 2 : i32, x = 1 : i32, y = 0 : i32}]} : (!neura.data<i64, i1>) -> !neura.data<i64, i1>
%3 = "neura.constant"() <{predicate = true, value = 0 : i64}> {mapping_locs = [{id = 0 : i32, resource = "tile", time_step = 0 : i32, x = 0 : i32, y = 0 : i32}]} : () -> !neura.data<i64, i1>
%4 = "neura.data_mov"(%3) {mapping_locs = [{id = 0 : i32, resource = "link", time_step = 0 : i32}]} : (!neura.data<i64, i1>) -> !neura.data<i64, i1>
%5 = "neura.grant_once"(%4) {mapping_locs = [{id = 1 : i32, resource = "tile", time_step = 1 : i32, x = 1 : i32, y = 0 : i32}]} : (!neura.data<i64, i1>) -> !neura.data<i64, i1>
%6 = "neura.constant"() <{predicate = true, value = 1 : i64}> {mapping_locs = [{id = 2 : i32, resource = "tile", time_step = 0 : i32, x = 2 : i32, y = 0 : i32}]} : () -> !neura.data<i64, i1>
%7 = "neura.data_mov"(%6) {mapping_locs = []} : (!neura.data<i64, i1>) -> !neura.data<i64, i1>
%8 = "neura.grant_once"(%7) {mapping_locs = [{id = 2 : i32, resource = "tile", time_step = 1 : i32, x = 2 : i32, y = 0 : i32}]} : (!neura.data<i64, i1>) -> !neura.data<i64, i1>
%9 = "neura.constant"() <{predicate = true, value = 3.000000e+00 : f32}> {mapping_locs = [{id = 14 : i32, resource = "tile", time_step = 2 : i32, x = 2 : i32, y = 3 : i32}]} : () -> !neura.data<f32, i1>
%10 = "neura.data_mov"(%9) {mapping_locs = [{id = 45 : i32, resource = "link", time_step = 2 : i32}]} : (!neura.data<f32, i1>) -> !neura.data<f32, i1>
%11 = "neura.grant_once"(%10) {mapping_locs = [{id = 10 : i32, resource = "tile", time_step = 3 : i32, x = 2 : i32, y = 2 : i32}]} : (!neura.data<f32, i1>) -> !neura.data<f32, i1>
%12 = "neura.constant"() <{predicate = true, value = 0.000000e+00 : f32}> {mapping_locs = [{id = 11 : i32, resource = "tile", time_step = 2 : i32, x = 3 : i32, y = 2 : i32}]} : () -> !neura.data<f32, i1>
%13 = "neura.data_mov"(%12) {mapping_locs = [{id = 36 : i32, resource = "link", time_step = 2 : i32}]} : (!neura.data<f32, i1>) -> !neura.data<f32, i1>
%14 = "neura.grant_once"(%13) {mapping_locs = [{id = 7 : i32, resource = "tile", time_step = 3 : i32, x = 3 : i32, y = 1 : i32}]} : (!neura.data<f32, i1>) -> !neura.data<f32, i1>
%15 = neura.reserve : !neura.data<i64, i1>
%16 = "neura.data_mov"(%2) {mapping_locs = []} : (!neura.data<i64, i1>) -> !neura.data<i64, i1>
%17 = "neura.phi"(%15, %16) {mapping_locs = [{id = 1 : i32, resource = "tile", time_step = 3 : i32, x = 1 : i32, y = 0 : i32}]} : (!neura.data<i64, i1>, !neura.data<i64, i1>) -> !neura.data<i64, i1>
%18 = neura.reserve : !neura.data<i64, i1>
%19 = "neura.data_mov"(%8) {mapping_locs = []} : (!neura.data<i64, i1>) -> !neura.data<i64, i1>
%20 = "neura.phi"(%18, %19) {mapping_locs = [{id = 2 : i32, resource = "tile", time_step = 2 : i32, x = 2 : i32, y = 0 : i32}]} : (!neura.data<i64, i1>, !neura.data<i64, i1>) -> !neura.data<i64, i1>
%21 = neura.reserve : !neura.data<f32, i1>
%22 = "neura.data_mov"(%11) {mapping_locs = [{id = 34 : i32, resource = "link", time_step = 3 : i32}]} : (!neura.data<f32, i1>) -> !neura.data<f32, i1>
%23 = "neura.phi"(%21, %22) {mapping_locs = [{id = 14 : i32, resource = "tile", time_step = 4 : i32, x = 2 : i32, y = 3 : i32}]} : (!neura.data<f32, i1>, !neura.data<f32, i1>) -> !neura.data<f32, i1>
%24 = neura.reserve : !neura.data<f32, i1>
%25 = "neura.data_mov"(%14) {mapping_locs = [{id = 21 : i32, resource = "link", time_step = 3 : i32}]} : (!neura.data<f32, i1>) -> !neura.data<f32, i1>
%26 = "neura.phi"(%24, %25) {mapping_locs = [{id = 6 : i32, resource = "tile", time_step = 4 : i32, x = 2 : i32, y = 1 : i32}]} : (!neura.data<f32, i1>, !neura.data<f32, i1>) -> !neura.data<f32, i1>
%27 = neura.reserve : !neura.data<i64, i1>
%28 = "neura.data_mov"(%5) {mapping_locs = [{id = 4 : i32, resource = "link", time_step = 1 : i32}]} : (!neura.data<i64, i1>) -> !neura.data<i64, i1>
%29 = "neura.phi"(%27, %28) {mapping_locs = [{id = 5 : i32, resource = "tile", time_step = 2 : i32, x = 1 : i32, y = 1 : i32}]} : (!neura.data<i64, i1>, !neura.data<i64, i1>) -> !neura.data<i64, i1>
%30 = "neura.data_mov"(%26) {mapping_locs = [{id = 20 : i32, resource = "link", time_step = 4 : i32}]} : (!neura.data<f32, i1>) -> !neura.data<f32, i1>
%31 = "neura.data_mov"(%23) {mapping_locs = [{id = 45 : i32, resource = "link", time_step = 4 : i32}]} : (!neura.data<f32, i1>) -> !neura.data<f32, i1>
%32 = "neura.fadd"(%30, %31) {mapping_locs = [{id = 10 : i32, resource = "tile", time_step = 5 : i32, x = 2 : i32, y = 2 : i32}]} : (!neura.data<f32, i1>, !neura.data<f32, i1>) -> !neura.data<f32, i1>
%33 = "neura.data_mov"(%29) {mapping_locs = [{id = 14 : i32, resource = "link", time_step = 2 : i32}]} : (!neura.data<i64, i1>) -> !neura.data<i64, i1>
%34 = "neura.data_mov"(%20) {mapping_locs = [{id = 7 : i32, resource = "link", time_step = 2 : i32}]} : (!neura.data<i64, i1>) -> !neura.data<i64, i1>
%35 = "neura.add"(%33, %34) {mapping_locs = [{id = 6 : i32, resource = "tile", time_step = 3 : i32, x = 2 : i32, y = 1 : i32}]} : (!neura.data<i64, i1>, !neura.data<i64, i1>) -> !neura.data<i64, i1>
%36 = "neura.data_mov"(%35) {mapping_locs = [{id = 17 : i32, resource = "link", time_step = 3 : i32}]} : (!neura.data<i64, i1>) -> !neura.data<i64, i1>
%37 = "neura.data_mov"(%17) {mapping_locs = [{id = 4 : i32, resource = "link", time_step = 3 : i32}]} : (!neura.data<i64, i1>) -> !neura.data<i64, i1>
%38 = "neura.icmp"(%36, %37) <{cmpType = "slt"}> {mapping_locs = [{id = 5 : i32, resource = "tile", time_step = 4 : i32, x = 1 : i32, y = 1 : i32}]} : (!neura.data<i64, i1>, !neura.data<i64, i1>) -> !neura.data<i1, i1>
%39 = "neura.data_mov"(%35) {mapping_locs = []} : (!neura.data<i64, i1>) -> !neura.data<i64, i1>
%40 = "neura.data_mov"(%38) {mapping_locs = [{id = 14 : i32, resource = "link", time_step = 4 : i32}]} : (!neura.data<i1, i1>) -> !neura.data<i1, i1>
%41 = neura.grant_predicate %39, %40 {mapping_locs = [{id = 6 : i32, resource = "tile", time_step = 5 : i32, x = 2 : i32, y = 1 : i32}]} : !neura.data<i64, i1>, !neura.data<i1, i1> -> !neura.data<i64, i1>
neura.ctrl_mov %41 -> %27 {mapping_locs = [{id = 17 : i32, resource = "link", time_step = 5 : i32}, {id = 20 : i32, resource = "register", time_step = 6 : i32}, {id = 20 : i32, resource = "register", time_step = 7 : i32}]} : !neura.data<i64, i1> !neura.data<i64, i1>
%42 = "neura.data_mov"(%32) {mapping_locs = [{id = 33 : i32, resource = "link", time_step = 5 : i32}, {id = 17 : i32, resource = "link", time_step = 6 : i32}, {id = 21 : i32, resource = "register", time_step = 7 : i32}, {id = 21 : i32, resource = "register", time_step = 8 : i32}]} : (!neura.data<f32, i1>) -> !neura.data<f32, i1>
%43 = "neura.data_mov"(%38) {mapping_locs = []} : (!neura.data<i1, i1>) -> !neura.data<i1, i1>
%44 = neura.grant_predicate %42, %43 {mapping_locs = [{id = 5 : i32, resource = "tile", time_step = 9 : i32, x = 1 : i32, y = 1 : i32}]} : !neura.data<f32, i1>, !neura.data<i1, i1> -> !neura.data<f32, i1>
neura.ctrl_mov %44 -> %24 {mapping_locs = [{id = 14 : i32, resource = "link", time_step = 9 : i32}]} : !neura.data<f32, i1> !neura.data<f32, i1>
%45 = "neura.data_mov"(%11) {mapping_locs = [{id = 31 : i32, resource = "link", time_step = 3 : i32}, {id = 27 : i32, resource = "link", time_step = 4 : i32}, {id = 32 : i32, resource = "register", time_step = 5 : i32}, {id = 32 : i32, resource = "register", time_step = 6 : i32}]} : (!neura.data<f32, i1>) -> !neura.data<f32, i1>
%46 = "neura.data_mov"(%38) {mapping_locs = [{id = 13 : i32, resource = "link", time_step = 4 : i32}, {id = 12 : i32, resource = "link", time_step = 5 : i32}, {id = 33 : i32, resource = "register", time_step = 6 : i32}]} : (!neura.data<i1, i1>) -> !neura.data<i1, i1>
%47 = neura.grant_pre dicate %45, %46 {mapping_locs = [{id = 8 : i32, resource = "tile", time_step = 7 : i32, x = 0 : i32, y = 2 : i32}]} : !neura.data<f32, i1>, !neura.data<i1, i1> -> !neura.data<f32, i1>
neura.ctrl_mov %47 -> %21 {mapping_locs = [{id = 24 : i32, resource = "link", time_step = 7 : i32}, {id = 30 : i32, resource = "link", time_step = 8 : i32}, {id = 41 : i32, resource = "link", time_step = 9 : i32}]} : !neura.data<f32, i1> !neura.data<f32, i1>
%48 = "neura.data_mov"(%8) {mapping_locs = [{id = 7 : i32, resource = "link", time_step = 1 : i32}, {id = 24 : i32, resource = "register", time_step = 2 : i32}, {id = 24 : i32, resource = "register", time_step = 3 : i32}, {id = 24 : i32, resource = "register", time_step = 4 : i32}, {id = 24 : i32, resource = "register", time_step = 5 : i32}, {id = 24 : i32, resource = "register", time_step = 6 : i32}]} : (!neura.data<i64, i1>) -> !neura.data<i64, i1>
%49 = "neura.data_mov"(%38) {mapping_locs = [{id = 16 : i32, resource = "link", time_step = 4 : i32}, {id = 28 : i32, resource = "link", time_step = 5 : i32}, {id = 33 : i32, resource = "link", time_step = 6 : i32}]} : (!neura.data<i1, i1>) -> !neura.data<i1, i1>
%50 = neura.grant_predicate %48, %49 {mapping_locs = [{id = 6 : i32, resource = "tile", time_step = 7 : i32, x = 2 : i32, y = 1 : i32}]} : !neura.data<i64, i1>, !neura.data<i1, i1> -> !neura.data<i64, i1>
neura.ctrl_mov %50 -> %18 {mapping_locs = [{id = 19 : i32, resource = "link", time_step = 7 : i32}]} : !neura.data<i64, i1> !neura.data<i64, i1>
%51 = "neura.data_mov"(%2) {mapping_locs = [{id = 4 : i32, resource = "link", time_step = 2 : i32}, {id = 21 : i32, resource = "register", time_step = 3 : i32}, {id = 21 : i32, resource = "register", time_step = 4 : i32}, {id = 21 : i32, resource = "register", time_step = 5 : i32}, {id = 21 : i32, resource = "register", time_step = 6 : i32}]} : (!neura.data<i64, i1>) -> !neura.data<i64, i1>
%52 = "neura.data_mov"(%38) {mapping_locs = []} : (!neura.data<i1, i1>) -> !neura.data<i1, i1>
%53 = neura.grant_predicate %51, %52 {mapping_locs = [{id = 5 : i32, resource = "tile", time_step = 7 : i32, x = 1 : i32, y = 1 : i32}]} : !neura.data<i64, i1>, !neura.data<i1, i1> -> !neura.data<i64, i1>
neura.ctrl_mov %53 -> %15 {mapping_locs = [{id = 15 : i32, resource = "link", time_step = 7 : i32}, {id = 4 : i32, resource = "register", time_step = 8 : i32}]} : !neura.data<i64, i1> !neura.data<i64, i1>
%54 = "neura.data_mov"(%38) {mapping_locs = [{id = 15 : i32, resource = "link", time_step = 4 : i32}]} : (!neura.data<i1, i1>) -> !neura.data<i1, i1>
%55 = "neura.not"(%54) {mapping_locs = [{id = 1 : i32, resource = "tile", time_step = 5 : i32, x = 1 : i32, y = 0 : i32}]} : (!neura.data<i1, i1>) -> !neura.data<i1, i1>
%56 = "neura.data_mov"(%32) {mapping_locs = [{id = 31 : i32, resource = "link", time_step = 5 : i32}, {id = 36 : i32, resource = "register", time_step = 6 : i32}]} : (!neura.data<f32, i1>) -> !neura.data<f32, i1>
%57 = "neura.data_mov"(%55) {mapping_locs = [{id = 4 : i32, resource = "link", time_step = 5 : i32}, {id = 16 : i32, resource = "link", time_step = 6 : i32}]} : (!neura.data<i1, i1>) -> !neura.data<i1, i1>
%58 = neura.grant_predicate %56, %57 {mapping_locs = [{id = 9 : i32, resource = "tile", time_step = 7 : i32, x = 1 : i32, y = 2 : i32}]} : !neura.data<f32, i1>, !neura.data<i1, i1> -> !neura.data<f32, i1>
%59 = "neura.data_mov"(%58) {mapping_locs = [{id = 30 : i32, resource = "link", time_step = 7 : i32}]} : (!neura.data<f32, i1>) -> !neura.data<f32, i1>
"neura.return"(%59) {mapping_locs = [{id = 13 : i32, resource = "tile", time_step = 8 : i32, x = 1 : i32, y = 3 : i32}]} : (!neura.data<f32, i1>) -> ()
}
}

[tancheng]

The mapping looks good to me. One data_mov could occupy multiple different links, indicating they need to traverse multiple tiles/links until the destination tile, I believe you understand this. And this implies one data_mov sometimes cannot be represented by single instructions, as it may require multiple instructions to reach the destination.

• A work around to help this is to duplicate data_mov into multiple data_mov ops, each op only holds single loc info. CCing @ShangkunLi for a potential pass implementation to ease the codegen life.

Similarly, for data_mov occupying multiple registers, usually, it won't change to another register (as you mentioned "repeated"). This indicates the data will stay inside one register multiple time_steps (cycles) before being consumed.

In our YAML, each instruction currently only has a single time_step and src/dst operand list. How should we represent the fact that a single value has multiple destinations at different timesteps (multiple links + repeated register writes)?

I didn't see your design yet. But I personally think we should generate instructions based on loc, i.e., using a data structure of map<<time_step%II, tile_id>, [operations]> or map<<tile_id, time_step%II>, [operations]>. In other words, we need assemble the IRs into this data structure first, then generate instructions, which will be robust and easier, rather than iterating each IR op, generating YAML instruction, then manipulate them.

I am not against your current design. But if you iterate IR op and generate instruction based on value, you'd better generate one or multiple instructions per value. And later on, you still need to merge instructions with same tile id and time step. For example, a value is from addOp at tile 5 on time step 9, an outport link of tile 5 is occupied by a data_mov on time step 9 as well (time step 3 and time step 15 can be viewed as time step 9 as well due to the compiled II = 6), the two instructions need to be merged into one in your yaml instruction format, as we don't separate router and PE, your one instruction indicating both computation and independent data routing. Is this already done?

So which one you prefer?

• Use your current design, generating multiple instructions per value, manipulate/merge them if needed
• A pass insert data_mov to guarantee each data_mov only contains single loc, then assemble all Ops into map<<tile_id, time_step%II>, [operations]>, then generate each instruction based on each entry of the map

I think the second option is cleaner, WDYT?

[n0thingNoob]

I agree with the second approach and can try using a data structure to implement it, since I am currently iterating over the IR. I am trying to do the merge thing right now. My current format is based on the Slack version, including details such as time_step and predicate. Let me show a small example:
array_config:
columns: 4
rows: 4
cores:
-
column: 0
row: 0
core_id: 0
entries:
-
entry_id: entry0
type: loop
instructions:
-
opcode: CONSTANT
time_step: 0
src_operands:
-
operand: #0
color: RED
dst_operands:
-
operand: EAST
color: RED
predicate: true
-
opcode: CONSTANT
time_step: 1
src_operands:
-
operand: #10
color: RED
dst_operands:
-
operand: EAST
color: RED
predicate: true

[tancheng]

It seems u don't even to merge inst as simulator allows multiple events at same cycle for same tile. U can make the call.

[ShangkunLi]

Nice catch, yeah, we didn't model such case carefully, CCing @ShangkunLi for fix.

For now, can you just give a random reg to it? say reg3? and hope this reg won't be over-write by others? you can use the last register available (reg15 or reg31) in your model architecture.

(if we do see overwriting happen after you generate the instructions, or during simulation, we can manually change it. how does this sound?) Meanwhile, @ShangkunLi can help with a proper fix these days?

Sorry for late reply, I will fix it ASAP.

[tancheng]

Nice catch, yeah, we didn't model such case carefully, CCing @ShangkunLi for fix.
For now, can you just give a random reg to it? say reg3? and hope this reg won't be over-write by others? you can use the last register available (reg15 or reg31) in your model architecture.
(if we do see overwriting happen after you generate the instructions, or during simulation, we can manually change it. how does this sound?) Meanwhile, @ShangkunLi can help with a proper fix these days?

Sorry for late reply, I will fix it ASAP.

Thanks @ShangkunLi, I think your fix would be related to the empty loc, i.e., loc = [ ], inside the data_mov. I am not exactly sure whether we need to have another pass to separate single data_mov into multiple data_mov, where the needed info can be easily done by a helper func inside the codegen pass.

[ShangkunLi]

Thanks @ShangkunLi, I think your fix would be related to the empty loc, i.e., loc = [ ], inside the data_mov. I am not exactly sure whether we need to have another pass to separate single data_mov into multiple data_mov, where the needed info can be easily done by a helper func inside the codegen pass.

I think the compiler should be in charge of register binding, so it should be better to allocate a register for the empty loc in data/ctrl_mov.