Code Generate Pass by n0thingNoob · Pull Request #88 · coredac/dataflow

n0thingNoob · 2025-07-20T21:10:32Z

Current work

Fixed source direction calculation for operations with data_mov operands
Added op_to_source_tile mapping to track original source locations
Enhanced ASM generation with correct Entry conditions and operand directions
Added test results in the Generated_Code folder

Future Enhancements Needed

Add comprehensive error handling and validation
Refactor duplicate code in JSON/ASM generation
Add configuration options for output customization
Implement unit tests and integration tests
Support more complex data flow patterns
Add debugging and diagnostic capabilities

Testing

Tested with test/test.mlir, the results of asm and json look right, but might be missing some details.

Files Changed

lib/NeuraDialect/Transforms/GenerateCodePass.cpp
Generated_Code/ (new folder with test results)

- Add PE ID system with PE(id) as keys - Organize instructions by PE coordinates (x, y) - Calculate data movement directions (N, S, E, W, NE, SE, NW, SW, Local) - Include complete instruction information (opcode, time_step, src_tile, dst_tile, operands) - Support constant value extraction - Generate structured JSON output for PE instructions

- Add example JSON output showing PE-based instruction organization - Demonstrates the structure with PE(id) keys and coordinate information - Shows complete instruction details including directions and time steps - Serves as reference for the code generation pass output format

…obust mapping handling

…source direction calculation

…prove return operation handling

Generated_Code/README.md

Generated_Code/test.mlir

lib/NeuraDialect/Transforms/GenerateCodePass.cpp

Generated_Code/generated-instructions.asm

tancheng · 2025-07-21T00:06:17Z

@ShangkunLi any idea why the build time out? no changes related to mapper in this PR.

n0thingNoob · 2025-07-21T00:24:39Z

Great, I still need more time to fix those errors. And I need to discuss some issue with @Jackcuii

ShangkunLi · 2025-07-21T04:00:31Z

@ShangkunLi any idea why the build time out? no changes related to mapper in this PR.

@n0thingNoob @tancheng I built and ran the code in this branch locally, the segment faults seem to come from the --generate-code pass.

n0thingNoob · 2025-07-21T16:08:23Z

@ShangkunLi any idea why the build time out? no changes related to mapper in this PR.

@n0thingNoob @tancheng I built and ran the code in this branch locally, the segment faults seem to come from the --generate-code pass.

ok, let me check what happened.

n0thingNoob · 2025-07-21T18:14:16Z

@ShangkunLi any idea why the build time out? no changes related to mapper in this PR.

@n0thingNoob @tancheng I built and ran the code in this branch locally, the segment faults seem to come from the --generate-code pass.

ok, let me check what happened.

@ShangkunLi How can I run the same test locally? I want to see what happened to my code

tancheng · 2025-07-21T18:41:36Z

@ShangkunLi any idea why the build time out? no changes related to mapper in this PR.

@n0thingNoob @tancheng I built and ran the code in this branch locally, the segment faults seem to come from the --generate-code pass.

ok, let me check what happened.

@ShangkunLi How can I run the same test locally? I want to see what happened to my code

dataflow/README.md

Line 51 in 493f4cb

$ ../build/tools/mlir-neura-opt/mlir-neura-opt --debug test.mlir

and include all the needed passes:

dataflow/test/neura/ctrl/branch_for.mlir

Lines 32 to 38 in 493f4cb

    
           // RUN:   --assign-accelerator \ 
        
           // RUN:   --lower-llvm-to-neura \ 
        
           // RUN:   --leverage-predicated-value \ 
        
           // RUN:   --transform-ctrl-to-data-flow \ 
        
           // RUN:   --insert-data-mov \ 
        
           // RUN:   --map-to-accelerator="mapping-strategy=heuristic" \ 
        
           // RUN:   --generate-code

n0thingNoob · 2025-07-21T20:46:40Z

Looks like my revision passed. I can keep working on other issues.

tancheng · 2025-07-21T21:33:12Z

Looks like my revision passed. I can keep working on other issues.

You still need to check your functionality in some tests like this

dataflow/test/neura/ctrl/branch_for.mlir

Lines 31 to 40 in 493f4cb

    
           // RUN: mlir-neura-opt %s \ 
        
           // RUN:   --assign-accelerator \ 
        
           // RUN:   --lower-llvm-to-neura \ 
        
           // RUN:   --leverage-predicated-value \ 
        
           // RUN:   --transform-ctrl-to-data-flow \ 
        
           // RUN:   --insert-data-mov \ 
        
           // RUN:   --map-to-accelerator="mapping-strategy=heuristic" \ 
        
           // RUN:   --generate-code 
        
           // RU: FileCheck %s --input-file=generated-instructions.json -check-prefix=INST

n0thingNoob · 2025-07-22T21:27:41Z

Looks like my revision passed. I can keep working on other issues.

You still need to check your functionality in some tests like this

dataflow/test/neura/ctrl/branch_for.mlir

Lines 31 to 40 in 493f4cb

// RUN: mlir-neura-opt %s \

// RUN: --assign-accelerator \

// RUN: --lower-llvm-to-neura \

// RUN: --leverage-predicated-value \

// RUN: --transform-ctrl-to-data-flow \

// RUN: --insert-data-mov \

// RUN: --map-to-accelerator="mapping-strategy=heuristic" \

// RUN: --generate-code

// RU: FileCheck %s --input-file=generated-instructions.json -check-prefix=INST

Yes, I am working on that.

lib/NeuraDialect/Transforms/GenerateCodePass.cpp

n0thingNoob · 2025-09-06T01:06:59Z

Updates: removed redundant IR walk() calls and unified traversal into a single function-level pass. Also standardized variable and function names (need reviews) @tancheng

tancheng

Great job Yijia~!

Plz fix the comments style and variable naming convention. Also, plz let me click on the "solve" once you fix all of them. Thanks.

lib/NeuraDialect/Transforms/GenerateCodePass.cpp

test/code_gen/test_code_generate.mlir

lib/NeuraDialect/Transforms/GenerateCodePass.cpp

test/code_gen/test_code_generate.mlir

lib/NeuraDialect/Transforms/GenerateCodePass.cpp

n0thingNoob · 2025-09-06T16:44:07Z

Great job Yijia~!

Plz fix the comments style and variable naming convention. Also, plz let me click on the "solve" once you fix all of them. Thanks.

Will do it ASAP. And for the last question:
Because not all operations have register mappings. The caller already handles this case properly with if (auto mapped_register_id = getMappedRegId(op)). We can add debug info here if necessary

This is the old README file for the test case

n0thingNoob · 2025-09-06T22:34:21Z

I am not sure why there is a conflict when pushing the first commit. I merged the latest main into architecture_spec to resolve it and removed the obsolete README that was only used by the old test. @tancheng

tancheng · 2025-09-06T22:40:10Z

I am not sure why there is a conflict when pushing the first commit. I merged the latest main into architecture_spec to resolve it and removed the obsolete README that was only used by the old test. @tancheng

I didn't get it. Will you provide a README or not?

n0thingNoob · 2025-09-06T23:40:57Z

I am not sure why there is a conflict when pushing the first commit. I merged the latest main into architecture_spec to resolve it and removed the obsolete README that was only used by the old test. @tancheng

I didn't get it. Will you provide a README or not?

Perhaps not, the branch test is a little bit complicated. Is the README necessary for this test?

tancheng · 2025-09-07T01:45:00Z

I am not sure why there is a conflict when pushing the first commit. I merged the latest main into architecture_spec to resolve it and removed the obsolete README that was only used by the old test. @tancheng

I didn't get it. Will you provide a README or not?

Perhaps not, the branch test is a little bit complicated. Is the README necessary for this test?

Hi @n0thingNoob, I just meant to have simple explanation maybe about only one tile for two time steps. You don't need explain what happen on all the tiles across all time steps. How does this sound? You can put your README.md back in your test folder, like what @HobbitQia (https://github.com/coredac/dataflow/tree/main/test/visualize) and @ShangkunLi (https://github.com/coredac/dataflow/tree/main/test) did. Sounds good to you?

n0thingNoob · 2025-09-07T03:21:04Z

I am not sure why there is a conflict when pushing the first commit. I merged the latest main into architecture_spec to resolve it and removed the obsolete README that was only used by the old test. @tancheng

I didn't get it. Will you provide a README or not?

Perhaps not, the branch test is a little bit complicated. Is the README necessary for this test?

Hi @n0thingNoob, I just meant to have simple explanation maybe about only one tile for two time steps. You don't need explain what happen on all the tiles across all time steps. How does this sound? You can put your README.md back in your test folder, like what @HobbitQia (https://github.com/coredac/dataflow/tree/main/test/visualize) and @ShangkunLi (https://github.com/coredac/dataflow/tree/main/test) did. Sounds good to you?

Sure.

test/code_gen/README.md

tancheng · 2025-09-07T16:14:37Z

You are aware of your always failed test, right? https://github.com/coredac/dataflow/actions/runs/17523525173/job/49770978606

And it should be an easy fix for you I think.

n0thingNoob · 2025-09-07T18:46:53Z

You are aware of your always failed test, right? https://github.com/coredac/dataflow/actions/runs/17523525173/job/49770978606

And it should be an easy fix for you I think.

Working on it.

test/code_gen/README.md

tancheng · 2025-09-08T01:05:35Z

okay to merge now @n0thingNoob? or anything else you want to fix?

n0thingNoob · 2025-09-08T01:43:06Z

okay to merge now @n0thingNoob? or anything else you want to fix?

Yes, let's do it for now.

Code Generate Pass

n0thingNoob added 6 commits July 19, 2025 19:39

feat: enhance GenerateCodePass with PE instruction organization and r…

a73637e

…obust mapping handling

merge: resolve conflicts and sync GenerateCodePass from main

3f9f560

example json file output from test.mlir

131bb17

Add Generated_Code folder with test results and fix GenerateCodePass …

1bd9c99

…source direction calculation

n0thingNoob changed the title ~~Architecture spec~~ Code Generate Pass Jul 20, 2025

Update GenerateCodePass: output files to Generated_Code folder and im…

1f0802e

…prove return operation handling

tancheng requested review from Jackcuii, ShangkunLi, itemkelvin, syifan, tancheng and yuqisun July 20, 2025 23:33

tancheng assigned n0thingNoob Jul 20, 2025

tancheng added the new feature New feature or request label Jul 20, 2025

tancheng reviewed Jul 21, 2025

View reviewed changes

fix some small issue, still working on file check and visualized demo

5813cd1

add protection for potential segmentation fault

d0647e6

add File Check to the test_code_generate.mlir

655f089

test file

0364a85

tancheng reviewed Sep 5, 2025

View reviewed changes

lib/NeuraDialect/Transforms/GenerateCodePass.cpp Outdated Show resolved Hide resolved

n0thingNoob added 2 commits September 5, 2025 20:53

optimize the code, modified some variable names and function names

cea312b

Reduce times of walking through all IR

37b19b8

n0thingNoob added 3 commits September 5, 2025 21:25

reivse some comments and helper function

691404b

FIX small output mistake

b9276a3

FIX small mistake

131d533

tancheng reviewed Sep 6, 2025

View reviewed changes

n0thingNoob and others added 3 commits September 6, 2025 18:01

Modify comments, variables, testcase

b5754c8

Merge branch 'main' into architecture_spec

5b0bffc

Delete test/code_gen/README.md

74ffd8b

This is the old README file for the test case

Create README.md for test

3671c38

tancheng reviewed Sep 7, 2025

View reviewed changes

test/code_gen/README.md Show resolved Hide resolved

test/code_gen/README.md Outdated Show resolved Hide resolved

test/code_gen/README.md Show resolved Hide resolved

Update README.md

1d11598

Update README.md

2dc7929

tancheng approved these changes Sep 7, 2025

View reviewed changes

test/code_gen/README.md Show resolved Hide resolved

fix failed test

730c5e4

n0thingNoob merged commit 65911ff into main Sep 8, 2025
1 check passed

ShangkunLi pushed a commit that referenced this pull request Mar 12, 2026

Merge pull request #88 from coredac/architecture_spec

c040307

Code Generate Pass

ShangkunLi pushed a commit that referenced this pull request Mar 12, 2026

Merge pull request #88 from coredac/architecture_spec

cf3c92b

Code Generate Pass

Conversation

n0thingNoob commented Jul 20, 2025 • edited by tancheng Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Current work

Future Enhancements Needed

Testing

Files Changed

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tancheng commented Jul 21, 2025

Uh oh!

n0thingNoob commented Jul 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ShangkunLi commented Jul 21, 2025

Uh oh!

n0thingNoob commented Jul 21, 2025

Uh oh!

n0thingNoob commented Jul 21, 2025

Uh oh!

tancheng commented Jul 21, 2025

Uh oh!

n0thingNoob commented Jul 21, 2025

Uh oh!

tancheng commented Jul 21, 2025

Uh oh!

n0thingNoob commented Jul 22, 2025

Uh oh!

Uh oh!

n0thingNoob commented Sep 6, 2025

Uh oh!

tancheng left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

n0thingNoob commented Sep 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

n0thingNoob commented Sep 6, 2025

Uh oh!

tancheng commented Sep 6, 2025

Uh oh!

n0thingNoob commented Sep 6, 2025

Uh oh!

tancheng commented Sep 7, 2025

Uh oh!

n0thingNoob commented Sep 7, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tancheng commented Sep 7, 2025

Uh oh!

n0thingNoob commented Sep 7, 2025

Uh oh!

Uh oh!

tancheng commented Sep 8, 2025

Uh oh!

Uh oh!

n0thingNoob commented Sep 8, 2025

Uh oh!

Reviewers

n0thingNoob commented Jul 20, 2025 •

edited by tancheng

Loading

n0thingNoob commented Jul 21, 2025 •

edited

Loading

n0thingNoob commented Sep 6, 2025 •

edited

Loading