feat: 添加minicpm3模型缺失的算子#16

Draft

onenewcode wants to merge 7 commits intoYdrMaster:mainfrom

Contributor

onenewcode commented Feb 14, 2025

No description provided.


          fix: 支持attention计算 q，k，v不相同的情况

120d30f

github-advanced-security bot found potential problems

View reviewed changes

github-advanced-security bot left a comment

clippy found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.


          feat: 添加Scale算子

f4a83f7

onenewcode force-pushed the dev branch from fc5a50d to f4a83f7 Compare

February 14, 2025 06:14

YdrMaster marked this pull request as draft

February 14, 2025 12:19

onenewcode force-pushed the dev branch 5 times, most recently from ec21c79 to 7fa85f7 Compare

February 18, 2025 03:25

github-advanced-security bot found potential problems

View reviewed changes

operators/src/scale/args.rs Fixed Show fixed Hide fixed

operators/src/scale/cuda/mod.rs Fixed Show fixed Hide fixed

operators/src/scale/cuda/mod.rs Fixed Show fixed Hide fixed

operators/src/scale/cuda/mod.rs Fixed Show fixed Hide fixed

operators/src/scale/cuda/mod.rs Fixed Show fixed Hide fixed

operators/src/scale/cuda/mod.rs Fixed Show fixed Hide fixed

operators/src/scale/cuda/mod.rs Fixed Show fixed Hide fixed

operators/src/scale/cuda/mod.rs Fixed Show fixed Hide fixed

operators/src/scale/cuda/mod.rs Fixed Show fixed Hide fixed

operators/src/scale/cuda/mod.rs Fixed Show fixed Hide fixed


          fix: 添加更多rope

ab65dfa

onenewcode force-pushed the dev branch 4 times, most recently from e754707 to ab65dfa Compare

February 20, 2025 07:46

github-advanced-security bot found potential problems

View reviewed changes

operators/src/attention_mla/args.rs

+                  pub dr: MaybeDyn<usize>,
+              }
+              impl<H: Hardware> Args<H> {

Check warning

Code scanning / clippy

associated function new_null is never used Warning

associated function new_null is never used

operators/src/attention_mla/args.rs

+              impl<H: Hardware> Args<H> {
+                  #[allow(clippy::too_many_arguments)]
+                  pub(crate) fn new_null(

Check warning

Code scanning / clippy

associated function new_null is never used Warning

associated function new_null is never used

operators/src/attention_mla/operator.rs Fixed Show fixed Hide fixed

operators/src/attention_mla/operator.rs Fixed Show fixed Hide fixed

operators/src/attention_mla/operator.rs

+              use ndarray_layout::ArrayLayout;
+              use std::marker::PhantomData;
+              pub struct Operator<Hardware, MatMul, Softmax, Rearrange> {

Check warning

Code scanning / clippy

field rearrange is never read Warning

field rearrange is never read

operators/src/attention_mla/operator.rs

+              pub struct Operator<Hardware, MatMul, Softmax, Rearrange> {
+                  mat_mul: MatMul,
+                  softmax: Softmax,
+                  rearrange: Rearrange,

Check warning

Code scanning / clippy

field rearrange is never read Warning

field rearrange is never read

operators/src/attention_mla/operator.rs

+                  fn scheme(
+                      &mut self,
+                      args: &Self::Args,

Check warning

Code scanning / clippy

unused variable: args Warning

unused variable: args

operators/src/attention_mla/operator.rs

+                  fn scheme(
+                      &mut self,
+                      args: &Self::Args,
+                      max_workspace_size: usize,

Check warning

Code scanning / clippy

unused variable: max_workspace_size Warning

unused variable: max_workspace_size


          feat: 添加mmla的吸收版本

6989b92

onenewcode force-pushed the dev branch from 23f8506 to 6989b92 Compare

February 24, 2025 04:59

github-advanced-security bot found potential problems

View reviewed changes

operators/src/attention_mla_cached/operator.rs

+                  fn scheme(
+                      &mut self,
+                      args: &Self::Args,

Check warning

Code scanning / clippy

unused variable: args Warning

unused variable: args

operators/src/attention_mla_cached/operator.rs

+                  fn scheme(
+                      &mut self,
+                      args: &Self::Args,
+                      max_workspace_size: usize,

Check warning

Code scanning / clippy

unused variable: max_workspace_size Warning

unused variable: max_workspace_size

operators/src/attention_mla_cached/operator.rs

+                          dt,
+                          nh,
+                          seq,
+                          att,

Check warning

Code scanning / clippy

unused variable: att Warning

unused variable: att

operators/src/attention_mla_cached/operator.rs

+                          seq,
+                          att,
+                          dkv,
+                          dv,

Check warning

Code scanning / clippy

unused variable: dv Warning

unused variable: dv

operators/src/attention_mla_cached/operator.rs

+                          mask,
+                          pos,
+                      } = args;
+                      let &[nh_skv, att_skv, dkv_skv] = kv_layout.strides() else {

Check warning

Code scanning / clippy

unused variable: nh_skv Warning

unused variable: nh_skv

operators/src/attention_mla_cached/operator.rs

+                      let &[nh_skv, att_skv, dkv_skv] = kv_layout.strides() else {
+                          unreachable!()
+                      };
+                      let &[nh_skr, att_skr, dr_skr] = kr_layout.strides() else {

Check warning

Code scanning / clippy

unused variable: dr_skr Warning

unused variable: dr_skr

operators/src/attention_mla_cached/operator.rs

+                      let &[nh_skr, att_skr, dr_skr] = kr_layout.strides() else {
+                          unreachable!()
+                      };
+                      let &[nh_sa, dv_sa, dkv_sa] = absorb_layout.strides() else {

Check warning

Code scanning / clippy

unused variable: nh_sa Warning

unused variable: nh_sa

operators/src/attention_mla_cached/operator.rs

+                      let &[nh_skr, att_skr, dr_skr] = kr_layout.strides() else {
+                          unreachable!()
+                      };
+                      let &[nh_sa, dv_sa, dkv_sa] = absorb_layout.strides() else {

Check warning

Code scanning / clippy

unused variable: dv_sa Warning

unused variable: dv_sa

operators/src/attention_mla_cached/operator.rs

+                      let &[nh_skr, att_skr, dr_skr] = kr_layout.strides() else {
+                          unreachable!()
+                      };
+                      let &[nh_sa, dv_sa, dkv_sa] = absorb_layout.strides() else {

Check warning

Code scanning / clippy

unused variable: dkv_sa Warning

unused variable: dkv_sa

operators/src/attention_mla_cached/operator.rs

+                      let &[nh_skrc, buf_skrc, dh_skrc] = kr_cache_layout.strides() else {
+                          unreachable!()
+                      };
+                      let ele = dt.nbytes();

Check warning

Code scanning / clippy

unused variable: ele Warning

unused variable: ele


          feat: 添加mla_cache

ab2c196

onenewcode force-pushed the dev branch from e44cff1 to ab2c196 Compare

February 24, 2025 09:25

1e4b2f5

github-advanced-security bot found potential problems

View reviewed changes

operators/src/rope/common_cpu/mod.rs

+                                  short,
+                                  origin_pos,
+                                  ..
+                              } => unsafe {

Check warning

Code scanning / clippy

unnecessary unsafe block Warning

unnecessary unsafe block


          finish

20f9c5a

YdrMaster force-pushed the main branch from 61dba6d to a645f82 Compare

February 27, 2025 11:19

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet