Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DO NOT MERGE] Dev/groupby order preservation poc #3000

Draft
wants to merge 44 commits into
base: master
Choose a base branch
from
Draft
Changes from 1 commit
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
3cae9e7
PoC complete. need to test
sak2002 Apr 19, 2022
73b61f9
Merge branch 'master' of https://github.com/sak2002/mars into dev/gro…
sak2002 Apr 19, 2022
aa39493
method auto will also use sort. added changes so that if partition do…
sak2002 Apr 20, 2022
9823ddf
Merge branch 'master' of https://github.com/mars-project/mars into de…
sak2002 Apr 21, 2022
98b995a
fixing unit tests
sak2002 Apr 22, 2022
0fcbc47
added unit tests for group by sort
sak2002 Apr 22, 2022
8186e64
fixed conflicts
sak2002 Apr 25, 2022
a7684d5
fixed conflicts
sak2002 Apr 25, 2022
f9f4e9c
sample output chunk shape does not rely on by
sakshamkumar-byt Apr 26, 2022
2f2f4d9
order preserve poc
sak2002 Apr 25, 2022
c93adb0
impl execute for indexing phase
sakshamkumar-byt Apr 28, 2022
ef8bfaa
Merge remote-tracking branch 'upstream/master' into dev/groupby-sort-poc
May 5, 2022
e75bfcf
Fix output_type of DataFrameGroupbyAgg map
May 5, 2022
3966d05
removed redundant comments
sakshamkumar-byt May 5, 2022
dd37724
changes for order preservation
sakshamkumar-byt May 5, 2022
0b379c9
pulled origin master. fixed merge conflict
sakshamkumar-byt May 5, 2022
83c49ad
Merge branch 'dev/groupby-sort-poc' of https://github.com/sak2002/mar…
sakshamkumar-byt May 5, 2022
dfbb6da
Made changes for sampling on the basis of series or dataframe output
sakshamkumar-byt May 5, 2022
03fb0e1
code cleanup
sakshamkumar-byt May 5, 2022
ff58e50
rebased to groupby-sort-poc. working poc for order preservation for d…
sakshamkumar-byt May 5, 2022
59c447f
Fix ut
May 6, 2022
19dc420
merge back groupby-sort-poc
sakshamkumar-byt May 8, 2022
e658810
fixed imports. removed redundant code
sakshamkumar-byt May 8, 2022
1126278
reformat code
sakshamkumar-byt May 8, 2022
49ee66f
code style check
sakshamkumar-byt May 8, 2022
3bd71ea
code style check
sakshamkumar-byt May 8, 2022
10591b6
added apache 2.0 licence
sakshamkumar-byt May 8, 2022
19cedfb
code cleanup
sakshamkumar-byt May 8, 2022
72c6c89
Merge branch 'master' of https://github.com/mars-project/mars into de…
sakshamkumar-byt May 8, 2022
df6f6fa
Added test for empty df
sakshamkumar-byt May 8, 2022
86bab10
removed unused import
sakshamkumar-byt May 8, 2022
68ef1ac
fixed code style issues
sakshamkumar-byt May 8, 2022
f893fc3
removed redundant code
sakshamkumar-byt May 8, 2022
9f88148
Merge branch 'dev/groupby-sort-poc' of https://github.com/sak2002/mar…
sakshamkumar-byt May 8, 2022
515f692
made changes to map and reduce stage to support series and dataframe
sakshamkumar-byt May 9, 2022
bb0236d
resolved a few review comments
sakshamkumar-byt May 9, 2022
daf4319
code style checks
sakshamkumar-byt May 9, 2022
0c8ca0b
code style checks
sakshamkumar-byt May 9, 2022
e98e585
resolved a few review comments
sakshamkumar-byt May 9, 2022
8ac2555
resolved review comment to direct output type to init
sakshamkumar-byt May 9, 2022
b09f47c
removed unused sort
sakshamkumar-byt May 9, 2022
ccbdebd
Merge remote-tracking branch 'upstream/master' into dev/groupby-sort-poc
May 9, 2022
ea997b2
Fix ut
May 9, 2022
6b01acb
merge back sort poc
sakshamkumar-byt May 10, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
pulled origin master. fixed merge conflict
sakshamkumar-byt committed May 5, 2022
commit 0b379c9376f022cbdec4db367eddef32a0206ba3
8 changes: 4 additions & 4 deletions mars/dataframe/groupby/aggregation.py
Original file line number Diff line number Diff line change
@@ -128,10 +128,10 @@ def _group_kurt(x, *args, **kwargs):


def build_mock_agg_result(
groupby: GROUPBY_TYPE,
groupby_params: typing.Dict,
raw_func: typing.Callable,
**raw_func_kw,
groupby: GROUPBY_TYPE,
groupby_params: Dict,
raw_func: Callable,
**raw_func_kw,
):
try:
agg_result = groupby.op.build_mock_groupby().aggregate(raw_func, **raw_func_kw)
You are viewing a condensed version of this merge commit. You can view the full changes here.