Skip to content

Commit b3c0005

Browse files
gfursingfursin
committed
formatted Python modules from the internal repository using autopep8
1 parent 80959de commit b3c0005

File tree

9 files changed

+650
-582
lines changed

9 files changed

+650
-582
lines changed

‎README.md

+13-17
Original file line numberDiff line numberDiff line change
@@ -13,28 +13,31 @@
1313

1414
[Collective Knowledge (CK, CM, CM4MLOps, CM4MLPerf and CMX)](https://cKnowledge.org)
1515
is an educational community project to learn how to run AI, ML and other emerging workloads
16-
in the most efficient and cost-effective way across diverse models, data sets, software and hardware.
16+
in the most efficient and cost-effective way across diverse models, data sets, software and hardware:
17+
[ [white paper](https://arxiv.org/abs/2406.16791) ].
18+
1719
It includes the following sub-projects.
1820

1921
### Collective Mind (CM)
2022

21-
[Collective Mind (CM)](https://github.com/mlcommons/ck/tree/master/cm) -
22-
a very lightweight Python-based framework with a unified CLI, Python API and minimal dependencies
23-
intended to help researchers and engineers automate their repetitive, tedious and time-consuming tasks
24-
to build, run, benchmark and optimize AI, ML and other applications and systems
25-
across diverse and continuously changing models, data, software and hardware.
23+
The [Collective Mind framework](https://github.com/mlcommons/ck/tree/master/cm)
24+
is a lightweight, Python-based toolset featuring
25+
a unified command-line interface (CLI), Python API, and minimal dependencies.
26+
It is designed to assist researchers and engineers in automating repetitive, time-consuming
27+
tasks such as building, running, benchmarking, and optimizing AI, machine learning,
28+
and other applications across diverse and continuously changing models, data, software and hardware.
2629

27-
Collective Mind is continuously enhanced through public and private CM4* Git repositories,
28-
which serve as the unified interface for various collections of reusable automations and artifacts.
30+
Collective Mind is continuously enhanced through public and private Git repositories
31+
with CM automation recipes and artifacts accessible via unified CM interface.
2932

3033
The CM architecture diagram is available for viewing
3134
[here](https://github.com/mlcommons/ck/tree/master/docs/specs/cm-diagram-v3.5.1.png).
3235

33-
### Collective Mind repositories
36+
### Notable Collective Mind repositories
3437

3538
#### CM4MLOps
3639

37-
[CM4MLOPS repository powered by CM](https://github.com/mlcommons/cm4mlops) -
40+
[CM4MLOPS repository powered by CM](https://github.com/mlcommons/ck/tree/master/cm-mlops) -
3841
a collection of portable, extensible and technology-agnostic automation recipes
3942
with a common CLI and Python API (CM scripts) to unify and automate
4043
all the manual steps required to compose, run, benchmark and optimize complex ML/AI applications
@@ -121,13 +124,6 @@ based on user feedback. Follow the project's progress [here]( https://github.com
121124

122125
* [Grigori Fursin](https://cKnowledge.org/gfursin) (FlexAI, cTuning)
123126

124-
## Maintainers
125-
126-
* Collective Mind (CM): [Grigori Fursin](https://cKnowledge.org/gfursin)
127-
* CM4MLOps repository: [Arjun Suresh](https://github.com/arjunsuresh) and [Anandhu Sooraj](https://github.com/anandhu-eng)
128-
* CMX (the next generation of CM and CM4MLOps): [Grigori Fursin](https://cKnowledge.org/gfursin)
129-
130-
131127
## Citing Collective Mind and Collective Knowledge
132128

133129
If you found the CM automation framework helpful, kindly reference this article:

‎cm-mlops/README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
**This repository is archived. In April 2024, we have separated this CM repo from the CM framework
2-
and moved it to a [standalone mlcommons@cm4mlops repository](https://github.com/mlcommons/cm4mlops/tree/dev)
2+
and moved it to a [standalone mlcommons@cm4mlops repository](https://github.com/mlcommons/cm4mlops)
33
following the suggesion from our users and MLCommons members.**
44

55
We keep this directory for backwards compatibility to let users reproduce past projects relying on this repository.

‎cm/CHANGES.md

+3
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
## V3.5.1.1
2+
- formatted Python modules from the internal repository using autopep8
3+
14
## V3.5.1
25
- added Collective Mind architecture diagram:
36
https://github.com/mlcommons/ck/tree/master/docs/specs/cm-diagram-v3.5.1.png

‎cm/cmind/repo/automation/automation/module.py

+42-34
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
from cmind.automation import Automation
88
from cmind import utils
99

10+
1011
class CAutomation(Automation):
1112
"""
1213
CM "automation" automation actions
@@ -35,12 +36,12 @@ def print_input(self, i):
3536
"""
3637

3738
import json
38-
print (json.dumps(i, indent=2))
39-
40-
return {'return':0}
39+
print(json.dumps(i, indent=2))
4140

41+
return {'return': 0}
4242

4343
############################################################
44+
4445
def add(self, i):
4546
"""
4647
Add CM automation.
@@ -66,36 +67,40 @@ def add(self, i):
6667

6768
console = i.get('out') == 'con'
6869

69-
parsed_artifact = i.get('parsed_artifact',[])
70+
parsed_artifact = i.get('parsed_artifact', [])
7071

71-
artifact_obj = parsed_artifact[0] if len(parsed_artifact)>0 else ('','')
72+
artifact_obj = parsed_artifact[0] if len(
73+
parsed_artifact) > 0 else ('', '')
7274

7375
module_name = 'module.py'
7476

7577
tags_list = utils.convert_tags_to_list(i)
76-
if 'automation' not in tags_list: tags_list.append('automation')
78+
if 'automation' not in tags_list:
79+
tags_list.append('automation')
7780

7881
# Add placeholder (use common action)
79-
i['out']='con'
80-
i['common']=True
82+
i['out'] = 'con'
83+
i['common'] = True
8184

82-
i['meta']={'automation_alias':self.meta['alias'],
83-
'automation_uid':self.meta['uid'],
84-
'tags':tags_list}
85+
i['meta'] = {'automation_alias': self.meta['alias'],
86+
'automation_uid': self.meta['uid'],
87+
'tags': tags_list}
8588

86-
if 'tags' in i: del(i['tags'])
89+
if 'tags' in i:
90+
del (i['tags'])
8791

8892
automation = i['automation']
89-
if automation!='.' and ',' not in automation:
93+
if automation != '.' and ',' not in automation:
9094
i['automation'] = automation + ',' + self.meta['uid']
9195

92-
r_obj=self.cmind.access(i)
93-
if r_obj['return']>0: return r_obj
96+
r_obj = self.cmind.access(i)
97+
if r_obj['return'] > 0:
98+
return r_obj
9499

95100
new_automation_path = r_obj['path']
96101

97102
if console:
98-
print ('Created automation in {}'.format(new_automation_path))
103+
print('Created automation in {}'.format(new_automation_path))
99104

100105
# Create Python module holder
101106
module_holder_path = new_automation_path
@@ -106,12 +111,12 @@ def add(self, i):
106111
# Copy module files
107112
for f in ['module_dummy.py']:
108113
f1 = os.path.join(self.path, f)
109-
f2 = os.path.join(new_automation_path, f.replace('_dummy',''))
114+
f2 = os.path.join(new_automation_path, f.replace('_dummy', ''))
110115

111116
if console:
112-
print (' * Copying {} to {}'.format(f1, f2))
117+
print(' * Copying {} to {}'.format(f1, f2))
113118

114-
shutil.copyfile(f1,f2)
119+
shutil.copyfile(f1, f2)
115120

116121
return r_obj
117122

@@ -137,33 +142,37 @@ def add_cmx(self, i):
137142

138143
# Prepare to call common function
139144
r = utils.process_input(i)
140-
if r['return']>0: return r
145+
if r['return'] > 0:
146+
return r
141147

142148
# Take only out from original control
143-
i['control']={'out':i['control']['out'],
144-
'common':True}
149+
i['control'] = {'out': i['control']['out'],
150+
'common': True}
145151

146152
tags_list = utils.convert_tags_to_list(i)
147-
if 'automation' not in tags_list: tags_list.append('automation')
153+
if 'automation' not in tags_list:
154+
tags_list.append('automation')
148155

149-
i['meta']={'automation_alias':self.meta['alias'],
150-
'automation_uid':self.meta['uid'],
151-
'tags':tags_list}
156+
i['meta'] = {'automation_alias': self.meta['alias'],
157+
'automation_uid': self.meta['uid'],
158+
'tags': tags_list}
152159

153-
if 'tags' in i: del(i['tags'])
160+
if 'tags' in i:
161+
del (i['tags'])
154162

155163
# Use yaml by default
156164
if 'yaml' not in i:
157165
i['yaml'] = True
158166

159167
# Pass to common action
160168
r_obj = self.cmind.x(i)
161-
if r_obj['return']>0: return r_obj
169+
if r_obj['return'] > 0:
170+
return r_obj
162171

163172
new_automation_path = r_obj['path']
164173

165174
if console:
166-
print ('Created automation in {}'.format(new_automation_path))
175+
print('Created automation in {}'.format(new_automation_path))
167176

168177
module_name = 'modulex.py'
169178

@@ -176,18 +185,17 @@ def add_cmx(self, i):
176185
# Copy module files
177186
for f in ['modulex_dummy.py']:
178187
f1 = os.path.join(self.path, f)
179-
f2 = os.path.join(new_automation_path, f.replace('_dummy',''))
188+
f2 = os.path.join(new_automation_path, f.replace('_dummy', ''))
180189

181190
if console:
182-
print (' * Copying {} to {}'.format(f1, f2))
191+
print(' * Copying {} to {}'.format(f1, f2))
183192

184-
shutil.copyfile(f1,f2)
193+
shutil.copyfile(f1, f2)
185194

186195
return r_obj
187196

188-
189-
190197
############################################################
198+
191199
def doc(self, i):
192200
"""
193201
Add CM automation.

0 commit comments

Comments
 (0)