-
Notifications
You must be signed in to change notification settings - Fork 0
/
outline.txt
29 lines (24 loc) · 934 Bytes
/
outline.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
Step 1: Download all data
FOR TOPIC IN TOPICS:
------------------------------------------------------ #Bash Script
Step 2: Cut all data
i. Divide each topic with unique IDs find number of posts --
ii. Extract only body
a. Topic-ID.csv
chemistry-0.csv
chemistry-n.csv
------------------------------------------------------#Rscript
Step 4: Clean body
i: Lowercase
ii: Remove Stop words Look up best methods in R
iii: Stem the words running -> run Look up best methods in R
iv: Finish cleaning Anything Else?
Step 5: Create word frequency matrix
i. Topic-ID-TF.csv #Output contains term frequency
Step 6: Save to output
------------------------------
Step 7: Aggregate matrices for all files in a topic
Step 8: Save final word-frequency matrix
------------------------------------------------------
Step 9: Visualize the final matrix
Either a single vector passed into Rscript