UofT-DSI | Production- Assignment 1

[hashimi07]

What changes are you trying to make? (e.g. Adding or removing code, refactoring existing code, adding reports)

• Added my solutions Assignment 1, including all required code for importing data, converting CSVs to Parquet, feature engineering with Dask, and calculating moving averages.
• Added comments and notes explaining my process and code.

What did you learn from the changes you have made?

• I learned how to efficiently process large stock datasets using Dask.
• Understood the advantages of Dask vs Pandas for handling big data and parallel computation.

Was there another approach you were thinking about making? If so, what approach(es) were you thinking of?

• I considered using only Pandas for the entire workflow, but chose Dask for scalability as suggested by the assignment guidelines.

Were there any challenges? If so, what issue(s) did you face? How did you overcome it?

• Faced challenges with environment variable setup and path issues for data import.
• The CSV to Parquet conversion took some time, but running the code step-by-step and verifying the Parquet output helped ensure everything was correct.
• Double-checked file paths and added comments for clarity.

How were these changes tested?

• Ran all notebook cells to ensure code executes successfully.
• Verified that Parquet files were created and features like moving averages and high-low ranges are present in the final DataFrame.
• Checked outputs with head() to confirm correct calculations.

A reference to a related issue in your repository (if applicable)

N/A

Checklist

• [xYes] I can confirm that my changes are working as intended

[Shiraz-Latif]

approved