Sarah prepare deliverables

README.md

@@ -2,11 +2,13 @@
 
 ## Project Background
 
-[Please add project background]
+Perpetual's goal as a nonprofit is to promote sustainability in large cities by reducing single-use disposable waste.  Currently, individual restaurants may be reluctant to change from disposables to reusables due to higher costs of purchasing the foodware and maintaining in-house washing facilities. This project aims to help perpetual lower the cost of establishing a system for washing and redistributing reusable foodware to participating food and drink establishments (FUEs) throughout a given city.
+
+Perpetual has partnered with Galveston, TX, for its first launch for this system. This quarter's team was responsible for creating the first working model for the collection and distribution process in Galveston and generalizing this city’s model into a pipeline for building models in future participating cities.
 
 ## Project Goals
 
-[Please add project background]
+[ to be filled ]
 
 ## Usage
 
@@ -31,35 +33,32 @@ If you prefer to develop inside a container with VS Code then do the following s
 3. Click the blue or green rectangle in the bottom left of VS code (should say something like `><` or `>< WSL`). Options should appear in the top center of your screen. Select `Reopen in Container`.
 
 
-
-
 ## Repository Structure
 
-### utils
-Project python code
-
-### notebooks
-Contains short, clean notebooks to demonstrate analysis.
+### code
+[ to be filled ]
 
 ### data
-
+[ to be filled ]
 Contains details of acquiring all raw data used in repository. If data is small (<50MB) then it is okay to save it to the repo, making sure to clearly document how to the data is obtained.
 
 If the data is larger than 50MB than you should not add it to the repo and instead document how to get the data in the README.md file in the data directory. 
 
 This [README.md file](/data/README.md) should be kept up to date.
 
+### notebooks
+[ to be filled ]
+
 ### output
-Should contain work product generated by the analysis. Keep in mind that results should (generally) be excluded from the git repository.
+Contains intermediary/resulting data, routes, visualizations, and feasibility reports output by code
 
-## Group Members
+### utils
+Contains files with commonly used functions stored in utility files
 
 =======
-yifan wu, [email protected]
-Jessica Cibrian ([email protected])
 ## Project Team
-
+Yifan Wu, ([email protected])
+Jessica Cibrian ([email protected])
 Sarah Walker ([email protected])
-### Contacts
-Huanlin Dai, [email protected]
+Huanlin Dai, ([email protected])
 

code/README.md

@@ -1,15 +1,34 @@
 # Code
 
+* `bike_conversion.py`: Converts truck-serviced points to bike-serviced points when near other bike-serviced points
+  * Aggregate pickup/dropoff loads to designated aggregation points
+  * Separate .csv outputs into bike-only and truck-only datasets
+* `extract_capacity_demands.py`: extract pickup/dropoff demands for all locations
+* `extract_supplementary_info.py`: extract supplementary info for all locations (to help with visualization)
 * `GenerateDistMatrix.py`: This is a python script to generate distance matrices for any dataset
+* `RouteViz.py`: Given a folder/directory of routes, create interactive visualizations (.html).
+* `GenerateSingleSourceofTruth.py`: 
+  This is a python script to clean the data and create the single source of truth dataframe that will be used for all location points in a given city.
 
-# RouteWithTime.py
+
+* `RouteWithTime.py` :
   data: optimized route data
 
   this file will using the data and request the time duration between each stop from mapbox direction API and than accumulates all 
   the time plus 5 min picking up or drop up time at each stop.
+
 
-  # GetWayPoint.py
+* `GetWayPoint.py` :
   data: optimized route data
-
-  this file will using the data and request the waypoints through the route along with the instructions for driveing
 
+  this file will using the data and request the waypoints through the route along with the instructions for driving
+
+### Pipeline
+
+1. bike_conversion.py
+2. RouteViz.py
+* `complete_feasibility_report.py`: 
+  this script interacts with the utils/config.ini file to grab inputs and outputs from the pipeline and update the feasibility report for each new trial 
+
+* `perpetual_pipeline.py`: 
+  This is the pipeline for our project.

code/RouteViz.py

@@ -1,10 +1,14 @@
-import configparser
+import ast
+import fnmatch
+import os
 
 import folium
 import networkx as nx
 import osmnx as ox
 import pandas as pd
 
+from utils.utils import read_config
+
 
 def find_bbox(coords):
     """
@@ -90,7 +94,7 @@ def calc_routes(graph, coords):
     return osmnx_to_latlon(graph, routes)
 
 
-def add_markers(f_map, route_df, color):
+def add_markers(f_map, points, color):
     """
     given a folium map, route data (includes location names), and a color (str),
     draw markers on the given map
@@ -102,20 +106,29 @@ def add_markers(f_map, route_df, color):
     """
 
     # icon_size = 100
-    for i in range(len(route_df)):
-        row = route_df.iloc[i]
+    for i in range(len(points)):
+        row = points.iloc[i]
         loc_name = row["Name"]
         y, x = row[["Latitude", "Longitude"]]
         address = row["Address"]
+        index = points.index[i]
         dropoff, pickup = row[["Weekly_Dropoff_Totes", "Daily_Pickup_Totes"]]
+        pickup_type = row["pickup_type"]
+        agg_point = row["Bike Aggregation Point"]
         popup_html = f"""
+                Index: {index}
+                <br>
                 Name: {loc_name}
                 <br>
                 Address: {address}
                 <br>
                 Dropoff (weekly): {dropoff}
                 <br>
                 Pickup (daily): {pickup}
+                <br>
+                Pickup Type: {pickup_type}
+                <br>
+                Aggregation Point: {agg_point}
                 """
         popup = folium.Popup(popup_html, max_width=700)
 
@@ -128,64 +141,65 @@ def add_markers(f_map, route_df, color):
 
 if __name__ == "__main__":
 
-    config = configparser.ConfigParser()
-    config.read("../utils/config_inputs.ini")
-    cfg = config["route.viz"]
+    cfg = read_config("../utils/config_inputs.ini", "route.viz")
 
     place = cfg["place"]
-    FUE_path = cfg["FUE_path"]
-    route_path = cfg["route_path"]
+    route_dir = cfg["route_dir"]
     latitude = float(cfg["latitude"])
     longitude = float(cfg["longitude"])
-    save_path = cfg["save_path"]
-
-    feu_galveston = pd.read_csv(FUE_path)
-    route_data = pd.read_csv(route_path)
     location = [latitude, longitude]
-    coords = route_data[["Longitude", "Latitude"]]
-    # route_2_data = pd.read_csv("../data/route2.csv")
-    # coords2 = route_2_data[["Longitude", "Latitude"]]
-
-    n, s, e, w = find_bbox(coords)
-    # n2, s2, e2, w2 = find_bbox(coords2)
+    colors = ast.literal_eval(cfg["colors"])
 
     graph = ox.graph_from_place(place, network_type="drive")
 
-    galv_graph = ox.truncate.truncate_graph_bbox(
-        graph,
-        n,
-        s,
-        e,
-        w,
-        truncate_by_edge=False,
-        retain_all=False,
-        quadrat_width=0.05,
-        min_num=3,
+    # directory = 'files'
+
+    all_fmap = folium.Map(
+        location=location, tiles="OpenStreetMap", zoom_start=11
     )
-    # galv_graph2 = ox.truncate.truncate_graph_bbox(
-    #     graph,
-    #     n2,
-    #     s2,
-    #     e2,
-    #     w2,
-    #     truncate_by_edge=False,
-    #     retain_all=False,
-    #     quadrat_width=0.05,
-    #     min_num=3,
-    # )
-
-    route = calc_routes(galv_graph, coords)
-    # route_2 = calc_routes(galv_graph2, coords2)
-
-    map = folium.Map(location=location, tiles="OpenStreetMap", zoom_start=11)
-    add_markers(map, route_data, "blue")
-    # add_markers(map, route_2_data, "red")
-    folium.PolyLine(locations=route, color="blue").add_to(map)
-    # folium.PolyLine(locations=route_2, color="red").add_to(map)
-
-    for y, x in route:
-        folium.CircleMarker(
-            location=[y, x], radius=2, weight=5, color="yellow"
-        ).add_to(map)
-
-    map.save(save_path)
+
+    i = 0
+    for root, _, files in os.walk(route_dir):
+        for filename in fnmatch.filter(files, "*.csv"):
+            filepath = os.path.join(root, filename)
+            if os.path.isfile(filepath):
+                name = filename[:-4]
+                print(f"mapping {name}")
+                route_data = pd.read_csv(filepath)
+                coords = route_data[["Longitude", "Latitude"]]
+                n, s, e, w = find_bbox(coords)
+
+                galv_graph = ox.truncate.truncate_graph_bbox(
+                    graph,
+                    n,
+                    s,
+                    e,
+                    w,
+                    truncate_by_edge=False,
+                    retain_all=False,
+                    quadrat_width=0.05,
+                    min_num=3,
+                )
+
+                route = calc_routes(galv_graph, coords)
+                # route_2 = calc_routes(galv_graph2, coords2)
+
+                color = colors[(i % len(colors))]
+
+                fmap = folium.Map(
+                    location=location, tiles="OpenStreetMap", zoom_start=11
+                )
+                add_markers(fmap, route_data, "blue")
+                add_markers(all_fmap, route_data, color)
+
+                folium.PolyLine(locations=route, color="blue").add_to(fmap)
+                folium.PolyLine(locations=route, color=color).add_to(all_fmap)
+
+                for y, x in route:
+                    folium.CircleMarker(
+                        location=[y, x], radius=2, weight=5, color="yellow"
+                    ).add_to(fmap)
+
+                fmap.save(route_dir + "/" "map_" + name + ".html")
+                i += 1
+    all_fmap.save(route_dir + "/" + "map_all_routes.html")

code/archive/README.md

@@ -2,22 +2,27 @@
 # Code archive
 
 
-* `optimize_cvrp_galv_old.py` : This is a python script to run the capacited vehicle routing problem specific to the 'FUE_Galveston.csv' (trial 1) dataset (note: this only includes pickup routing)
+* `optimize_cvrp_galv_old.py` : 
+  This is a python script to run the capacited vehicle routing problem specific to the 'FUE_Galveston.csv' (trial 1) dataset (note: this only includes pickup routing)
 
-* `capacitatedVRP.py` : This is a python script to run the capacited vehicle routing problem specific to the 'FUE_Galveston.csv' (trial 1) dataset (note: this only includes pickup routing)
+* `capacitatedVRP.py` : 
+  This is a python script to run the capacited vehicle routing problem specific to the 'FUE_Galveston.csv' (trial 1) dataset (note: this only includes pickup routing)
 
-# Capacity_galveston.py
+* `Capacity_galveston.py` :
   run GenerateDistMatrix.py first, the output will stored in the ../data file.
   using both distance_matrices file and capacity_list file (does not include in this branch, but you can find it in the main),
   ortools helped us generates the optimized routes.
   adjust the time constraints, we will get more optimized result.
 
 
-* `ClusteringGAL.py` : This is a python script to create clusters of the Galveston dataset based on pickup or dropoff quantities 
+* `ClusteringGAL.py` : 
+  This is a python script to create clusters of the Galveston dataset based on pickup or dropoff quantities 
 
-* `cvrp_galv_dropoff_only.py` : This is a python script to run the capacited vehicle routing problem specific to tracking dropoff capacities only (generalized so you can change arguments specific to your question)
+* `cvrp_galv_dropoff_only.py` : 
+  This is a python script to run the capacited vehicle routing problem specific to tracking dropoff capacities only (generalized so you can change arguments specific to your question)
 
-* `cvrp_galv_pickup_only.py` : This is a python script to run the capacited vehicle routing problem specific to tracking pickup capacities only (generalized so you can change arguments specific to your question)
+* `cvrp_galv_pickup_only.py` : 
+  This is a python script to run the capacited vehicle routing problem specific to tracking pickup capacities only (generalized so you can change arguments specific to your question)