Adding defog Restaurants database for testing #449

john-sanchez31 · 2025-10-27T21:31:57Z

Resolves #448

knassre-bodo

Revisions left for restaurant code

knassre-bodo · 2025-10-30T17:30:16Z

tests/test_pydough_functions/defog_test_functions.py

+    What is the total count of restaurants in each city?
+    """
+    return locations.PARTITION(name="city", by=city_name).CALCULATE(
+        city_name, total_count=NDISTINCT(locations.restaurant_id)


Should be able to just do COUNT(locations) (since each restaurant_id value is unique)

knassre-bodo · 2025-10-30T17:32:28Z

tests/test_metadata/defog_graphs.json

+            "column name": "county",
+            "data type": "string",
+            "description": "The name of the county",
+            "sample values": ["New York", "San Francisco"],


These county names are ALSO city names. Let's include more sample values that are unambiguously county names (make sure to update the other graphs): Miami-Dade, Cook

knassre-bodo · 2025-10-30T17:33:37Z

tests/test_metadata/defog_graphs.json

+        "name": "geographies",
+        "type": "simple table",
+        "table path": "main.geographic",
+        "unique properties": [
+          "city_name"


Since each row represents a unique city, I'd advise changing the name of the collection to cities, and altering the description accordingly.

knassre-bodo · 2025-10-30T17:34:53Z

tests/test_metadata/defog_graphs.json

+        "type": "simple table",
+        "table path": "main.location",
+        "unique properties": [
+          "restaurant_id"


Wouldn't each combination of (house_number, street_name, city_name) also be unique?

Do you mean adding in unique properties this ["restaurant_id", ["house_number", "street_name", "city_name"]] or [["restaurant_id", "house_number", "street_name", "city_name"]] ? Will this change impact how the current queries are written?

The first one:

"restaurant_id", ["house_number", "street_name", "city_name"]]

And no it won't necessarily change any of the current queries.

knassre-bodo · 2025-10-30T17:36:34Z

tests/test_metadata/defog_graphs.json

+    "relationships": [
+      {
+        "type": "simple join",
+        "name": "locations",


I'd call this restaurant_locations since it is mapping a city to each restaurant location within the city.

knassre-bodo · 2025-10-30T17:52:36Z

tests/test_pydough_functions/defog_test_functions.py

+    high_rated_restaurants = COUNT(
+        restaurants.WHERE((rating > 4.0) & (LOWER(city_name) == "new york"))
+    )
+    low_rated_restaurants = COUNT(
+        restaurants.WHERE((rating < 4.0) & (LOWER(city_name) == "new york"))
+    )
+    return Restaurants.CALCULATE(
+        ratio=(
+            high_rated_restaurants
+            / KEEP_IF(low_rated_restaurants, low_rated_restaurants > 0)
+        )
+    )


Same as before:

Suggested change

high_rated_restaurants = COUNT(

restaurants.WHERE((rating > 4.0) & (LOWER(city_name) == "new york"))

)

low_rated_restaurants = COUNT(

restaurants.WHERE((rating < 4.0) & (LOWER(city_name) == "new york"))

)

return Restaurants.CALCULATE(

ratio=(

high_rated_restaurants

/ KEEP_IF(low_rated_restaurants, low_rated_restaurants > 0)

)

)

nyc_restaurants = restaurants.WHERE(LOWER(city_name) == "new york")

n_hi_rating = SUM(nyc_restaurants.rating > 4.0)

n_lo_rating= SUM(nyc_restaurants.rating > 4.0)

return Restaurants.CALCULATE(

ratio=(

n_hi_rating

/ KEEP_IF(n_lo_rating, n_lo_rating != 0)

)

)

knassre-bodo · 2025-10-30T18:00:13Z

tests/test_pydough_functions/defog_test_functions.py

+    vegan_sf_rest = COUNT(
+        restaurants.WHERE(
+            (LOWER(food_type) == "vegan") & (LOWER(city_name) == "san francisco")
+        )
+    )
+    no_vegan_sf_rest = COUNT(
+        restaurants.WHERE(
+            (LOWER(food_type) != "vegan") & (LOWER(city_name) == "san francisco")
+        )
+    )
+    return Restaurants.CALCULATE(
+        ratio=(vegan_sf_rest / KEEP_IF(no_vegan_sf_rest, no_vegan_sf_rest > 0))
+    )


Same as before:

Suggested change

vegan_sf_rest = COUNT(

restaurants.WHERE(

(LOWER(food_type) == "vegan") & (LOWER(city_name) == "san francisco")

)

)

no_vegan_sf_rest = COUNT(

restaurants.WHERE(

(LOWER(food_type) != "vegan") & (LOWER(city_name) == "san francisco")

)

)

return Restaurants.CALCULATE(

ratio=(vegan_sf_rest / KEEP_IF(no_vegan_sf_rest, no_vegan_sf_rest > 0))

)

sf_restaurants = restaurants.WHERE(LOWER(city_name) == "san francisco")

n_vegan = SUM(LOWER(sf_restaurants.rating) == "vegan")

n_non_vegan= SUM(LOWER(sf_restaurants.rating) != "vegan")

return Restaurants.CALCULATE(

ratio=(

n_hi_rating

/ KEEP_IF(n_lo_rating, n_lo_rating != 0)

)

)

knassre-bodo · 2025-11-03T18:27:25Z

tests/test_pydough_functions/defog_test_functions.py

+    italian_la_rest = COUNT(
+        restaurants.WHERE(
+            (LOWER(food_type) == "italian") & (LOWER(city_name) == "los angeles")
+        )
+    )
+    no_italian_la_rest = COUNT(restaurants.WHERE((LOWER(city_name) == "los angeles")))
+    return Restaurants.CALCULATE(
+        ratio=(italian_la_rest / KEEP_IF(no_italian_la_rest, no_italian_la_rest > 0))
+    )


Same as before:

la_restaurants = restaurants.WHERE(LOWER(city_name) == "los angeles") n_la_italian = SUM(la_restaurants.food_type == "italian") n_la = COUNT(la_restaurants) return Restaurants.CALCULATE( ratio=( n_la_italian / KEEP_IF(n_la, n_la != 0) ) )

knassre-bodo · 2025-11-03T18:29:28Z

tests/test_pydough_functions/defog_test_functions.py

+        geographies.WHERE(COUNT(restaurants) > 0)
+        .PARTITION(name="regions", by=region)
+        .CALCULATE(rest_region=region, avg_rating=AVG(geographies.restaurants.rating))
+        .ORDER_BY(region.ASC())


Can replace geographies.WHERE(COUNT(restaurants) > 0) with geographies.WHERE(HAS(restaurants))

knassre-bodo · 2025-11-03T18:31:18Z

tests/test_pydough_functions/defog_test_functions.py

+    What's the name and food type of all the restaurants located on Market St in
+    San Francisco?
+    """
+    return locations.WHERE((LOWER(street_name) == "market st")).CALCULATE(


Suggested change

return locations.WHERE((LOWER(street_name) == "market st")).CALCULATE(

return locations.WHERE((LOWER(street_name) == "market st") & (LOWER(city_name) == "san francisco")).CALCULATE(

hadia206

Good job. Thanks John

hadia206 · 2025-11-11T18:36:01Z

tests/test_pydough_functions/defog_test_functions.py

    # Find the treatments from the doctors within the specialty in the past 6 months
    recent_treatments = doctors.prescribed_treatments.WHERE(
-        DATEDIFF("months", start_date, DATETIME("now")) <= 6
+        start_date >= DATETIME("now", "-6 months", "start of day")


Why this change?

john-sanchez31 added 6 commits October 27, 2025 15:25

adding defog restaurants data

01b3816

sqlite metadata

4efbfa2

10 gen test added

9ff71d0

all defog restaurants test added

fa43a5b

running tests [run all]

ef4d02a

breakpoint deleted [run all]

9f32785

john-sanchez31 marked this pull request as ready for review October 30, 2025 17:29

john-sanchez31 requested review from a team, hadia206, juankx-bodo and knassre-bodo and removed request for a team October 30, 2025 17:29

john-sanchez31 added 2 commits October 31, 2025 09:49

dermtreatment_basic1 test fixed

bbdf79d

testing [run all]

5b8fe65

knassre-bodo reviewed Nov 3, 2025

View reviewed changes

metadata and queries fixed [run all]

27b2507

john-sanchez31 requested a review from knassre-bodo November 5, 2025 20:35

john-sanchez31 added 2 commits November 6, 2025 08:44

testing [run all]

5497013

one more try [run all]

a310b4e

knassre-bodo approved these changes Nov 10, 2025

View reviewed changes

hadia206 approved these changes Nov 11, 2025

View reviewed changes

Merge branch 'main' into John/defog_restaurants [run all]

91ca1c2

john-sanchez31 merged commit 5fb0b18 into main Nov 11, 2025
19 checks passed

john-sanchez31 deleted the John/defog_restaurants branch November 11, 2025 23:07

	return locations.WHERE((LOWER(street_name) == "market st")).CALCULATE(
	return locations.WHERE((LOWER(street_name) == "market st") & (LOWER(city_name) == "san francisco")).CALCULATE(

Adding defog Restaurants database for testing #449

Adding defog Restaurants database for testing #449

Uh oh!

Conversation

john-sanchez31 commented Oct 27, 2025

Uh oh!

knassre-bodo left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

knassre-bodo Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hadia206 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

knassre-bodo Oct 30, 2025 •

edited

Loading