[STORY] Explore ingredients of vegetarian Indian food

[dharashah410]

Ideation

• Decide story pitch with domain experts
  • Does the story idea add value?
  • Is data available?

Prepare

• Data collection and cleaning
• Story insight
• Get story reviewed from domain experts (Is there even a story here?)

Plan

• Structure of the story
• Design the visualization
• Get story reviewed from domain experts (Is the story plan making sense?)

Execute

• Code the visualization
• Write the essay
• Get story reviewed from domain experts

Publish

• Prepare social media images
• Publish
  • Post on site
  • Post on Instagram
  • Post on Facebook
  • Post on Twitter

[dharashah410]

https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0139539

[dharashah410]

1. Clean up

Hi @rjpraj, first up - let us clean up the names and nomenclature so the data structure is readable.

ingredient_table.csv

• Rename CSV from ingredient_table.csv to ingredients.csv
• Change column name from Ingrident_id to id
• Change column name from ingridents to name

recipe_name.csv

• ID cannot start from 0
• Rename CSV from recipe_name.csv to recipes.csv
• Rename column recipe_id to id
• Rename column Recipe Name to name
• Rename column name desc to description
• Rename column name cook_time to cook_time_in_minutes
• Rename column name prep_time to preparation_time_in_minutes
• Rename column name total_time to total_time_in_minutes

quantity_table.csv

• Rename CSV from quantity_table.csv to ingredient_quantities.csv
• Rename column Recipe_Id to recipe_id
• Rename column Ingredient_Id to ingredient_id

nutrition.csv

• Rename CSV from nutrition.csv to nutrition_values.csv
• Rename column Recipe_Id to recipe_id
• Rename column Nutrient to key
• Rename column Values to value

method_table.csv

• Rename CSV table from method_table.csv to steps.csv
• Rename column Recipe_Id to recipe_id
• Rename column Step to step
• Rename column Desc to description

2. Upload the Code

• Upload your python script code

[dharashah410]

Finish above task... meanwhile, I will come up with new todos. Below is my draft that I am still working on....

#5 Things we need to improve after above todos are done

If we take the example of https://www.tarladalal.com/Achaari-Aloo-Roll-(-Wraps-and-Rolls)-32665r

• To the recipes table, we need to add how much does this recipe make? -> Makes 4 rolls
• Add a tags (name) and taggable table (recipe_id, tag_id) and store -> Rolls, Saute, RefrigeratorNon Stick Kadai, Veg
• From the breadcrum we need to store --> Starters / Snacks, Rolls in the tags

[rjpraj]

I am making the changes suggested but since recipe_id and ingredient_id are acting as the foreign key both should have a unique name rather than using "id" as the primary key in both recipe and ingredients table. Or should I change it to id only?

[dharashah410]

Yes, change it to "id" only. Also, I opened 2 issues above.

[dharashah410]

1. Improve the data from Tarla Dalal.com

recipe_name.csv

• Add a new column called image_url and store URL of the image recipe
• Add a new column called number_of_views and store how many times was it viewed --> 150491 times
• Clean data in cook_time_in_minutes to ensure there is no units e.g. minutes or hours. And all values that have hours get standardized to minutes.
• Clean data in preparation_time_in_minutes to ensure there is no units e.g. minutes or hours. And all values that have hours get standardized to minutes.
• Clean data in total_time_in_minutes to ensure there is no units e.g. minutes or hours. And all values that have hours get standardized to minutes.

quantity_table.csv

• Add column called "unit" and split value in quantity column. Units like cup, tbsp, tsp, etc. should go into new column and quantity column should only have number

nutrition.csv

• Add column called "unit" and split value in Values column. Units like cal, g, etc. should go into new column and Values column should only have number

ingredient_table.csv

• They should be unique. E.g. In https://github.com/SamhitaWiki/food-science/blob/master/tarla-dalal-com/ingredients.csv Cinnamon is present in 4 rows. Cinnamon should only be 1 row.
• Split name into two columns name (English) and Hindi name
• Title case all values in name column (Eg: Fenugreek seeds become Fenugreek Seeds)

2. Upload the Code and Data

• Upload your python script code

3. Improve data from Wikipedia

ingredient_table.csv

• Add a new column called Wikipedia link. Search Wikipedia / WikiData to find respective ingredient
• Add a new column called Wikipedia image. Store the primary image of the page in this column.

4. Upload the Code

• Upload your python script code

[rjpraj]

I will be making changes so that the ingredients are unique in the CSV files, but I had a doubt in cases where the form of the ingredient is different it should be treated as a different ingredient ? for example Cinnamon sticks and Cinnamon powder should be different ingredients right?

[dharashah410]

https://www.nytimes.com/2019/02/12/well/live/why-do-south-asians-have-such-high-rates-of-heart-disease.html

[dharashah410]

@rjpraj We want Cinnamon sticks and Cinnamon powder to be treated as the same thing since we are analyzing the data instead of helping readers cook recipes

[dharashah410]

@rjpraj

• Green Asparagus and Asparagus are in 2 rows. It should be 1 row for Asparagus.
• Similarly, there are 5 different rows for Raw Mangoes -- Raw Mangoes, Raw Mango Cubes, Chopped Raw Mango, Grated Raw Mangoes. All these are same ingredient called "Raw Mango"
• Certain Wikipedia links for ingredients are broken. There is no page available for Turmeric Powder, in that case add link to Turmeric (core ingredient and not derived one). Eg: https://en.wikipedia.org/wiki/Turmeric Powder. Clean up for other such ingredients as well
• There should be no Hindi names present in "name" column Eg: Kabuli Chana (White Chick Peas).
• Remove Hindi name from brackets in "name" column

Did you upload all your latest python scripts?

[rjpraj]

I have tried to remove all the redundancy in the ingredients name but since this is a manual and time-consuming process I may have missed out on some. Please let me know here in such a case.

[dharashah410]

@rjpraj

• Replace Chopped Onions with "Onions", Chopped Green Chillies with "Green Chillies", Grated Ginger with "Ginger", Mashed Potatoes with "Potatoes", Boiled Green Peas with "Green Peas", Sliced Ladies Finger with " Ladies Finger", Chopped Tomatoes with "Tomatoes", Paneer Cubes with "Paneer"
• I cannot see row for "Raw Mango". Did it make it as "Mango"? Please note: "Mango" and "Raw Mango" are 2 different core ingredients.

Once this is done, replace the broken Wikipedia links and add primary image from Wikipedia for each ingredient

[rjpraj]

I have made all the changes suggested. There are no more broken links and I have added the image URL for the ingredients as well.

[dharashah410]

• Rename Ingredients.csv file to "ingredients.csv" (lower case)
• Rename column Wikipedia Links to "wikipedia_links"
• Rename column Hindi name to "hindi_name"
• Capitalise first letter of all Hindi names
• Rename column Image Link to "wikipedia_image"
• Upload the latest python script

[nehasharma1806]

https://medium.com/@yanhann10/searching-for-the-secret-ingredient-of-good-recipes-205d4537f11e
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6242282/
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3083808/
https://www.aace.com/sites/all/files/aapi_guide_to_nutrition_health_and_diabetes.pdf
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3083808/
https://drive.google.com/file/d/1eqQ578gHiPoIaHaVYjQa_3sFe_LzGhm1/view

[dharashah410]

@rjpraj

Data collection

• Scrape data for remainder recipes (B-Z)

Data cleanup

• Remove duplicate ingredients from the table
• Create a mapping array which removes words like chopped, grated, sliced etc
• Ensure that Hindi names are not present in English name columns
• Dynamically generate Wikipedia links from ingredients name
• Figure out the pattern to match remaining Wiki URLs (later)
• Add image URLs of primary image from Wikipedia

Scripts

• Write the code to pull out data whenever new recipe is added.
• Write the code to pull only newly added recipes and not scrape all the data again