Touchless Chef

50.001 1D Project

A POC Recipe Android Application using Google Mediapipe Hands Tracking model for smart touchless scrolling.

Deliverables:

• Download and install our .apk HERE to try it out
• Slides
• Poster
• Video

A. Acknowledgement:

• Tran Nguyen Bao Long @TNBL: Project Lead
  • define software architecture
  • documentation
• Li Xueqing @cnmnrll: Project Design
  • set basic Recipe, Ingredients and Instructions models model
  • documentation
• Melodie Chew En Qi @melmelchew: Marketing + Frontend
  • RecyclerView and NavigationView
  • adapter for RecyclerView
• Melvin Lim Boon Han @melvinlimbh: Marketing + Frontend
  • Fragments for different cuisines home
  • Fragments for creating recipes RecipeCreate
  • all resources and layouts res
• Han Jin @dlmszhhh: Backend Acitivity
  • basic recipe activities acitivity
• Wang Zhouran @wzrwzr23: Backend Database
  • setup SQLite database and corresponding Data Access Objects dao and adapter
• Liang Junyi @LiangJunyi-010 Backend HandTracking
  • implement HandTracking using Google Mediapipe Hands model for touchless scrolling in ViewRecipeActivity

B. Description

Overview of Design Pattern used:

• Adapter Design Pattern (for SQLite database and RecyclerView)
• Singleton Design Pattern (for SQLite database)
• Observer Design Pattern (for CreateRecipeActivity)
• Factory Design Pattern (for Cuisine Fragment)
• Template Method Design Pattern (for Create Recipe Fragment)

0. Model

• Utilize Parcelable interface:
  • speed up data transfer using Java Serializable
  • easily pass data as Bundle object through Activity
• Recipe:
  • id: primary key for SQLite
  • name: name of recipe
  • description: short description of recipe
  • category: category/cuisine (Vietnamese/Chinese)
  • mealType: type of meals (Breakfast, Dinner,...)
  • time: estimated to prepare the dish
  • imagePath: local URI path to Recipe image
  • ingredients: list of ingredients
  • instructions: list of instructions
• Instruction:
  • id: primary key for SQLite
  • recipeID: foreign key
  • content: instruction body
• Ingredient
  • id: primary key for SQLite
  • recipeID: foreign key
  • name: ingredient name

1. Database

• We use local database SQLite and utilize abstract class SQLiteOpenHelper to create and manage our database in SQLiteDbCRUD

• Adapter in DatabaseAdapter
  • using Adapter Design Pattern as main API for Recipe CRUD operations
  • using Singleton Design Pattern to create a single instance of our database to be shared within all activities
• Data Access Objects (DAO):
  • provide an API for our app to set/get data from SQLite database
  • map SQLite Config with corresponding model attributes
  • Config is a nested class improving encapsulation since it is only used within dao package for CRUD operations

2. Activity

• Both MainActivity and ViewRecipeActivity inherits a Toolbar (explain in section B5a) from abstract class MenuActivity
• MainActivity
  • implements interface BaseFragment.FragmentListener that define blueprint for methods to:
    • display recipe: onShowRecipe which navigates to ViewRecipeActivity (an Explicit Intent)
    • delete recipe: onDeleteRecipe
  • consists of 3 main views:
    • ImageView show background image for selected Cuisines
    • DrawerLayout for NavigationView (explain in section B5b)
    • NestedScrollView for displaying of recipes for different Cuisine Fragments (explain in section B4a)
• ViewRecipeActivity:
  • display chosen recipe from the data passed as parcel from MainActivity
  • where HandsTracking feature (explain in section B3) is located
• CreateRecipeActivity
  • implements 3 Recipe Create Fragment listener, each corresponding to 1 step in creating the recipe (explain in section B4b)
  • using Observer Design Pattern, changing Fragment when reaching subsequent step in creating Recipe

3. Hand Tracking

• This feature is implemented in ViewRecipeActivity
  • 2 main Buttons Start Tracking and Stop Tracking to enable/disable the feature, which open up a glSurfaceView in the RelativeLayout at the bottom of the page
  • glSurfaceView is an instance of SolutionGlSurfaceView<HandsResult> provided by Mediapipe Library, similar to OpenGL that allows writing words and drawing on webcam output.
  • we detect whether the index finger is pointing up or down and from there, post a Runnable to our ScrollView (that is displaying the Ingredients and Instructions) to perform a smooth scroll correspoding to the gesture.

4. Fragment

a. Cuisine Fragment under fragment/home

• Both ChineseFragment and VietnameseFragment extends from abstract class BaseFragment
  • using Factory Design Pattern (BaseFragment is the factory, ChineseFragment and VietnameseFragment are the products) allow us to easily scale to multiple cuisines in the future
  • the only abstract method is getFragmentLayout that each product should define to generate its own correct fragment
  • both using RecyclerView to display its recipes (explain in section B5c)

b. Recipe Create Fragment under fragment/recipe

• 3 fragments correspond to 3 steps in creating a new Recipe:
  • RecipeCreateImageFragment: (1st step) to add image, description, choose meal type and time
  • RecipeCreateIngredientFragment: (2nd step) to add ingredients
  • RecipeCreateInstructionFragment: (3rd step) to add instructions

• All 3 Create Recipe Fragments extends the abstract class NavigableFragment:
  • blueprint for method onNext that define which fragment to navigate to after completing each creating new recipe step
  • using Template Method Design Pattern with NavigableFragment as the template
• RecipeCreateImageFragment can access the local storage to upload new image for recipe (an Implicit Intent)

5. View and Layout

a. Toolbar

• Both MainActivity and ViewRecipeActivity needs to define its own Toolbar according to the blueprint defined in the abstract class MenuActivity
• MainActivity Toolbar:
  • includes NavigationView (explain in section B5b below)
  • includes menu to add a "New recipe"
• ViewRecipeActivity Toolbar:
  • support Edit and Delete recipe functionalities

b. Navigation View and DrawerLayout

• make use of a switch case to switch to a different cuisine inside MainActivity
• once a new cuisine is selected, the corresponding Fragment (ChineseFragment or VietnameseFragment) will be added into NestedScrollView to show recipe CardView

c. RecyclerView

• We user RecyclerView at 2 places:
  • to display recipes in the Cuisine Fragment (BaseFragment)
  • to display the added ingredients and instructions when creating new recipe in Recipe Create Fragment
• To implement RecyclerView we need to implement the corresponding adapter as shown in adapter/recipe (Adapter Design Pattern)
  • the 3 main functions in each adapter to override are:
    • onCreateViewHolder that inflate a new view in the parent layout to allow for display of listed items
    • onBindViewHolder that binds the data to be displayed in as listed items
    • getItemCount that return the size of all the items to be displayed
  • in the corresponding Fragment, before we set the above adapter, we also need to set a LinearLayoutManager for the RecyclerView to reside in
  • code examples:

private RecyclerView instructionRecyclerView;
instructionRecyclerView.setHasFixedSize(true);
instructionRecyclerView.setLayoutManager(new LinearLayoutManager(getActivity()));
instructionRecyclerView.setAdapter(instructionAdapter);

C. Appendix

0. Reference:

• We made use of a some very useful Youtube tutorials + Github repos to build a normal Cooking Recipe app before adding modifications and our hand tracking feature:
  • Coding with Evan
  • Muhammad Shahid Niazi
  • aza0092/Cooking-Recipe-Android-App

1. User Story Development

• BDD: Behaviour-Driven Design
• SMART User Stories: Specific, Measurable, Achievable, Relevant, and Time- boxed