Google Assistant with Python Fulfillment
How to read this articleYou can read through sequentially but I would like to share alternative way to learn.- If you would like to begin with coding; start by Walkthrough along with GitHub -> Architecture- If you would like to have an overview project first; begin with Architecture -> Action on Google -> Action on GoogleDisclaimer: I've not explained in detail when in come to code. I highly suggest you to read through GitHub along the Walkthrough sectorGithub: https://github.com/kanin-kearpimy/L-L-Chatbot--action-on-google-
Order food via web UI is a common use case in software development nowadays. Design buttons, table of food, summary for orders, etc. What if, on the other hand, we can grab some food with less UI / text on it?
What if, we can just use our voice and a couple of clicks to buy some food? If you are interested, Chatbot with Google Assistant is one of the easier ways to get this up and running in no time.
Action on Google
Action on Google is a service that allows the developer to link their own customized feature to Google Assistant, utilizing Conversational Actions.
An essential component in this project are described below:
- Scene: Major building block for the customized conversational model. (We implement some conversational model here to link it with our webhook)
- Intent: Task that the Assistant carries out. Some probably track word/phrase from a user by NLU to identify which “scene” to be executed
- Type: configuration for data structured extraction.
- Webhook: Fulfillment service which flexibly interacts with Assistant. To receive data and store it in the database then response ending message for example.
Architecture
We’re going to build Noodle Helper which can order noodles via Google Assistant and export order data to shop in form of JSON
Basic System Architecture
Project Structure
Breaking down into 2 main parts: /api, and /sdk
/sdk: directory where a customized conversational model has been implemented
/api: webhook codebase is written by Python to handle Assistant requests, store databases, and proper responses.
We should consider /sdk director as Action on Google main base which is composed of
/actions: Action on Google configuration file.
/custom: customized script for
- /global: configure global intent, scene. for example, how to handle first enter app (MAIN), the user wants to exit (CANCEL), the user put unknown data (NO_INPUT).
- /intents: custom intents, training phases (we are going to talk about this soon).
- /scenes: building block handler.
- /types: structure extractor.
/webhook: trigger handler to send a request to Python FastAPI.
Walkthrough
We should have a look at actions.intent.MAIN.yaml and modify speech and transitionToScene for controlling first Scene entry.
in Shop.yaml scene we’ll handle incoming action with suggested choice James Shop
This scene will describe the user to choose Shop and handle intent by james_shop which has its unique training phase for NLU. if voice input matches correctly. Google Assistant will lead to the next James_Collection scene.
James_Collection will trigger a request to webhook (/api) to store user choice in-memory database and display menu from shop_item response (please have a look at GitHub for more information)
After selecting the menu, Assistant will lead the user to Taste scene which handles suggested choices for tastes and trigger webhook to store tastes in in-memory data
The next scene is Size to handle Size customized user’s decision
When everything is done, the Assistant will sum up all data to allow the user to review and make a final decision on the shop in Placeholder.
If you noticed Placeholder isn’t the last scene. This project has Next scene for requesting users whether want to choose another shop.
Result order will display to Shop as JSON.
Next Step Challenge
- Testing: To overall test is challenging. Because we have divided codebase on Action on Google and FastAPI.
- CI/CD: As I mentioned about the divided codebase. How to set pipeline to handle?
Reference:
