IVRobot – An IVR-based ChatBot for Small Business Owners

January 14, 2021

IVR-based chatbot that can easily be used by small business owners to improve their digital presence, attract more customers, and enhance their customer support

Product Description

An integrated voice chatbot that can easily be used by small business owners to improve their digital presence, attract more customers and enhance their customer support.

Problems the product intends to solve

Enhance digital presence for small businesses
Automate answering of popular/repeat queries
Save time spent in answering calls by in-store staff thereby increasing productivity
Provide easy online customer service for small stores which can increase their customer volume and loyalty

Target customers/users

Customer: Small shop owners
User: People calling these shops

Why is this an important problem?

On one hand we have people who prefer calling up before visiting the store of their choice. Based on user interviews, we discovered that these users are spread across different age groups and call because of any/all of the following:

They don’t trust what is on the store website (if there is any)
They like to get a real-time status but calling in
They don’t like text based chatbots

On the other end, we spoke to the store owners who highlighted that during Covid-19 times, the incoming calls have increased. As a result, the amount of time store staff spend on these calls, hampers productivity. Most of the inquiries are repetitive. We have collected this information and mapped it into features of the IVRobot. There are 3 types of inquiries that are answered: Store Open hours, ongoing discounts, specific product availability with an option of talking to the store staff in case callers have additional questions.

Advantages

As a caller, they do not have to dial multiple extensions (like in traditional IVR systems) to get their query answered
As a SMB owner, they get more time to do the actual work in running the shop without losing out on digital interactions with their customer base
Analytics and customizable console mean store owners can derive actionable insights on their customer behavior

Structure

IVR

A IVR (interactive voice response) system using Twilio’s Programmable API that get user’s inquery and send it to endpoint then reply as the endpoint’s response.

Endpoint

A backend that handle user’s inquery based on store’s information (discounts/promotions, open/close hours, product availability, etc.)

NLP

A Python NLP engine to verify user’s inquery (e.g. product availability)

store-owner-console

A store owner console that manage the customers’ call and analyze of it (e.g. # of calls, popular inquery, etc)

Architecture

Prerequisite

Node.js/Express
MongoDB
Twilio API
ngrok (for the testing localhost to public url)

API Integration

Google Speech-to-Text API (Speech-to-text)
Twilio Programmable Voice API (Text-to-speech)

Repository

https://github.com/IkwhanChang/Twilio-IVR-Chatbot

Demo

Step-by-step Installation

Enable Google Cloud API

To use the speech-to-text, you should have a Google Cloud API key. We’re using Google Cloud’s Speech-to-Text to transcribe speech (especially phone audio) coming from Twilio, a leading cloud communications PaaS. We’re discovered that it’s now easier than ever to integrate live call data with Google Cloud’s Speech-to-Text using Twilio’s Media Streams.

The new TwiML <stream> command streams call audio to a websocket server. This makes it simple to move your call audio from your business phone system into an AI platform that can transcribe that data in real time and use it for use cases like helping contact center agents and admins, as well as store it for later analysis.

When you combine this new functionality with Google Cloud’s Speech-to-Text abilities and other infrastructure and analytics tools like BigQuery, you can create an extremely scalable, reliable and accurate way of getting more value from your audio.

Architecture

The overall architecture for creating this flow looks something like what you see below (We modified this architecture without using GCP.) Twilio creates and manages the inbound phone number. Twilio’s new Stream command takes the audio from an incoming phone call and sends it to a configured websocket which runs on a simple App Engine flexible environment. From there, sending the audio along as it comes to Cloud Speech-to-Text is not very challenging. Once a transcript is created, it’s stored in BigQuery (in our project, we directly receive it to our IVR system) where real-time analysis can be performed.

To use speech-to-text API, you must have a valid Google Cloud API that requires a credit card number but it will provide you $300 free credit. After you have a GCP account, create a project, then go to the API browser and Enable the Speech-to-Text API for that project.

Afterward, create a service account. Then, download a private key as JSON. You can view and manage these resources at any time in the Cloud Console. Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the path of the JSON file that contains your service account key. This variable only applies to your current shell session, so if you open a new session, set the variable again. We’ll use this JSON key in our IVR.

Twilio ML

You should have an account of Twilio. Once you sign up as a developer, Twilio provides you a test phone number. Check https://www.twilio.com/console and sign up. Then, you can have Twilio’s phone number. For that, go developer console and add phone number (that will charge you $1 per month)

NGROK/WEBHOOK

Ngrok is a Go program, distributed as a single executable file for all major desktop platforms. This is super rad – no additional frameworks to install or other dependencies. Grab the version for your development system of choice and simply unzip the file somewhere on your computer. Download: https://ngrok.com/download

In a Terminal window, navigate to the directory where you unzipped ngrok. Going forward you may want to add ngrok to your system path. That would allow you to access the ngrok command from any directory, but for now we will run the ngrok utility directly from this location.

We now need to start ngrok, telling it which port we want to expose to the public Internet. In the Terminal, type ./ngrok http 3000 on Linux and Mac, and just ngrok http 3000 on Windows. After starting up, you should see something like the following:

See that URL on the ngrok domain? That’s the brand new home of your local web app on the public Internet. In a browser window, open up https://[your generated ID].ngrok.io/voice – this should display the same XML found at http://localhost:3000/voice – verify this is the case.

Now, we can configure our Twilio number to hit our laptop when it receives an inbound call or text. In your Twilio account dashboard, go to your list of incoming numbers and choose one to work with. Enter your ngrok URL, including the “/voice” route, in the Messaging Request URL field as shown. Also, make sure to change the HTTP request type to a “GET” rather than a “POST”. Save your changes.

Go to Twilio Account -> Programmable Voice API -> Click your number

Firebase (Testing only)

We mainly use MongoDB for our real version. But for testing purpose, you may use Firebase for convenience. If you have a Google Cloud account, you may have a Firebase account easily. Go to firebase.google.com and create project, then add apps in Project Overview -> Settings

Click Add app and create a key, then keep your config file like as below:

We’ll use it for IVR and Store Owner Console.

MongoDB (Real)

You should have mongodb instance either local one or Mongodb Atlas (free resource is available. Check https://www.mongodb.com/cloud/atlas ) Once you get it, please change the mongoDbUri (in the index.ts for IVR, Endpoint, and Store Console)

Run IVR system

We will provide a docker compose file but if you want to run without dockerized ones, you can run our system as separately.

First, clone our repository

You have to run those things: IVR, endpoint, store-owner-console

IVR

IVR is our main router handling inquiry from Twilio and replies whatever we ask. Please follow as below:

Go to ivr folder, and run npm install
Go ivr > handler.js file, then change json file to your firebase key (you can download GCP console)

Run GOOGLE_APPLICATION_CREDENTIALS=[YOUR_SPEECH_TO_TEXT_GOOGLE_CLOUD_KEY] node index.js

Now you can have your IVR router as default port of 3000.

Endpoint

Endpoint handles the REST request regarding merchant, customer, inquiries, promotions, and items. Please follow as below:

Go to endpoint folder, and run “npm install”
Go to src > index.ts and change mongoDbUri to your own mongoDB connection string.

Run npm start

Now you can have your IVR endpoint at localhost with a default port of 3001.

Regex matching rules for incoming inquiry:

If the user inquiry contains keyword “menu”, return menu response
If the user inquiry contains keywords “discount” or “promotion”, return promotion response
If the user inquiry contains keywords “open”, “close”, “hour” or “hours”, return store hour response
If the user inquiry contains keyword “have”, return store inventory response.

NLP server

Go to /nlp folder
Install pip by running “python get-pip.py”, and use pip to install flask and nltk
Run “python proper_noun_api.py” and the nlp server will start at 0.0.0.0:3007

Store Owner Console

Store owner console is an integrated dashboard that can handle your customer and store information for the store owner only. To run, please follow the instructions:

Go to endpoint folder, and run “npm install”
Go to src > firebase.js and change firebase config as your one.

Run npm start

Now you can have a store console. Go http://localhost:3000/ and you can see login screen:

To use it, you must first register your store account.

The most important part is you should add your store phone number and Twilio Phone Number.

License

MIT / Carnegie Mellon University – Silicon Valley

Developers

Ikwhan Chang
Manuja Gokulan
Wei Li