Abstract— Intelligent Assistants arebecoming an essential part of our life they are present in smartphones likeiPhone has Siri Android has Google Assistant and now they are becoming a corepart of computer operating systems i.e. Windows has Cortana, Linux has Stella.
TheseIntelligent Assistants are aimed to make life easier by helping the user inroutine tasks like a Human Personal Assistant do. But they are unable to takeplace of Human Personal Assistants, one of the main reason behind this is thatthese Intelligent Assistants cannot understand Human emotions. In this paper wehave proposed a method that will make these Intelligent Assistants capable ofunderstanding Human Emotions. In the proposed method we are using open sourceIntelligent Assistant “Open Assistant”, as the user calls the Assistant a real-timepicture of the user will be taken using OpenCV library and we use logisticregression algorithm to train on a provided dataset and evaluate the new image.It is detected whether the user is smiling or not then this information ispassed to Open Assistant to make him able to react according to user’s emotion.
1. IntroductionIn present day Intelligent Assistantsare becoming a core part of our life. These personal assistants can remind youabout your daily tasks, they can send emails, even they can order lunch foryou. These Intelligent Assistant are equipped with a bit of artificialintelligence (AI), they can understand human language, respond to generalqueries and their artificial intelligence is also used to ensure that theirresponses are in line with the expectations of the user. But these Assistantsare lacking the understanding of human emotions. In 1968, Albert Mehrabian 1pointed out that in human to human interaction 7% of communication iscontributed by verbal cues, 38% is contributed by vocal cues and major portion55% is contributed by facial expressions. And these facial expressions are usedto understand emotions.
If a machine can identify human emotions, it canunderstand human behavior better, thus improving the task efficiency 2. A lotof work has been done for making these intelligent assistants efficient buttechniques for making them capable of understanding human emotions are yet notavailable.In this paper, we have proposed atechnique that will make these intelligent Assistants capable of understandingHuman emotions. For this we are using an open source Intelligent Assistant”Open Assistant” as the user call the Open Assistant the webcam will takeuser’s picture then OpenCV computer vision library and logisticregression algorithm would be used to detect whether the user is smiling or neutralthen this information is forwarded to Open Assistant for executing particularcode.
This technique can be used for any Intelligent Assistant to make it moreefficient and humanoid.2. Related WorkThe technology giants like Google,Microsoft are working for making their Intelligent Assistant humanoid forbettering the tasks efficiency. There work is not available publicly but thegeneral perspective about their work for Intelligent Assistants is that, theyare using data stored on user devices and their daily device usage patterns formaking Intelligent Assistant responses inline with the expectations of user. Atthe time of writing this paper no research work was found for making theIntelligent Assistant capable of understanding human emotions. 3.
MethodologyThe following is the algorithmfor making Intelligent Assistant capable of understanding human-emotions: Fig: Block Diagramfor emotion recognition systema) OpenCVThe Intelligent Assistant startsby a keyword spoken by the user, as the user speaks the keyword a picture of theuser with either a smiling or neutral facial expression using a webcam istaken, this is done before the Intelligent Assistant done any processing. Thenour program uses an algorithm adopted from the OpenCV library to localize themouth area.b) VectorizationWe resize the image such that theoutput is 28 pixels by 10 pixels image only containing the person’s mouth andsurrounding areas.
The images are converted to grayscale and then flattenedinto a vector of length 280, with each entry representing the grayscale of apixel.c) Logistic RegressionWe built a logistic regressionprogram that will take the user-provided image vector and determine whetherthat person was smiling or not. First, the logistic regression is built to takean input of dimension 280. It applies a set of weights to that input and thenyields a single scalar. Whether the activation is closer to 0 or 1 determineswhether the model will say that the original person was smiling or not.Before the logistic regression canclassify the user-provided image, we trained the model using gradient descent.We used 64 neutral images and 43 smiling images from online datasets to trainthe model and finetune the weights.
With the suitable weights and biases, wecan input the user’s processed mouth image into the model and the network canpredict whether that person was smiling or not. d) Open AssistantThedifferent responses of Open Assistant are classified according to the emotionof the user like one portions of code for neutral and one portion for happy,when the smile is detected the code of Open Assistant for happy user isexecuted. 4. ConclusionIn this paper we have used OpenCVcomputer vision library, Logistic Regression program and Open Assistant to proposea method that can provide emotion understanding to Intelligent Assistant whichare becoming an essential part of our lives.In the future work more emotionscan be detected giving capability to Intelligent Assistant to react moreeffectively. REFERENCES 1Mehrabian, A. Communication without words, Psychology Today, volume 2, pp52-55, 1968.2 Suchitra, Suja P and Shika Taripathi,Real-Time Emotion Recognition from Facial Images using Raspberry Pi II, 2016 3rdInternational Conference on Signal Processing and Integrated Networks (SPIN)