J. Cent. South Univ. (2016) 23: 160-168

DOI: 10.1007/s11771-016-3059-3

Emotional inference by means of Choquet integral and λ-fuzzy measurement in consideration of ambiguity of human mentality

KWON Il-kyoung¹, LEE Sang-yong²

1. Division of Computer Science & Engineering, Kongju National University,

1223-24 CheonAn-Daero CheonAn-Si ChungCheongNam-Do, 330-717, Korea;

2. Department of Computer Science & Engineering, Kongju National University,

1223-24 CheonAn-Daero CheonAn-Si ChungCheongNam-Do, 330-717, Korea

Central South University Press and Springer-Verlag Berlin Heidelberg 2016

Abstract: Research on human emotions has started to address psychological aspects of human nature and has advanced to the point of designing various models that represent them quantitatively and systematically. Based on the findings, a method is suggested for emotional space formation and emotional inference that enhance the quality and maximize the reality of emotion-based personalized services. In consideration of the subjective tendencies of individuals, AHP was adopted for the quantitative evaluation of human emotions, based on which an emotional space remodeling method is suggested in reference to the emotional model of Thayer and Plutchik, which takes into account personal emotions. In addition, Sugeno fuzzy inference, fuzzy measures, and Choquet integral were adopted for emotional inference in the remodeled personalized emotional space model. Its performance was evaluated through an experiment. Fourteen cases were analyzed with 4.0 and higher evaluation value of emotions inferred, for the evaluation of emotional similarity, through the case studies of 17 kinds of emotional inference methods. Matching results per inference method in ten cases accounting for 71% are confirmed. It is also found that the remaining two cases are inferred as adjoining emotion in the same section. In this manner, the similarity of inference results is verified.

Key words: fuzzy measure; fuzzy integral; emotional model; emotion space; AHP; fuzzy inference system; Choquet integral

1 Introduction

Advancement in IT sectors leads mankind to easier, faster, and more convenient processes in life, as well as maximized reality. In fact, IT contributes to consistent advancement not only in performance but also in the automation and simplification of complicated environments. As research specifying emotional information communication and technology (ICT) industry based on emotional user experience (UX) is actively conducted, IT service environments are also developing toward the realization of high quality service in reflection of personal tendencies and emotions. Particularly for personalized services that reflect individuals’ subjective tendencies, many studies are conducted on various intelligence systems. As to the reflection of individuals’ subject tendencies, it is necessary to consider individuals’ emotional aspects in addition to pattern inference, learning, and prediction based on mere profile analysis results. The resulting data will contribute to expanding the range of available services and improving quality. To this end, methods to infer human emotions and create personalized emotional spaces appropriate for emotion-based personalized services are necessary.

Studies on emotional representation are divided mainly into psychological emotional models and emotional engineering models. The “Wheel of Emotion” mode, a psychological emotional model proposed by Plutchik, classifies basic human emotions into eight categories and creates various combinations. The “Ekman Basic Emotion” model classifies common emotions regardless of cultures and races. It proposes a model that simulates six basic emotions that people might have in similar situations. Parrot classifies emotions based on the 3-step tree structure; it specifies six emotions—love, joy, surprise, anger, sadness, and fear—with each consisting different steps. HUMAINE suggested emotion annotation and representation language (EARL), an emotional classification model, to represent emotional expression languages with an extensible markup language (XML)-based ontology. Thayer suggested a valence-arousal (V–A) emotional model, which is a two-dimensional emotional space model. In this model, arousal indicates the transference from excitement to calmness as the intensity of emotion increases from “Exciting” to “Calm” valences, indicating the extent of positive or negative feelings. As the values increase, positiveness increases accordingly. Such psychological or emotional engineering emotion models suggest various types of methods for challenging problems by expressing and classifying human emotions linguistically.

In this work, an emotional inference model is presented to establish and use personalized emotional spaces based on the model proposed by Thayer and Plutchik among various emotional models. To model a personalized emotional space, it is vital to reflect individuals’ subjective tendencies or intents. Hence, an emotional space is created through AHP, a subjective decision-making method. In addition, Sugeno fuzzy inference method, λ-fuzzy measure, and Choquet fuzzy integral are used for emotional inference and transference modeling.

2 Related researches

2.1 AHP

AHP is the decision-making support model developed by SAATY [1-2], which systematically analyzes decision-making processes and draws the importance of various evaluation items by means of pairwise comparison to be able to make a reasonable assessment on solutions. AHP specially analyzes qualitative issues quantitatively to support decision- making systematically and arranges complex and ambiguous problems into different hierarchies. It also evaluates the importance of each item by dividing them into 9-point scales as seen in Table 1 by means of a one- to-one comparison for partial relationship. In this way, the person who makes a decision can obtain a clearer and more certain decision.

AHP can be conducted in five steps: 1) hierarchy generation, 2) inputting for assessment by pairwise comparison, 3) calculation of relative importance on items to be evaluated, 4) verification of consistency of decision, and 5) opinion integration and calculation of overall importance. In particular, calculation of importance and measurement of consistency are to calculate elements preferred by users to verify consistency in the importance imposed by users. If the relative importance of n numbers of elements to be compared within one hierarchy is W_i, element a_ij of  matrix A in pairwise comparison can be reasoned as follows:

                     (1)

Therefore, matrix A with element a_ij can be presented as follows:

                    (2)

In the case that respondent fails to respond consistently to the relative importance of each assessment item in the pairwise comparison matrix, correctness will be reduced. Therefore, it needs to verify the importance obtained by pairwise comparison with a consistency index (I) and ensure consistency in consistency ratio (R). Consistency index and consistency ratio are

                               (3)

where λ_max is maximum eigen value and C is random index. I has a low value as the responses of respondents are consistent. If the ratio is below 0.1, it is regarded as having consistency.

This work adopts AHP to determine the relative weight of each satisfaction indicator of vouchers including food kind, price, discount rate, location of available stores (relative distance) in order to reflect users’ subjective preferences in the evaluation of item satisfaction rates.

Table 1 Nine-point scale of pairwise comparison

2.2 Fuzzy theory

The fuzzy theories are divided into fuzzy set theories and fuzzy measure theories depending on the level of ambiguity. Fuzzy set theories address vagueness; that is to say, a status whose boundary is not clearly defined, while fuzzy measure theories address the status of ambiguity where there are various possibilities, but to which an object belongs is uncertain [3-4]. As theoretical methods for emotional inference, this study adopts both Sugeno inference method that belongs to a fuzzy set theory and λ-fuzzy measure and Choquet fuzzy integral that belongs to fuzzy measure theory [5-11].

The fuzzy measure theory is set to measure a vague object subjectively. When a set function g that matches subsets A and B is randomly chosen among the universal set X with the actual values in section [0, 1], it meets the following condition:

Condition 1: g(f)=0, g(X)=1 (Boundary condition)

Condition 2: then g(A)≤g(B) (Monotonicity)

Condition 3: … or …

then (Continuity)

λ-fuzzy measure (g_λ) that will be used in this work introduces the parameter (λ) into the fuzzy measure as follows: Here, λ-fuzzy measure (g_λ) is of monotonicity.

         (4)

Here, λ indicates the synergism or offset as some values are positive and some are negative. When λ=0, λ-fuzzy measure becomes the possibility scale. This can be expressed as follows:

(5)

For the comprehensive evaluation of the objects by means of the fuzzy measure, the fuzzy integral is adopted. As to Choquet integral, when the function of the universal set X={x} is defined as h: X→[0, ∞), the fuzzy measure is g: 2^x→[0,1]. As to g, the fuzzy measure of function h, Choquet integral, is defined as follows:

                        (6)

where H_a={x|h(x)≥a}.

In general, since X={x₁, …, x_n}, the set of items to be evaluated is a finite set and x_i is determined so that function h(x_i) becomes h(x₁)=a₁≥h(x₂)=a₂≥h(x₃)…≥ h(x_n)=a_n. Here, the comprehensive evaluation can be expressed as follows:

                  (7)

The process of Choquet integral may be illustrated in Fig. 1.

3 AHP-based emotional space modeling

Personalization of emotional space aims to quantitatively express various emotions that a certain individual may have in daily activities. For instance, as physiological feelings of “Angry” and “Happy” are expressed quantitatively, it is possible to establish an individual’s emotional model accordingly. Hence, in this work, AHP is applied to model an emotional space personalized based on Thayer’s V–A emotion model and Plutchik’s emotional model, as illustrated in Fig. 2.

Fig. 1 Choquet integral process

Fig. 2 Thayer’s V–A emotion model (a) and Plutchik’s emotional model (b)

The most significant difference between the two emotional models is the kinds of emotions that can be expressed.

Plutchik’s model can express only 12 kinds of emotions while Thayer’s model can express 32 kinds of emotions [12]. The theory of colors also may be applied to emotions for combinations of different emotions as follows [7, 13-15].

Each model is divided into four sections based on the valence–arousal factors to establish a personalized emotional space model. AHP is used to determine the relative weight of each emotion so that the intensity of each internal and psychological emotion is quantified. For example, when the following question is raised: Evaluate the relative intensity of positive aspects of “Happy” and “Angry”, it is possible to quantify the language of emotional expression, and the involved person’s will is reflected.

The question above helps to evaluate the intensity of positiveness of one’s own feelings, “Happy” and “Anger”, in daily activities numerically based on the 9-point-scale of Saaty. Such a survey-based pairwise analysis of emotional expression terms was conducted for all emotional expression terms, and the quantitative values of each emotion were determined. In this work, the relative weight of each emotion related to valence and positive values as well as arousal’s exciting values is presented.

The following is the result of AHP evaluation on positive elements of 12 emotional expression terms defined in Thayer’s emotional model (Table 2). Some values are the same but divided into black and red. For instance, the relative weights of “Excited” and “Calm” (Cal.) values, indicated in black, are 3.0. This means that the weight of “Excited” is 3 points higher than that of “Calm”. In contrast, the ones indicated in red mean that the weight is 1/3 of the other.

One object in AHP evaluation shown in Table 2 indicates the highest intensity with regard to “Pleased” in terms of positiveness, and then “Happy” and “Peaceful” in the order. In the same principle, the relative weights of each emotion with regard to “Excited” in the arousal axis are 0.079 (Pleased), 0.125 (Happy), 0.249 (Excited), 0.148 (Annoyed), 0.191 (Angry), 0.060 (Nervous), 0.042 (Sad), 0.018 (Bored), 0.014 (Sleepy), 0.015 (Calm), 0.023 (Peaceful), and 0.036 (Relaxed). In short, the intensity level of emotions regarding “Exciting” is “Excited”, “Angry”, and “Annoyed” in the order.

Among the 32 different emotions defined in Plutchik’s emotional model, the middle group of emotions that belong to the same color was analyzed in the AHP evaluation for such emotion as “Trust”, “Joy”, “Anticipation”, “Anger”, “Disgust”, “Sadness”, “Surprise”, and “Fear”. Once the middle group is evaluated, the stronger and weaker emotions are predictable.

Table 3 shows the result of the AHP evaluation on Plutchik’s emotional model. In terms of “Positive” elements, “Trust” and “Joy” in the order, and in terms of “Exciting” elements, “Joy” and “Anger” in the order are evaluated as strong emotions. The values indicate that as “Positive” and “Exciting” values increase, “Joy” will be selected.

Based on the weights of each emotion determined through AHP, a 2D-type personalized emotional space was modeled in reference to valence–arousal values. Table 4 shows the normalized values between 0 and 1, which express the weights of each emotion, the results of AHP evaluation, with a V–A emotional space model. Here, “Negative” and “Calm” values are 1 less than “Positive” and “Exciting” values, respectively. As shown in Fig. 2, V-A emotional space is divided into four sections, and normalization is implemented for each of them. For instance, both “Trust” and “Joy” belong to the 1st quarter. As the sum of two values is set to 1, the expression is as

V_Trust=W_Trust/(W_Trust+W_Joy)                       (8)

V_Joy=W_Joy/(W_Trust+W_Joy)                         (9)

where V_Trust and V_Joy are the normalized values of “Trust” and “Joy” respectively. W_Trust and W_Joy are the weights of “Trust” and “Joy”, respectively.

In the same manner, regulation is implemented for each section, and the results are given in Table 4. The normalized values indicated in red, shown in Table 4, are expressed in Space A in Fig. 3.

Table 2 Relative weights of emotions based on Thayer’s emotional model (positive aspects)

Table 3 AHP evaluation result on Plutchik’s emotional model (positive/exciting aspects)

Not all of the 32 emotions expressed by Plutchik’s model in Fig. 3 are displayed. Such emotions as “Love” indicated with a dotted line are marked in the locations of colored emotions’ average values. The object of this model displays certain emotions such as “Joy”, “Angry”, “Sad”, “Surprise”, and “Disgust” quite evidently. “Fear” and “Anticipation” are not clearly shown even if V–A values change while such emotions as “Joy” are revealed immediately.

4 Design and establishment of an emotional inference system

4.1 Sugeno inference based system

This work includes personalized emotional inference by means of Sugeno inference method, λ-fuzzy measure, and Choquet integral. Sugeno inference method aims to establish a fuzzy set for all emotions expressed in an emotional space as well as input necessary for inference to determine the extent to which each emotion belongs to a certain subset. This makes it possible to create a personalized inference model by means of adaptive neuro fuzzy inference system. The fuzzy measure theory must determine the probability scale or belief measure of which emotion the inferred one based on input would be [16].

The steps of Sugeno fuzzy inference system establishment for emotional inference are shown in Fig. 4.

In this work, the inference system illustrated in  Fig. 5 is established according to the steps specified in Fig. 4. In Sugeno inference, the variables in the first half are given in a fuzzy set, and those in the second half in a linear expression are given according to the linear reasoning.

Types of rules used in Sugeno inference system are as follows:

If x is A and y is B, Then z is C

where A, B and C are fuzzy numbers, x and y are antecedent variables, and z is a consequent variable.

Table 4 V-A Emotional space modeling in reference to Plutchik’s emotional model AHP evaluation value

Fig. 3 AHP-based personalized V–A emotional space:

Fig. 4 Steps of Sugeno fuzzy inference system establishment

The following rule is an example of those actually used in this work. The first variable indicates the V–A value, and the second indicates the fuzzy membership function for emotional values.

If valence is positive-high and arousal is exciting- high

Then Happy is Happy-Strong

Once the input variable, output variable, and inference rules are defined, the following four steps are taken for inference.

The following shows inference steps when rules R₁ and R₂ are given:

R₁: If x is A₁ and y is B₁, Then z is C₁

R₂: If x is A₂ and y is B₂, Then z is C₂

Step 1: Suitability of each rule for the confirmed input (x₀, y₀) is determined by the following expression:

R₁ suitability:

R₂ suitability:

The smaller value between suitability to A_i and suitability to B_i is taken (min).

Step 2: The suitability gained in Step 1 is reflected in the latter part fuzzy set to produce the inference results of each rule. Here, the smaller value (min) between the value gained in Step 1 and the suitability to C_i is taken:

R₁ inference result:

R₂ inference result:

Step 3: The final inference result is as follows. Among the suitability values for each rule, the largest value (max) is taken:

Step 4: In the defuzzification step, when there are two input variables and l input variables as in this work, the inference is as follows:

R_i : IF x is A_i and y is B_i , THEN z_i is C_i , i=1, 2, …, l

where i is the rule number; l is the number of rules; A_i and B_i are fuzzy sets; C_i is the real number value.

When the suitability of the former part of rule I is  z_i*, the final inference result of each rule is as follows:

Fig. 5 Emotional inference system based on Sugeno inference (Plutchik’s emotional space)

4.2 Fuzzy measure and fuzzy integral based inference system

With a personalized emotional space modeled, if the V–A value at which emotional transference occurs in the space is given, the fuzzy measure and fuzzy integral are used to predict how emotions are transferred. The procedures are shown in Fig. 6.

When positive, negative, exciting, and calm values are given in a personalized emotional space, ambiguity inference is implemented to reason what type of emotions develops. Hence, inference results may be different depending on the type of basic emotional space. As to Sugeno inference system suggested earlier, the whole system needs to be adjusted for a personalized emotional space to be established and reflected shortly, which makes personalization quite difficult. In contrast, the fuzzy measure-based inference system involves adjustment to the personalized emotional evaluation values only, which makes the process easier.

5 Experiment and evaluation

By means of the two methods suggested above, emotional inference was implemented, and the results were comparatively analyzed. The personalized emotional space illustrated in Table 4 was used as the V–A emotional space. Evaluation on transferred emotions was conducted 17 times (E1-E17). Input E1 is {9, 2, 8, 1}, which indicates the size of exciting, calm, positive, and negative values in the order.

Figures 7 and 8 show that the results of the fuzzy measure based inference method in a personalized Thayer emotional space and the results of Sugeno inference method are quite similar to each other. Difference in the size of some emotions or inferred emotions is interpreted as the result of personalization. In both systems, when the emotional intensity is 0.3, the emotions are marked as neutral.

Figure 9 shows the result of emotional inference in application of Plutchik’s emotional model.

Unlike Thayer’s emotional model, Plutchik’s model involves 32 different types of emotions and infers new emotions by combining two or more of them. In steps E3-E7, the inferred emotions are all “Surprise”, but the maximum value of intensity is twice of the minimum value. In this case, emotions can be divided into “Amazement”, “Surprise”, and “Distraction”, depending on the intensity of emotions according to Pultchik’s emotional model.

“Love” and “Remose”, illustrated with polygonal lines in Fig. 9, are a combination of certain emotions, and these belong to the opposite type of emotions in the model. This inference is possible through an appropriate operation of emotions that are combined. In this work, the average value of the simplest form of emotions is used for inference.

It is demonstrated that it is possible to model emotional inference through a personalized emotional space with various types of fuzzy inference systems in it. In addition, the same task is performed in various emotional space models to confirm two-dimensional emotional spaces appropriate for each service type, which makes it possible to establish a fast inference system. It is demonstrated that combinations of certain emotions can be processed in reference to the intensity of emotions. In the case of Plutchik’s emotional space, it is possible to combine and infer new emotions in connection with the chromatic theory. Conversion into a three-dimensional emotional model is also expected to be possible.

Fig. 6 Emotional inference steps by means of fuzzy measure and fuzzy integral

Fig. 7 Results of fuzzy measure and fuzzy integral based emotional inference by means of personalized Thayer emotional space model

Fig. 8 Results of emotional inference by means of Sugeno inference system in application of Thayer emotional space model

Fig. 9 Results of emotional inference in application of Plutchik’s emotional model

6 Conclusions

1) The construction of emotional space and inference method is proposed using AHP, fuzzy measure, and fuzzy integral-based hybrid fuzzy method, which are subjective decision making support tools. It is confirmed that V–A emotional space modeling is possible using the FIS emotional modeling and inference system by the inference method of Sugeno. Emotional space modeling is also ascertained reflecting subjective inclination according to personalized emotional space construction using AHP if hybrid fuzzy method is used.

2) A case study is carried on 17 kinds using the inference method of Sugeno in the Thayer’s emotional space model and the hybrid fuzzy method-based inference method. Consequently, the same emotion is confirmed to be inferred in 10 kinds of cases corresponding to 71% among 14 kinds of cases, which are not neutral emotions with the inferred 4.0 and bigger emotional evaluation value. The remaining two cases are ascertained to be the same sectional adjoining emotion.

3) Through all these, it is confirmed that there is no difference in the inference results of each inference method in view of 85% of the similarity level in the results of each inference method.

4) The hybrid fuzzy method-based emotional inference method is identified to conduct complex emotional inference and expression unlike the inference method of Sugeno. In fact, the hybrid fuzzy method has a merit that arithmetic operations required for emotional inference can be carried out quickly.

5) Although the method of Sugeno is judged to build a personal services emotional space model according to neuro fuzzy system construction, additional studies on the learning method in the personalized emotional space are needed concerning hybrid fuzzy method-based emotional inference.

References

[1] SAATY T L. The analytical hierarchy process [M]. New York: McGraw-Hill, 1980: 281.

[2] SAATY T L. How to make a decision: The analytic hierarchy process [J]. European Journal of Operational Research, 1990, 48: 9-26.

[3] ZADEH L A. Fuzzy sets [J]. Information and Control, 1965, 8(3): 338-353.

[4] Zadeh L A. Fuzzy sets and systems [C]// Proc Symp on Systems Theory, Polytechnic Institute of Brooklyn. New York, 1965: 29-37.

[5] QIN Y, ZHANG X, YING H. A Hmm-based fuzzy affective model for emotional speech synthesis [C]// 2nd International Conference on Signal Processing Systems (ICSPS). Dalian, 2010: 525-528.

[6] KOVCS S. Fuzzy reasoning and fuzzy automata in user adaptive emotional and information retrieval systems [C]// Proceedings of the IEEE International Conference On Systems, Man and Cybernetics. Hammamet, 2002: 6.

[7] SALMERON J L. Fuzzy cognitive maps for artificial emotions forecasting [J]. Applied Soft Computing, 2012, 12(2): 3704-3710.

[8] EIICHIRO T. On identification methods of λ-fuzzy measures using weights and λ [J]. Japanese Journal of Fuzzy Sets and Systems, 2000, 12(5): 665-676.

[9] SUGENO M, TERANO T. A model of learning based on fuzzy information [J]. Kybernetes, 1977, 6(3): 157-166.

[10] KANG Ying-shi, WANG Hai-ning. Uncertain Choquet integral operator and its application to product design selection [J]. JCIT: Journal of Convergence Information Technology, 2011, 6(6): 1- 6.

[11] FENG Hui-min, LI Xue-fei, CHEN Ai-xia, WANG Bin. The relationship between diversity and the accuracy of classifier fusion based on Choquet integral [J]. JCIT: Journal of Convergence Information Technology, 2013, 8(1): 44-51.

[12] THAYER R E. The biopsychology of mood and arousal [M]. Oxford, UK: Oxford University Press, 1989: 234.

[13] Plutchik R. Emotion: theory, research, and experience. Vol. 1: Theories of emotion [M]. New York: Academic, 1980: 423.

[14] Plutchik r. Emotions and life: Perspectives from psychology, biology, and evolution [M]. Washington, DC: American Psychological Association, 2002: 381.

[15] Plutchik r, Hope C r. Circumplex models of personality and emotions [M]. Washington, DC: American Psychological Association, 1997: 484.

[16] MALKAWI M, MURAD O. Artificial neuro fuzzy logic system for detecting human emotions [J]. Human-Centric Computing and Information Sciences 2013, 3(1): 2 -13.

(Edited by YANG Bing)

Foundation item: Project(2012R1A1A2042625) supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education

Received date: 2015-01-27; Accepted date: 2015-05-14

Corresponding author: LEE Sang-yong, PhD; Tel: +82-41-521-9226; E-mail: sylee@kongju.ac.kr