Filetype PDF | Posted on 28 Sep 2022 | 3 years ago
Authentication
digital resources
193x Filetype PDF File size 0.47 MB Source: www.scitepress.org
Automatic Detection and Classification of Cognitive Distortions in
Journaling Text
Mai Mostafa, Alia El Bolock and Slim Abdennadher
German University in Cairo, Egypt
Keywords: Cognitive Distortions, Cognitive Behavioral Therapy, Mental Health, Machine Learning, Deep Learning,
Natural Language Processing.
Abstract: Cognitive distortions are negative thinking patterns that people adopt. Left undetected, it could lead to
developing mental health problems. The goal of cognitive behavioral therapy is to correct and change
cognitive distortions that in turn help with the recovery from mental illnesses such as depression and
anxiety, overcoming addictions, and facing common life challenges. The aim of this study is to provide a
machine learning solution for the automatic detection and classification of common cognitive distortions
from journaling texts. Relatively few works have focused on exploring machine learning solutions and tools
in the context of cognitive-behavioral therapy. And, given the rising popularity of online therapy programs,
this tool could be used for instant feedback, and would also be a helpful service for therapists and
psychiatrists to initiate and ease the detection of cognitive distortions. In this study, we provide a novel
dataset that we used to train machine learning and deep learning algorithms. We then employed the best-
performing model in an easy-to-use user interface.
1 INTRODUCTION distortions covered in this study are provided in
table 1 (de Oliveira, 2012).
Cognitive distortions describe the dysfunctional core In many cases, cognitive distortions result in
beliefs and misconceptions a person might have, that feelings such as anxiety and depression. Beck’s
control the way people feel towards themselves and cognitive theory for depression suggests that people
the world around them. These maladaptive with inaccurate and negative core beliefs are more
cognitions highly influence the way people react susceptible to depression. This cognitive theory is
emotionally, psychologically, and how they behave based on the grounds that an individual’s affect and
(Beck, 2011). For example, “The plant I just got behavior are largely determined by the way in which
died, I will never have a beautiful garden because they structure the world (Beck, 1987). Cognitive
everything will die” is a type of cognitive distortion, Behavioral Therapy (CBT) is a therapeutic approach
because it reached a conclusion about a single that is derived from Cognitive Therapy model theory
isolated negative event, and applied that conclusion (Beck, 1976; Beck, 1987) that helps patients
on all future plants. Cognitive distortions are recognize and identify their own thinking errors and
commonly grouped into 15 types (Beck, 1976). distorted view of reality. They are then helped to
However, there is no evidence-based way to classify correct these thinking errors, and are taught
cognitive distortions. And it’s important to recognize cognitive and behavioral skills so that they can
that there is a degree of overlap between them. develop more accurate beliefs and adopt a healthier
Moreover, a single sentence can exhibit multiple way of making sense of the world around them.
types of cognitive distortions. For example, “I failed CBT was attributed to help with the treatment of
this interview, I’ll probably fail all interviews I get” anxiety disorders, somatoform disorders, bulimia,
can be classified as overgeneralization, as well as anger control problems, and general stress (Hofmann
magnification, and catastrophizing. For these et al., 2012). This approach holds people
reasons, we have decided to pick only a couple of accountable to their own thoughts and feelings, and
types of cognitive distortions for the purpose of this rather than only delve into the past to know the
reasons for their thought fallacies, the goal is to
study. Definitions and examples of the cognitive
444
Mostafa, M., El Bolock, A. and Abdennadher, S.
Automatic Detection and Classification of Cognitive Distortions in Journaling Text.
DOI: 10.5220/0010713000003058
In Proceedings of the 17th International Conference on Web Information Systems and Technologies (WEBIST 2021), pages 444-452
ISBN: 978-989-758-536-4; ISSN: 2184-3252
c
Copyright
2021 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved
Automatic Detection and Classification of Cognitive Distortions in Journaling Text
Table 1: Cognitive distortions, definitions and examples.
Cognitive distortions Definitions Examples
1 Overgeneralization I take isolated cases and generalize “Every time I have a day off from
them widely by means of words such work, it rains.” “You only pay
as “always”, “never”, “everyone”, etc. attention to me when you want sex”.
2 Should statements I tell myself that events, people’s “I should have been a better mother”.
(also “musts”, behaviors, and my own attitudes “He should have married Ann instead
“oughts”, “have tos”) “should” be the way I expected them of Mary”. “I shouldn’t have made so
to be and not as they really are. many mistakes.”
identify and correct them. Recently, online therapy 2.1.1 Crowdsourcing
programs have gained a lot of popularity. These
programs are developed to accompany, or replace in- Crowdsourcing platforms are used as a means to
person CBT (Ruwaard et al., 2012) One of the main collect data from a large group of paid participants.
reasons that make it unique and important is because For the purpose of collecting texts portraying
it can be more frequently accessed, which was found cognitive distortions, participants are given a brief
to be one major component for the effectiveness of description of a cognitive distortion, then asked to
CBT and leads to a more rapid recovery (Bruijniks mention a situation or event, where they have
et al., 2015). exhibited that type of thinking (Shickel et al., 2019).
This study is conducted to develop methods for 2.1.2 Online Therapy Partnerships
the automatic detection and classification of
cognitive distortions found in mental health journals. datasets have been collected in partnership with
It will be of assistance to therapists in online therapy Koko (Morris et al., 2015). Which is an online
programs. Providing detection and instant feedback therapy program that is based on peer-to-peer
and allowing them to scale more easily. Only a few therapy. As well as TAO, an online therapy program
machine learning studies were conducted in relation implemented in various universities across the USA.
to mental health. Fewer in the context of cognitive As part of the program, students are requested to fill
behavioral therapy. The goal of this study is to out journals and logs to track their progress. Texts
collect a novel dataset to be used to explore ways to collected from actual journals are argued to be a
detect and classify cognitive distortions, and provide more accurate representation of the cognitive
machine learning and deep learning methods for the distortion than those collected by crowdsourcing.
detection and classification of two common Since the authors of those text passages weren’t
cognitive distortions. As well as develop a user specifically asked to recall a situation where they
interface to visualize the performance of the tool and exhibited a certain way of thinking (Shickel et al.,
put it to use. Which would be highly beneficial and 2016; Shickel et al., 2019; RojasBarahona et al.,
easy for therapists to use in online therapy programs. 2018).
2 RELATED WORK 2.1.3 Social Media APIs
Social media and Twitter in particular is an ideal
2.1 Data Collection platform to collect data from. As it provides texts
with the same natural expression of cognitive
There is a wide variety of choices when it comes to distortions as those in journals. Meaning that the
data collection. Most papers studying sentiment authors of the texts are not asked to specifically
analysis and emotion recognition have used already recall a situation where they felt they were thinking
existing datasets that are publicly available to in a specific manner. In addition to the easy, free of
conduct their research. Unfortunately, due to the fact charge use of the application programming interface
that cognitive distortion detection and classification (API), it can provide big volumes of data in a short
is still not widely researched, we haven’t been able amount of time. Due to the popularity of the
to find an available dataset. In this subsection, we platform itself, and the ease of data collection, many
discuss multiple sources for data collection. academic research studies have employed the
445
WEBIST2021-17thInternational Conference on Web Information Systems and Technologies
Twitter API to build their dataset. (Hu et al., 2019) Where the NLP unit receives input from the user and
(Mozeticˇ et al., 2016)(Cliche, 2017)(Chatterjee et builds a natural language query. The reasoning
al., 2019). (Campan et al., 2018) Have shown that subsystem with the help of the ontology evaluates
using Twitter API is a reliable way of collecting data the query and delivers a natural language answer.
for research purposes. (Shiv-hare and Khethawat, 2012; Minu and
R.Ezhilarasi, 2012) were able to classify emotions
2.2 Methods for Detection and from natural language texts based on an emotion
Classification hierarchy defined by the ontology. Ontologies are
also utilized to understand and recognize the way of
Cognitive distortion detection and classification speaking when feeling a certain emotion, and to get
tasks are similar to the tasks of emotion detection the similarity between sentences, not just to classify
and sentiment analysis. In a way, emotion the emotion based on keywords (Haggag et al.,
classification and cognitive distortion classification 2015).
are tasks to classify different negative sentiments. 2.2.2 Learning-based Approach
We have compiled and referred to a few studies in
these areas in this section. Traditional Learning: The automatic detection and
2.2.1 Rule-based Approach classification of emotions from texts are in great
demand. A lot of papers have studied multiple
Rule-based knowledge consists of grammatical and approaches and techniques to be able to perform
logical rules to follow. The approach may rely on such a task. One of the methods is classifiers such as
dictionaries, lexicons, and ontologies. Support Vector Machine (SVM) that are trained to
Keyword Recognition: The task is to find be able to detect emotions (Teng et al.,
occurrences of certain keywords in a sentence. These 2006)(Balabantaray et al., 2012)(Hasan et al., 2014).
keywords are stored in a constructed dictionary or (Asghar et al., 2020) applied and compared different
lexicon.(Bracewell et al., 2006) presented an machine learning algorithms, which are Na¨ıve
emotion dictionary, where emotion words and Bayes, Random Forest, Support Vector Machine
phrases were gathered from different sources (SVM), Logistic regression, K-Nearest neighbor,
including news articles. These words were then and XG boost to try and suggest the algorithm with
labeled either positive or negative. An emotion the best text classification results. The algorithm that
classification algorithm is then used on news articles performed best with respect to the accuracy, recall,
to classify the overall sentiment. The algorithm and precision was the logistic regression algorithm.
counts the number of positive and negative emotion Detecting and classifying cognitive distortions is an
words, and a simple equation is used to determine important task for the improvement of online
the article’s emotion. therapy services. Both tasks of detecting whether a
Ontological Knowledge: Gruber defined an text contained cognitive distortions or not, and
ontology as “an explicit specification of a classifying a text known to contain a cognitive
conceptualization”(Gruber, 1993). Ontologies offer distortion into one of fifteen cognitive distortions
meaning to terms and address the relationship have been performed. After testing out multiple
between them. Most medical ontology applications classifiers, it was found that logistic regression
follow a symptom-treatment or symptom-diagnosis performs best for a relatively small data set (Shickel
categorization. Some are used to assist health et al., 2019).
professionals in clinical decisions by making Deep Learning: Given a large data set, deep
evidence-based inferences. These inferences are learn-ing techniques can outperform and scale more
delivered by providing knowledge through the effectively with data, than traditional machine
ontology regarding treatments, symptoms, diagnosis, learning techniques. In addition, given the fact that it
and prevention methods(Yamada et al., 2020), requires less feature extraction and engineering, it is
therefore require limited options for input. increasingly being adopted for natural language
Nonetheless, ontologies were used to assist with processing tasks. One such task is SemEval 2017
natural language processing (NLP) applications task 4. Which includes Twitter sentiment
when it comes to categorizing a natural language classification on a 5-point scale (Rosenthal et al.,
text, or with Artificial Intelligence (AI) chatbots. 2017). The best performing system belonged to
One such ontology is introduced in (Estival et al., (Cliche, 2017) which uses Long Short-Term
2004) as part of a virtual environment project. Memory (LSTM) and Convolutional Neural
446
Automatic Detection and Classification of Cognitive Distortions in Journaling Text
Network (CNN) models. For the participation of entries were collected using the API, and they were
(Baziotis et al., 2018) in SemEval 2018 Task 1, reviewed for relevance and labeled.
which included determining the existence of none, Web Crawling: Examples of cognitive
one or more out of 11 emotions in Twitter texts. distortions are provided on most websites and blogs
Bidirectional LSTM were trained by a fairly large about cognitive behavioral therapy. we collected
data set of around 60,000 annotated tweets. LSTM some of these examples, as well as examples
models were also used by (Cachola et al., 2018) who provided in research papers. (Beck, 1970; Yurica
focused on the effect of using vulgar words and and DiTomasso, 2005; de Oliveira, 2012).
expressions on the perceived sentiment. Survey: We also constructed and distributed a
Using a large data set, deep learning models survey. We first presented the participants with a
were trained, and unsupervised learning for a large short description of the cognitive distortion and
quantity of unlabeled data was utilized to classify provided two examples. We then asked the
cognitive distortions, as well as emotions and participants to recall a time in their own lives when
situations (RojasBarahona et al., 2018). they exhibited the described pattern of thinking, and
provide examples of what they might have said to
themselves, or to others. We encouraged participants
3 METHODS to provide multiple examples or paraphrase the same
example. The survey was distributed on different
3.1 Data Collection and Annotation social media platforms, and participants were
requested to share it. In total, we were able to collect
Due to the fact that cognitive distortion detection 147 entries from 49 responses. These responses were
and classification tasks are not widely researched reviewed for relevance and labeled.
topics, there is no publicly available dataset HappyDB Dataset: We utilized (Asai et al.,
containing text with labeled cognitive distortions. 2018) data set to collect non distorted texts.
Hence, we collected and annotated a novel dataset. HappyDB was collected using crowdsourcing,
The dataset contains text passages labeled into one where the workers were asked to answer either:
of three categories. Namely, overgeneralization, ”what made you happy in the last 24 hours?” or,
should statement, and non-distorted. A ”what made you happy in the last 3 months?” We
summarization of the dataset is provided in table 2. added 1101 answers to our dataset and labeled them
Each collected entry was reviewed for relevance and as nondistorted. These entries were again reviewed
annotated by the authors and a life coach with a for relevance. It’s important for the research to
Meta coaching certification. The life coach was collect nondistorted texts, as the goal is to create a
presented with the text data in a shared excel sheet. tool that can automatically detect cognitive
The sheet contained the sentences, the given label, distortions. So providing plenty of nondistorted
and a checkbox. There was another column next to examples was crucial to be able to separate distorted
the checkbox that was left blank to be filled with the and nondistorted texts.
correct label in case the given label was incorrect. Preprocessing: We performed common
Corrections to the dataset were applied according to preprocessing techniques, including converting all
the excel sheet. text to lower case and removing punctuation and
Twitter API: We decided to collect data from emojis. For the machine learning models, a couple
Twitter. The social media platform provides an easy- of vectorizers were used. Namely, tf-idf vectorizer,
touse API that can be deployed to collect big and count vectorizer. These vectorizers transformed
volumes of data in a short amount of time. Using the our dataset textual entries into sparse vectors.
API, we only collected the body of the tweet, no Multiple n-gram ranges were tested using these
demographics or any other information about the vectorizers, to find that, in general, unigrams and
author of the tweet were collected. Search words bigrams performed the best. We also utilized
were required for filtering relevant tweets. From the multiple dense embeddings that are most popular in
examples provided by (de Oliveira, 2012), we have similar NLP tasks for the machine learning models,
been able to deduce a pattern or form that sentences such as GloVe, Bert, and Flair. For our deep
exhibiting a certain cognitive distortion usually learning models, we train 100 and 300 dimensions
acquire. One example, “Every time I have a day off for GloVe embeddings, as well as BERT
from work, it rains” the sentence form that could be embeddings.
derived is “Every time . . . , it . . . ” Where
something negative happens after “it”. Overall, 1122
447
no reviews yet
Please Login to review.
