Afsaneh Doryab

Carnegie Mellon University, Human-Computer Interaction Institute, Faculty Member

Followers

160

Following

Co-authors

Public Views

I am a Systems Scientist in the Human-Computer Interaction Institute at Carnegie Mellon University in Pittsburgh Pennsylvania. I am interested in ubiquitous and context-aware computing, technologies for measuring and motivating health-related behaviors, and mobile technologies that permit longitudinal measurement of physical, psychological, and behavioral states of individuals. My research is focused on computational modeling of human behavior from streams of data collected through mobile, wearable, and embedded sensors. Examples of my work in the health domain include modeling of surgical activities inside the operating room, detecting behavior change in people with depression, predicting mania-depression episodes in bipolar disorder, estimating symptom severity in cancer patients. My recent work in Context-aware Peer-to-Peer economic exchange is focused on connecting communities of people through mobile technology to enable successful and meaningful service transactions especially in low income communities. In my work, I draw on methods from Machine Learning, Data Mining, Statistics, and Human-Computer Interaction.
Supervisors: Ph.D. advisor: Prof. Jakob E. Bardram and Postdoc hosts: Prof. Jason Hong, Prof. John Zimmerman, Prof. Anind Dey
Address: 5000 Forbes Ave, Pittsburgh, Pennsylvania, 15213, USA

less

InterestsView All (8)

Uploads

Papers by Afsaneh Doryab

Activity-aware recommendation for collaborative work in operating rooms

This paper presents a recommender system for teams of med-ical professionals working collaborativ... more This paper presents a recommender system for teams of med-ical professionals working collaboratively in hospital oper-ating rooms. The system recommends relevant virtual ac-tions, such as retrieval of information resources and initia-tion of communication with professionals outside the operat-ing rooms. Recommendations are based on the current state of the ongoing operation as recognised from sensor data using machine learning techniques. The selection and non-selection of virtual actions during operations are interpreted as implicit feedback and used to update the weight matrices that guide recommendations. A pilot user study involving medical pro-fessionals indicates that the adaptation mechanism is effective and that the system provides adequate recommendations.

Download

BeWell+: Multi-dimensional Wellbeing Monitoring with Community-guided User Feedback and Energy Optimization

Smartphone sensing and persuasive feedback design is enabling a new generation of wellbeing appli... more Smartphone sensing and persuasive feedback design is enabling a new generation of wellbeing applications capable of automatically monitoring multiple aspects of physical and mental health. In this paper, we present BeWell+ the next generation of the BeWell smartphone health app, which continuously monitors user behavior along three distinct health dimensions, namely sleep, physical activity, and social interaction. BeWell promotes improved behavioral patterns via feedback rendered as an ambient display on the smartphone’s wallpaper. With BeWell+, we introduce new wellbeing mechanisms to address challenges identified during the initial deployment of the BeWell app; specifically, (i) community adaptive wellbeing feedback, which automatically generalize to diverse user communities (e.g., elderly, young adults, children) by balancing the need to promote better behavior yet remains realistic to the user’s goals; and, (ii) wellbeing adaptive energy allocation, which prioritizes monitoring...

Download

Clinical surfaces - activity-based computing for distributed multi-display environments in hospitals

Abstract. A multi-display environment (MDE) is made up of co-located and networked personal and p... more Abstract. A multi-display environment (MDE) is made up of co-located and networked personal and public devices that form an integrated workspace ena-bling co-located group work. Traditionally, MDEs have, however, mainly been designed to support a single “smart room”, and have had little sense of the tasks and activities that the MDE is being used for. This paper presents a novel ap-proach to support activity-based computing in distributed MDEs, where dis-plays are physically distributed across a large building. CLINICAL SURFACES was designed for clinical work in hospitals, and enables context-sensitive re-trieval and browsing of patient data on public displays. We present the design and implementation of CLINICAL SURFACES, and report from an evaluation of the system at a large hospital. The evaluation shows that using distributed pub-lic displays to support activity-based computing inside a hospital is very useful for clinical work, and that the apparent contradiction between mainta...

Download

Machine Learning Modeling of Adaptive and Maladaptive Personality Traits from Passively Sensed Behavior

Continuous passive sensing of daily behavior from mobile devices has the potential to identify be... more Continuous passive sensing of daily behavior from mobile devices has the potential to identify behavioral patterns associated with different aspects of human characteristics. This paper presents novel analytic approaches to extract and understand these behavioral patterns and their impact on predicting adaptive and maladaptive personality traits. Our machine learning analysis extends previous research by showing that both adaptive and maladaptive traits can be predicted from passively sensed behavior providing initial evidence for the utility of this type of data to study personality and its pathology. The analysis also provides insights into the underlying behavior patterns that link adaptive and maladaptive variants consistent with contemporary models of personality pathology.

Download

Context-Aware Coproduction: Implications for Recommendation Algorithms

Coproduction is an important form of service exchange in local community where members perform an... more Coproduction is an important form of service exchange in local community where members perform and receive services among each other on non-profit basis. Local coproduction systems enhance community connections and re-energize neighborhoods but face difficulties matching relevant and convenient transaction opportunities. Context-aware recommendations can provide promising solutions, but are so far limited to matching spatio-temporal and static user contexts. By analyzing data from a transportation-share app during a 3-week study with 23 participants, we extend the design scope for context-aware recommendation algorithms to include important community-based parameters such as sense of community. We find that inter- and intra-relationships between spatio-temporal and community-based social contexts significantly impact users’ motivation to request or provide service. The results provide novel insights for designing context-aware recommendation algorithms for community coproduction ser...

Detection of Racial Bias from Physiological Responses

Despite the evolution of norms and regulations to mitigate the harm from biases, harmful discrimi... more Despite the evolution of norms and regulations to mitigate the harm from biases, harmful discrimination linked to an individual’s unconscious biases persists. Our goal is to better understand and detect the physiological and behavioral indicators of implicit biases. This paper investigates whether we can reliably detect racial bias from physiological responses, including heart rate, conductive skin response, skin temperature, and micro-body movements. We analyzed data from 46 subjects whose physiological data was collected with Empatica E4 wristband while taking an Implicit Association Test (IAT). Our machine learning and statistical analysis show that implicit bias can be predicted from physiological signals with 76.1% accuracy. Our results also show that the EDA signal associated with skin response has the strongest correlation with racial bias and that there are significant differences between the values of EDA features for biased and unbiased participants.

Download

Daily Trajectory Prediction Using Temporal Frequent Pattern Tree

Adaptive Humanoid Robots for Pain Management in Children

Accurate pain assessment and management is particularly important in children exposed to prolonge... more Accurate pain assessment and management is particularly important in children exposed to prolonged or repeated acute pain including procedural pain because of elevated risk for adverse outcomes such as traumatic medical stress, intense pain response for subsequent pain and also developing chronic pain. Our current work in progress tries to help pain management in children through developing intelligent adaptive humanoid robots as a multi-modal non pharmacological intervention. Our current work increases the interactive capabilities of Nao humanoid robots by using the camera and microphone to assess pain and emotion in children undergoing procedural treatment through combining detection models for facial expression, voice quality, and adapt the robot's verbal and non-verbal interactive responses accordingly for optimal distraction through adaptive behavioral models. By combining two different methods of obtaining emotion predictive probabilities using facial expression and stress...

Identifying Symptoms Using Technology

The widespread usage of technology and personal devices can help track the state of well-being an... more The widespread usage of technology and personal devices can help track the state of well-being and mental illnesses in the general population. This chapter provides an overview of the technology used in mental health to identify symptoms with a particular focus on depression. Detection of physiological and behavioral signals such as facial expressions, vocal prosody, sleep patterns, and social behavior change that reveal symptoms of depression is discussed.

The Impact of Humanoid Affect Expression on Human Behavior in a Game-Theoretic Setting

With the rapid development of robot and other intelligent and autonomous agents, how a human coul... more With the rapid development of robot and other intelligent and autonomous agents, how a human could be influenced by a robot's expressed mood when making decisions becomes a crucial question in human-robot interaction. In this pilot study, we investigate (1) in what way a robot can express a certain mood to influence a human's decision making behavioral model; (2) how and to what extent the human will be influenced in a game theoretic setting. More specifically, we create an NLP model to generate sentences that adhere to a specific affective expression profile. We use these sentences for a humanoid robot as it plays a Stackelberg security game against a human. We investigate the behavioral model of the human player.

Download

Concurrent Activity Recognition For Clinical Work

We present an approach to learning to recognize concurrent activities based on multiple data stre... more We present an approach to learning to recognize concurrent activities based on multiple data streams. One example is recognition of concurrent activities in hospital operating rooms based on multiple wearable and embedded sensors. This problem differs from standard time series classification in that there is no natural single target dimension, as multiple activities are performed at the same time. Hence, most existing approaches fail. The key innovations that allow us to tackle this problem is (1) learning to recognize base activities from raw sensor data, (2) creating artificial joint activities from base activities using frequent pattern mining and (3) handling temporal dependency using virtual evidence boosting.

Download

A Deep Learning Framework for Prediction of Readmission Risk After Cancer Surgery from Mobile Data Streams

Hospital readmissions cost the US healthcare system billions of dollars annually; they are associ... more Hospital readmissions cost the US healthcare system billions of dollars annually; they are associated with high mortality rates and listed as the source of stress and suffering for both patients and family members. Accurate and early prediction of readmission risk could enable management of emerging postoperative complications before they escalate into readmissions. However, it is challenging to predict who will be readmitted to the hospital following surgery ahead of time. Traditional approaches to readmission risk stratification rely on static administrative and medical record data and generally classify all surgical oncology patients at high-risk. There are, however, different factors related to day to day behavior and activities that may contribute to or signal increased or decreased risk of readmission after discharge. In our research, we leverage the capability of mobile sensing and deep learning to measure daily readmission risk in cancer patients after discharge. Using data ...

Understanding health and behavioral trends of successful students through machine learning models

This study analyzes patterns of physical, mental, lifestyle, and personality factors in college s... more This study analyzes patterns of physical, mental, lifestyle, and personality factors in college students in different periods over the course of a semester and models their relationships with students’ academic performance. The data analyzed was collected through smartphones and Fitbit. The use of machine learning models derived from the gathered data was employed to observe the extent of students’ behavior associated with their GPA, lifestyle, physical health, mental health, and personality attributes. A mutual agreement method was used in which rather than looking at the accuracy of results, the model parameters and weights of features were used to find common behavioral trends. From the results of the model creation, it was determined that the most significant indicator of academic success defined as a higher GPA, was the places a student spent their time. Lifestyle and personality factors were deemed more significant than mental and physical factors. This study will provide insi...

Download

Digital Biomarkers of Symptom Burden Self-Reported by Perioperative Patients Undergoing Pancreatic Surgery: Prospective Longitudinal Study

JMIR Cancer, 2021

Background Cancer treatments can cause a variety of symptoms that impair quality of life and func... more Background Cancer treatments can cause a variety of symptoms that impair quality of life and functioning but are frequently missed by clinicians. Smartphone and wearable sensors may capture behavioral and physiological changes indicative of symptom burden, enabling passive and remote real-time monitoring of fluctuating symptoms Objective The aim of this study was to examine whether smartphone and Fitbit data could be used to estimate daily symptom burden before and after pancreatic surgery. Methods A total of 44 patients scheduled for pancreatic surgery participated in this prospective longitudinal study and provided sufficient sensor and self-reported symptom data for analyses. Participants collected smartphone sensor and Fitbit data and completed daily symptom ratings starting at least two weeks before surgery, throughout their inpatient recovery, and for up to 60 days after postoperative discharge. Day-level behavioral features reflecting mobility and activity patterns, sleep, sc...

Download

Towards a Computational Framework for Automated Discovery and Modeling of Biological Rhythms from Wearable Data Streams

Lecture Notes in Networks and Systems, 2021

Modeling biological rhythms helps understand the complex principles behind the physical and psych... more Modeling biological rhythms helps understand the complex principles behind the physical and psychological abnormalities of human bodies, to plan life schedules, and avoid persisting fatigue and mood and sleep alterations due to the desynchronization of those rhythms. The first step in modeling biological rhythms is to identify their characteristics, such as cyclic periods, phase, and amplitude. However, human rhythms are susceptible to external events, which cause irregular fluctuations in waveforms and affect the characterization of each rhythm. In this paper, we present our exploratory work towards developing a computational framework for automated discovery and modeling of human rhythms. We first identify cyclic periods in time series data using three different methods and test their performance on both synthetic data and real fine-grained biological data. We observe consistent periods are detected by all three methods. We then model inner cycles within each period through identifying change points to observe fluctuations in biological data that may inform the impact of external events on human rhythms. The results provide initial insights into the design of a computational framework for discovering and modeling human rhythms.

Download

Wild by Design: Workshop on Designing Ubiquitous Health Monitoring Technologies for Challenging Environments

Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers, 2021

Recent years have shown an emergence of ubiquitous technologies that aim to monitor a person's he... more Recent years have shown an emergence of ubiquitous technologies that aim to monitor a person's health in their day to day. However, albeit focused at a real world setting and technically able, most research is still limited in its real-world coverage, suitability, and adoption. In this workshop, we will focus on the challenges of real world health monitoring deployments to produce forward-looking insights that can shape the way researchers and practitioners think about health monitoring, in platforms and systems that account for the complex environments where they are bound to be used. • Human-centered computing → Ubiquitous and mobile computing.

Download

Leveraging Collaborative-Filtering for Personalized Behavior Modeling

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2021

The prevalence of mobile phones and wearable devices enables the passive capturing and modeling o... more The prevalence of mobile phones and wearable devices enables the passive capturing and modeling of human behavior at an unprecedented resolution and scale. Past research has demonstrated the capability of mobile sensing to model aspects of physical health, mental health, education, and work performance, etc. However, most of the algorithms and models proposed in previous work follow a one-size-fits-all (i.e., population modeling) approach that looks for common behaviors amongst all users, disregarding the fact that individuals can behave very differently, resulting in reduced model performance. Further, black-box models are often used that do not allow for interpretability and human behavior understanding. We present a new method to address the problems of personalized behavior classification and interpretability, and apply it to depression detection among college students. Inspired by the idea of collaborative-filtering, our method is a type of memory-based learning algorithm. It l...

Critical thinking of mobile sensing for health

XRDS: Crossroads, The ACM Magazine for Students, 2021

What you need to consider before collecting, processing, and analyzing mobile data for health app... more

HHAR-net: Hierarchical Human Activity Recognition using Neural Networks

Activity recognition using built-in sensors in smart and wear-able devices provides great opportu... more Activity recognition using built-in sensors in smart and wear-able devices provides great opportunities to understand and detect human behavior in the wild and gives a more holistic view of individuals' health and well being. Numerous computational methods have been applied to sensor streams to recognize different daily activities. However, most methods are unable to capture different layers of activities concealed in human behavior. Also, the performance of the models starts to decrease with increasing the number of activities. This research aims at building a hierarchical classification with Neural Networks to recognize human activities based on different levels of abstraction. We evaluate our model on the Extrasensory dataset; a dataset collected in the wild and containing data from smartphones and smartwatches. We use a two-level hierarchy with a total of six mutually exclusive labels namely, "lying down", "sitting", "standing in place", "w...

Download

Leveraging Routine Behavior and Contextually-Filtered Features for Depression Detection among College Students

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2019

The rate of depression in college students is rising, which is known to increase suicide risk, lo... more The rate of depression in college students is rising, which is known to increase suicide risk, lower academic performance and double the likelihood of dropping out of school. Existing work on finding relationships between passively sensed behavior and depression, as well as detecting depression, mainly derives relevant unimodal features from a single sensor. However, co-occurrence of values in multiple sensors may provide better features, because such features can describe behavior in context. We present a new method to extract contextually filtered features from passively collected, time-series mobile data via association rule mining. After calculating traditional unimodal features from the data, we extract rules that relate unimodal features to each other using association rule mining. We extract rules from each class separately (e.g., depression vs. non-depression). We introduce a new metric to select a subset of rules that distinguish between the two classes. From these rules, w...

format_quoteContextually filtered features improve model performance by 9.7% over unimodal features, verified across multiple datasets.format_quote

Download

Afsaneh Doryab

Uploads

Papers by Afsaneh Doryab

Log In