Yiqiang Chen

Semantic-Discriminative Mixup for Generalizable Sensor-based Cross-domain Activity Recognition

arXiv (Cornell University), Jun 14, 2022

It is expensive and time-consuming to collect sufficient labeled data to build human activity rec... more It is expensive and time-consuming to collect sufficient labeled data to build human activity recognition (HAR) models. Training on existing data often makes the model biased towards the distribution of the training data, thus the model might perform terribly on test data with different distributions. Although existing efforts on transfer learning and domain adaptation try to solve the above problem, they still need access to unlabeled data on the target domain, which may not be possible in real scenarios. Few works pay attention to training a model that can generalize well to unseen target domains for HAR. In this paper, we propose a novel method called Semantic-Discriminative Mixup (SDMix) for generalizable cross-domain HAR. Firstly, we introduce semantic-aware Mixup that considers the activity semantic ranges to overcome the semantic inconsistency brought by domain differences. Secondly, we introduce the large margin loss to enhance the discrimination of Mixup to prevent misclassification brought by noisy virtual labels. Comprehensive generalization experiments on five public datasets demonstrate that our SDMix substantially outperforms the state-of-the-art approaches with 6% average accuracy improvement on cross-person, cross-dataset, and cross-position HAR. CCS Concepts: • Human-centered computing → Ubiquitous computing; • Computing methodologies → Transfer learning.

Download

Deep Transfer Learning for Cross-domain Activity Recognition

arXiv (Cornell University), Jul 20, 2018

Download

Deep learning for sensor-based activity recognition: A survey

Pattern Recognition Letters, Mar 1, 2019

To create your highlights, please type the highlights against each \item command. It should be sh... more To create your highlights, please type the highlights against each \item command. It should be short collection of bullet points that convey the core findings of the article. It should include 3 to 5 bullet points (maximum 85 characters, including spaces, per bullet point.) • We survey deep learning based HAR in sensor modality, deep model, and application. • We comprehensively discuss the insights of deep learning models for HAR tasks. • We extensively investigate why deep learning can improve the performance of HAR. • We also summarize the public HAR datasets frequently used for research purpose. • We present some grand challenges and feasible solutions for deep learning based HAR.

Download

Local and Global Alignments for Generalizable Sensor-Based Human Activity Recognition

ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May 23, 2022

Instance-Wise Dynamic Sensor Selection for Human Activity Recognition

Proceedings of the ... AAAI Conference on Artificial Intelligence, Apr 3, 2020

Human Activity Recognition (HAR) is an important application of smart wearable/mobile systems for... more Human Activity Recognition (HAR) is an important application of smart wearable/mobile systems for many humancentric problems such as healthcare. The multi-sensor synchronous measurement has shown better performance for HAR than a single sensor. However, the multi-sensor setting increases the costs of data transmission, computation and energy. Therefore, the efficient sensor selection to balance recognition accuracy and sensor cost is the critical challenge. In this paper, we propose an Instance-wise Dynamic Sensor Selection (IDSS) method for HAR. Firstly, we formalize this problem as minimizing both activity classification loss and sensor number by dynamically selecting a sparse subset for each instance. Then, IDSS solves the above minimization problem via Markov Decision Process whose policy for sensor selection is learned by exploiting the instancewise states using Imitation Learning. In order to optimize the parameters of the activity classification model and the sensor selection policy, an algorithm named Mutual DAgger is proposed to alternatively enhance their learning process. To evaluate the performance of IDSS, we conduct experiments on three real-world HAR datasets. The experimental results show that IDSS can effectively reduce the overall sensor number without losing accuracy and outperforms the state-of-theart methods regarding the combined measurement of accuracy and sensor number.

Download

A Class Incremental Extreme Learning Machine for Activity Recognition

Cognitive Computation, Apr 3, 2014

Automatic activity recognition is an important problem in cognitive systems. Mobile phone-based a... more Automatic activity recognition is an important problem in cognitive systems. Mobile phone-based activity recognition is an attractive research topic because it is unobtrusive. There are many activity recognition models that can infer a user’s activity from sensor data. However, most of them lack class incremental learning abilities. That is, the trained models can only recognize activities that were included in the training phase, and new activities cannot be added in a follow-up phase. We propose a class incremental extreme learning machine (CIELM). It (1) builds an activity recognition model from labeled samples using an extreme learning machine algorithm without iterations; (2) adds new output nodes that correspond to new activities; and (3) only requires labeled samples of new activities and not previously used training data. We have tested the method using activity data. Our results demonstrated that the CIELM algorithm is stable and can achieve a similar recognition accuracy to the batch learning method.

OKRELM: online kernelized and regularized extreme learning machine for wearable-based activity recognition

International Journal of Machine Learning and Cybernetics, Apr 9, 2017

AR dataset and experimental results show the efficiency and effectiveness of OKRELM.

Download

Generalizable Low-Resource Activity Recognition with Diverse and Discriminative Representation Learning

arXiv (Cornell University), May 25, 2023

Human activity recognition (HAR) is a time series classification task that focuses on identifying... more Human activity recognition (HAR) is a time series classification task that focuses on identifying the motion patterns from human sensor readings. Adequate data is essential but a major bottleneck for training a generalizable HAR model, which assists customization and optimization of online web applications. However, it is costly in time and economy to collect large-scale labeled data in reality, i.e., the low-resource challenge. Meanwhile, data collected from different persons have distribution shifts due to different living habits, body shapes, age groups, etc. The low-resource and distribution shift challenges are detrimental to HAR when applying the trained model to new unseen subjects. In this paper, we propose a novel approach called Diverse and Discriminative representation Learning (DDLearn) for generalizable low-resource HAR. DDLearn simultaneously considers diversity and discrimination learning. With the constructed self-supervised learning task, DDLearn enlarges the data diversity and explores the latent activity properties. Then, we propose a diversity preservation module to preserve the diversity of learned features by enlarging the distribution divergence between the original and augmented domains. Meanwhile, DDLearn also enhances semantic discrimination by learning discriminative representations with supervised contrastive learning. Extensive experiments on three public HAR datasets demonstrate that our method significantly outperforms state-of-art methods by an average accuracy improvement of 9.5% under the low-resource distribution shift scenarios, while being a generic, explainable, and flexible framework. Code is available at: . • Human-centered computing → Ubiquitous computing; • Computing methodologies → Transfer learning.

Download

Generalizable Low-Resource Activity Recognition with Diverse and Discriminative Representation Learning

Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Human activity recognition (HAR) is a time series classification task that focuses on identifying... more Human activity recognition (HAR) is a time series classification task that focuses on identifying the motion patterns from human sensor readings. Adequate data is essential but a major bottleneck for training a generalizable HAR model, which assists customization and optimization of online web applications. However, it is costly in time and economy to collect large-scale labeled data in reality, i.e., the low-resource challenge. Meanwhile, data collected from different persons have distribution shifts due to different living habits, body shapes, age groups, etc. The low-resource and distribution shift challenges are detrimental to HAR when applying the trained model to new unseen subjects. In this paper, we propose a novel approach called Diverse and Discriminative representation Learning (DDLearn) for generalizable low-resource HAR. DDLearn simultaneously considers diversity and discrimination learning. With the constructed self-supervised learning task, DDLearn enlarges the data diversity and explores the latent activity properties. Then, we propose a diversity preservation module to preserve the diversity of learned features by enlarging the distribution divergence between the original and augmented domains. Meanwhile, DDLearn also enhances semantic discrimination by learning discriminative representations with supervised contrastive learning. Extensive experiments on three public HAR datasets demonstrate that our method significantly outperforms state-of-art methods by an average accuracy improvement of 9.5% under the low-resource distribution shift scenarios, while being a generic, explainable, and flexible framework. Code is available at: .

Download

Semantic-Discriminative Mixup for Generalizable Sensor-based Cross-domain Activity Recognition

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

It is expensive and time-consuming to collect sufficient labeled data to build human activity rec... more It is expensive and time-consuming to collect sufficient labeled data to build human activity recognition (HAR) models. Training on existing data often makes the model biased towards the distribution of the training data, thus the model might perform terribly on test data with different distributions. Although existing efforts on transfer learning and domain adaptation try to solve the above problem, they still need access to unlabeled data on the target domain, which may not be possible in real scenarios. Few works pay attention to training a model that can generalize well to unseen target domains for HAR. In this paper, we propose a novel method called Semantic-Discriminative Mixup (SDMix) for generalizable cross-domain HAR. Firstly, we introduce semantic-aware Mixup that considers the activity semantic ranges to overcome the semantic inconsistency brought by domain differences. Secondly, we introduce the large margin loss to enhance the discrimination of Mixup to prevent misclass...

Download

Disagreement-based class incremental random forest for sensor-based activity recognition

Knowledge-Based Systems, 2021

Less Annotation on Personalized Activity Recognition Using Context Data

2016 Intl IEEE Conferences on Ubiquitous Intelligence & Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People, and Smart World Congress (UIC/ATC/ScalCom/CBDCom/IoP/SmartWorld), 2016

Miscellaneous mini-wearable devices (e.g. wristbands, smartwatches, armbands) have emerged in our... more Miscellaneous mini-wearable devices (e.g. wristbands, smartwatches, armbands) have emerged in our life, capable of recognizing activities of daily living, monitoring health information and so on. Conventional activity recognition (AR) models deployed inside these devices are generic classifiers learned offline from abundant data. Transferring generic model to user-oriented model requires time-consuming human effort for annotations. To solve this problem, we propose SS-ARTMAP-AR, a self-supervised incremental learning AR model updated from surrounding information such as Bluetooth, Wi-Fi, GPS, GSM data without user's annotation effort. Experimental results show that SS-ARTMAP-AR can gradually adapt individual users and become more incremental intelligence.

Download

Stratified Transfer Learning for Cross-domain Activity Recognition

2018 IEEE International Conference on Pervasive Computing and Communications (PerCom), 2018

In activity recognition, it is often expensive and time-consuming to acquire sufficient activity ... more In activity recognition, it is often expensive and time-consuming to acquire sufficient activity labels. To solve this problem, transfer learning leverages the labeled samples from the source domain to annotate the target domain which has few or none labels. Existing approaches typically consider learning a global domain shift while ignoring the intra-affinity between classes, which will hinder the performance of the algorithms. In this paper, we propose a novel and general cross-domain learning framework that can exploit the intra-affinity of classes to perform intra-class knowledge transfer. The proposed framework, referred to as Stratified Transfer Learning (STL), can dramatically improve the classification accuracy for cross-domain activity recognition. Specifically, STL first obtains pseudo labels for the target domain via majority voting technique. Then, it performs intra-class knowledge transfer iteratively to transform both domains into the same subspaces. Finally, the labels of target domain are obtained via the second annotation. To evaluate the performance of STL, we conduct comprehensive experiments on three large public activity recognition datasets (i.e. OPPORTU-NITY, PAMAP2, and UCI DSADS), which demonstrates that STL significantly outperforms other state-of-the-art methods w.r.t. classification accuracy (improvement of 7.68%). Furthermore, we extensively investigate the performance of STL across different degrees of similarities and activity levels between domains. And we also discuss the potential of STL in other pervasive computing applications to provide empirical experience for future research.

Download

SensoryGANs: An Effective Generative Adversarial Framework for Sensor-based Human Activity Recognition

2018 International Joint Conference on Neural Networks (IJCNN), 2018

This study focuses on improving the performance of human activity recognition when a small number... more This study focuses on improving the performance of human activity recognition when a small number of sensor data are available under some special practical scenarios and resource-limited environments, such as some high-risk projects, anomaly monitoring and actual tactical scenarios. The Human Activity Recognition (HAR) based on wearable sensors is an attractive research topic in machine learning and ubiquitous computing over the last few decades, and has extremely practicality in health surveillance, medical assistance, personalized services, etc. However, with the limitation of sensor sampling rate, sustainability, deployment, and other restricted conditions, it is difficult to collect enough and resultful sensor data anywhere, anytime. Therefore, the HAR based on wearable sensors always faces the challenges of the low-data regime under some practical scenarios, which leads to a low accuracy of activity recognition and needs to be solved urgently. Currently, the Generative Adversarial Networks (GANs) provide a powerful method for training resultful generative models that could generate very convincing verisimilar images. The framework of GANs and its variants shed many lights on improving the performance of HAR. In this paper, we propose a new generative adversarial networks framework called SensoryGANs that can effectively generate available sensor data used for HAR. To the best of our knowledge, SensoryGANs is the first unbroken generative adversarial networks applied in generating sensor data in the HAR research field. Firstly, we tried exploring and devising three activity-special GANs models for three human daily activities. Secondly, these specific models are trained with the guidance of unbroken vanilla GANs. Thirdly, the trained generators from adversarial optimization process are used to generate synthetic sensor data. Finally, the synthetic sensor data from SensoryGANs are used to enrich the original authentic sensor datasets, which can improve the performance of target activity recognition model. Meanwhile, we propose three visual evaluation methods for assessing synthetic sensor data produced by the trained generators in SensoryGANs models. Experimental results show that SensoryGANs models have the capability of capturing the implicit distribution of real sensor data of human activity, and then the synthetic sensor data generated by SensoryGANs models Have a potential for improving human activity recognition.

Deep Transfer Learning for Cross-domain Activity Recognition

Proceedings of the 3rd International Conference on Crowd Science and Engineering, 2018

Human activity recognition plays an important role in people's daily life. However, it is o en ex... more Human activity recognition plays an important role in people's daily life. However, it is o en expensive and time-consuming to acquire su cient labeled activity data. To solve this problem, transfer learning leverages the labeled samples from the source domain to annotate the target domain which has few or none labels. Unfortunately, when there are several source domains available, it is di cult to select the right source domains for transfer. e right source domain means that it has the most similar properties with the target domain, thus their similarity is higher, which can facilitate transfer learning. Choosing the right source domain helps the algorithm perform well and prevents the negative transfer. In this paper, we propose an e ective Unsupervised Source Selection algorithm for Activity Recognition (USSAR). USSAR is able to select the most similar K source domains from a list of available domains. After this, we propose an e ective Transfer Neural Network to perform knowledge transfer for Activity Recognition (TNNAR). TNNAR could capture both the time and spatial relationship between activities while transferring knowledge. Experiments on three public activity recognition datasets demonstrate that: 1) e USSAR algorithm is e ective in selecting the best source domains. 2) e TNNAR method can reach high accuracy when performing activity knowledge transfer.

Download

Deep learning for sensor-based activity recognition: A survey

Pattern Recognition Letters, 2018

To create your highlights, please type the highlights against each \item command. It should be sh... more To create your highlights, please type the highlights against each \item command. It should be short collection of bullet points that convey the core findings of the article. It should include 3 to 5 bullet points (maximum 85 characters, including spaces, per bullet point.) • We survey deep learning based HAR in sensor modality, deep model, and application. • We comprehensively discuss the insights of deep learning models for HAR tasks. • We extensively investigate why deep learning can improve the performance of HAR. • We also summarize the public HAR datasets frequently used for research purpose. • We present some grand challenges and feasible solutions for deep learning based HAR.

Download

FSELM: fusion semi-supervised extreme learning machine for indoor localization with Wi-Fi and Bluetooth fingerprints

Soft Computing, 2018

Recently, the problem of indoor localization based on WLAN signals is attracting increasing atten... more Recently, the problem of indoor localization based on WLAN signals is attracting increasing attention due to the development of mobile devices and the widespread construction of networks. However, no definitive solution for achieving a low-cost and accurate positioning system has been found. In most traditional approaches, solving the indoor localization problem requires the availability of a large number of labeled training samples, the collection of which requires considerable manual effort. Previous research has not provided a means of simultaneously reducing human calibration effort and improving location accuracy. This paper introduces fusion semi-supervised extreme learning machine (FSELM), a novel semi-supervised learning algorithm based on the fusion of information from Wi-Fi and Bluetooth Low Energy (BLE) signals. Unlike previous semi-supervised methods, which consider multiple signals individually, FSELM fuses multiple signals into a unified model. When applied to sparsely calibrated localization problems, our proposed method is advantageous in three respects. First, it can dramatically reduce the human calibration effort required when using a semi-supervised learning framework. Second, it utilizes fused Wi-Fi and BLE fingerprints to markedly improve the location accuracy. Third, it inherits the beneficial properties of ELMs with regard to training and testing speeds because the input weights and biases of hidden nodes can be generated randomly. As demonstrated by experimental results obtained on practical indoor localization datasets, FSELM possesses a better semi-supervised manifold learning ability and achieves higher location accuracy than several previous batch supervised learning approaches (ELM, BP and SVM) and semi-supervised learning approaches (SELM, S-RVFL and FS-RVFL). Moreover, FSELM needs less training and testing time, making it easier to apply in practice. We conclude through experiments that FSELM yields good results when applied to a multi-signal-based semi-supervised learning problem. The contributions of this paper can be summarized as follows: First, the findings indicate that effective multi-data fusion can be achieved not only through data-layer fusion, feature-layer fusion and decision-layer fusion but also through the fusion of constraints within a model. Second, for semi-supervised learning problems, it is necessary to combine the advantages of different types of data by optimizing the model’s parameters.

Mapping learning in eigenspace for harmonious caricature generation

Proceedings of the 14th ACM international conference on Multimedia, 2006

... The artist can create people&amp;amp;#x27;s caricature artificially according to one&... more

Cross-People Mobile-Phone Based Activity Recognition

Activity recognition using mobile phones has great potential in many applications including mobil... more Activity recognition using mobile phones has great potential in many applications including mobile healthcare. In order to let a person easily know whether he is in strict compliance with the doctor's exercise prescription and adjust his exercise amount accordingly, we can use a smart-phone based activity reporting system to accurately recognize a range of daily activities and report the duration of each activity. A triaxial accelerometer embedded in the smart phone is used for the classification of several activities, such as staying still, walking, running, and going upstairs and downstairs. The model learnt from a specific person often cannot yield accurate results when used on a different person. To solve the cross-people activity recognition problem, we propose an algorithm known as TransEMDT (Transfer learning EMbedded Decision Tree) that integrates a decision tree and the k-means clustering algorithm for personalized activity-recognition model adaptation. Tested on a real...

Download

Speech Driven MPEG-4 Based Face Animation via Neural Network

Advances in Multimedia Information Processing — PCM 2001, 2001

... Hani Yehia [1] presents the method to estimate face motion from speech acoustic by training n... more

Uploads

Papers by Yiqiang Chen

Log In