CAPHAR: context-aware personalized human activity recognition using associative learning in smart environments

Sunder Ali Khowaja1, Bernardo Nugroho Yahya1, Seok-Lyong Lee1
1Department of Industrial and Management Engineering, Hankuk University of Foreign Studies, Global Campus, Yongin-Si, South Korea

Tóm tắt

AbstractThe existing action recognition systems mainly focus on generalized methods to categorize human actions. However, the generalized systems cannot attain the same level of recognition performance for new users mainly due to the high variance in terms of human behavior and the way of performing actions, i.e. activity handling. The use of personalized models based on similarity was introduced to overcome the activity handling problem, but the improvement was found to be limited as the similarity was based on physiognomies rather than the behavior. Moreover, human interaction with contextual information has not been studied extensively in the domain of action recognition. Such interactions can provide an edge for both recognizing high-level activities and improving the personalization effect. In this paper, we propose the context-aware personalized human activity recognition (CAPHAR) framework which computes the class association rules between low-level actions/sensor activations and the contextual information to recognize high-level activities. The personalization in CAPHAR leverages the individual behavior process using a similarity metric to reduce the effect of the activity handling problem. The experimental results on the “daily lifelog” dataset show that CAPHAR can achieve at most 23.73% better accuracy for new users in comparison to the existing classification methods.

Từ khóa


Tài liệu tham khảo

Hong J-H, Ramos J, Dey AK (2016) Toward personalized activity recognition systems with a semipopulation approach. IEEE Trans Human Mach Syst 46:101–112. https://doi.org/10.1109/THMS.2015.2489688

Rabbi M, Aung MH, Zhang M, Choudhury T (2015) MyBehavior: automatic personalized health feedback from user behaviors and preferences using smartphones. In: Proceedings of the 2015 ACM International Joint Conference on pervasive and ubiquitous computing—UbiComp’15. ACM Press, New York. pp 707–718

Ordóñez F, Roggen D (2016) Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors 16:115. https://doi.org/10.3390/s16010115

Wen J, Zhong M, Wang Z (2015) Activity recognition with weighted frequent patterns mining in smart environments. Expert Syst Appl 42:6423–6432. https://doi.org/10.1016/j.eswa.2015.04.020

Kaltz J, Wolfgang JZ, Lohmann S (2005) Context-aware web engineering: modeling and applications. RIA Revue d’Intelligence Artif Spec Issue Appliying Context 19:439–458

Khowaja SA, Yahya BN, Lee S-L (2017) Hierarchical classification method based on selective learning of slacked hierarchy for activity recognition systems. Expert Syst Appl 88:165–177. https://doi.org/10.1016/j.eswa.2017.06.040

Villalonga C, Razzaq M, Khan W et al (2016) Ontology-based high-level context inference for human behavior identification. Sensors 16:1617. https://doi.org/10.3390/s16101617

Rajasethupathy K, Scime A, Rajasethupathy KS, Murray GR (2009) Finding “persistent rules”: combining association and classification results. Expert Syst Appl 36:6019–6024. https://doi.org/10.1016/j.eswa.2008.06.090

Khowaja SA, Prabono AG, Setiawan F et al (2018) Contextual activity based Healthcare Internet of Things, Services, and People (HIoTSP): an architectural framework for healthcare monitoring using wearable sensors. Comput Networks. https://doi.org/10.1016/j.comnet.2018.09.003

Filippoupolitis A, Oliff W, Takand B, Loukas G (2017) Location-enhanced activity recognition in indoor environments using off the shelf smart watch technology and BLE beacons. Sensors 17:1230. https://doi.org/10.3390/s17061230

Mafrur R, Nugraha IGD, Choi D (2015) Modeling and discovering human behavior from smartphone sensing life-log data for identification purpose. Human Centric Comput Inf Sci 5:31. https://doi.org/10.1186/s13673-015-0049-7

Atzmueller M, Hayat N, Trojahn M, Kroll D (2018) Explicative human activity recognition using adaptive association rule-based classification. In: IEEE International Conference on Future IoT Technologies (Future IoT). IEEE, New York. pp 1–6

Liu Y, Nie L, Han L, et al (2015) Action2Activity: recognizing complex activities from sensor data. In: Twenty-Fourth International Joint Conference on artificial intelligence. pp 1617–1623

Liu L, Cheng L, Liu Y, et al (2016) Recognizing complex activities by a probabilistic interval-based model. In: Thirtieth AAAI Conference on artificial intelligence. pp 1266–1272

Palmes P, Pung HK, Gu T et al (2010) Object relevance weight pattern mining for activity recognition and segmentation. Pervasive Mob Comput 6:43–57. https://doi.org/10.1016/j.pmcj.2009.10.004

Chen L, Nugent CD, Wang H (2012) A knowledge-driven approach to activity recognition in smart homes. IEEE Trans Knowl Data Eng 24:961–974. https://doi.org/10.1109/TKDE.2011.51

Sztyler T, Carmona J, Völker J, Stuckenschmidt H (2016) Self-tracking reloaded: applying process mining to personalized health care from labeled sensor data. In: Transactions on perti nets and other models of concurrency. pp 160–180

Markovikj D, Gievska S, Kosinski M, Stillwell DJ (2013) Mining Facebook data for predictive personality modeling. In: Seventh International AAAI Conference on weblogs and social media. pp 23–26

Friasmartinez E, Magoulas G, Chen S, Macredie R (2005) Modeling human behavior in user-adaptive systems: recent advances using soft computing techniques. Expert Syst Appl 29:320–329. https://doi.org/10.1016/j.eswa.2005.04.005

Prabono AG, Lee S-L, Yahya BN (2019) Context-based similarity measure on human behavior pattern analysis. Soft Comput 23:5455–5467. https://doi.org/10.1007/s00500-018-3198-6

Fernández-Llatas C, Benedi J-M, García-Gómez J, Traver V (2013) Process mining for individualized behavior modeling using wireless tracking in nursing homes. Sensors 13:15434–15451. https://doi.org/10.3390/s131115434

Hernandez M, Scarr S, Sharma M (2020) The Korean clusters: how coronavirus cases exploded in South Korean churches and hospitals. In: Reuters Graph. https://graphics.reuters.com/CHINA-HEALTH-SOUTHKOREA-CLUSTERS/0100B5G33SB/index.html. Accessed 4 Mar 2020

Tapia EM, Intille SS, Larson K (2004) Activity recognition in the home using simple and ubiquitous sensors. In: International Conference on pervasive computing. pp 158–175

Chang Chien Y-W, Chen Y-L (2010) Mining associative classification rules with stock trading data—a GA-based method. Knowl Based Syst 23:605–614. https://doi.org/10.1016/j.knosys.2010.04.007

Pach F, Gyenesei A, Abonyi J (2008) Compact fuzzy association rule-based classifier. Expert Syst Appl 34:2406–2416. https://doi.org/10.1016/j.eswa.2007.04.005

Qodmanan HR, Nasiri M, Minaei-Bidgoli B (2011) Multi objective association rule mining with genetic algorithm without specifying minimum support and minimum confidence. Expert Syst Appl 38:288–298. https://doi.org/10.1016/j.eswa.2010.06.060

Yan X, Zhang C, Zhang S (2009) Genetic algorithm-based strategy for identifying association rules without specifying actual minimum support. Expert Syst Appl 36:3066–3076. https://doi.org/10.1016/j.eswa.2008.01.028

Gu T, Chen S, Tao X, Lu J (2010) An unsupervised approach to activity recognition and segmentation based on object-use fingerprints. Data Knowl Eng 69:533–544. https://doi.org/10.1016/j.datak.2010.01.004

Rashidi P, Cook DJ, Holder LB, Schmitter-Edgecombe M (2011) Discovering activities to recognize and track in a smart environment. IEEE Trans Knowl Data Eng 23:527–539. https://doi.org/10.1109/TKDE.2010.148

Lühr S, West G, Venkatesh S (2007) Recognition of emergent human behaviour in a smart home: a data mining approach. Pervasive Mob Comput 3:95–116. https://doi.org/10.1016/j.pmcj.2006.08.002

Yassine A, Singh S, Alamri A (2017) Mining human activity patterns from smart home big data for health care applications. IEEE Access 5:13131–13141. https://doi.org/10.1109/ACCESS.2017.2719921

Hela S, Amel B, Badran R (2018) Early anomaly detection in smart home: a causal association rule-based approach. Artif Intell Med 91:57–71. https://doi.org/10.1016/j.artmed.2018.06.001

Liu Y, Nie L, Liu L, Rosenblum DS (2016) From action to activity: sensor-based activity recognition. Neurocomputing 181:108–115. https://doi.org/10.1016/j.neucom.2015.08.096

Marimuthu P, Perumal V, Vijayakumar V (2019) OAFPM: optimized ANFIS using frequent pattern mining for activity recognition. J Supercomput 75:5347–5366. https://doi.org/10.1007/s11227-019-02802-z

Ni Q, García Hernando A, de la Cruz I (2015) The Elderly’s independent living in smart homes: a characterization of activities and sensing infrastructure survey to facilitate services development. Sensors 15:11312–11362. https://doi.org/10.3390/s150511312

Gong H, Xing K, Du W (2018) A user activity pattern mining system based on human activity recognition and location service. In: IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS). IEEE, pp 1–2

Cao L, Wang Y, Zhang B et al (2018) GCHAR: an efficient Group-based Context—aware human activity recognition on smartphone. J Parallel Distrib Comput 118:67–80. https://doi.org/10.1016/j.jpdc.2017.05.007

Zhang W, Qin L, Zhong W, et al (2019) Framework of sequence chunking for human activity recognition using wearables. In: Proceedings of the 2019 International Conference on image, video and signal processing—IVSP 2019. ACM Press, New York. pp 93–98

Lee H, Ahn C, Choi N et al (2019) The effects of housing environments on the performance of activity-recognition systems using wi-fi channel state information: an exploratory study. Sensors 19:983. https://doi.org/10.3390/s19050983

Aminikhanghahi S, Cook DJ (2019) Enhancing activity recognition using CPD-based activity segmentation. Pervasive Mob Comput 53:75–89. https://doi.org/10.1016/j.pmcj.2019.01.004

Zhang Y, Tian G, Zhang S, Li C (2020) A knowledge-based approach for multiagent collaboration in smart home: from activity recognition to guidance service. IEEE Trans Instrum Meas 69:317–329. https://doi.org/10.1109/TIM.2019.2895931

Civitarese G, Bettini C, Sztyler T et al (2019) newNECTAR: collaborative active learning for knowledge-based probabilistic activity recognition. Pervasive Mob Comput 56:88–105. https://doi.org/10.1016/j.pmcj.2019.04.006

Zhang Shuai, McClean SI, Scotney BW (2012) Probabilistic learning from incomplete data for recognition of activities of daily living in smart homes. IEEE Trans Inf Technol Biomed 16:454–462. https://doi.org/10.1109/TITB.2012.2188534

Stikic M, Larlus D, Ebert S, Schiele B (2011) Weakly supervised recognition of daily life activities with wearable sensors. IEEE Trans Pattern Anal Mach Intell 33:2521–2537. https://doi.org/10.1109/TPAMI.2011.36

Maekawa T, Kishino Y, Sakurai Y, Suyama T (2013) Activity recognition with hand-worn magnetic sensors. Pers Ubiquitous Comput 17:1085–1094. https://doi.org/10.1007/s00779-012-0556-8

Bianchi V, Bassoli M, Lombardo G et al (2019) IoT wearable sensor and deep learning: an integrated approach for personalized human activity recognition in a smart home environment. IEEE Internet Things J 6:8553–8562. https://doi.org/10.1109/JIOT.2019.2920283

Siirtola P, Koskimäki H, Röning J (2019) Personalizing human activity recognition models using incremental learning. arXiv:1905.12628

Burns DM, Whyne CM (2020) Personalized activity recognition with deep triplet embeddings. arXiv:2001.05517.

Vaizman Y, Ellis K, Lanckriet G (2017) Recognizing detailed human context in the wild from smartphones and smartwatches. IEEE Pervasive Comput 16:62–74. https://doi.org/10.1109/MPRV.2017.3971131

Shen C, Li Y, Chen Y et al (2018) Performance analysis of multi-motion sensor behavior for active smartphone authentication. IEEE Trans Inf Forensics Secur 13:48–62. https://doi.org/10.1109/TIFS.2017.2737969

Jalali L, Oh H, Moazeni R, Jain R (2016) Human behavior analysis from smartphone data streams. In: International Workshop on human behavior understanding. pp 68–85

Soleimani E, Nazerfard E (2019) Cross-subject transfer learning in human activity recognition systems using generative adversarial networks. arXiv:1903.12489

Huang P-C, Lee S-S, Kuo Y-H, Lee K-R (2010) A flexible sequence alignment approach on pattern mining and matching for human activity recognition. Expert Syst Appl 37:298–306. https://doi.org/10.1016/j.eswa.2009.05.057

Riboni D, Bettini C (2011) COSAR: hybrid reasoning for context-aware activity recognition. Pers Ubiquitous Comput 15:271–289. https://doi.org/10.1007/s00779-010-0331-7

Wang Z, Wu D, Gravina R et al (2017) Kernel fusion based extreme learning machine for cross-location activity recognition. Inf Fusion 37:1–9. https://doi.org/10.1016/j.inffus.2017.01.004

Guan Y, Plötz T (2017) Ensembles of deep LSTM learners for activity recognition using wearables. Proc ACM Interactive, Mobile Wearable Ubiquitous Technol 1:1–28. https://doi.org/10.1145/3090076

Cook D (2012) Learning setting-generalized activity models for smart spaces. IEEE Intell Syst 27:32–38. https://doi.org/10.1109/MIS.2010.112

Hodges MR, Pollack ME (2007) An ‘Object-Use Fingerprint’: the use of electronic sensors for human identification. In: UbiComp 2007: International Conference on ubiquitous computing. Springer Berlin Heidelberg, Berlin, Heidelberg. pp 289–303

Chen Z, Chen G (2008) Building an associative classifier based on fuzzy association rules. Int J Comput Intell Syst 1:262–273. https://doi.org/10.1080/18756891.2008.9727623

Tax N, Sidorova N, van der Aalst WMP (2018) Discovering more precise process models from event logs by filtering out chaotic activities. J Intell Inf Syst. https://doi.org/10.1007/s10844-018-0507-6

Hwang I, Jang YJ (2017) Process mining to discover shoppers’ pathways at a fashion retail store using a wifi-base indoor positioning system. IEEE Trans Autom Sci Eng 14:1786–1792. https://doi.org/10.1109/TASE.2017.2692961

Verbeek E, van der Aalst WMP (2000) Woflan 2.0 A Petri-Net-based workflow diagnosis tool. In: International Conference on application and theory of Petri Nets. pp 475–484

Buijs JCAM, van Dongen BF, van der Aalst WMP (2012) On the role of fitness, precision, generalization and simplicity in process discovery. In: OTM Confederated International Conferences “On the Move to Meaningful Internet Systems.” pp 305–322

Roggen D, Calatroni A, Rossi M, et al (2010) Collecting complex activity datasets in highly rich networked sensor environments. In: 2010 Seventh International Conference on Networked Sensing Systems (INSS). IEEE, New York. pp 233–240

Kingma DP, Ba LJ (2015) Adam: A Method for stochastic optimization. In: International Conference on Learning Representations (ICLR). pp 1–11

Trabelsi D, Mohammed S, Chamroukhi F et al (2013) An unsupervised approach for automatic activity recognition based on hidden Markov Model regression. IEEE Trans Autom Sci Eng 10:829–835. https://doi.org/10.1109/TASE.2013.2256349

van der Aalst WM, Bolt A, van Zelst SJ (2017) RapidProM: mine your processes and not just your data. arXiv:1703.03740.

Yao S, Hu S, Zhao Y, et al (2017) DeepSense: a unified deep learning framework for time-series mobile sensing data processing. In: Proceedings of the 26th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, pp 351–360

Zhao Y, Yang R, Chevalier G et al (2018) Deep residual Bidir-LSTM for human activity recognition using wearable sensors. Math Probl Eng 2018:1–13. https://doi.org/10.1155/2018/7316954