Benchmarking distance-based partitioning methods for mixed-type data
Tóm tắt
Clustering mixed-type data, that is, observation by variable data that consist of both continuous and categorical variables poses novel challenges. Foremost among these challenges is the choice of the most appropriate clustering method for the data. This paper presents a benchmarking study comparing eight distance-based partitioning methods for mixed-type data in terms of cluster recovery performance. A series of simulations carried out by a full factorial design are presented that examined the effect of a variety of factors on cluster recovery. The amount of cluster overlap, the percentage of categorical variables in the data set, the number of clusters and the number of observations had the largest effects on cluster recovery and in most of the tested scenarios. KAMILA, K-Prototypes and sequential Factor Analysis and K-Means clustering typically performed better than other methods. The study can be a useful reference for practitioners in the choice of the most appropriate method.
Tài liệu tham khảo
Ahmad A, Dey L (2007) A k-mean clustering algorithm for mixed numeric and categorical data. Data & Knowledge Engineering 63(2):503–527
Ahmad A, Khan SS (2019) Survey of state-of-the-art mixed data clustering algorithms. IEEE Access 7:31,883–31,902
Arabie P (1994) Cluster analysis in marketing research. Blackwell, Oxford, pp 160–189
Audigier V, Husson F, Josse J (2016) A principal component method to impute missing values for mixed data. Adv Data Anal Classif 10(1):5–26
Boulesteix AL, Hatz M (2017) Benchmarking for clustering methods based on real data: A statistical view. In: Palumbo F, Montanari A, Vichi M (eds) Data Science. Springer International Publishing, Cham, pp 73–82
Boulesteix AL, Lauer S, Eugster MJ (2013) A plea for neutral comparison studies in computational sciences. PLoS ONE 8(e61):562
De Soete G, Carroll JD (1994) K-means clustering in a low-dimensional Euclidean space, Springer, 212–219
Dolnicar S, Grün B (2008) Challenging “factor-cluster segmentation’’. J Travel Res 47(1):63–71
Ferreira L, Hitchcock DB (2009) A comparison of hierarchical methods for clustering functional data. Communications in Statistics - Simulation and Computation 38(9):1925–1949
Foss A, Markatou M, Ray B et al (2016) A semiparametric method for clustering mixed data. Mach Learn 105(3):419–458
Foss AH, Markatou M (2018) kamila: Clustering mixed-type data in R and Hadoop. J Stat Softw 83:1–44
Foss AH, Markatou M, Ray B (2019) Distance metrics and clustering methods for mixed-type data. Int Stat Rev 87(1):80–109
Gower JC (1971) A general coefficient of similarity and some of its properties. Biometrics 27:857–871
Hennig C (2020) Package ‘fpc’. URL https://cran.r-project.org/web/packages/fpc/fpc.pdf
Hennig C (2022) An empirical comparison and characterisation of nine popular clustering methods. Adv Data Anal Classif 16:201–229
Hennig C, Liao TF (2013) How to find an appropriate clustering for mixed-type variables with application to socio-economic stratification. J Roy Stat Soc: Ser C (Appl Stat) 62(3):309–369
Huang Z (1997) Clustering large data sets with mixed numeric and categorical values. In: Proceedings of the 1st Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD), Citeseer, 21–34
Hubert L, Arabie P (1985) Comparing partitions. J Classif 2(2):193–218
Hunt L, Jorgensen M (2011) Clustering mixed data. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 1(4):352–361
Javed A, Lee BS, Rizzo DM (2020) A benchmark study on time series clustering. Machine Learning with Applications 1(100):001
Jimeno J, Roy M, Tortora C (2021) Clustering mixed-type data: A benchmark study on KAMILA and K-Prototypes. In: Chadjipadelis T, Lausen B, Markos A et al (eds) Data Analysis and Rationality in a Complex World. Springer International Publishing, Cham, pp 83–91
Kaufman L, Rousseeuw PJ (1990) Finding Groups in Data: An Introduction to Cluster Analysis. John Wiley & Sons, chap 2:68–125
Kiers HA (1991) Simple structure in component analysis techniques for mixtures of qualitative and quantitative variables. Psychometrika 56(2):197–212
Maechler M, Rousseeuw P, Struyf A et al (2021) cluster: Cluster Analysis Basics and Extensions. URL https://CRAN.R-project.org/package=cluster, R package version 2.1.2)
Maitra R, Melnykov V (2010) Simulating data to study performance of finite mixture modeling and clustering algorithms. J Comput Graph Stat 19(2):354–376
Markos A, Iodice D’Enza A, van de Velden M (2019) Beyond tandem analysis: Joint dimension reduction and clustering in R. J Stat Softw 91:1–24
Markos A, Moschidis O, Chadjipantelis T (2020) Sequential dimension reduction and clustering of mixed-type data. International Journal of Data Analysis Techniques and Strategies 12(3):228–246
Meilă M, Heckerman D (2001) An experimental comparison of model-based clustering methods. Mach Learn 42:9–29
Melnykov V, Maitra R (2010) Finite mixture models and model-based clustering. Statistics Surveys 4:80–116
Melnykov V, Chen WC, Maitra R (2012) MixSim: An R package for simulating data to study performance of clustering algorithms. J Stat Softw 51(12):1–25
Milligan GW (1980) An examination of the effect of six types of error perturbation on fifteen clustering algorithms. Psychometrika 45:325–342
Modha DS, Spangler WS (2003) Feature weighting in k-means clustering. Mach Learn 52(3):217–237
Murtagh F (2015) A Brief History of Cluster Analysis. In: Hennig C, Meila M, Murtagh F et al (eds) Handbook of Cluster Analysis. Chapman & Hall/CRC, 21–33
Pagès J (2014) Multiple Factor Analysis By Example Using R. Chapman and Hall/CRC, chap 3:67–78
Preud’Homme G, Duarte K, Dalleau K et al (2021) Head-to-head comparison of clustering methods for heterogeneous data: a simulation-driven benchmark. Sci Rep 11(1):1–14
Saccenti E, Timmerman ME (2016) Approaches to sample size determination for multivariate data: Applications to PCA and PLS-DA of omics data. J Proteome Res 15(8):2379–2393
Saraçli S, Doğan N, Doğan İsmet (2013) Comparison of hierarchical cluster analysis methods by cophenetic correlation. Journal of Inequalities And Applications 2013:1–8
Shireman EM, Steinley D, Brusco MJ (2016) Local optima in mixture modeling. Multivar Behav Res 51(4):466–481
Steinley D (2004) Properties of the Hubert-Arabie Adjusted Rand Index. Psychol Methods 9(3):386–396
Steinley D (2006) Profiling local optima in k-means clustering: developing a diagnostic technique. Psychol Methods 11(2):178–192
Szepannek G (2018) clustMixType: User-Friendly Clustering of Mixed-Type Data in R. The R Journal 10(2):200–208
Van der Hoef H, Warrens MJ (2019) Understanding information theoretic measures for comparing clusterings. Behaviormetrika 46:353–370
Van Mechelen I, Boulesteix AL, Dang R et al (2018) Benchmarking in cluster analysis: A white paper arxiv:1809.10496v2
van de Velden M, Iodice D’Enza A, Palumbo F (2017) Cluster correspondence analysis. Psychometrika 82(1):158–185
van de Velden M, Iodice D’Enza A, Markos A (2019) Distance-based clustering of mixed data. Wiley Interdisciplinary Reviews: Computational Statistics 11(3):e1456
Vichi M, Kiers HA (2001) Factorial k-means analysis for two-way data. Computational Statistics & Data Analysis 37(1):49–64
Vichi M, Vicari D, Kiers HA (2019) Clustering and dimension reduction for mixed variables. Behaviormetrika 46(2):243–269
Vinh NX, Epps J, Bailey J (2010) Information theoretic measures for clusterings comparison: Variants, properties, normalization and correction for chance. The Journal of Machine Learning Research 11:2837–2854
Yamamoto M, Hwang H (2014) A general formulation of cluster analysis with dimension reduction and subspace separation. Behaviormetrika 41(1):115–129