Feature representation via graph-regularized entropy-weighted nonnegative matrix factorization

Document Type : Original Article

Authors

1 Department of Applied Mathematics, University of Kurdistan, Sanandaj, Iran

2 Department of Computer Engineering‎, ‎University of Kurdistan‎, ‎Sanandaj‎, ‎Iran

3 School of Engineering, RMIT University, Melbourne, Australia

Abstract

Feature extraction plays a crucial role in dimensionality reduction in machine learning applications. Nonnegative Matrix Factorization (NMF) has emerged as a powerful technique for dimensionality reduction; however, its equal treatment of all features may limit accuracy. To address this challenge, this paper introduces Graph-Regularized Entropy-Weighted Nonnegative Matrix Factorization (GEWNMF) for enhanced feature representation. The proposed method improves feature extraction through two key innovations: optimizable feature weights and graph regularization. GEWNMF uses optimizable weights to prioritize the extraction of crucial features that best describe the underlying data structure. These weights, determined using entropy measures, ensure a diverse selection of features, thereby enhancing the fidelity of the data representation. This adaptive weighting not only improves interpretability but also strengthens the model against noisy or outlier-prone datasets. Furthermore, GEWNMF integrates robust graph regularization techniques to preserve local data relationships. By constructing an adjacency graph that captures these relationships, the method enhances its ability to discern meaningful patterns amid noise and variability. This regularization not only stabilizes the method but also ensures that nearby data points appropriately influence feature extraction. Thus, GEWNMF produces representations that capture both global trends and local nuances, making it applicable across various domains. Extensive experiments on four widely used datasets validate the efficacy of GEWNMF compared to existing methods, demonstrating its superior performance in capturing meaningful data patterns and enhancing interpretability.

Keywords

Main Subjects

References

[1] I. Ahmed, T. Galoppo, X. Hu, and Y. Ding, Graph regularized autoencoder and its application in unsu[1]pervised anomaly detection, IEEE Trans. Pattern Anal. Mach. Intell., 44 (2022), pp. 4110–4124.
[2] V. D. Blondel, N. D. Ho, and P. V. Dooren, Weighted nonnegative matrix factorization and face feature extraction, Image Vis. Comput., 26 (2008), pp. 1–17.
[3] D. Cai, X. He, and J. Han, Document clustering using locality preserving indexing, IEEE Trans. Knowl. Data Eng., 17 (2005), pp. 1624–1637.
[4] D. Cai, X. He, J. Han, and T. S. Huang, Graph regularized nonnegative matrix factorization for data representation, IEEE Trans. Pattern Anal. Mach. Intell., 33 (2010), pp. 1548–1560.
[5] M. Chen, M. Gong, and X. Li, Feature weighted non-negative matrix factorization, IEEE Trans. Cybern., 53 (2021), pp. 1093–1105.
[6] P. Deng, T. Li, H. Wang, S. J. Horng, Z. Yu, and X. Wang, Tri-regularized nonnegative matrix tri-factorization for co-clustering, Knowl.-Based Syst., 226 (2021), p. 107101.
[7] P. Deng, T. Li, H. Wang, D. Wang, S. Horng, and R. Liu, Graph regularized sparse non-negative matrix factorization for clustering, IEEE Trans. Comput. Soc. Syst., (2022), pp. 1–12.
[8] C. Ding, T. Li, and M. I. Jordan, Convex and semi-nonnegative matrix factorizations, IEEE Trans. Pattern Anal. Mach. Intell., 32 (2010), pp. 45–55.
[9] C. Ding, T. Li, W. Peng, and H. Park, Orthogonal nonnegative matrix t-factorizations for clustering, in Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2006, pp. 126–135.
[10] Y. Dong, H. Che, M. F. Leung, C. Liu, and Z. Yan, Centric graph regularized log-norm sparse non[1]negative matrix factorization for multi-view clustering, Signal Process., 217 (2024), pp. 109–341.
[11] H. Gao, F. Nie, W. Cai, and H. Huang, Robust capped norm nonnegative matrix factorization: Capped norm nmf, in Proceedings of the 24th ACM International Conference on Information and Knowledge Manage[1]ment, 2015, pp. 871–880.
[12] N. Guan, T. Liu, Y. Zhang, D. Tao, and L. S. Davis, Truncated cauchy non-negative matrix factorization, IEEE Trans. Pattern Anal. Mach. Intell., 41 (2017), pp. 246–259.
[13] R. Hamon, V. Emiya, and C. Fevotte, Convex nonnegative matrix factorization with missing data, in IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP), 2016, pp. 1–6.
[14] A. B. Hamza and D. J. Brady, Reconstruction of reflectance spectra using robust nonnegative matrix factorization, IEEE Trans. Signal Process., 54 (2006), pp. 3637–3642.
[15] S. Huang, H. Wang, T. Li, T. Li, and Z. Xu, Robust graph regularized nonnegative matrix factorization for clustering, Data Min. Knowl. Discov., 32 (2018), pp. 483–503.
[16] S. Huang, Z. Xu, Z. Kang, and Y. Ren, Regularized nonnegative matrix factorization with adaptive local structure learning, Neurocomputing, 382 (2020), pp. 196–209.
[17] Q. i. Huang, X. Yin, S. Chen, Y. Wang, and B. Chen, Robust nonnegative matrix factorization with structure regularization, Neurocomputing, 412 (2020), pp. 72–90.
[18] Y. Jia, S. Kwong, J. Hou, and W. Wu, Semi-supervised non-negative matrix factorization with dissimilarity and similarity regularization, IEEE Trans. Neural Netw. Learn. Syst., 31 (2019), pp. 2510–2521.
[19] Y. D. Kim and S. Choi, Weighted nonnegative matrix factorization, in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, 2009, pp. 19–24.
[20] D. D. Lee and H. S. Seung, Learning the parts of objects by non-negative matrix factorization, Nature, 401 (1999), pp. 788–791.
[21] H. Liu, Z. Wu, X. Li, D. Cai, and T. S. Huang, Constrained nonnegative matrix factorization for image representation, IEEE Trans. Pattern Anal. Mach. Intell., 34 (2012), pp. 1299–1311.
[22] A. Lotfi, P. Moradi, and H. Beigy, Density peaks clustering based on density backbone and fuzzy neigh[1]borhood, Pattern Recognit., 107 (2020), p. 107449.
[23] X. Ma, W. Zhao, and W. Wu, Layer-specific modules detection in cancer multi-layer networks, IEEE/ACM Trans. Comput. Biol. Bioinform., 20 (2022), pp. 1170–1179.
[24] F. Nie, X. Wang, and H. Huang, Clustering and projected clustering with adaptive neighbors, in Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, USA, 2014, Association for Computing Machinery, pp. 977–986.
[25] P. Paatero, Least squares formulation of robust, non-negative factor analysis, Chemom. Intell. Lab. Syst., 37 (1997), pp. 23–35.
[26] A. Suleman, A convex semi-nonnegative matrix factorization approach to fuzzy k-means clustering, Fuzzy Sets Syst., 270 (2015), pp. 90–110.
[27] J. Tang and H. Feng, Robust local-coordinate non-negative matrix factorization with adaptive graph for robust clustering, Inf. Sci., 610 (2022), pp. 1058–1077.
[28] C. Wang, J. Wang, Z. Gu, J. M. Wei, and J. Liu, Unsupervised feature selection by learning exponential weights, Pattern Recognit., 148 (2024), pp. 110–183.
[29] D. Wang, T. Li, and C. Ding, Weighted feature subset non-negative matrix factorization and its applications to document understanding, in IEEE International Conference on Data Mining, 2010, pp. 541–550.
[30] Y. X. Wang and Y. J. Zhang, Nonnegative matrix factorization: a comprehensive review, IEEE Trans. Knowl. Data Eng., 25 (2013), pp. 1336–1353.
[31] J. Wei, C. Tong, B. Wu, H. Qiang, Q. Shouliang, Y. Yudong, and T. Yueyang, An entropy weighted nonnegative matrix factorization algorithm for feature representation, IEEE Trans. Neural Netw. Learn. Syst., 34 (2022), pp. 1–11.
[32] W. Wu, Y. Jia, S. Wang, R. Wang, H. Fan, and S. Kwong, Positive and negative label-driven nonnegative matrix factorization, IEEE Trans. Circuits Syst. Video Technol., 31 (2021), pp. 2698–2710.
[33] W. Wu, S. Kwong, J. Hou, Y. Jia, and H. H. S. Ip, Simultaneous dimensionality reduction and classifica[1]tion via dual embedding regularized nonnegative matrix factorization, IEEE Trans. Image Process., 28 (2019), pp. 3836–3847.
[34] Y. Wu, Y. Wang, L. Hu, and J. Hu, Dcgnn: Adaptive deep graph convolution for heterophily graphs, Inf. Sci., 666 (2024), pp. 120–427.
[35] X. Zhang, D. Chen, H. Yu, G. Wang, H. Tang, and K. Wu, Improving nonnegative matrix factorization with advanced graph regularization, Inf. Sci., 597 (2022), pp. 125–143.
[36] J. Zhao and G. F. Lu, Clean affinity matrix learning with rank equality constraint for multiview subspace clustering, Pattern Recognit., 134 (2023), pp. 109–118.
[37] Z. Zhou, G. Si, H. Sun, K. Qu, and W. Hou, A robust clustering algorithm based on the identification of core points and k-nn kernel density estimation, Expert Syst. Appl., 195 (2022), pp. 116–573.
AUT Journal of Mathematics and Computing
Volume 5, Issue 4
2024
Pages 289-304

Files

Share

How to cite

Statistics