DeepMCGCN: Multi-channel Deep Graph Neural Networks
Tóm tắt
Graph neural networks (GNNs) have shown powerful capabilities in modeling and representing graph structural data across various graph learning tasks as an emerging deep learning approach. However, most existing GNNs focus on single-relational graphs and fail to fully utilize the rich and diverse relational information present in real-world graph data. In addition, deeper GNNs tend to suffer from overfitting and oversmoothing issues, leading to degraded model performance. To deeply excavate the multi-relational features in graph data and strengthen the modeling and representation abilities of GNNs, this paper proposes a multi-channel deep graph convolutional neural network method called DeepMCGCN. It constructs multiple relational subgraphs and adopts multiple GCN channels to learn the characteristics of different relational subgraphs separately. Cross-channel connections are utilized to obtain interactions between different relational subgraphs, which can learn node embeddings richer and more discriminative than single-channel GNNs. Meanwhile, it alleviates overfitting issues of deep models by optimizing convolution functions and adding residual connections between and within channels. The DeepMCGCN method is evaluated on three real-world datasets, and the experimental results show that its node classification performance outperforms that of single-channel GCN and other benchmark models, which improves the modeling and representation capabilities of the model.
Từ khóa
Tài liệu tham khảo
Scott, J., Carrington, P.J.: The SAGE handbook of social network analysis. SAGE publications, London (2011)
Gligorijević, V., Barot, M., Bonneau, R.: deepNF: deep network fusion for protein function prediction. Bioinformatics 34(22), 3873–3881 (2018). https://doi.org/10.1093/bioinformatics/bty440
Newman, M.E.J.: Scientific collaboration networks. I. Network construction and fundamental results. Phys. Rev. E 64(1), 016131 (2001). https://doi.org/10.1103/PhysRevE.64.016131
Farahani, R.Z., Miandoabchi, E., Szeto, W.Y., Rashidi, H.: A review of urban transportation network design problems. Eur. J. Oper. Res.Oper. Res. 229(2), 281–302 (2013). https://doi.org/10.1016/j.ejor.2013.01.001
Xiao, S., Wang, S., Dai, Y., Guo, W.: Graph neural networks in node classification: survey and evaluation. Mach. Vis. Appl. 33, 1–19 (2022). https://doi.org/10.1007/s00138-021-01251-0
Luan, S., Hua, C., Xu, M., Lu, Q. C., Zhu, J. Q., Chang, X. W., Fu, J., Leskovec, J., Precup, D.: When do graph neural networks help with node classification: Investigating the homophily principle on node distinguishability (2023). arXiv preprint arXiv:2304.14274. https://doi.org/10.48550/arXiv.2304.14274
Wang, C., Pan, S., Yu, P.C., Hu, R.Q., Long, G.D., Zhang, C.Q.: Deep neighbor-aware embedding for node clustering in attributed graphs. Pattern Recognit. 122, 108230 (2022). https://doi.org/10.1016/j.patcog.2021.108230
Khan, M.F., Bibi, M., Aadil, F., Lee, J.W.: Adaptive node clustering for underwater sensor networks. Sensors. 21(13), 4514 (2021). https://doi.org/10.3390/s21134514
Zhang, M., Chen, Y.: Link prediction based on graph neural networks. In: NeurIPS 2018. MIT Press, Oxford (2018)
Guo, Z., Shiao, W., Zhang, S., Liu, Y.Z., Chawla, N.V., Shah, N., Zhao, T.: Linkless link prediction via relational distillation. In: PMLR 2023, vol. 202, pp. 12012–12033 (2023).
Yang, Y.L., Ye, Z.L., Zhao, H.X., Meng, L.: A graph representation learning framework predicting potential multivariate interactions. Int. J. Comput. Intell. Syst. 16(1), 1–16 (2023). https://doi.org/10.1007/s44196-023-00329-z
Chen, X., Jia, S., Xiang, Y.: A review: knowledge reasoning over knowledge graph. Expert Syst. Appl. 141, 112948 (2020). https://doi.org/10.1016/j.eswa.2019.112948
Liu, S., Qin, Y.F., Xu, M., Kolmanič, S.: Knowledge graph completion with triple structure and text representation. Int. J. Comput. Intell. Syst. 16(1), 95 (2023). https://doi.org/10.1007/s44196-023-00271-0
Ying, R., He, R., Chen, K., Eksombatchai, P., Hamilton, W., Leskovec, J.: Graph convolutional neural networks for web-scale recommender systems. In: ACM SIGKDD’24, pp. 974–983 (2018). https://doi.org/10.1145/3219819.3219890
Wu, C., Liu, S., Zeng, Z., Chen, M., Alhudhaif, A., Tang, X.Y., Alenezi, F., Alnaim, N., Peng, X.C.: Knowledge graph-based multi-context-aware recommendation algorithm. Inf. Sci. 595, 179–194 (2022). https://doi.org/10.1016/j.ins.2022.02.054
Gao, C., Zheng, Y., Li, N., Li, Y., Qin, Y., Piao, J., Quan, Y., Chang, J., Jin, D., He, X., Li, Y.: A survey of graph neural networks for recommender systems: challenges, methods, and directions. ACM Trans. Web 1(1), 1–51 (2023). https://doi.org/10.1145/3568022
Zhang, Y., Li, C., Cai, J., Liu, Y., Wang, H.: BKGNN-TI: a bilinear knowledge-aware graph neural network fusing text information for recommendation. Int. J. Comput. Intell. Syst. 15(1), 95 (2022). https://doi.org/10.1007/s44196-022-00154-w
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., Yu, P.S.: A comprehensive survey on graph neural networks. IEEE Trans. Neural Netw. Learn. Syst. 32(1), 4–24 (2020). https://doi.org/10.1109/TNNLS.2020.2978386
Kipf, T. N., Welling, M.: Semi-supervised classification with graph convolutional networks (2016). arXiv preprint arXiv:1609.02907. https://doi.org/10.48550/arXiv.1609.02907
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y. S.: Graph attention networks (2017). arXiv preprint arXiv:1710.10903. https://doi.org/10.48550/arXiv.1710.10903
Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. In: NeurIPS 2017. MIT Press, Oxford (2017)
Wu, F., Souza, A., Zhang, T. Y, Fifty, C., Yu, T., Weinberger, K.: Simplifying graph convolutional networks. In: PMLR 2019, vol. 97, pp. 6861–6871 (2019).
Zeng, D.Y., Liu, W. L., Chen, W.Y, Zhou, L., Zhang, M.L., Qu, H.: Substructure aware graph neural networks. In: AAAI’2023. AAAI Press. vol. 37(9), pp. 11129–11137 (2023). https://doi.org/10.1609/aaai.v37i9.26318
Sriramulu, A., Fourrier, N., Bergmeir, C.: Adaptive dependency learning graph neural networks. Inf. Sci. 625, 700–714 (2023). https://doi.org/10.1016/j.ins.2022.12.086
Peng, L., Hu, R., Kong, F., Gan, J.Z., Mo, Y.J., Shi, X.S., Zhu, X.F.: Reverse graph learning for graph neural network. IEEE Trans. Neural Netw. Learn. Syst. (2022). https://doi.org/10.1109/TNNLS.2022.3161030
Liu, Z., Yang, D., Wang, Y.J., Lu, M.J., Li, R.R.: EGNN: graph structure learning based on evolutionary computation helps more in graph neural networks. Appl. Soft Comput.Comput. 135, 110040 (2023). https://doi.org/10.1016/j.asoc.2023.110040
Zou, M.H., Gan, Z.X., Cao, R.Z., Guan, C., Leng, S.Y.: Similarity-navigated graph neural networks for node classification. Inf. Sci. 633, 41–69 (2023). https://doi.org/10.1016/j.ins.2023.03.057
Zhong, Z., Li, C.T., Pang, J.: Hierarchical message-passing graph neural networks. Data. Min. Knowl. Discov. 37(1), 381–408 (2023). https://doi.org/10.1007/s10618-022-00890-9
Oskarsson, J., Sidén, P., Lindsten, F.: Temporal graph neural networks for irregular data. In: PMLR 2023, pp. 4515–4531 (2023).
Islam, M.I.K., Khanov, M., Akbas, E.: MPool: motif-based graph pooling. In: PAKDD 2023. Springer Nature, Switzerland, pp. 105–117 (2023). https://doi.org/10.1007/978-3-031-33377-4_9
Bo, D.Y., Shi, C., Wang, L.L., Liao, R.J.: Specformer: Spectral graph neural networks meet transformers. In: ICLR 2023. (2023). https://openreview.net/forum?id=0pdSt3oyJa1
Dudzik, A.J., Veličković, P.: Graph neural networks are dynamic programmers. In: NeurIPS 2022, vol. 35, pp. 20635–20647 (2022).
Lin, R.J., Du, S.D., Wang, S.P., Guo, W.Z.: Multi-channel augmented graph embedding convolutional network for multi-view clustering. IEEE Trans. Netw. Sci. Eng. 10(4), 2239–2249 (2023)
Zhu, X.F., Li, C.H., Guo, J.F., Dietze, S.: CNIM-GCN: consensus neighbor interaction-based multi-channel graph convolutional networks. Expert Syst. Appl. 226, 120178 (2023). https://doi.org/10.1016/j.eswa.2023.120178
Zhai, R., Zhang, L.B., Wang, Y.Q., Song, Y.L.: A multi-channel attention graph convolutional neural network for node classification. J. Supercomput.Supercomput. 79(4), 3561–3579 (2023). https://doi.org/10.1007/s11227-022-04778-9
Chao, H., Cao, Y.M., Liu, Y.L.: Multi-channel EEG emotion recognition using Residual Graph Attention Neural Network. Front. Neurosci.Neurosci. 17, 1135850 (2023). https://doi.org/10.3389/fnins.2023.1135850
Li, J.C., Lu, G.G., Wu, Z.T., Ling, F.Q.: Multi-view representation model based on graph autoencoder. Inf. Sci. 632, 439–453 (2023). https://doi.org/10.1016/j.ins.2023.02.092
Xu, K., Li, C., Tian, Y., et al.: Representation learning on graphs with jumping knowledge networks. In: PMLR 2018, vol. 80, pp. 5453–5462 (2018).
Li, Q., Han, Z., Wu, X. M.: Deeper insights into graph convolutional networks for semi-supervised learning. In: AAAI’2018, pp. 32(1) (2018). https://doi.org/10.1609/aaai.v32i1.11604.
Li, G. H., Xiong, C. X., Thabet, A., Ghanem, B.: Deepergcn: all you need to train deeper GCNS (2020). arXiv preprint arXiv:2006.07739. https://doi.org/10.48550/arXiv.2006.07739
Rong, Y., Huang, W., Xu, T., Huang, J. Z.: Dropedge: towards deep graph convolutional networks on node classification (2019). arXiv preprint arXiv:1907.10903. https://doi.org/10.48550/arXiv.1907.10903
Chen, M., Wei, Z.W., Huang, Z.F., Ding, B.L., Li, Y.L.: Simple and deep graph convolutional networks. In: PMLR 2020, pp. 1725–1735 (2020).
Gao, Z., Gama, F., Ribeiro, A.: Wide and deep graph neural network with distributed online learning. IEEE Trans. Signal Process. 70, 3862–3877 (2022). https://doi.org/10.1109/TSP.2022.3192606
Feng, G.S., Wang, H.Z., Wang, C.N.: Search for deep graph neural networks. Inf. Sci. (2023). https://doi.org/10.1016/j.ins.2023.119617
Perozzi, B., Al-Rfou, R., Skiena, S.: Deepwalk: online learning of social representations. In: ACM SIGKDD’14, pp. 701–710 (2014). https://doi.org/10.1145/2623330.2623732
Tang J, Qu M, Wang M, et al. Line: large-scale information network embedding. In: WWW’15, pp. 1067–1077 (2015). https://doi.org/10.1145/2736277.2741093
Grover, A., Leskovec, J.: node2vec: scalable feature learning for networks. In: ACM SIGKDD’16, pp. 855–864 (2016). https://doi.org/10.1145/2939672.2939754