Mosaic LSM: A Liquid State Machine Approach for Multimodal Longitudinal Data Analysis

Sugam Budhraja; Balkaran Singh; Maryam Doborjeh; Zohreh Doborjeh; Samuel Tan; Edmund Lai; Wilson Goh; Nikola Kasabov

doi:10.1109/IJCNN54540.2023.10191256

Mosaic LSM: A Liquid State Machine Approach for Multimodal Longitudinal Data Analysis

Sugam Budhraja, Balkaran Singh, Maryam Doborjeh, Zohreh Doborjeh, Samuel Tan, Edmund Lai, Wilson Goh, Nikola Kasabov^*

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Citations (Scopus)

Abstract

In this paper, we present a novel Liquid State Machine (LSM) based approach for modelling of multimodal longitudinal data: the Mosaic LSM. Our model harnesses the strengths of multiple LSMs, each designed to capture the temporal patterns of a specific data modality. This temporal information is then added to the raw data to create a composite representation that encompasses both the multimodal and the longitudinal aspects of the data. We demonstrate the performance of our approach on a real-world dataset that contains clinical, cognitive, and genetic modalities with the aim of predicting the Ultra-High Risk (UHR) status in individuals, six months in advance. Our results show that the Mosaic LSM outperforms traditional machine learning models, achieving an outstanding Matthew's Correlation Coefficient of 0.84 and prediction accuracy of 92.4%. Overall, our work highlights the potential of Mosaic LSM as a powerful tool for disease prognosis, and its ability to leverage both the multimodality and temporality of the data to improve performance.

Original language	English
Title of host publication	IJCNN 2023 - International Joint Conference on Neural Networks, Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781665488679
DOIs	https://doi.org/10.1109/IJCNN54540.2023.10191256
Publication status	Published - 2023
Externally published	Yes
Event	2023 International Joint Conference on Neural Networks, IJCNN 2023 - Gold Coast, Australia Duration: Jun 18 2023 → Jun 23 2023

Publication series

Name	Proceedings of the International Joint Conference on Neural Networks
Volume	2023-June

Conference

Conference	2023 International Joint Conference on Neural Networks, IJCNN 2023
Country/Territory	Australia
City	Gold Coast
Period	6/18/23 → 6/23/23

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

ASJC Scopus Subject Areas

Software
Artificial Intelligence

Keywords

Liquid State Machine
Multimodal Learning
Prognosis
Spiking Neural Networks
Time-Series
Ultra-High Risk

Access to Document

10.1109/IJCNN54540.2023.10191256

Cite this

Budhraja, S., Singh, B., Doborjeh, M., Doborjeh, Z., Tan, S., Lai, E., Goh, W., & Kasabov, N. (2023). Mosaic LSM: A Liquid State Machine Approach for Multimodal Longitudinal Data Analysis. In IJCNN 2023 - International Joint Conference on Neural Networks, Proceedings (Proceedings of the International Joint Conference on Neural Networks; Vol. 2023-June). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IJCNN54540.2023.10191256

@inproceedings{70e7c1740bfb4ecbb711c2682e4651a5,

title = "Mosaic LSM: A Liquid State Machine Approach for Multimodal Longitudinal Data Analysis",

abstract = "In this paper, we present a novel Liquid State Machine (LSM) based approach for modelling of multimodal longitudinal data: the Mosaic LSM. Our model harnesses the strengths of multiple LSMs, each designed to capture the temporal patterns of a specific data modality. This temporal information is then added to the raw data to create a composite representation that encompasses both the multimodal and the longitudinal aspects of the data. We demonstrate the performance of our approach on a real-world dataset that contains clinical, cognitive, and genetic modalities with the aim of predicting the Ultra-High Risk (UHR) status in individuals, six months in advance. Our results show that the Mosaic LSM outperforms traditional machine learning models, achieving an outstanding Matthew's Correlation Coefficient of 0.84 and prediction accuracy of 92.4%. Overall, our work highlights the potential of Mosaic LSM as a powerful tool for disease prognosis, and its ability to leverage both the multimodality and temporality of the data to improve performance.",

keywords = "Liquid State Machine, Multimodal Learning, Prognosis, Spiking Neural Networks, Time-Series, Ultra-High Risk",

author = "Sugam Budhraja and Balkaran Singh and Maryam Doborjeh and Zohreh Doborjeh and Samuel Tan and Edmund Lai and Wilson Goh and Nikola Kasabov",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 2023 International Joint Conference on Neural Networks, IJCNN 2023 ; Conference date: 18-06-2023 Through 23-06-2023",

year = "2023",

doi = "10.1109/IJCNN54540.2023.10191256",

language = "English",

series = "Proceedings of the International Joint Conference on Neural Networks",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "IJCNN 2023 - International Joint Conference on Neural Networks, Proceedings",

address = "United States",

}

Budhraja, S, Singh, B, Doborjeh, M, Doborjeh, Z, Tan, S, Lai, E, Goh, W & Kasabov, N 2023, Mosaic LSM: A Liquid State Machine Approach for Multimodal Longitudinal Data Analysis. in IJCNN 2023 - International Joint Conference on Neural Networks, Proceedings. Proceedings of the International Joint Conference on Neural Networks, vol. 2023-June, Institute of Electrical and Electronics Engineers Inc., 2023 International Joint Conference on Neural Networks, IJCNN 2023, Gold Coast, Australia, 6/18/23. https://doi.org/10.1109/IJCNN54540.2023.10191256

Mosaic LSM: A Liquid State Machine Approach for Multimodal Longitudinal Data Analysis. / Budhraja, Sugam; Singh, Balkaran; Doborjeh, Maryam et al.
IJCNN 2023 - International Joint Conference on Neural Networks, Proceedings. Institute of Electrical and Electronics Engineers Inc., 2023. (Proceedings of the International Joint Conference on Neural Networks; Vol. 2023-June).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Mosaic LSM

T2 - 2023 International Joint Conference on Neural Networks, IJCNN 2023

AU - Budhraja, Sugam

AU - Singh, Balkaran

AU - Doborjeh, Maryam

AU - Doborjeh, Zohreh

AU - Tan, Samuel

AU - Lai, Edmund

AU - Goh, Wilson

AU - Kasabov, Nikola

PY - 2023

Y1 - 2023

N2 - In this paper, we present a novel Liquid State Machine (LSM) based approach for modelling of multimodal longitudinal data: the Mosaic LSM. Our model harnesses the strengths of multiple LSMs, each designed to capture the temporal patterns of a specific data modality. This temporal information is then added to the raw data to create a composite representation that encompasses both the multimodal and the longitudinal aspects of the data. We demonstrate the performance of our approach on a real-world dataset that contains clinical, cognitive, and genetic modalities with the aim of predicting the Ultra-High Risk (UHR) status in individuals, six months in advance. Our results show that the Mosaic LSM outperforms traditional machine learning models, achieving an outstanding Matthew's Correlation Coefficient of 0.84 and prediction accuracy of 92.4%. Overall, our work highlights the potential of Mosaic LSM as a powerful tool for disease prognosis, and its ability to leverage both the multimodality and temporality of the data to improve performance.

AB - In this paper, we present a novel Liquid State Machine (LSM) based approach for modelling of multimodal longitudinal data: the Mosaic LSM. Our model harnesses the strengths of multiple LSMs, each designed to capture the temporal patterns of a specific data modality. This temporal information is then added to the raw data to create a composite representation that encompasses both the multimodal and the longitudinal aspects of the data. We demonstrate the performance of our approach on a real-world dataset that contains clinical, cognitive, and genetic modalities with the aim of predicting the Ultra-High Risk (UHR) status in individuals, six months in advance. Our results show that the Mosaic LSM outperforms traditional machine learning models, achieving an outstanding Matthew's Correlation Coefficient of 0.84 and prediction accuracy of 92.4%. Overall, our work highlights the potential of Mosaic LSM as a powerful tool for disease prognosis, and its ability to leverage both the multimodality and temporality of the data to improve performance.

KW - Liquid State Machine

KW - Multimodal Learning

KW - Prognosis

KW - Spiking Neural Networks

KW - Time-Series

KW - Ultra-High Risk

UR - http://www.scopus.com/inward/record.url?scp=85169565203&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85169565203&partnerID=8YFLogxK

U2 - 10.1109/IJCNN54540.2023.10191256

DO - 10.1109/IJCNN54540.2023.10191256

M3 - Conference contribution

AN - SCOPUS:85169565203

T3 - Proceedings of the International Joint Conference on Neural Networks

BT - IJCNN 2023 - International Joint Conference on Neural Networks, Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 18 June 2023 through 23 June 2023

ER -

Budhraja S, Singh B, Doborjeh M, Doborjeh Z, Tan S, Lai E et al. Mosaic LSM: A Liquid State Machine Approach for Multimodal Longitudinal Data Analysis. In IJCNN 2023 - International Joint Conference on Neural Networks, Proceedings. Institute of Electrical and Electronics Engineers Inc. 2023. (Proceedings of the International Joint Conference on Neural Networks). doi: 10.1109/IJCNN54540.2023.10191256

Mosaic LSM: A Liquid State Machine Approach for Multimodal Longitudinal Data Analysis

Abstract

Publication series

Conference

Bibliographical note

ASJC Scopus Subject Areas

Keywords

Access to Document

Other files and links

Fingerprint

Cite this