Off-campus SJSU users: To download campus access theses, please use the following link to log into our proxy server with your SJSU library user name and PIN.
Publication Date
Fall 2024
Degree Type
Thesis - Campus Access Only
Degree Name
Master of Science (MS)
Department
Computer Engineering
Advisor
Stas Tiomkin; Gautam Kumar; Magdalini Eirinaki
Abstract
The study of causality, i.e., the notion that A is caused by B or A influences B has evolved through the years. In information theory terms, it is expressed in terms of Directed Information (DI), as introduced by Massey. DI is the measure of information flow between two causal processes with feedback. This metric is asymmetric, meaning A influencing B is not the same as B influencing A. In real world scenarios, the calculation of this value is intractable as it requires knowledge of the underlying probability distribution of the processes, which is very rarely known. We only observe samples from the processes. There have been many algorithms that aim to calculate it from the samples, but they are not accurate. Only recently has it been shown that this can be estimated using neural networks with aptly named algorithm Directed Information Neural Estimation (DINE). The DINE algorithm has been studied to maximize channel capacity for channels with feedback. It uses LSTM to estimate DI. LSTMs are limited in their ability to capture long term dependencies, and they are slow since the processing is done sequentially. In this work, we propose a transformer based architecture for DINE which addresses this limitation.
Recommended Citation
Shrestha, Enosh, "Extension of Directed Information Neural Estimation With Transformers" (2024). Master's Theses. 5608.
DOI: https://doi.org/10.31979/etd.87yq-dbrf
https://scholarworks.sjsu.edu/etd_theses/5608
