Abstract:
"Despite the significant progress made in Deep Learning (DL) in areas such as Natural Language Processing (NLP) and Reinforcement Learning (RL), Time Series (TS) forecasting has yet to witness a comparable breakthrough due to inherent algorithmic limitations. Consequently, systems developed using these methods are constrained in forecasting performance.
Researchers have proposed various approaches to address this challenge, with neural Ordinary Differential Equations (ODEs) and Liquid Time-constant (LTC) networks among the most promising. While neural ODEs introduce the concept of continuous-time and depth models, their performance has been underwhelming compared to traditional neural networks like Long Short-Term Memory (LSTM). On the other hand, LTC networks have shown impressive results, but their outdated architecture of ODEs limits their usability and performance. In this project, the author proposes a novel algorithm that combines the adaptability of LTC networks with Stochastic Differential Equations (SDEs). The architecture achieved state-of-the-art performance in TS forecasting, as demonstrated by the highly popular Bitcoin (BTC) price prediction problem.
The proposed algorithm, called Liquid Time-stochasticity (LTS), combines the adaptability of the LTC network with SDEs. By introducing SDEs, the LTS can handle small noises and immediate changes, resulting in a more stable and robust implementation of continuous-time and depth models. The algorithm uses traditional Backpropagation Through Time (BPTT) to produce accurate results, although it comes at the cost of higher memory usage. Applying LTS on a BTC price forecasting problem yielded a promising result, with a percentage error of 2.5% ± 0.1. Future research could explore using ad joint sensitivities to push the algorithm’s efficiency and scalability."
