Questions tagged [lstm]
A Long Short Term Memory (LSTM) is a neural network architecture that contains recurrent NN blocks that can remember a value for an arbitrary length of time.
805 questions
0 votes
0 answers
34 views
Training process for LSTM based sequence labelling
I'm training an LSTM to predict a binary anomaly sequence from multi-dimensional, irregularly sampled input sequences. While CNNs perform adequately, I'm struggling to get good performance from my ...
4 votes
1 answer
81 views
Weight Gradient Dimensions in LSTM Backpropagation
In an LSTM(regression), the output gate is defined as: $$o_t = \sigma\left(W_o x_t + U_o h_{t-1} + b_o \right),$$ where: $W_o \in \mathbb{R}^{m \times d}$ is the input weight matrix, $U_o \in \mathbb{...
3 votes
2 answers
125 views
Question on RNNs lookback window when unrolling
I will use the answer here as an example: https://stats.stackexchange.com/a/370732/78063 It says "which means that you choose a number of time steps $N$, and unroll your network so that it ...
0 votes
0 answers
36 views
How Is the Final Prediction Derived from the LSTM Hidden State?
I am working on using an LSTM model to predict electricity spot prices. I understand how the LSTM cell works internally, including the role and dimensions of the gates and hidden states. What I’m ...
1 vote
1 answer
59 views
Is it possible to ignore the past values of the response variable in an LSTM model with multiple predictor variables?
I have an LSTM model to predict a variable by considering multiple variables. (Say the target variable is river discharge and the predictors are rainfall, temperature, evapotranspiration etc.) There ...
4 votes
1 answer
217 views
Do i.i.d. assumptions extend to datasets of independently generated sequences in modern sequence models (e.g., RNNs)?
In standard machine learning settings with cross-sectional data, it's common to assume that data points are independently and identically distributed (i.i.d.) from some fixed data-generating process (...
0 votes
0 answers
17 views
Why does my GNN-LSTM model fail to generalize with full training data for a spatiotemporal prediction task? [duplicate]
I'm working on a spatiotemporal prediction problem where I want to forecast a scalar value per spatial node over time. My data spans multiple spatial grid locations with daily observations. Data Setup ...
6 votes
0 answers
168 views
Time series predictions with LSTM
I have collection of TEC data.My data sample for example the day1,day2,day3,day4. Case1: I have the following task to do: Training by the consecutive 3 days to predict the each 4th day. Each day data ...
0 votes
0 answers
33 views
Short and Long terms in LSTM
I have read somewhere (sorry, couldn't find the reference anymore), that $C_t$ (check figure below) represents the "long term" in LSMT. The reason was because "The information goes ...
0 votes
0 answers
132 views
How to properly split train/val sets for time series LSTM prediction with multiple unique items?
I am working on a time series prediction problem using an LSTM model. My dataset consists of 27 different items, each with unique IDs, and roughly the same number of samples per item. There are around ...
1 vote
0 answers
60 views
Neural networks - irregular time shifts of output compared to inputs in given time series data sets
I have some time series data with multiple features. The output is shifted (I mean the times at which I have the output values are shifted from the corresponding inputs and also irregularly). I have ...
0 votes
0 answers
43 views
Time series forecast with dynamic input features
At the period $T$, I want to forecast the target variables $V_{T+1}, ..., V_{T+60}$. My independent variables are $X$ and $f_1, ..., f_{60}$. $f_i$ is actually a forecast of variable $f$ from the $i$ ...
2 votes
1 answer
72 views
Varying sequence lengths between classes in LSTM
I am working on a project where the goal is to predict whether students in an online course will drop out of the course. The course is divided into 20 course weeks. For each week, I have certain kinds ...
0 votes
0 answers
9 views
LSTM Variational auto encoder not learning with Pytorch [duplicate]
I'm trying to train an LSTM Variational Autoencoder, but I cannot figure out why the model is not making any progress, the loss gets stuck immediately. Here is my code and training loop. The sequences ...
0 votes
0 answers
78 views
LSTM Limitations for Time Series Forecasting
I am working on a project where I generate synthetic data which is the sum of 5 random sine functions sampled every 0.01s (and I add mean reverting brownian motion noise to the data). ...