Welcome to the world of Temporal Distribution Matching (TDM)!
What is TDM?
Temporal Distribution Matching is a method for matching the distributions of the discovered periods to build a time series prediction model. It is used in the AdaRNN architecture, which is a type of recurrent neural network model.
Why use TDM?
The TDM module is designed to learn the common knowledge shared by different periods via matching their distributions. This allows the learned model to generalize well on unseen test data compared with the methods which only rely on local or statistical information.
How does TDM work?
In AdaRNN, Temporal Distribution Matching aims to adaptively match the distributions between the RNN cells of two periods while capturing the temporal dependencies. TDM introduces the importance vector $\mathbf{\alpha}$ to learn the relative importance of hidden states inside the RNN. For each pair of periods, there is an $\mathbf{\alpha}$, and it is used to dynamically reduce the distribution divergence of cross-periods.
Given a period-pair, the loss of temporal distribution matching is formulated as a sum of the distribution importance between the periods at each state.
All the hidden states of the RNN can be easily computed by following the standard RNN computation. The state computation involves the computation of a next hidden state based on a previous state.
The final objective of temporal distribution matching (one RNN layer) is a sum of the distribution distances of all pairwise periods.
Temporal Distribution Matching is a powerful and adaptive method for matching the distributions of the discovered periods to build an effective time series prediction model. By allowing the learned model to generalize well on unseen test data, TDM is a valuable tool for those working in the field of data analysis and machine learning.
