Implementation questions

[gofysuil]

Hi Joseph,

Thanks for posting this notebook about WaveNet implementation in the context of pure time-series forecast. It is very helpful for me to understand the model. However I have the following questions about your implementation in the notebook:

1. Why is there no activation function applied to the Conv1D layers? It looks like all non-linearity of the model comes from the last fully connected layer.
2. Should the dilation_rates be (2, 4, 8, ...) according to the diagram?
3. Since only last 14 output values are used in the calculation of loss, can we truncate the input sequence (encoding_interval) to the width of the receptive field (128) without affecting training?

Thanks!

[gofysuil]

To answer 1: In full implementation ("TS_Seq2Seq_Conv_Full") gated activations are included in the dilated Conv1D layers.

[gofysuil]

To answer 2: I misunderstood dilation_rate in ConvNets and confused it with strides. I found a good explanation of this topic here: https://theblog.github.io/post/convolution-in-autoregressive-neural-networks/