How does SingleVarianceNetwork participate in network training?

[xuyaojian123]

i have some questions i want to ask you. There is only one parameter of variance in this network. How does it participate in the training of the loss function? Why does the network converge when 1/s approaches 0? Many Thanks to you!

[xuyaojian123]

anyone who can help me answer these questions？/(ㄒoㄒ)/~~

[kzhang2]

The network converges when 1/s approaches 0 because at that point the parameterized sigmoid function becomes a step function. Essentially, the step function should take on the value 0 outside the surface and 1 inside the surface (I might be getting this flipped).

[xuyaojian123]

Thank you very much, but I still don't get it. Registering a parameter 'variance' in SingleVarianceNetwork, self. register_parameter('variance', nn.Parameter(torch.tensor(init_val))). But I don't see anywhere else where the parameter 'variance' is changed and how it participates in the network training.

[weihan1]

the variance parameter participates in the forward pass during the alpha calculation.