Can the loglikelhood cost go negative ?

[malzantot]

Hi,

I have modified the code to handle another type of data where the output gaussians are uni-variate. However, I noticed while training the model that the log likelihood cost goes negative and keeps decreasing ? I wonder if this negative likelihood is sensible , as while your training your model too sometimes I notice negative cost values reported for one or two batches ? Or it means that I have potentially made some bugs while modifying the code.

Thanks,
Moustafa

[hardmaru]

Hi Yes the log loss can go negative Does the model generate sensible handwriting after you have trained it? If you read the paper ok VRNN (Chung etc al 2015), one of their baseline methods is a univariate Gaussian.

…

On Wed, Dec 21, 2016 at 12:59 PM Moustafa Alzantot ***@***.***> wrote: Hi, I have modified the code to handle another type of data where the output gaussians are uni-variate. However, I noticed while training the model that the log likelihood cost goes negative and keeps decreasing ? I wonder if this negative likelihood is sensible , as while your training your model too sometimes I notice negative cost values reported for one or two batches ? Or it means that I have potentially made some bugs while modifying the code. Thanks, Moustafa — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#12>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGBoHpa7kXreUvseKDbTPQt6QVzFW5gmks5rKLITgaJpZM4LSk6_> .

[malzantot]

Hi,

Thanks a lot for your reply. I will have a look at the paper.
Your code output looks OK, but my experiments with other data and the univariate gaussian does not produce reasonable output yet.

I was wondering what is the significance of negative log likelihood. ? Since the likelihood is a multiplication of probability values ( 0 <= p(x) <= 1' ), I expected (-infty <= log p(x) <= 0') and therefore, (0 <= log p(x) <= infty) .

[soi]

@malzantot was wondering about this as well, please take a look at this to clarify: https://stats.stackexchange.com/questions/4220/can-a-probability-distribution-value-exceeding-1-be-ok

[malzantot]

@soi I found it later as well, it is because this is a PDF (probability density function) which can have a value > 1, unlike the PMF (probability mass function) which is always in [0, 1].