Python action. Py prompts for error

[gzchenjiajun]

tensorflow.python.framework.errors_impl.InvalidArgumentError: No OpKernel was registered to support Op 'CudnnRNN' used by node cu_dnnlstm/CudnnRNN (defined at action.py:28) with these attrs: [seed=0, dropout=0, input_mode="linear_input", T=DT_FLOAT, direction="unidirectional", rnn_mode="lstm", seed2=0, is_training=true]
Registered devices: [CPU]
Registered kernels:

     [[cu_dnnlstm/CudnnRNN]]

[gzchenjiajun]

Device: MacBook Pro 2016

[imsoo]

@gzchenjiajun,

CudnnLSTM is a GPU-only operation. (it depends on CuDNN)
So we need to transform CuDNNLSTM to LSTM. (CuDNNLSTM/LSTM weights are interchangeable)

Delete action.py Line 26 and insert below code.
(Creating a standard model with the same architecture and load weights.)

fight_detection/server/action.py

Line 26 in 4eb4930

model = keras.models.load_model("weights/action.h5")

from tensorflow.keras import layers
lambda_loss_amount = 0.0015
model = tf.keras.Sequential([
   # relu activation
   layers.Dense(n_hidden, activation='relu',
       kernel_initializer='random_normal',
       bias_initializer='random_normal',
       batch_input_shape=(batch_size, n_steps, n_input)
   ),

   # cuDNN
   # layers.CuDNNLSTM(n_hidden, return_sequences=True,  unit_forget_bias=1.0),
   # layers.CuDNNLSTM(n_hidden,  unit_forget_bias=1.0),

   layers.LSTM(n_hidden, return_sequences=True,  unit_forget_bias=1.0),
   layers.LSTM(n_hidden,  unit_forget_bias=1.0),

   layers.Dense(n_classes, kernel_initializer='random_normal',
       bias_initializer='random_normal',
       kernel_regularizer=tf.keras.regularizers.l2(lambda_loss_amount),
       bias_regularizer=tf.keras.regularizers.l2(lambda_loss_amount),
       activation='softmax'
   )
])
model.load_weights("weights/action.h5")

[gzchenjiajun]

This repo is an LSTM, I want to use it as real-time streaming video detection (RTSP), is it feasible?
Any Suggestions for modification?

@imsoo

[imsoo]

@gzchenjiajun

In this project LSTM is expensive but Pose Estimation (CNN) is more expensive. It spend most of time (About 95 percent)

So if your machine can process pose estimation in real time, I think it may feasible.
You can check that pose estimation works fine in this repository.

In my case, I tested it in the following environment. (Note : Only one client.)
Client <------ Web Cam Stream ----> GCP Server (Nvidia K80)

I think that following case will need more resource.

• Add more client.
• Improve pose estimation output. (change input image size 200, 200 -> 400, 400)
• Improve action recognition model. (add layer ...)

[gzchenjiajun]

If I want to realize real-time image detection of RTSP stream, do you have any Suggestions?

[gzchenjiajun]

If I want to realize real-time image detection of RTSP stream, do you have any Suggestions? @imsoo

[imsoo]

@gzchenjiajun

Sorry for my late reply.

I've never used RTSP. I regret to say that I am unable to help you.

But I read about RTSP and then I'll let you know if good idea comes of it.

[gzchenjiajun]

Ok, thank you very much!
It appears that this repo does not currently support RTSP live streaming.
So if I want to predict the sequence of images in a folder, is that ok?

@imsoo

[imsoo]

@gzchenjiajun

Unfortunately This repo doesn't support sequence of images.
Only support video (MP4, AVI ...) or web cam stream.
You need to convert an image sequence into a video before you start.

[gzchenjiajun]

Ok, I will try it with the video, and I will ask you if I have any questions. Thank you
@imsoo

[gzchenjiajun]

I would like to ask if there is any recommended that can realize real-time fight detection?
@imsoo

[imsoo]

@gzchenjiajun

Sorry I'm not sure what's meant by "warehouse".

I think that "warehouse" means repository or dataset. Is that right? if so, Below dataset looks good.

And in case of repository, I don't know actually which one is the best. I think it all depends on your purpose.

• Dataset

https://www.kaggle.com/mohamedmustafa/real-life-violence-situations-dataset
https://github.com/mchengny/RWF2000-Video-Database-for-Violence-Detection

[gzchenjiajun]

@gzchenjiajun

Unfortunately This repo doesn't support sequence of images.
Only support video (MP4, AVI ...) or web cam stream.
You need to convert an image sequence into a video before you start.

What is the format of msg data? #17
I want to convert it into msg format based on the stream camera. How should I do it?