Client ERROR #7
Comments
|
I change main.cpp as you described in this
|
|
I think Try below changes.
/*
opencv/modules/highgui/src/cap_gstreamer.cpp#L462-L479
*/
void CvVideoWriter_GStreamer::init()
{
encs[CV_FOURCC('D','R','A','C')]=(char*)"diracenc";
encs[CV_FOURCC('H','F','Y','U')]=(char*)"ffenc_huffyuv";
encs[CV_FOURCC('J','P','E','G')]=(char*)"jpegenc";
encs[CV_FOURCC('M','J','P','G')]=(char*)"jpegenc";
encs[CV_FOURCC('M','P','1','V')]=(char*)"mpeg2enc";
encs[CV_FOURCC('M','P','2','V')]=(char*)"mpeg2enc";
encs[CV_FOURCC('T','H','E','O')]=(char*)"theoraenc";
encs[CV_FOURCC('V','P','8','0')]=(char*)"vp8enc";
encs[CV_FOURCC('H','2','6','4')]=(char*)"x264enc";
encs[CV_FOURCC('X','2','6','4')]=(char*)"x264enc";
encs[CV_FOURCC('X','V','I','D')]=(char*)"xvidenc";
encs[CV_FOURCC('F','F','Y','U')]=(char*)"y4menc";
//encs[CV_FOURCC('H','F','Y','U')]=(char*)"y4menc";
pipeline=0;
buffer=0;
} |
|
@imsoo thanks for giving respond I have open four terminals and run: sink work like this: ventilator: worker : darknet_client, keep running all the time :
the output video and json is empty : what should I do to fix it? |
|
In this repo, we need to make below pipeline. So, you need to run ./sink
./ventilator
./worker cfg/openpose.cfg weight/openpose.weight -gpu 0 -pose
./action.py
./darknet_client ...
If you want to run this(fight detection based on YOLO object detection), try below command using darknet_server not this repo. ./sink
./ventilator
./worker cfg/fight.cfg weights/fight.weights names/fight.names -gpu 0 -thresh 0.2
./darknet_client ...If this is confusing, let me know and I will try to explain. |
|
@imsoo thanks for your reply! I want to run this: There are not "openpose.weight" in weight file , only action.h5 |
|
I see |
|
when I run my keras version is 2.2.4 what should I do？ thx |
|
@imsoo I am really interested in this repo |
|
Sorry for my late reply. I was late finding the cause of the problem. First, I've tested the same environment on my computer and got a same error. import tensorflow as tf
print(tf.__version__)
>>> 1.9.0
from tensorflow import keras
print(keras.__version__)
>>> 2.1.6-tfI've found this issue. import tensorflow as tf
print(tf.__version__)
>>> 2.1.0
from tensorflow import keras
print(keras.__version__)
>>> 2.2.4-tfPlease upgrade tensorflow and try again.
|
|
@imsoo TensorFlow 2.1.0 seems to be incompatible with my system environment: Cuda 9.0. I worried about the global environment if I change to Cuda 10.1 Is there any easier way to fix it? |
|
I understand it well. Could you try changed code below? This code create a standard model with the same architecture and load weights files. Changed code (action.py)import tensorflow as tf
import numpy as np
import zmq
import io
import time
from tensorflow import keras
from tensorflow.keras.backend import set_session
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
sess = tf.Session(config=config)
set_session(sess)
sess.run(tf.global_variables_initializer())
'''
## TF 2.0
from tensorflow.compat.v1.keras.backend import set_session
tf.compat.v1.disable_eager_execution()
config = tf.compat.v1.ConfigProto()
config.gpu_options.allow_growth = True
sess = tf.compat.v1.Session(config=config)
set_session(sess)
sess.run(tf.compat.v1.global_variables_initializer())
'''
n_input = 24 # num input parameters per timestep
n_steps = 32
n_hidden = 34 # Hidden layer num of features
n_classes = 4
batch_size = 1024
from tensorflow.keras import layers
lambda_loss_amount = 0.0015
model = tf.keras.Sequential([
# relu activation
layers.Dense(n_hidden, activation='relu',
kernel_initializer='random_normal',
bias_initializer='random_normal',
batch_input_shape=(batch_size, n_steps, n_input)
),
# cuDNN
layers.CuDNNLSTM(n_hidden, return_sequences=True, unit_forget_bias=1.0),
layers.CuDNNLSTM(n_hidden, unit_forget_bias=1.0),
# layers.LSTM(n_hidden, return_sequences=True, unit_forget_bias=1.0),
# layers.LSTM(n_hidden, unit_forget_bias=1.0),
layers.Dense(n_classes, kernel_initializer='random_normal',
bias_initializer='random_normal',
kernel_regularizer=tf.keras.regularizers.l2(lambda_loss_amount),
bias_regularizer=tf.keras.regularizers.l2(lambda_loss_amount),
activation='softmax'
)
])
model.load_weights("weights/action.h5")
def load_X(msg):
buf = io.StringIO(msg)
X_ = np.array(
[elem for elem in [
row.split(',') for row in buf
]],
dtype=np.float32
)
blocks = int(len(X_) / 32)
X_ = np.array(np.split(X_,blocks))
return X_
# load
input_ = np.zeros((batch_size, n_steps, n_input), dtype=np.float32)
print("model loaded ...")
context = zmq.Context()
socket = context.socket(zmq.REP)
socket.bind("ipc://action")
while True:
msg = socket.recv()
msg = msg.decode("utf-8")
recv_ = load_X(msg)
for i in range(len(recv_)):
input_[i] = recv_[i]
startTime = time.time()
pred = model.predict_classes(input_, batch_size = batch_size)
endTime = time.time() - startTime
print("time : ", endTime)
pred_str = ""
for i in range(len(recv_)):
pred_str += str(pred[i])
print("result : ", pred_str)
socket.send_string(pred_str) |
|
@imsoo It works ！ thanks！ But It seems only got one result, then broken: ./ventilator and ./worker seems to work fine ./sink broken All output Error:nd@nd-All-Series:~/fight_detection/server$ ./sink |
|
Could you change fight_detection/server/src/frame.hpp Line 16 in f6a435c |
|
@imsoo Sink doesn't break now after rebuild all But:
|
|
Sorry, a client needs change too. |
|
@imsoo |
👍
Did you set GPU option when you build a darknet library? # AlexeyAB/darknet/Makefile
GPU=1
CUDNN=1
CUDNN_HALF=0
OPENCV=1
AVX=0
OPENMP=1
LIBSO=1
ZED_CAMERA=0 # ZED SDK 3.0 and above
ZED_CAMERA_v2_8=0 # ZED SDK 2.X |
|
@imsoo |
|
@imsoo I check the video output (ignore the wrong pixel, which caused there are not "kick" detected, anything wrong? |
|
Sorry, action recognition performance is not good especially kick action. I think it needs more train. (with dataset and add more feature vector) But try below thing to improve result.
fight_detection/server/cfg/openpose.cfg Lines 2 to 3 in f6a435c
fight_detection/server/action.py Line 26 in f6a435c |
|
thx a lot🤝 I will try it later |
|
Everything works fine when I run But I get empty output_video file_size when I want to continue to detect another video (by simply run It seems to disconnect between client and action when I finish a video? thx! |
|
Oh sorry, I forgot to tell you. After one video is over, we need to re-run sink. The sink retains the track_frame to send frames of the video to the client in order. fight_detection/server/src/sink.cpp Line 140 in f6a435c So after one video is over, track_frame be a total number of frames. But sink doesn't know if it's the end. This causes a problem. If the client starts sending frames of another video, sink think that this frame is not the right order. fight_detection/server/src/sink.cpp Lines 159 to 169 in f6a435c To solve this problem, I think we need to add something to mark the end of the video. This is my TODO but I haven't done it yet. |
|
@imsoo thx anyway By the way，what is the meaning of the number of bounding boxes?
|
|
It's an ID given as a result of object(person) tracking.
|
|
hi @imsoo ，I notice that fight detection is base on:
I have some questions:
I would be appreciated if you help me out of these |
In Sink process, We run this pipeline.
Tracker receive bounding boxes (is produced which is a result of pose estimation) and produces object unique IDs. Plus, Tracker manage time series data about each person. So we can get unique data (joint time series data) about each person. This data is used to recognize the action of each person. For instance, (not the actual outcome.)
Yes, It's when a tracker misses them or they disappear. |
|
@imsoo
how Crash detention work？ （maybe bounding box IOU？） |
|
Compare the position of the wrist and elbow with the other person's bounding box. fight_detection/server/src/people.cpp Lines 147 to 175 in f6a435c In Sink process, The comparison is a simply nested for loop. For instance,
ID1 : Walking (Do not check) fight_detection/server/src/sink.cpp Lines 268 to 284 in f6a435c |
Server and client make successfully
But Something gets wrong when I run ./darknet_client:
Is that OpenCV version problem? any help would be appreciated
The text was updated successfully, but these errors were encountered: