Python API support?

[protossw512]

Hi, I am wondering if there is a way to run it in python. I tried to build Python wrapper, and it turned out I have to modify some code in order to make tracking arg pass to correct place. However, the Python api seems only returns the result of a single image, instead of performing tracking.

Here is the code I modified in order to make Python working:

diff --git a/python/openpose/openpose_python.cpp b/python/openpose/openpose_python.cpp
index 655ca89b..af8d5616 100644
--- a/python/openpose/openpose_python.cpp
+++ b/python/openpose/openpose_python.cpp
@@ -151,8 +151,11 @@ public:
                 (float)FLAGS_hand_alpha_heatmap, (float)FLAGS_hand_render_threshold};
             opWrapper->configure(wrapperStructHand);
             // Extra functionality configuration (use WrapperStructExtra{} to disable it)
+            const op::WrapperStructTracking wrapperStructTracking{
+                FLAGS_tracking}; // Raaj: Add your flags in here
+            opWrapper->configure(wrapperStructTracking);
             const WrapperStructExtra wrapperStructExtra{
-                FLAGS_3d, FLAGS_3d_min_views, FLAGS_identification, FLAGS_tracking, FLAGS_ik_threads};
+                FLAGS_3d, FLAGS_3d_min_views, FLAGS_identification, FLAGS_ik_threads};
             opWrapper->configure(wrapperStructExtra);
             // Output (comment or use default argument to disable any output)
             const WrapperStructOutput wrapperStructOutput{

Here is my python script:

# From Python
# It requires OpenCV installed for Python
import sys
import cv2
import os
import glob
import numpy as np
from sys import platform
from tqdm import tqdm
import argparse
import pdb

try:
    # Import Openpose (Windows/Ubuntu/OSX)
    dir_path = os.path.dirname(os.path.realpath(__file__))
    try:
        # Windows Import
        if platform == "win32":
            # Change these variables to point to the correct folder (Release/x64 etc.)
            sys.path.append(dir_path + '/../../python/openpose/Release');
            os.environ['PATH']  = os.environ['PATH'] + ';' + dir_path + '/../../x64/Release;' +  dir_path + '/../../bin;'
            import pyopenpose as op
        else:
            # Change these variables to point to the correct folder (Release/x64 etc.)
            sys.path.append('../../python');
            # If you run `make install` (default path is `/usr/local/python` for Ubuntu), you can also access the OpenPose/python module from there. This will install OpenPose and the python library at your desired installation path. Ensure that this is in your python path in order to use it.
            # sys.path.append('/usr/local/python')
            from openpose import pyopenpose as op
    except ImportError as e:
        print('Error: OpenPose library could not be found. Did you enable `BUILD_PYTHON` in CMake and have this Python script in the right folder?')
        raise e

    # Flags
    parser = argparse.ArgumentParser()
    args = parser.parse_known_args()

    # Custom Params (refer to include/openpose/flags.hpp for more parameters)
    params = dict()
    params["model_pose"] = "BODY_21A"
    params["tracking"] = 1
    params["render_pose"] = 1
    params["model_folder"] = "/xxx/openpose_multi/openpose/models/"

    # Add others in path?
    for i in range(0, len(args[1])):
        curr_item = args[1][i]
        if i != len(args[1])-1: next_item = args[1][i+1]
        else: next_item = "1"
        if "--" in curr_item and "--" in next_item:
            key = curr_item.replace('-','')
            if key not in params:  params[key] = "1"
        elif "--" in curr_item and "--" not in next_item:
            key = curr_item.replace('-','')
            if key not in params: params[key] = next_item

    # Construct it from system arguments
    # op.init_argv(args[1])
    # oppython = op.OpenposePython()

    # Starting OpenPose
    opWrapper = op.WrapperPython()
    opWrapper.configure(params)
    opWrapper.start()

    image_root = "/xxx/"
    save_root = "/xxx/"
    if not os.path.exists(save_root):
        os.makedirs(save_root)
    image_paths = glob.glob(image_root + "*.jpg")
    for image_path in tqdm(image_paths):
        # Process Image
        datum = op.Datum()
        imageToProcess = cv2.imread(image_path)
        datum.cvInputData = imageToProcess
        opWrapper.emplaceAndPop([datum])
        image_output = datum.cvOutputData
        pdb.set_trace()
        landmarks = datum.poseKeypoints
        cv2.imwrite(os.path.join(save_root, os.path.basename(image_path)[:-4] + "_pose.png"), image_output)
        np.save(os.path.join(save_root, os.path.basename(image_path)[:-4] + "_pose.npy"), landmarks)

except Exception as e:
    print(e)
    sys.exit(-1)

It seems to me that I have to pass a video into Python wrapper to make tracking work, but I have no idea how. It would be of great help if you could provide some instructions on that. I am expecting to get tracking IDs from landmarks detected.

[BroderickHigby]

Hey @protossw512, did you get the example working using BODY_21A? It seems that you should also make sure that you're not passing --number_people_max 1 anywhere.

[protossw512]

Hi @BroderickHigby, the above example is working on BODY_21A, but the results only return keypoints results, I have no idea how to get tracking information from op.Datum().

[kilianbreathnach]

Hi, I am trying to achieve the same. It seems from the definition in include/openpose/core/datum.hpp, that the tracking can be done using the poseIds array in the Datum struct. However, if we try to access this array from python, the following error occurs:

TypeError: Unable to convert function return value to a Python type! The signature was
	(self: openpose.pyopenpose.Datum) -> op::Array<long long>

It seems this is due to a lack of functionality in the pybind code, as discussed in this issue on the original openpose repo: CMU-Perceptual-Computing-Lab#1162

[sh0w]

hi!

i am also trying to use OpenPose+STAF with the Python API because I am interested in the poseIds... (multi person tracking info)

in my python script, datum.poseKeypoints and datum.poseScores both work fine.

however, datum.poseIds throws
TypeError: Unable to convert function return value to a Python type! The signature was (self: openpose.pyopenpose.Datum) -> op::Array<long long>

Has anyone been able to get Openpose+STAF+Python API running? (@soulslicer)? or should I try @BroderickHigby's fork next?

thank you !!

[yraaj]

I no longer am working on this project. If youre using C++, yes the data is stored in an op:Array<long long> object which unfortunately cannot be cast as only op::Array<float> is castable.

You could implement a custom handle for that type i guess:

https://github.com/soulslicer/openpose/blob/master/python/openpose/openpose_python.cpp#L373

[sh0w]

hi raaj!
thank you so much.
it works now, even with the python api, and is about 20% faster than openpose without STAF :)

[rush9838465]

@sh0w Can you tell me the method you use? Thank you.

[sh0w]

@rush9838465 hi, you can check out my fork: https://github.com/sh0w/openpose (make sure you check out the staf branch). (you might have to download the BODY_21A model manually)
check out the python example 04 to see how to set the tracking params for STAF and get the tracking IDs (poseIDs).

unfortunately – I'm not sure why – the openpose python API with STAF only processes every 2nd image.
a simple hack (but probably not ideal) is to just call this line twice if you use STAF:

opWrapper.emplaceAndPop([datum])
opWrapper.emplaceAndPop([datum])

[rush9838465]

@sh0w Thank you very much!