Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Onnx2RKNN Conversion Stucks at UpsamplingHypothesis #250

Open
yildizyakup opened this issue Jan 6, 2025 · 0 comments
Open

Onnx2RKNN Conversion Stucks at UpsamplingHypothesis #250

yildizyakup opened this issue Jan 6, 2025 · 0 comments

Comments

@yildizyakup
Copy link

In the below section, I have given my code to transform my depth estimation onnx model to rknn model but in conversion steps it stucks and get in infinite loop to finish the conversion. I'll provide the output of the conversion process at the below of the code.

from rknn.api import RKNN
import time
import onnx
from onnxsim import simplify

rknn = RKNN(verbose=True)
rknn.config(
    mean_values=[[0] * 2],
    std_values=[[255] * 2],
    optimization_level=1,
    target_platform='rk3588',
    remove_reshape=True,  # Explicitly remove reshape operations
    model_pruning=True,
    single_core_mode=True
)
onnx_model = onnx.load('models/eth3d/saved_model_480x640/model_float32.onnx')

for input_tensor in onnx_model.graph.input:
    input_tensor.type.tensor_type.shape.dim[0].dim_value = 1
model_simplified, check = simplify(onnx_model)
if check:
    onnx.save(model_simplified, "static_model_simplified.onnx")

print('--> Loading model')
ret = rknn.load_onnx(model="//media/yakup/New Volume/ONNX-HITNET-Stereo-Depth-estimation-main/static_model_simplified.onnx")
if ret != 0:
    print('Load model failed!')
    exit(ret)
print('Done')

print('--> Building model with quantization')
start_time = time.time()
ret = rknn.build(do_quantization=False, 
                 dataset="/home/yakup/Downloads/courtyard_dslr_jpg/courtyard/dslr_calibration_jpg/images.txt")
                 
if ret != 0:
    print('Build model failed!')
    exit(ret)
build_time = time.time() - start_time
print(f'Build completed in {build_time:.2f} seconds')

print('\n--> Evaluating runtime performance')
ret = rknn.eval_perf()
if ret != 0:
    print('Performance evaluation failed!')
else:
    print('Performance evaluation completed')

print('\n--> Exporting RKNN model')
ret = rknn.export_rknn('model_float32_480x640_eth3d_finalpass_xl.rknn')
if ret != 0:
    print('Export RKNN model failed!')
    exit(ret)
print('Model exported successfully')

rknn.release()  

D     input_align_4D_add: remove node = ['level4/shared/prop0/WarpImageWithHypotheses/warp_and_aggregate_sad/sub_2add'], add node = ['level4/shared/prop0/WarpImageWithHypotheses/warp_and_aggregate_sad/sub_2add']
D     convert_sub_to_expand_sub: remove node = [], add node = ['level4/shared/prop0/WarpImageWithHypotheses/warp_and_aggregate_sad/Reshape_1_rs_expand']
D     squeeze_to_4d_leakyrelu: remove node = [], add node = ['level4/shared/prop0/WarpImageWithHypotheses/warp_and_aggregate_sad/sub_3_rs', 'level4/shared/prop0/WarpImageWithHypotheses/warp_and_aggregate_sad/Abs_2-rs']
D     convert_sub_to_expand_sub: remove node = [], add node = ['level4/shared/prop0/WarpImageWithHypotheses/warp_and_aggregate_sad/Reshape_1_rs#1_expand']
D     squeeze_to_4d_leakyrelu: remove node = [], add node = ['level4/shared/prop0/WarpImageWithHypotheses/warp_and_aggregate_sad/sub_2_rs', 'level4/shared/prop0/WarpImageWithHypotheses/warp_and_aggregate_sad/Abs_1-rs']
D     convert_sub_to_expand_sub: remove node = [], add node = ['level4/shared/prop0/WarpImageWithHypotheses/warp_and_aggregate_sad/Reshape_1_rs#2_expand']
D     squeeze_to_4d_leakyrelu: remove node = [], add node = ['level4/shared/prop0/WarpImageWithHypotheses/warp_and_aggregate_sad/sub_1_rs', 'level4/shared/prop0/WarpImageWithHypotheses/warp_and_aggregate_sad/Abs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level0/UpsampleHypotheses/add-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level1/UpsampleHypotheses/add-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level2/UpsampleHypotheses/add-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level3/UpsampleHypotheses/add-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level0/UpsampleHypotheses/add-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level1/UpsampleHypotheses/add-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level2/UpsampleHypotheses/add-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level3/UpsampleHypotheses/add-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level0/UpsampleHypotheses/add-rs-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level1/UpsampleHypotheses/add-rs-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level2/UpsampleHypotheses/add-rs-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level3/UpsampleHypotheses/add-rs-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level0/UpsampleHypotheses/add-rs-rs-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level1/UpsampleHypotheses/add-rs-rs-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level2/UpsampleHypotheses/add-rs-rs-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level3/UpsampleHypotheses/add-rs-rs-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level0/UpsampleHypotheses/add-rs-rs-rs-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level1/UpsampleHypotheses/add-rs-rs-rs-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level2/UpsampleHypotheses/add-rs-rs-rs-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level3/UpsampleHypotheses/add-rs-rs-rs-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level0/UpsampleHypotheses/add-rs-rs-rs-rs-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level1/UpsampleHypotheses/add-rs-rs-rs-rs-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level2/UpsampleHypotheses/add-rs-rs-rs-rs-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level3/UpsampleHypotheses/add-rs-rs-rs-rs-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level0/UpsampleHypotheses/add-rs-rs-rs-rs-rs-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level1/UpsampleHypotheses/add-rs-rs-rs-rs-rs-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level2/UpsampleHypotheses/add-rs-rs-rs-rs-rs-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level3/UpsampleHypotheses/add-rs-rs-rs-rs-rs-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level0/UpsampleHypotheses/add-rs-rs-rs-rs-rs-rs-rs-rs-rs-rs']
D     squeeze_to_4d_add: remove node = [], add node = ['level1/UpsampleHypotheses/add-rs-rs-rs-rs-rs-rs-rs-rs-rs-rs']
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant