无线表格的推理cpu 连续跑10次平均将进0.93，不只500ms呢,GPU 1s

[cqray1990]

[Joker1212]

emmm,请问是什么cpu呢，我这边是mac m1测的，gpu的话理论上还是onnx模型本身结构支持不够好，有一个简单的办法是进行量化，或者可以尝试slanet-plus当做无线表格模型哈，是因为cls模型精度不够才没有把它当无线模型用

[Joker1212]

可以参考这个，稍微改一下就行

from pathlib import Path
import onnx
from onnxruntime.quantization import quantize_dynamic, QuantType, quant_pre_process
def quantize_model(root_dir_str, model_dir, pre_fix):

    original_model_path = f"{pre_fix}.onnx"
    quantized_model_path = f"{pre_fix}_quantized.onnx"
    # quantized_model_path = original_model_path
    original_model_path = f"{root_dir_str}/{model_dir}/{original_model_path}"
    quantized_model_path = f"{root_dir_str}/{model_dir}/{quantized_model_path}"
    quant_pre_process(original_model_path, quantized_model_path, auto_merge=True)
    # 进行动态量化
    quantize_dynamic(
        model_input=original_model_path,
        model_output=quantized_model_path,
        weight_type=QuantType.QUInt8
    )

    # 验证量化后的模型
    quantized_model = onnx.load(quantized_model_path)
    onnx.checker.check_model(quantized_mo

[cqray1990]

from pathlib import Path import onnx from onnxruntime.quantization import quantize_dynamic, QuantType, quant_pre_process def quantize_model(root_dir_str, model_dir, pre_fix):

original_model_path = f"{pre_fix}.onnx"
quantized_model_path = f"{pre_fix}_quantized.onnx"
# quantized_model_path = original_model_path
original_model_path = f"{root_dir_str}/{model_dir}/{original_model_path}"
quantized_model_path = f"{root_dir_str}/{model_dir}/{quantized_model_path}"
quant_pre_process(original_model_path, quantized_model_path, auto_merge=True)
# 进行动态量化
quantize_dynamic(
    model_input=quantized_model_path,
    model_output=quantized_model_path,
    weight_type=QuantType.QUInt8
)

# 验证量化后的模型
quantized_model = onnx.load(quantized_model_path)
onnx.checker.check_model(quantized_mo

量化模型肯定影响效果，哈哈

[Joker1212]

理论上影响很小其实，你可以量化了试试你的场景受影响的情况

[cqray1990]

理论上影响很小其实，你可以量化了试试你的场景受影响的情况

合合TextIn的无线表格效果很通用，不知道他们怎么做的

[Joker1212]

好像没有公开他们的实现，只知道他们也是区分了有线和无线场景分别训练的模型，看起来可能是transformer的序列