Writing a custom scraper for VRM / GLTF

[fire]

Hi,

I have done work to get a machine learning pipeline to guess the bone name for a 3d model from training sets of 3d skeleton models.

The article has a demo of a scraper. For this purpose, instead of an API, a GitHub repo of binary 3d models (GLB and VRM) can be converted to TSV.

https://github.com/fire/avatar-wellness/blob/main/addons/catboost/catboost.gd

Is there a way to turn a script like "./convert_vrm_to_tsv.sh" and use it with Git History?

References:

1. https://simonwillison.net/2021/Dec/7/git-history/.
2. https://github.com/fire/avatar-wellness/blob/main/addons/catboost/model/train.tsv Concatenated inputs
3. Outputs are the probability that a particular bone is named out of all the possible standard bone names.
4. Has been modified on contact with the problem https://v-sekai.github.io/v-sekai-proposals/log4brains/adr/20211101-rank-skeleton-bone-names-with-catboost
5. Processing https://github.com/fire/avatar-wellness/blob/main/addons/catboost/catboost.gd

Results

These bone names results will be compared against manually graded entries.

## Results.
[leftFoot, J_Bip_L_Foot, -0]
[rightFoot, J_Bip_R_Foot, -0]
[neck, J_Bip_C_Neck, -0]
[leftUpperArm, J_Bip_L_UpperArm, -0]
[rightUpperArm, J_Bip_R_UpperArm, -0]
[leftLowerArm, J_Bip_L_LowerArm, -0]
[hips, J_Bip_C_Hips, -0.1]
[leftUpperLeg, J_Bip_L_UpperLeg, -0.1]
[rightUpperLeg, J_Bip_R_UpperLeg, -0.1]
[leftLowerLeg, J_Bip_L_LowerLeg, -0.1]
[rightLowerLeg, J_Bip_R_LowerLeg, -0.1]
[spine, J_Bip_C_Spine, -0.1]
[chest, J_Bip_C_Chest, -0.1]
[rightLowerArm, J_Bip_R_LowerArm, -0.2]
[leftHand, J_Bip_L_Hand, -0.2]
[rightHand, J_Bip_R_Hand, -0.3]
[head, J_Bip_C_Head, -1]
[leftShoulder, J_Bip_L_Shoulder, -0]
[rightShoulder, J_Bip_R_Shoulder, -0]
[leftToes, J_Bip_L_ToeBase, -0]
[rightToes, J_Bip_R_ToeBase, -0]
[leftThumbDistal, J_Bip_L_Thumb3, -0]
[leftIndexProximal, J_Bip_L_Index1, -0]
[leftIndexIntermediate, J_Bip_L_Index2, -0]
[leftIndexDistal, J_Bip_L_Index3, -0]
[leftMiddleDistal, J_Bip_L_Middle3, -0]
[leftRingProximal, J_Bip_L_Ring1, -0]
[leftRingIntermediate, J_Bip_L_Ring2, -0]
[leftRingDistal, J_Bip_L_Ring3, -0]
[rightThumbProximal, J_Bip_R_Thumb1, -0]
[rightThumbDistal, J_Bip_R_Thumb3, -0]
[rightIndexProximal, J_Bip_R_Index1, -0]
[rightIndexIntermediate, J_Bip_R_Index2, -0]
[rightMiddleProximal, J_Bip_R_Middle1, -0]
[rightMiddleIntermediate, J_Bip_R_Middle2, -0]
[rightMiddleDistal, J_Bip_R_Middle3, -0]
[rightRingProximal, J_Bip_R_Ring1, -0]
[rightRingIntermediate, J_Bip_R_Ring2, -0]
[rightRingDistal, J_Bip_R_Ring3, -0]
[upperChest, J_Bip_C_UpperChest, -0]
[leftThumbProximal, J_Bip_L_Thumb1, -0.1]
[leftThumbIntermediate, J_Bip_L_Thumb2, -0.1]
[leftMiddleProximal, J_Bip_L_Middle1, -0.1]
[leftMiddleIntermediate, J_Bip_L_Middle2, -0.1]
[leftLittleProximal, J_Bip_L_Little1, -0.1]
[leftLittleIntermediate, J_Bip_L_Little2, -0.1]
[leftLittleDistal, J_Bip_L_Little3, -0.1]
[rightIndexDistal, J_Bip_R_Index3, -0.1]
[rightLittleProximal, J_Bip_R_Little1, -0.1]
[rightLittleDistal, J_Bip_R_Little3, -0.1]
[rightThumbIntermediate, J_Bip_R_Thumb2, -0.2]
[rightLittleIntermediate, J_Bip_R_Little2, -0.2]
[leftEye, J_Adj_L_FaceEye, -3.1]
[rightEye, J_Adj_R_FaceEye, -5.1]
[jaw, HairJoint-29ee833a-ea1a-4fcb-8c9b-3533fa46aeea, -7.1]
Returned results 55

[simonw]

git-history is specifically designed for creating databases of multiple changes made to some data over time - I'm not sure if that's quite right for what you're doing here.

Is the problem here that you need to compare the machine learning results against the manually graded entries?

You may find sqlite-utils more useful, as it gives you tools for loading TSV/CSV files directly into a SQLite database: https://sqlite-utils.datasette.io/en/stable/cli.html#inserting-csv-or-tsv-data

[fire]

Thanks for the pointers.