Skip to content

Commit

Permalink
Added FIL model GRPC protocol test on triton for Kserve (red-hat-data…
Browse files Browse the repository at this point in the history 
…-services#2028)
  • Loading branch information
@rpancham
rpancham authored Dec 4, 2024
1 parent 4c346dd commit d273f6f
Show file tree
Hide file tree
Showing 4 changed files with 52 additions and 0 deletions.
1 change: 1 addition & 0 deletions ods_ci/tests/Resources/Files/triton/kserve-triton-fil-grpc-input.json
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"id":"test1","model_name":"fil","model_version":"1","inputs":[{"name":"input__0","datatype":"FP32","shape":[1,393]}],"raw_input_contents":["/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAkGBwgHBgkIBwgKCgkLDRYPDQwMDRsUFRAWIB0iIiAdHx8kKDQsJCYxJx8fLT0tMTU3Ojo6Iys/RD84QzQ5OjcBCgoKDQwNGg8PGjclHyU3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3N//AABEIAJQBBQMBIgACEQEDEQH/xAAcAAABBQEBAQAAAAAAAAAAAAAEAAIDBQYBBwj/xABHEAACAQMDAQYCBwYDBQYHAAABAgMABBEFEiExBhMiQVFhcYEHFDKRobHBFSNSYtHwQoLhFjNyktIkU2OisvElRHODk8LD/8QAGgEAAgMBAQAAAAAAAAAAAAAAAAIBAwQFBv/EAC0RAAIBBAECBQMDBQAAAAAAAAABAgMEERIhMUEFExQiUTJSYXGBoUJDU2KR/9oADAMBAAIRAxEAPwCoke1Vf98YI3yXbgs3n0+FWWm6c0DIMy7ZFBTvz9jP/pHTgVValfWFtLBGxFz3YP1eUDJHPRgego0X/ddnomR0ViRGFJGVyevxA9eleblGWqS7m+L9z+R19LLNMPrdz3duyHmIBi2c/fzUAmtHuxDDqAULgxo52F24wDnoPP1qr1bUY7qCCz0t3kC58ssxHv8AAZozSdPtL5kvbi37oQDDoW4JAJ6efkflTaKMcyKmtp4iFz93HiMRLM58TBc/azgrjz61W3d1ZnUGjaBG75QsuwnKYBwOPh+FFapqEVmqLbSxySPLnbGB+7B8hjpxnzqru7OdLk3dscpt3yYOWiBxjcfImnox7s1erzHQoobUy3778qmSu9uVB9P79Pauww6jJcPHaKzRxgsPDjgefqM1K15MtyJI2AK4Ozn5/Hz60dPqUt6ZRZpBFEQVk3eAsvqT+nXit+ZZMz8pxfPJS3Mx/wB5CIlDgEAkZOOtBOSdzSRnvATy3THpR0zd2ixWrRN3TE97tPiHp8PlR+n6Fq+oRtdTbUtzkhjnxH0C1btGC5ZnUXJ4QJbX6fs5FnijIj2BRt4J9cUrqcSMbmW87wupcAHDD0J6AfAVYWXZ+5F01rcgQKh3MxIGQPMZoKGwW61iOJXM0cs6qDkbmXjnA48vKoUoSfBMoTissvNNnlgsri0neXvUA3t3e4x556mhrDWBFDNHPuGVxuV8Bc9cf0qe/uFvtQ+r2sLfV2uP3gY7Q2TnA+VVkOlXEdx3yqJUL4DcENn4/EVVrF8SFy0Mkit5FkbvjIkSMu1QBjBO3J6H40XPLdQWsd5HK8vG1DMctn1HmRxQ1horsE/ertZxn1BBxzn/AFou/s4LeUW1tveU4HdId2V88j1pm1lLJKfcrbjVZ47XYYo/EOHC8nnPP44+NI9oLy4Qxd5gsQqRkZVBkHIPy+4mhmtn2bnhMa52k+VQiJohvJCsSVBLdKt0g+weZNLGS1/aaXJht75Wfa37x9+MD2Ht/X1rS6dHYwwB7ZmkR12gt1I9/uPFYmytlkvBDPwJfDuY+Z6H4f1rQ6dbRRQqFzMrHaHzgE+3mcVRXjHXVBOrKo9p9S6R7hwFhCrkjvXKrnAz4QPOooNSM7CO3iBjVyhJHI8+R5dKdGVs7V1tQ0x8W5ghDAn/AA0O0a9xHGxkhR33s4IPJHI9xgDmsesQycvLhIpAl1Kqyc7hGclfIEGhNS+oyTo090Y3KknC7m3eefbHn7UobcWtybw3YmVFwhZftHyHyoLUtQDW0i94huEz0G0gf4twPn/WtEIYa1Eb+QiO2srWMPIRLIx4zlc88cGq/WhPDOJZ4UlimkGSshKsOmPanHUBFbRRp3ThYlIJA59eatNPWO8Ek3dQMhl4O0Dd5ZIzx581bzB7SFI7e7keNZltGDFdqsZGxn4Hrj0rl5LcCJxPNEiSqCQ2AAfNR99dm01YX76Ca5CIcFFJYA56Lnp8KHazd3NzcN9YkbJhiiXIz0yccDnmkWreUS2GQNBYS/VEEyOgCudpYtkDNRWkVrFP9fiOFZjlJBk5x6eXNRWdtetatLdKrxyE8Sycgex8z04ofUoBCjKxdZYxlcNxk+Q/XNNrlvnqQWl5cm2g79J40OOYQACwJ9uP1qvu5YtSYuSsbtkhVXqccZqutruSKKMNAjwxkjDL4ckY59ahDXTTO0ZznjCjA58hVkKKiGSK9WWOYoyAMOueaVSiHHNxGwJ6DgkfGuVoRBp9Qs559RUNb7JJJMrHjIUHzOfvofXbmN3MCBTHBgCSNcbgOMZ8+K0NxcWBtZbkTd0UJ5A3nOM5HPTPnVei6SIpBHcW02TvbcSMDHQ8cf1rj05t8tdDTNrlIG0eG0M8LR3AedsuYgAR14znrjrjjpV21jDBCYomdpH3Mr8Hp6epJrKSz2SyyRwwSCAMY03EMcH16dTn8K0j69FA8cALRwpCStwsR3eIABQpx7+dTXhNtYFhJcg8sFvDcd5LvMrwurIw56cMPj61U6ndfVdQigt9vcqgDgEMD8fL+lR3l495ks02UXCuVAAB4AoIwyicps8WCXUYJA65NXwpNcyEbXYbJLK27bGoIX0xwD5VDJseRVjj2Iybjn2AqytbdDDM2HkmIwqjp8/brzUSW4hSITjaE3DcByM4yD8wMfOroyRGSuljkVFco2zPhJGK0d12uf8AZsUdrLi4ZF77wkBMEnw56+h+dU2pXDIghjbMS4YqTnOfSgpGjUIz52k8gH8Kd0o1EtkNTqyp5wXF1rM+pxKgHdsgIJRiCwPkfaoNNvXtBJJEoBTqccZoG1dF1RXUYjLhSOoANH6nC6zRRp4VLFgB0Pzqzy4x4SIlJz5YXo87F3fjxtu4XOCOPuxREebpTHHHM8mfCpQ59AdvsKbZQGLFvaMZJyweYnhIePXzPtVskz242QuxOMMw6k+9bLTw513tPhGerX16A9noN33YiuI4hEvKd9MAy568Lmpoez+Ls3F1fxuxJ4SEnr08+vvUM9zMAN0kn/Nj8qDNw7nBZzn1c11Y+G2keWm/3M7uKj4LSPs5pkeVMl5IG6r0APqKlTQNDRs/s+4kbzLStzVNeQzwQJJNC6pJ9lznBoOGJ7hsRxGQ+gGatjSto/TTQrc5dZGtSx0iPATRAdvILBic0QDaomxNIXbnpsasTMFibY8AVh1DDFRd4o6RqKfFv/jRGJ/czeiS2QNt0gKG64RuaiMth/j0lRx6MPzrDidlPgLKf5WIp4v7lPs3NynwmYfrS4tu9JE6z+41NxBoEqHvtPkUZz4ZmXpQ7aVoty7GKG4UMfGWlz6efWqSC8upWAluZZF54Zia0WiIksUhcZVDjHqcc/oPlUSp2rpuSppE+9P6iN9G0URGCO9MTMuNrqrc/nUVr2d+qWChJBenPPdsQOvUrVzJbxSoydzHjHHgHWhDaWjhWji7l8fahJQ/hXPq2VOosJY/QsjV+WVcAuUkK4cYkwuDkIvuenr70r147wgtIoljGMM2FfgnGKNvI7pVzJvu4843IAJlH5N+BpC2t77TpHt0aZI2Q95jxDyOVPIxnoa5FxZzoPMuhpp1FNcFRZC6nsluEaJQ48JDHgt0YqPPOOarZ7eSbbdCO4XbgHKYVecZyPetrZaMkU31ePfBFGwXAPLkHGc+9LtJZCK3l+ryMhIwApyOevHn1qmM8SLdXg84lRnBMasY1O7ax6epxRNlFLOyRCUxAZ2k5Kn++a0kHYp0AluLoRKQU2hTvbIPv51Yx9kVtrcrdTqwA5QHqePL0pncU84TF8tmM1XRXsZEe6u4074EjajEHHXypV6LNpKywwKCgVEwoIB/Ou1KqrA3lMxd6bK6mWKzdxK+S0rnqPTj1x+NCT2LWpcJI7ITtJX/ABexHxoO5llhkEcYDGIghwuOcUP9YnlnV5AeGDEdM0RpNLGeBOQ0m3SKNAoWWMne5B+XHpROta/DdwMqQKHOE7wccA+f3fdQEzFgZZAAZCSR6UBKm5sjG0nofPmrPLjJ5YFtNHcvFGuwlHwGMZzwOlWOh2feXIbunExyiHIRfnnJOaK0LTxf22Zi8cOcLHGxHTqM/cPnWhstJ0+2m79ImMkYJUNIxAJ8uf74rLUqrDiPCm5LJWW6QQRXMzKO8UbXVJTjZnpn168VX61La5eGA4XChvHuyfLGPjVxc2NrFaO5wJA+47pDgt64+dZe9i7yF3VSG3Lnb64OKrpUsvbIzg0gW4MgtwjxhYlbxsqjOeMflVZcLhtkbbgOTx0NaCw01b+7WCJSUA3StgnH9/rV9f8AZqzstLuLuKAOYk3BWJJJ+Fdq1s51ouaaSXyZJ1YwkovqYu0083FozqH7/dheRjPlmtHqHdXd4ltpzFmjUGa4I8MZxzj3qot43acwW4C3Djxt5Qr+prR2lh3Fkwt0Ihj5eQ9Xbz+daLa02ltPohalXRcCgjjt4vq9sNqLySTkk+pPrXEmUHATcSODTFnflYlBz54zXO6cj94VjB659PurqyfGq6GTGeWNuWxbNO2PGdqey1XspCA560bqcMkjKY+7aBRj922cfKgmA7sru5688YqtzxwOkFXGsXVxafVZu7aIAAZXkY6YoWzupYJswkBsUMxI44++lFIInEnU44pZPjgdYO3k8s9w0k+O8PB4qDNOmkMjlj1NQ5pc/IBVqytKqMAuc+KppYD3LybugyMetAKdrK38Joq7nV4lCMd2eQOKqlnOUOkPsfCrueg/v+lafs+caXA/BMjMx+8/0rMHMOnGQjG4ZFaLsTC93pUEO8rLuOOOinJz/frWmUWoxg+/JXJ8NlzuIfIIx1FAudlzIn+EnK/A1Zy6U8J/dyzM3XBAP6VT3ffx3ypPGwG3CvtI6etXuhrHbJnhPZ4DFOfCc4PpUU9pLIfrOnukN8h5zwkuPJh+vlU0BDjGMH3qZo2ikUjaQ/Bz5HyNZqjhNay5LYqUXshul67FOrq9sILmKRRNEy8r6nrzjqMUQboz3Uk8cSgKccrg4H95qsjto7rVZp27wJ3Qj3KcHIOdw9xR53WiK2FYAAGVRz88+uK4F34fOkt4L2nVoXEZ+2Q251RnErLhWjBxgYYnHBP+lS297M8ju7Hbt+1jHp6/05odpIm2F4SJGfgEjkgH8eaj2SKJFjd173ALS+Irweh6Vyvama8rbAW15tC/9m3jbgHO2lUEUMQRBcyojbRzI2c/IDilTjYPOIG7xXaVixPPB8yafLaD6oZ9x4kKhfYedW8nZ2S3uEDTK6xsCUSNiWAPTPSrCDsxqF40adyqRGQsTJxhScnzrQ68OzMS5Zm9TtZLa2tjKpUyIGA9jVYiu86pGMknCrjOTXqnars7dazHbQ2jwQpCOsnX0FUkH0fOg3XWoRhh02KTR6qko5bFaww7TIDb2cEa7sKPFnhufw/9qKhZiuxG3Nk5JP8AfvVhHY29rbLEs7hwAM7OtQy2MpjDW8cruMjc4Crz86wQjKtLFNNsZTUOrM9rd4VmiiCKFBJPPXp1+6qdArQ3RZ9hYr3akHnrn7vX3rZQ6dbwXTXdyBPcYwCfsp8B6+9GteRP9uOI4/lr1Fr4HUlTTm8fgyVL1KXCPP7D6zbO227kiTzMOQauUfvbR45dVum7wE/bBJ9ARj/WtQJ4XU74IceYKConvYYeIo4l/wCFQK7NtYzorVcmapcRnyYvTtL1KE/9mtXkRyC5Knc3rzitL3Wqy2Udq1m8cSLjBKA/fmppdWc8BqGbUnP+KtMLJ9GVTrKXYYujXSLtEaqP/qCmto9x6p85BTWv3PnmuLPO/IOBWj0qK92Rtok6tuUxg+oeuR6XdQ52PCM9fsmiVH/ezlf+GpUktE5YPIf5modvD4I3kCfUrwfZmgJ9Cq0HNo7OxeaSIE9fGBVrLfWoBHdL8MmhHv4M/uLKIn1ZRR6aL/pJU5FPLp9tGeZFJ/lYmoTZwk4VHNXDzXM3GIo19FTFQskKZM833UO1prqkOpyBLSDThMq3lsBDnxvukZh8FVhmn6mNHcxjS9Pu1C53tMxw3TGBuOB1qWKWCR9ttA8p9dp/v8KfLFedQiRqP/DLH8ax1KVCMtl1LYuTKTVpLhrY95D3cQ4VB/WuaPPrUsQjsrk2yRqAH24OPQECjr6WeHYTGzsxwrzEHH+UcVYaBcPPas98FTa+F8O3IrHWgqkk8l0XhAqWGuSZL65cjP8ABnn8f0oi17P3IlWa4v7uYrz+8ckH8au0uII1zHgj41DcaykUbbWTdjwgDJqt04Lqv5DL7BEcWAB3hGPID9aKSOLjvMn4nNZpe0LmLLKgOT9o4rg1DUrriCNlT+JhtA+ZpnXpR4issXy5vqasXNlC3dYAYDdg8cUpNV0+HIlniUHjrmsrHpbXLiW7vy7DjbCcgD51YwafpttDLMbZCI1LMXOelMqs5LKWP1FcYp8ia/Q99PbThsHZCJVGDk+vlxXY7x49sNw0Uc+SCgXICjrj2pnYuxivUnNwvgXaR8c5+7itEez1i3Cr3ZByCMgqfXNeJrzhGq0/k68Wl0M/GoieTvFLljuAAPApVbnQSvD6gw5OCyBvyx/ZpUvnQG8xF2sCZ3YUv7jpSkUqMbhUQRwQQx3evrTu6ZhlmJB9q56UiraQ0Mueu4ipIomklUFV3NwPOuSRAjCeE4xnFM00l+0EVnuMmIixQetaLWh51VRfQWTaINUn/ZlxsuPAH4RwCQ/tn19qrbnVlKlYlLMf4uMVeaXfQyXE+n39jcyZZg0NxDuE2ATgEgen41RdstPbR+z0+oNp9zZy4wiFgyls85zkgfOvd2dzaW0NFHhfBgqUqk2VT3Dv9osffNRm4IrDyavqgRZGm2o+4r1wcdfP3rn7b1NM4ugcE9fYD+tb141Q+x/wV+jn8m5+sN6n50TbWN1d2ct7GFFvExV2ZwBnGcfcRWLtO02owyATJbzqGwQw6+fxrddou0mkaZoOnTafp6yT6kneBGYhVxjOfvqZeL05RzBYK3bTTwVLE5Iz0po5zkg1npu1t+wLR21ogAzgpnjOPam/7T6nnlLIYbb0b/qpn43QXGGOrSZpVI8iM/CkWf8Ai/Os5F2q1FWEjQWLDZuwyE8Zx5k81KO1d0qtL9UtCccgp4emeNu34VK8bt2+UwdpUxwXnjPRqayPnGflmtxZ6TpEtmk8upKAY1dxBGq7cj1bdWgsezPZg28c8do16zruBmlaYe2FHhH3VMvGqKXtTYsbab6nkio0jYjRnPooz+VGxaXqEgBS0lGfMjH516vNp8MVtPJbRRQRRDAWNAME+34/KqK4uoHjktHkhkdz+7IjIZifIEe1Z34y30iWq1Z51qNpqFt4DGu707wZqlMpVsyrvZTjxHp8q9H+kE2lvpUVu/czMZcwpgd59nrkc46ZzXmtwxkbAGT1OKW1vK1w3KSwh6lKNPjOWWGn6nGlwgkBQbuueKuGW6n1Rp4px9VRB4Q2c+ox+Oaxl3KtvCGkzk9EXqaFtNRJl2AtDu48LnB9jVNe9jGeCY0m0aLUpJGu3Vb893u+xGORUKxHvC628shPmVJH9KFWea0uQ0Yz0ynOD86U097dM2ZGRT0UHAFZ3W3XLG0aCZp5x4ZcJ7M4XHyFD/WkDeKR29oxt/HrQ62UrHLMoPr1qdbAY8UrfIVnlKGcjpMkTUDAuLa1iiPXvDl2+80PNeXU5/eSs2euefwomOxgX7Rc/E0SkEEY8EYFVyuFgZQ+SqaSeIhkkZfcGriDVZpdPktpsYcbBIT9r41xwhXBQHmqPUEF2ziPIWMkLzxUeoeuES4LJ6D2GyLqZfLu/Py54rYrjJHJNYH6NrhpIFSQ+JWKN6kAcVvvCwOATjyry9+35zLOTpG8+GMtjrilXI2LLndz5+VKsYcj+5HqR93H4UhCNuO8apRXcmvRO0ov+ks2ZEI1UHnoOOKyKt9bumuIp5InZvtxNg+2c8fhWyfOxvXBxXlhaWKYtHIVYE8g12PBralTnP25yjHdOTisPBqpLK8eY3Dag8sykETOp7zjp4gwrl9DqGp6fPay6pdyRupDQvt2E9R0A86b2Z1f61P9SviFkdcRyeTH0PvTtWkksJQyDkHzFduFnQk3HTk5kq1ZPGTyrUYJ7QCyu4mV4JWK7vNTjj8M0LHEWy7H4KvU8Zr1q2de0V1DZNaRSzyeFN+MdPfpRU/0UXUhytjEhznwSYrmVvD403jdL8M3U7lyXMTxwRlz4QftHA6nkZq319GOiaCzDhLdl/EEfnXpEH0T3MMyyS2TyAEHAnx0+dX1z2MmubMWs/Z6J0X7P77G34c1XG3ioyTmufyE672i1F8fg+fmHnkcgVJ3YwSPRjXrLfRHcMxxZOo9O8HH406f6MotMtmudQgKQqQCzNuxk49feqvSf7r/AKO7hY+lnlkEELOI3kaKRnADEZUDGefMfH3rQ9nOxOsdowEgj7m1Vm3XLjwnP8PrW903SezOnEE2SzMMZXuxjPzrYWnaTvykMEHdJwqn0q/0M4ctZM075NYiUZ+jWeVNzagXbyR9xVR6DBxj5VdaVZ9oNDtY7K3jtXt4ydu2Ujr16g+ta63ljgtDNcsqKq5ZmPAHrWL1bt7Zq7pp0TXGOA/2EPw86ppUJ1p6wjkaVacYJ7Ac2ia8upTajHqEdpJLH3ZhjIaMLxxtK48uD1qqvLHUbMF/2qyt/HFCob/mOfwxXLztfq1ySIUigB/hXJ+81R3l1qFwpM80jfgK69v4Y081EjLK4qS7lJfQIJZQjOzHq8jbmPzNVcghTCvhj5UVfkqSQxz8aqrgN9VndPtKhOasvq6px1iaqMW+pVXNwstwXOTjgfCudzHPHleHz91CMeMj1xU9o471Vc4V+N3pXmds9TfjBpLGUzadGzEb18DEjzFSfA8UJZAmF4T9pZATj7jVxDa7SQ2ABUJsjAIBxmnBaN228Z8TAmoJdQt4gQhRT69ahpsnVLuNCc48/SpO7YDJFDW9ykznutx8yxFENuccHIxVcotdSVhgt3J3Mcsg+0iZHxqohzBAzN9gRkL6M2M/oPvo3VW22r4PJIFVdxG7WpmB/d7xGcHjp6eVOkQbX6NlJllbH2nDj/lAr0NDtD+uOlYr6MYjHZXEkaMdshjywx5A1tFmBj5GWbjivP3rzXY2rxkUZ48TP91KmkSSMdq428fapVkwRiQYCK7mmYPtS5r1I4/qOK8k0+8v9V1s6VYWtvueaRUaQHy3MST7BSflXrOeK8smgm0bWLtoEDSi5eVDnGMnOPfg4+ZrbZyqrPlPkpqxi/qAZ77UbG/aCW0QTxPjcpPhI8wfxr0TvYdd0NbvcqyKP3mTgKw65rzjUZbq8v2vLxXaaRiXkZf9Kms4Bf2sthcZSKYhonPRJBnGfbqPn7V07epc7Nz7GStSpySwzSdl9X0ux7X6a76lbLGs4DO0gCqOep6Cvcotf0Wb/c6vp8n/AA3SH8jXyVNpUksu23TY6nZJG5+wf6Uy709bWXuZSgdR4uRg/l61z76tOtUzNYaNNGChHCZ9gpqFi/2L22b4Sqf1qVZoW+xLGR7MK+O7fT45BwC3wA9PjU/7KXoE/wDKOaxYLT7ADL5MD8DWa+keQRdidVkJACRBsk+jCvmX9kjH+7+e0V0aSMjCgHqPD1qY+15IaysHoVjcR3Mm2OWNz5ANk1uuzemhWE8ueOgPma+fhpxa5EaBdwH2R1rWaZq+t6PpNzY3F/cbJlwIJG3NGvqD1GfTNdz1tS6WkVg5dSzUXnJr+2/bK1v7o6essp0+E+PuhxMwP4qKzY7U6JC4VkuB7iPP61nWR3OyNSfTH9aagDFC2GLDIHUqPLjk/hTVq0rWEY011LIW1Oo/czWw9quz7nAS9c46CCiLvV9JAWJ1ubWSQZQXUDRFvhkc1mdCvV0y+gu1gSZYrhHZSM7gpzj+/StL2o1rTda0e5thG5nmmVoXm8KwDOSST54yODWaHiNwpZfI7sqXYyl9IJHbHrVPqrNBAkanJmyGA8wPL8aNYkAA8jGAw8/eqrUZSt5EyuA0agjP3/0qb+o5rJbRjrwCmCSEmKeJo227lWQEZ96hQ7eFPOeW8qstb1OTVp4rl1CMkQiwCTwKCs7ZrppVCliEyAPXyrlv8GhFxYyL9dhfPhkhAJHmwyD+Ioq4uruWRkTIX19aiNotvPZqBgOJJACegDlfzU1MbpVcqJ44h5u36DzpqS5IkyEWNzMMu3HnzUyabGqbo1a4k8lUfrTv2vp1vyzSXD+uOKhn7VMARBbKPQuc4rViC+qX7Iq57ItLWzuQik28fXoW4H3VBqE0ttKq3zwEHoI26f5etZ6616/uMh5mUHyQbaru9bfuVirdcg85+NU1HRxiER4qfcu9ZlMhVQpUFCwDcH0oKSVzsjd2EbMp4PUcCho5GbcZGZmPmxyaISKSdImQDCAg5rM+Cw9P+jnZ+xrpw5VXuGwPYACtVEAgCjJ2jOB5V5x2c1660jTktooEZNxbLr1yat17ZXXdgPBHx5gkGuNcW9SdRyS6lsZLGDZBS4ypf3xSrHjtbcgA92gz6muVn9JV+Cco3AFOIXbxUWacDXeEO4rJ9r9Lfd9fjACcB19K1g6UyZElieKRA6sMFT0Ip6VaVKW0SHFNYPLZ7+K1UAnxn7K9M/Oqe81m73nu2BXHI2g5rSdr+zNzbxNNaoZ7UHPg+1H8R5j4V59NuBOHOfSttXxG4rd8FKoQiXo1izuGWW772K4AwXU/a+NPe406Yl2vyXPXI6/+YVlyx8zzTevSkd1Ul9WGTol0NajWAPF+CCPKMf8AXT+7sz9i/G1eQGjJx+JrIBSfI04IfSl85d4oNX8muEVqAf8A4hbjPXMcn/QaeqWq4I1C1bb08Eg//nWRCCulsDGKPOX2INX8s10VzaafObqGTv7sLhWCFUX35x+QqSHXbeEHv9PhuJWO4ySznJPw6Viyx8jiuFvnTxu5Q+hY/QV0Yy68npWl9otDu5Eg1CNbJn+y5k/d/Mjp8TxWjn7P2QTvfqyOh6EMW49eteIfKiFvLtFCJczKi9FEhAHyrRDxOf8AcWTPOzy/Y2j1G6sbSAHureJcfy5/OqW7cKTjAx0wAKxIvbj/ABu7/wCY08TO46kfFq0vxdYwoEQtGusi+uX3bOcnHn8aoLyXdeu/VUP5eVckVwM7jg+9RMmVwAD8K5lxX855NcYao7czCaZnijEUbcbQehonS5JElHdggkg8eZHT5A80MkW5gOfuo1HaGNolYjI2k+g64HzqjoOS61qBupEaMkJHGsER8yq8k/Mkn/NVVubpRRgeV8kH2qeHTmehAVu0mnLGf8NXkGlMxAXJz5AVotL7GXN1EJW2xJnjvAc/dijoBhVt5G6CpBp07dBXq9n2JsoxmeWSVvQHaKt7fQNMt+VtEJ8t3NK5xJ1Z4xBo17I4WNQSenX9M1fad2T14eLucQt1wpP4ECvWVijjO2OJU/4FAqTZ4PUevnSuf4G1MVpXZC5dk+sLOi/yhSf/AFCtPbditPTxTPPJx6hastuFHtT1mmUeByPnSylJkpJEUfZjSlUA2Yf3ZmNdqf63KPtNzSpcSJ4B1PHSl7ikCKeMYpiBoruwnqfnThjnyrgDbvlUAQyQiVCDgr7jOazmpdjNOv2aVu8jcnkrjH3YrTopGDTm58qAMI30c2ZJxdSD/Itcb6ObUDw3c3/KtbsL7U8KMZ8qMsMI82l+jqYf7m+jK/zRkfrQzfR1qBbC3VqfizD9K9RCjJ9PKm7AW4qdmGEeVN9H2qgkB7Rsf+Iw/wD1ph+j7Vh5WxPtKf6V64oxkEDrXDgtgDoaNmRhHkB7Aaz/ANzB8pRTP9gtZH/y8Z/+8texlfbNc2gYyKndhhHj3+wOteUEK/GUV0fR/q46pb//AJf9K9gwCcYFd24PSjdhqeMv2F1hTxbBx/LIv60l7B60TkWpX4yKP1r2VgBy3SkAD0xUbMNTyFewOu46W6j0aQfpUi/RrrTN4p7JAf5mP5CvWyAUOR0NcOMA5IFGzJ1PMoPozvsfvdWhTP8ABAW/Mij7b6M7dD+/1K4cfyRqv9a3/AXmlwDj8aNmGEZa37C6RDt3ieTHq/8ATFWMPZrSYMEWSHH8QzVvSOf4qMthhA8VpBEP3MKR49EAzUmzPJp/iI61wgj0NQSM2qPPmujBPPFO8J8sUnGACKCBZA4xzSGM48q6OeTTlTIJ44oAaeabg5wOlP4AHOc+VdwBUgRsu48UqYd4JxSoAYKevWlSoAkxlT8KapLNg+QpUqAHKvh6mugYpUqgBFiD99dbhdvUe9KlQAlpf4qVKgkaxwwp54KnPUClSoIHkkcDoKY55FKlQA1+GGK7ExIOaVKoJE5wK6DlDSpUAdPp7UxueD0rlKpATE7fnTvUetKlQQMBp1KlQA6kelKlQAwdaegyxB9KVKgCNuGNdLEMgHnXKVSB2NiVc8Z3YrrcilSoAbnFKlSqAP/Z"]}
1 change: 1 addition & 0 deletions ods_ci/tests/Resources/Files/triton/kserve-triton-fil-grpc-output.json
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"outputs":[{"shape":["1","2"],"parameters":{},"name":"output__0","datatype":"FP32","contents":null}],"raw_output_contents":["JmB8P4b2Zzw="],"parameters":{},"model_name":"fil","model_version":"1","id":"test1"}
3 changes: 3 additions & 0 deletions ods_ci/tests/Resources/Files/triton/triton_onnx_gRPC_servingruntime.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,9 @@ spec:
name: triton
priority: 1
version: "2"
- autoSelect: true
name: xgboost
version: "1"
- autoSelect: true
name: python
version: "1"
Expand Down
47 changes: 47 additions & 0 deletions ...l_serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve.robot
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,9 @@ ${KERAS_RUNTIME_NAME}= triton-keras-rest
${KERAS_RUNTIME_FILEPATH}= ${RESOURCES_DIRPATH}/triton_keras_rest_servingruntime.yaml
${INFERENCE_REST_INPUT_KERAS}= @tests/Resources/Files/triton/kserve-triton-keras-rest-input.json
${EXPECTED_INFERENCE_REST_OUTPUT_FILE_KERAS}= tests/Resources/Files/triton/kserve-triton-keras-rest-output.json
${INFERENCE_GRPC_INPUT_FIL}= tests/Resources/Files/triton/kserve-triton-fil-grpc-input.json
${EXPECTED_INFERENCE_GRPC_OUTPUT_FILE_FIL}= tests/Resources/Files/triton/kserve-triton-fil-grpc-output.json
${FIL_MODEL_NAME}= xgboost
${KERAS_RUNTIME_NAME_GRPC}= triton-keras-grpc
${KERAS_GRPC_RUNTIME_FILEPATH}= ${RESOURCES_DIRPATH}/triton_keras_gRPC_servingruntime.yaml
${INFERENCE_GRPC_INPUT_KERAS}= tests/Resources/Files/triton/kserve-triton-keras-gRPC-input.json
Expand Down Expand Up @@ -401,6 +404,50 @@ Test FIL Model Rest Inference Via UI (Triton on Kserve) # robocop: off=too-lo
... Clean All Models Of Current User
... AND
... Delete Serving Runtime Template From CLI displayed_name=triton-kserve-rest

Test FIL Model Grpc Inference Via UI (Triton on Kserve) # robocop: off=too-long-test-case
[Documentation] Test the deployment of an fil model in Kserve using Triton
[Tags] Sanity RHOAIENG-15823
Open Data Science Projects Home Page
Create Data Science Project title=${PRJ_TITLE} description=${PRJ_DESCRIPTION}
... existing_project=${FALSE}
Open Dashboard Settings settings_page=Serving runtimes
Upload Serving Runtime Template runtime_filepath=${ONNX_GRPC_RUNTIME_FILEPATH}
... serving_platform=single runtime_protocol=gRPC
Serving Runtime Template Should Be Listed displayed_name=${ONNX_GRPC_RUNTIME_NAME}
... serving_platform=single
Recreate S3 Data Connection project_title=${PRJ_TITLE} dc_name=model-serving-connection
... aws_access_key=${S3.AWS_ACCESS_KEY_ID} aws_secret_access=${S3.AWS_SECRET_ACCESS_KEY}
... aws_bucket_name=ods-ci-s3oject_title=${PRJ_TITLE}
Deploy Kserve Model Via UI model_name=${FIL_MODEL_NAME} serving_runtime=triton-kserve-grpc
... data_connection=model-serving-connection path=triton/model_repository/ model_framework=xgboost - 1
... token=${TRUE}
Wait For Pods To Be Ready label_selector=serving.kserve.io/inferenceservice=${FIL_MODEL_NAME}
... namespace=${PRJ_TITLE}
${EXPECTED_INFERENCE_GRPC_OUTPUT_FIL}= Load Json File file_path=${EXPECTED_INFERENCE_GRPC_OUTPUT_FILE_FIL}
... as_string=${TRUE}
${EXPECTED_INFERENCE_GRPC_OUTPUT_FIL}= Load Json String ${EXPECTED_INFERENCE_GRPC_OUTPUT_FIL}
${EXPECTED_INFERENCE_GRPC_OUTPUT_FIL}= Evaluate json.dumps(${EXPECTED_INFERENCE_GRPC_OUTPUT_FIL})
Log ${EXPECTED_INFERENCE_GRPC_OUTPUT_FIL}
Open Model Serving Home Page
${host_url}= Get Model Route Via UI model_name=${FIL_MODEL_NAME}
${host}= Evaluate re.search(r"${PATTERN}", r"${host_url}").group(1) re
Log ${host}
${token}= Get Access Token Via UI single_model=${TRUE} model_name=fil project_name=${PRJ_TITLE}
${inference_output}= Query Model With GRPCURL host=${host} port=443
... endpoint=inference.GRPCInferenceService/ModelInfer
... json_body=@ input_filepath=${INFERENCE_GRPC_INPUT_FIL}
... insecure=${True} protobuf_file=${PROTOBUFF_FILE} json_header="Authorization: Bearer ${token}"
Log ${inference_output}
${inference_output}= Evaluate json.dumps(${inference_output})
Log ${inference_output}
${result} ${list}= Inference Comparison ${EXPECTED_INFERENCE_GRPC_OUTPUT_FIL} ${inference_output}
[Teardown] Run Keywords Get Kserve Events And Logs model_name=${FIL_MODEL_NAME}
... project_title=${PRJ_TITLE}
... AND
... Clean All Models Of Current User
... AND
... Delete Serving Runtime Template From CLI displayed_name=triton-kserve-grpc

*** Keywords ***
Triton On Kserve Suite Setup
Expand Down

0 comments on commit d273f6f

Please sign in to comment.