Webhooks - example

[JovanVeljanoski]

Hi,

Would it be possible to add the simplest possible example of how one would create or "listen to" webhooks with solara?
Thank you!

[JovanVeljanoski]

In addition, would be nice if we can add additional (custom) endpoints to the solara server, and implement it into the app.

[JovanVeljanoski]

Ok, i am pretty sure I am doing something very unholy here, but let me share an example.
(based on #670)

from copy import deepcopy

import solara
import solara.lab
from starlette.responses import JSONResponse
import asyncio



added = False

webhooks_result = solara.reactive('initial value')


async def update_webhook_value(message):
    old_value = webhooks_result.value
    message = deepcopy(message)
    print(f'Updating webhook value from "{old_value}" to "{message}"')
    webhooks_result.set(message)
    print(f'Updated webhook value to: "{webhooks_result.value}"')


async def webhook_handler(request):
    try:
        payload = await request.json()
        print('Request received!')
        print(payload.get('message', 'no message'))
        payload = deepcopy(payload)
        asyncio.create_task(update_webhook_value(payload.get('message', 'no message')))
        return JSONResponse({
            "status": "success",
            "message": "Webhook received",
            "data": payload
        })
    except Exception as e:
        return JSONResponse({
            "status": "error",
            "message": str(e)
        }, status_code=400)


def add_webhook_handler():
    global added
    if not added:
        import solara.server.starlette
        solara.server.starlette.app.router.add_route("/webhook", webhook_handler, methods=["POST"])
        added = True


@solara.component
def Page():
    add_webhook_handler()
    value = solara.use_reactive('')

    def fetch_webhook_value():
        print('Fetching webhook value')
        print(f'The webhook value is: {webhooks_result.value}')
        value.set(webhooks_result.value)
        webhooks_result.set('what is this?')

    with solara.Column(align="center"):
        with solara.Card(style="width: 400px; margin: auto;"):
            solara.Text("Hello")
            solara.Markdown("This is a markdown block")
            solara.Markdown(f"Webhook added: {added}")


            solara.Markdown("### Latest Webhook Data")
            solara.Markdown(f'{webhooks_result.value}')
            solara.Button(label='Fetch webhook value', on_click=fetch_webhook_value)
            solara.Text(f'Value: {value.value}')

Then one can hit the endpoint with

 curl -X POST http://localhost:8765/webhook \
  -H "Content-Type: application/json" \
  -d '{"message": "Hello from webhook!"}'

So i am almost happy with this! I can query the endpoint, and the endpoint can trigger functions inside the codebase.
However, I met a problem:

In the example above, the global reactive variable webhooks_result seems to ... have a double life. I.e. it looks like exists in two contexts. When I hit the webhook endpoint with curl commands, i see its value updating correctly. Since I print to terminal the old value, and after setting the new value, I see that the changes happen correctly.

However the UI does not know of any of these changes! First I thought that a re-render is not triggered upon updating the global reactive variable, but looks like something beyond that.

If by some other (solara proper process), i update the webhooks_result variable (say but pressing a button what sets a fixed value), I see that the previous value has nothing to do with the values obtained via the webhook, but the initial value was there. Upon setting, if you hit the webhook endpoint, the reactive variable has no knowledge of the interactions with the solara UI.

So in short.. webhooks_result seems to live in two contexts: the solara/reactor render context, and something to do with starlette context. Same variable (at least same name) same starting point, but that'sit..

How do I connect these two? Is it even possible?
Or am I trully doing something unholy that.. should not be done (but i really need it.. well the webhooks functionality).

Thank you and many thanks!

[maartenbreddels]

You got pretty far! The issue is that a reactive var is a container for multiple virtual kernels, so it should be set for each kernel (i.e. all pages connect).

from copy import deepcopy

import solara
import solara.lab
from starlette.responses import JSONResponse
import asyncio
import solara.server.kernel_context



added = False

webhooks_result = solara.reactive('initial value')

async def update_webhook_value(message):
    for kernel_id, context in solara.server.kernel_context.contexts.items():
        with context:
            old_value = webhooks_result.value
            message = deepcopy(message)
            print(f'Updating webhook value from "{old_value}" to "{message}". Kernel: {kernel_id}')
            webhooks_result.value = message
            print(f'Updated webhook value to: "{webhooks_result.value}"')


async def webhook_handler(request):
    try:
        payload = await request.json()
        print('Request received!')
        print(payload.get('message', 'no message'))
        payload = deepcopy(payload)
        await update_webhook_value(payload.get('message', 'no message'))
        return JSONResponse({
            "status": "success",
            "message": "Webhook received",
            "data": payload
        })
    except Exception as e:
        return JSONResponse({
            "status": "error",
            "message": str(e)
        }, status_code=400)


def add_webhook_handler():
    global added
    if not added:
        import solara.server.starlette
        solara.server.starlette.app.router.add_route("/webhook", webhook_handler, methods=["POST"])
        added = True


@solara.component
def Page():
    counter = solara.use_reactive(0)
    add_webhook_handler()
    print("Render", solara.get_kernel_id(), webhooks_result.value, counter.value)
    solara.Text(f'Value: {webhooks_result.value}')
    solara.Button("Increment", on_click=lambda: counter.set(counter.value + 1))

Something like this should work, but it for some reason does not render it to the screen. At least it might satisfy your curiosity!

[JovanVeljanoski]

Thanks for the response!

I get the idea.. but still does not work.
Also in your example, an interesting thing (not sure if it is expected): add a solara.Text(f'Counter: {counter.value}') to the Page component (so to see the counter value).

When you click the counter value, there is the new value printed on the terminal and the page component updates.
Then do a curl command hitting the webhook endpoint (The UI is not updated at this point still..). But after that.. if you click pressing the "Increment" button, the terminal will print the correct new value, but the UI will not reflect the change, i.e. the render will be stuck on whatever state it was prior to the curl command..

This is a curious case indeed. Would be nice if it is solved at some point in the future, with a more elegant api. Webhooks are quite common (I am learning..) so it is very helpful when interacting with other services for more serious use-case...

Thanks! I will report back if I figure this out in the meantime..

[JovanVeljanoski]

I think I got it to work (sort of). See the example below.

When I git the webhook endpoint with a curl command, the global reactive variable gets updated, but the UI does not re-render. If you do something else (another of action, like any of the other buttons) that trigger a re-render, you can see the new value.

No idea how this example is different from what you showed above..

Anyway, this is good enough for now I think. I would be fantastic if down the line webhooks are officially supported!
Thanks

from copy import deepcopy
import random
import string

import solara
import solara.lab
import solara.server.kernel_context
from starlette.responses import JSONResponse




added = False

webhooks_result = solara.reactive('initial value')

async def update_webhook_value(message):
    for kernel_id, context in solara.server.kernel_context.contexts.items():
        with context:
            old_value = webhooks_result.value
            message = deepcopy(message)
            print(f'Updating webhook value from "{old_value}" to "{message}". Kernel: {kernel_id}')
            webhooks_result.value = message
            print(f'Updated webhook value to: "{webhooks_result.value}"')


async def webhook_handler(request):
    try:
        payload = await request.json()
        print('Request received!')
        print(payload.get('message', 'no message'))
        payload = deepcopy(payload)
        await update_webhook_value(payload.get('message', 'no message'))
        return JSONResponse({
            "status": "success",
            "message": "Webhook received",
            "data": payload
        })
    except Exception as e:
        return JSONResponse({
            "status": "error",
            "message": str(e)
        }, status_code=400)


def add_webhook_handler():
    global added
    if not added:
        import solara.server.starlette
        solara.server.starlette.app.router.add_route("/webhook", webhook_handler, methods=["POST"])
        added = True


@solara.component
def Page():
    add_webhook_handler()
    value = solara.use_reactive('')
    increment = solara.use_reactive(0)
    print("Render", solara.get_kernel_id(), webhooks_result.value, value.value)

    def fetch_webhook_value():
        print('Fetching webhook value')
        print(f'The webhook value is: {webhooks_result.value}')
        value.set(webhooks_result.value)
        random_string = ''.join(random.choices(string.ascii_letters + string.digits, k=5))
        what = f"what is this? {random_string}"
        value.set(what)

    with solara.Column(align="center"):
        with solara.Card(style="width: 400px; margin: auto;"):
            solara.Text("Hello")
            solara.Markdown("This is a markdown block")
            solara.Markdown(f"Webhook added: {added}")

            with solara.Column():
                solara.Markdown("### Latest Webhook Data")
                solara.Markdown(f'{webhooks_result.value}')
                solara.Button(label='Fetch webhook value', on_click=fetch_webhook_value)
                solara.Text(f'Value: {value.value}')
                solara.Button(label='Increment', on_click=lambda: increment.set(increment.value + 1))
                solara.Text(f'Increment: {increment.value}')

[maartenbreddels]

Seeing @Ben-Epstein also wanted to have this working, I've take another look at this. It appeared the problem was due to #919 which should be released soon.

Together with this code:

from copy import deepcopy
import random
import string

import solara
import solara.lab
import solara.server.kernel_context
from starlette.responses import JSONResponse

from starlette.requests import Request
from anyio import to_thread



added = False

webhooks_result = solara.reactive(None)

def update_webhook_value(message):
    # we update the reactive value for all kernels (i.e. connected browser pages)
    for kernel_id, context in solara.server.kernel_context.contexts.items():
        with context:
            old_value = webhooks_result.value
            message = deepcopy(message)
            print(f'Updating webhook value from "{old_value}" to "{message}". Kernel: {kernel_id}')
            webhooks_result.value = message


async def webhook_handler(request: Request):
    try:
        payload = await request.json()

        print('Request received!')
        print(payload.get('message', 'no message'))
        payload = deepcopy(payload)
        # if we call this directly, we run in the same thread as the uvicorn/starlette server (or anyio thread)
        # that the websocket is related to, which causes issues (it will raise an exception in solara >= 1.43)
        # using anyio, we run it in a separate thread
        await to_thread.run_sync(update_webhook_value, payload.get('message', 'no message'))
        return JSONResponse({
            "status": "success",
            "message": "Webhook received",
            "data": payload
        })
    except Exception as e:
        return JSONResponse({
            "status": "error",
            "message": str(e)
        }, status_code=400)


def add_webhook_handler():
    # workaround since we cannot import solara.server.starlette directly
    global added
    if not added:
        import solara.server.starlette
        solara.server.starlette.app.router.add_route("/webhook", webhook_handler, methods=["POST"])
        added = True


@solara.component
def Page():
    add_webhook_handler()
    
    with solara.Column(align="center", style={"height": "100%", "justify-content": "center"}):
        with solara.Card(style="width: 400px; margin: auto;"):
            with solara.Column():
                if webhooks_result.value is None:
                    solara.Markdown("No data yet")
                else:
                    solara.Markdown(f'{webhooks_result.value}')

And this curl command:

$ curl -X POST http://localhost:8877/webhook -d '{"message": "Hello from webhook!"}' -H "Content-Type: application/json" 

Should be a good working example.

Once released, I'll close the issue, if there are more issues, feel free to let me know!