Human-in-the-loop

[tgm185z]

Hi, very nice work! It has been a great help in getting me into the world of agents and langgraph and I am still getting to know this project.

I'm building a simple chatbot to manage restaurant reservations and I'm combining it with your front-end. I use human-in-the-loop to ask the user for confirmation in the use of certain tools, any suggestions on how to apply it to your project? Thanks a lot.

[JoshuaC215]

Hey, thanks for the kind words! Really glad to hear this has been helpful.

There are a few options for how to handle the confirmation flow:

1. I suspect the simplest would be to treat it as a structured conversation step. So you might send a certain message back to the client at the end state which indicates approval is needed, and show the user a button or dialog to confirm when it sees that message, then if you get approval the agent workflow proceeds as a new execution on the same thread (maybe following a separate branch of logic). It would make your graph a bit more complex but not need any architecture change
2. Another approach is to use a breakpoint in the agent as described in LangGraph docs (see below). On an interrupt, return a structured response as above. If the user confirms (via a new POST call), then continue. You’d need to adapt the service to handle this interrupt case.
3. The most complex would be to enable bidirectional communication between server and client in the same call. Then you could use a “get_confirmation” node in a simple agent graph which sends back the response and waits for confirmation in the same call. Most likely you’d need to change from SSE to websocket to make that work, it would bring some other challenges.

I’d probably start with (1) if I was implementing this myself for a simple case. If I had multiple human in the loop steps, I might use (2) since it requires a bigger up front change but seems easier to generalize.

Hope this helps!

https://langchain-ai.github.io/langgraph/concepts/human_in_the_loop/

[tgm185z]

Thank you very much for your detailed reply.
I will follow your advice and start with (1), as this is a simple confirmation, keep it simple would be the best.

[phfifofum]

Hi Josh,

Great toolkit, been using it to guide me past few weeks.

I have a question around implementation number 2 above. I have an issue restarting the graph after an interrupt, perhaps you can shed some light or offer a workaround to my problem.

My set up is a graph backend, langserve/fast API endpoint and streamlit front end.

1. Invoke the graph with astream_events via streamlit and handle events as normal.
2. Move through the graph nodes.
3. Emit custom event asking for user approval
4. Submit user approval state to a user_approval post endpoint
5. Make the checkpoints available in the endpoint file, retrieve the latest state and the update the state with approval.
6. Then I try to restart the graph with astream_events

I dont know if I am accessing state or handling the requests in the cleanest way. Although the graph appears to update with my method.

The biggest issue however is restarting the graph and trying to keep some consistency in the front end. I can restart the graph from the endpoint file using astream_events but this doesnt stream events to the front end and would need some type of websocket or similar connection bloating my UI code.

I am struggling to use any method to invoke with 'None' from streamlit as its failing validation deep in the langserve libraries by the looks of it and I cant muster up a workaround atm.

Any help or signposting would be appreciated.

[JoshuaC215]

Hey @phfifofum thanks for the kind words

Did I understand right that you're using LangServe (https://github.com/langchain-ai/langserve) and calling it from the Streamlit app using the LangServe client?

AFAIK LangServe does not fully support serving LangGraph agents. This was the first thing I attempted too and ran into problems, hence why I developed this toolkit as an alternative.

If that is indeed the case, I'm afraid I won't be able to offer much support since I haven't spent time working with LangServe. If I misunderstood what you're doing, let me know.

Alternatively, if you use agent-service-toolkit, I suspect the steps (1)-(6) you described would work. You'd just need to add an endpoint for POSTing the approval and updating the thread state, and then call astream() again with the same thread_id (similar to the pseudo-code here).

If you end up trying to do this with agent-service-toolkit feel free to post a mock / simple version as a draft PR, would love to play with it.

[phfifofum]

Ah that seems like a blocker tbf. Refactoring into the toolkit strucutre shouldnt take too long so let me try that and report back.

[tgm185z]

Hello,

Just here to update on how I'm doing with human-in-the-loop.
I ended up deciding that option 2 seems to be the most useful for what I need.
I'll explain briefly how I'm doing with it. I'm using this repository to create a chatbot to manage bookings, for this I provide the graph with a set of tools. Some of these tools modify databases and I need the user to give me confirmation that the data the graph modifies is correct. Following langgraph tutorials I have created a set of tools that can be used without permission and another set that require permission (i.e. two different nodes) and placed an interrupt_before in the graph before the node that needs confirmation.

Following your advice, I have made some modifications to the repository. It's probably not completely generalizable and it's not perfectly optimized, but it's starting to work more or less. I am encountering problems in the front-end part though. For some reason, the AsyncGenerator are not closing properly and even if it finishes if the user asks another question the old generator and the new one exist at the same time and it breaks the execution. I'm debugging this part right now to make sure if the problem is in the integration with streamlit or somewhere else, since I don't have enough knowledge about this part.

I leave here a summary of the modifications that I have been making in case they are useful or you consider to be able to give me some advice of improvement/generalisation:

1. Created InterruptMessage class. I decided to implement it as a new class instead of a ChatMessage type in order to have a set of message formatting functions, so I had to modify quite a few of the methods already created in order to support it, however, I thought it was best because I ended up adding different features that I thought were out of ChatMessage.
2. In service.message_generator, after astream_event, I check the status of the graph and if it has been interrupted I return an InterruptMessage with the corresponding information.
3. In streamlit front-end draw_messages() when waiting for the response from the tools I check if what I receive is an InterruptMessage, if it is, I show a yes/no button and a custom message for each tool asking for confirmation.
4. In service and client I have created a method /iinvoke in order to continue the execution depending on the confirmation. If it is a yes I execute ainvoke with None, and if it is a no I execute ainvoke with a manual ToolMessage (as indicated in the langgraph tutorials). From what I've seen, it's not possible to continue the execution with astream_events but you have to resort to an invoke.

Maybe these help, anyway, I'm still debugging the problem I mentioned above.

[AmooreFuentes]

@tgm185z could you solve the issue?

[tgm185z]

Hi,

I just added a PR with a first version. #117
Currently it's only for stream. I've created a new class InterruptMessage because it seems appropriate to treat it as a completely different entity from chat message, although it might be possible to integrate it as a custom type. It will probably need some cleanup.
Open to improvements and suggestions.