diff --git a/notebook/agentchat_agentops.ipynb b/notebook/agentchat_agentops.ipynb
index 293efa8e4bd6..71106e45d3c4 100644
--- a/notebook/agentchat_agentops.ipynb
+++ b/notebook/agentchat_agentops.ipynb
@@ -7,7 +7,7 @@
"collapsed": false
},
"source": [
- "# AgentOps"
+ "# Agent Tracking with AgentOps"
]
},
{
@@ -15,7 +15,7 @@
"id": "a447802c88c8a240",
"metadata": {},
"source": [
- "![logo](https://raw.githubusercontent.com/AgentOps-AI/agentops/35d5682866921a9e28d8ef66ae3c3b3d92d8fa6b/img/logo.png)\n",
+ "\n",
"\n",
"[AgentOps](https://agentops.ai/?=autogen) provides session replays, metrics, and monitoring for AI agents.\n",
"\n",
@@ -27,8 +27,11 @@
"id": "b354c068",
"metadata": {},
"source": [
- "### Dashboard\n",
- "![Agent Dashboard](https://github.com/AgentOps-AI/agentops/assets/14807319/158e082a-9a7d-49b7-9b41-51a49a1f7d3d)"
+ "### Overview Dashboard\n",
+ "\n",
+ "\n",
+ "### Session Replays\n",
+ ""
]
},
{
@@ -39,7 +42,7 @@
"## Adding AgentOps to an existing Autogen service.\n",
"To get started, you'll need to install the AgentOps package and set an API key.\n",
"\n",
- "AgentOps automatically configures itself when it's initialized. This means your agents will be tracked and logged to your AgentOps account right away."
+ "AgentOps automatically configures itself when it's initialized meaning your agent run data will be tracked and logged to your AgentOps account right away."
]
},
{
@@ -69,7 +72,7 @@
"\n",
"By default, the AgentOps `init()` function will look for an environment variable named `AGENTOPS_API_KEY`. Alternatively, you can pass one in as an optional parameter.\n",
"\n",
- "Create an account and API key at [AgentOps.ai](https://agentops.ai/)"
+ "Create an account and obtain an API key at [AgentOps.ai](https://agentops.ai/settings/projects)"
]
},
{
@@ -87,12 +90,14 @@
"name": "stderr",
"output_type": "stream",
"text": [
- "π AgentOps: \u001B[34m\u001B[34mSession Replay: https://app.agentops.ai/drilldown?session_id=8bfaeed1-fd51-4c68-b3ec-276b1a3ce8a4\u001B[0m\u001B[0m\n"
+ "π AgentOps: \u001b[34m\u001b[34mSession Replay: https://app.agentops.ai/drilldown?session_id=8bfaeed1-fd51-4c68-b3ec-276b1a3ce8a4\u001b[0m\u001b[0m\n"
]
},
{
"data": {
- "text/plain": "UUID('8bfaeed1-fd51-4c68-b3ec-276b1a3ce8a4')"
+ "text/plain": [
+ "UUID('8bfaeed1-fd51-4c68-b3ec-276b1a3ce8a4')"
+ ]
},
"execution_count": 1,
"metadata": {},
@@ -104,7 +109,7 @@
"\n",
"from autogen import ConversableAgent, UserProxyAgent, config_list_from_json\n",
"\n",
- "agentops.init(api_key=\"7c94212b-b89d-47a6-a20c-23b2077d3226\") # or agentops.init(api_key=\"...\")"
+ "agentops.init(api_key=\"...\")"
]
},
{
@@ -144,19 +149,19 @@
"name": "stdout",
"output_type": "stream",
"text": [
- "\u001B[33magent\u001B[0m (to user):\n",
+ "\u001b[33magent\u001b[0m (to user):\n",
"\n",
"How can I help you today?\n",
"\n",
"--------------------------------------------------------------------------------\n",
- "\u001B[33muser\u001B[0m (to agent):\n",
+ "\u001b[33muser\u001b[0m (to agent):\n",
"\n",
"2+2\n",
"\n",
"--------------------------------------------------------------------------------\n",
- "\u001B[31m\n",
- ">>>>>>>> USING AUTO REPLY...\u001B[0m\n",
- "\u001B[33magent\u001B[0m (to user):\n",
+ "\u001b[31m\n",
+ ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
+ "\u001b[33magent\u001b[0m (to user):\n",
"\n",
"2 + 2 equals 4.\n",
"\n",
@@ -168,7 +173,7 @@
"output_type": "stream",
"text": [
"π AgentOps: This run's cost $0.000960\n",
- "π AgentOps: \u001B[34m\u001B[34mSession Replay: https://app.agentops.ai/drilldown?session_id=8bfaeed1-fd51-4c68-b3ec-276b1a3ce8a4\u001B[0m\u001B[0m\n"
+ "π AgentOps: \u001b[34m\u001b[34mSession Replay: https://app.agentops.ai/drilldown?session_id=8bfaeed1-fd51-4c68-b3ec-276b1a3ce8a4\u001b[0m\u001b[0m\n"
]
}
],
@@ -185,7 +190,7 @@
"# Create the agent that represents the user in the conversation.\n",
"user_proxy = UserProxyAgent(\"user\", code_execution_config=False)\n",
"\n",
- "# Let the assistant start the conversation. It will end when the user types exit.\n",
+ "# Let the assistant start the conversation. It will end when the user types \"exit\".\n",
"assistant.initiate_chat(user_proxy, message=\"How can I help you today?\")\n",
"\n",
"# Close your AgentOps session to indicate that it completed.\n",
@@ -204,13 +209,13 @@
},
{
"cell_type": "markdown",
- "source": [
- "![Session_Overview](https://github.com/AgentOps-AI/agentops/assets/14807319/d7228019-1488-40d3-852f-a61e998658ad)"
- ],
+ "id": "cbd689b0f5617013",
"metadata": {
"collapsed": false
},
- "id": "cbd689b0f5617013"
+ "source": [
+ "![session replay](https://github.com/AgentOps-AI/agentops/blob/main/docs/images/external/app_screenshots/session-overview.png?raw=true)"
+ ]
},
{
"cell_type": "markdown",
@@ -236,70 +241,70 @@
"name": "stderr",
"output_type": "stream",
"text": [
- "π AgentOps: \u001B[34m\u001B[34mSession Replay: https://app.agentops.ai/drilldown?session_id=880c206b-751e-4c23-9313-8684537fc04d\u001B[0m\u001B[0m\n"
+ "π AgentOps: \u001b[34m\u001b[34mSession Replay: https://app.agentops.ai/drilldown?session_id=880c206b-751e-4c23-9313-8684537fc04d\u001b[0m\u001b[0m\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
- "\u001B[33mUser\u001B[0m (to Assistant):\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
"\n",
"What is (1423 - 123) / 3 + (32 + 23) * 5?\n",
"\n",
"--------------------------------------------------------------------------------\n",
- "\u001B[31m\n",
- ">>>>>>>> USING AUTO REPLY...\u001B[0m\n",
- "\u001B[33mAssistant\u001B[0m (to User):\n",
+ "\u001b[31m\n",
+ ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
+ "\u001b[33mAssistant\u001b[0m (to User):\n",
"\n",
- "\u001B[32m***** Suggested tool call (call_aINcGyo0Xkrh9g7buRuhyCz0): calculator *****\u001B[0m\n",
+ "\u001b[32m***** Suggested tool call (call_aINcGyo0Xkrh9g7buRuhyCz0): calculator *****\u001b[0m\n",
"Arguments: \n",
"{\n",
" \"a\": 1423,\n",
" \"b\": 123,\n",
" \"operator\": \"-\"\n",
"}\n",
- "\u001B[32m***************************************************************************\u001B[0m\n",
+ "\u001b[32m***************************************************************************\u001b[0m\n",
"\n",
"--------------------------------------------------------------------------------\n",
- "\u001B[35m\n",
- ">>>>>>>> EXECUTING FUNCTION calculator...\u001B[0m\n",
- "\u001B[33mUser\u001B[0m (to Assistant):\n",
+ "\u001b[35m\n",
+ ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
"\n",
- "\u001B[33mUser\u001B[0m (to Assistant):\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
"\n",
- "\u001B[32m***** Response from calling tool (call_aINcGyo0Xkrh9g7buRuhyCz0) *****\u001B[0m\n",
+ "\u001b[32m***** Response from calling tool (call_aINcGyo0Xkrh9g7buRuhyCz0) *****\u001b[0m\n",
"1300\n",
- "\u001B[32m**********************************************************************\u001B[0m\n",
+ "\u001b[32m**********************************************************************\u001b[0m\n",
"\n",
"--------------------------------------------------------------------------------\n",
- "\u001B[31m\n",
- ">>>>>>>> USING AUTO REPLY...\u001B[0m\n",
- "\u001B[33mAssistant\u001B[0m (to User):\n",
+ "\u001b[31m\n",
+ ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
+ "\u001b[33mAssistant\u001b[0m (to User):\n",
"\n",
- "\u001B[32m***** Suggested tool call (call_prJGf8V0QVT7cbD91e0Fcxpb): calculator *****\u001B[0m\n",
+ "\u001b[32m***** Suggested tool call (call_prJGf8V0QVT7cbD91e0Fcxpb): calculator *****\u001b[0m\n",
"Arguments: \n",
"{\n",
" \"a\": 1300,\n",
" \"b\": 3,\n",
" \"operator\": \"/\"\n",
"}\n",
- "\u001B[32m***************************************************************************\u001B[0m\n",
+ "\u001b[32m***************************************************************************\u001b[0m\n",
"\n",
"--------------------------------------------------------------------------------\n",
- "\u001B[35m\n",
- ">>>>>>>> EXECUTING FUNCTION calculator...\u001B[0m\n",
- "\u001B[33mUser\u001B[0m (to Assistant):\n",
+ "\u001b[35m\n",
+ ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
"\n",
- "\u001B[33mUser\u001B[0m (to Assistant):\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
"\n",
- "\u001B[32m***** Response from calling tool (call_prJGf8V0QVT7cbD91e0Fcxpb) *****\u001B[0m\n",
+ "\u001b[32m***** Response from calling tool (call_prJGf8V0QVT7cbD91e0Fcxpb) *****\u001b[0m\n",
"433\n",
- "\u001B[32m**********************************************************************\u001B[0m\n",
+ "\u001b[32m**********************************************************************\u001b[0m\n",
"\n",
"--------------------------------------------------------------------------------\n",
- "\u001B[31m\n",
- ">>>>>>>> USING AUTO REPLY...\u001B[0m\n"
+ "\u001b[31m\n",
+ ">>>>>>>> USING AUTO REPLY...\u001b[0m\n"
]
},
{
@@ -316,94 +321,94 @@
"name": "stdout",
"output_type": "stream",
"text": [
- "\u001B[33mAssistant\u001B[0m (to User):\n",
+ "\u001b[33mAssistant\u001b[0m (to User):\n",
"\n",
- "\u001B[32m***** Suggested tool call (call_CUIgHRsySLjayDKuUphI1TGm): calculator *****\u001B[0m\n",
+ "\u001b[32m***** Suggested tool call (call_CUIgHRsySLjayDKuUphI1TGm): calculator *****\u001b[0m\n",
"Arguments: \n",
"{\n",
" \"a\": 32,\n",
" \"b\": 23,\n",
" \"operator\": \"+\"\n",
"}\n",
- "\u001B[32m***************************************************************************\u001B[0m\n",
+ "\u001b[32m***************************************************************************\u001b[0m\n",
"\n",
"--------------------------------------------------------------------------------\n",
- "\u001B[35m\n",
- ">>>>>>>> EXECUTING FUNCTION calculator...\u001B[0m\n",
- "\u001B[33mUser\u001B[0m (to Assistant):\n",
+ "\u001b[35m\n",
+ ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
"\n",
- "\u001B[33mUser\u001B[0m (to Assistant):\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
"\n",
- "\u001B[32m***** Response from calling tool (call_CUIgHRsySLjayDKuUphI1TGm) *****\u001B[0m\n",
+ "\u001b[32m***** Response from calling tool (call_CUIgHRsySLjayDKuUphI1TGm) *****\u001b[0m\n",
"55\n",
- "\u001B[32m**********************************************************************\u001B[0m\n",
+ "\u001b[32m**********************************************************************\u001b[0m\n",
"\n",
"--------------------------------------------------------------------------------\n",
- "\u001B[31m\n",
- ">>>>>>>> USING AUTO REPLY...\u001B[0m\n",
- "\u001B[33mAssistant\u001B[0m (to User):\n",
+ "\u001b[31m\n",
+ ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
+ "\u001b[33mAssistant\u001b[0m (to User):\n",
"\n",
- "\u001B[32m***** Suggested tool call (call_L7pGtBLUf9V0MPL90BASyesr): calculator *****\u001B[0m\n",
+ "\u001b[32m***** Suggested tool call (call_L7pGtBLUf9V0MPL90BASyesr): calculator *****\u001b[0m\n",
"Arguments: \n",
"{\n",
" \"a\": 55,\n",
" \"b\": 5,\n",
" \"operator\": \"*\"\n",
"}\n",
- "\u001B[32m***************************************************************************\u001B[0m\n",
+ "\u001b[32m***************************************************************************\u001b[0m\n",
"\n",
"--------------------------------------------------------------------------------\n",
- "\u001B[35m\n",
- ">>>>>>>> EXECUTING FUNCTION calculator...\u001B[0m\n",
- "\u001B[33mUser\u001B[0m (to Assistant):\n",
+ "\u001b[35m\n",
+ ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
"\n",
- "\u001B[33mUser\u001B[0m (to Assistant):\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
"\n",
- "\u001B[32m***** Response from calling tool (call_L7pGtBLUf9V0MPL90BASyesr) *****\u001B[0m\n",
+ "\u001b[32m***** Response from calling tool (call_L7pGtBLUf9V0MPL90BASyesr) *****\u001b[0m\n",
"275\n",
- "\u001B[32m**********************************************************************\u001B[0m\n",
+ "\u001b[32m**********************************************************************\u001b[0m\n",
"\n",
"--------------------------------------------------------------------------------\n",
- "\u001B[31m\n",
- ">>>>>>>> USING AUTO REPLY...\u001B[0m\n",
- "\u001B[33mAssistant\u001B[0m (to User):\n",
+ "\u001b[31m\n",
+ ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
+ "\u001b[33mAssistant\u001b[0m (to User):\n",
"\n",
- "\u001B[32m***** Suggested tool call (call_Ygo6p4XfcxRjkYBflhG3UVv6): calculator *****\u001B[0m\n",
+ "\u001b[32m***** Suggested tool call (call_Ygo6p4XfcxRjkYBflhG3UVv6): calculator *****\u001b[0m\n",
"Arguments: \n",
"{\n",
" \"a\": 433,\n",
" \"b\": 275,\n",
" \"operator\": \"+\"\n",
"}\n",
- "\u001B[32m***************************************************************************\u001B[0m\n",
+ "\u001b[32m***************************************************************************\u001b[0m\n",
"\n",
"--------------------------------------------------------------------------------\n",
- "\u001B[35m\n",
- ">>>>>>>> EXECUTING FUNCTION calculator...\u001B[0m\n",
- "\u001B[33mUser\u001B[0m (to Assistant):\n",
+ "\u001b[35m\n",
+ ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
"\n",
- "\u001B[33mUser\u001B[0m (to Assistant):\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
"\n",
- "\u001B[32m***** Response from calling tool (call_Ygo6p4XfcxRjkYBflhG3UVv6) *****\u001B[0m\n",
+ "\u001b[32m***** Response from calling tool (call_Ygo6p4XfcxRjkYBflhG3UVv6) *****\u001b[0m\n",
"708\n",
- "\u001B[32m**********************************************************************\u001B[0m\n",
+ "\u001b[32m**********************************************************************\u001b[0m\n",
"\n",
"--------------------------------------------------------------------------------\n",
- "\u001B[31m\n",
- ">>>>>>>> USING AUTO REPLY...\u001B[0m\n",
- "\u001B[33mAssistant\u001B[0m (to User):\n",
+ "\u001b[31m\n",
+ ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
+ "\u001b[33mAssistant\u001b[0m (to User):\n",
"\n",
"The result of the calculation is 708.\n",
"\n",
"--------------------------------------------------------------------------------\n",
- "\u001B[33mUser\u001B[0m (to Assistant):\n",
+ "\u001b[33mUser\u001b[0m (to Assistant):\n",
"\n",
"\n",
"\n",
"--------------------------------------------------------------------------------\n",
- "\u001B[31m\n",
- ">>>>>>>> USING AUTO REPLY...\u001B[0m\n",
- "\u001B[33mAssistant\u001B[0m (to User):\n",
+ "\u001b[31m\n",
+ ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
+ "\u001b[33mAssistant\u001b[0m (to User):\n",
"\n",
"TERMINATE\n",
"\n",
@@ -415,7 +420,7 @@
"output_type": "stream",
"text": [
"π AgentOps: This run's cost $0.001800\n",
- "π AgentOps: \u001B[34m\u001B[34mSession Replay: https://app.agentops.ai/drilldown?session_id=880c206b-751e-4c23-9313-8684537fc04d\u001B[0m\u001B[0m\n"
+ "π AgentOps: \u001b[34m\u001b[34mSession Replay: https://app.agentops.ai/drilldown?session_id=880c206b-751e-4c23-9313-8684537fc04d\u001b[0m\u001b[0m\n"
]
}
],
@@ -474,7 +479,7 @@
" description=\"A simple calculator\", # A description of the tool.\n",
")\n",
"\n",
- "# Let the assistant start the conversation. It will end when the user types exit.\n",
+ "# Let the assistant start the conversation. It will end when the user types \"exit\".\n",
"user_proxy.initiate_chat(assistant, message=\"What is (1423 - 123) / 3 + (32 + 23) * 5?\")\n",
"\n",
"agentops.end_session(\"Success\")"
@@ -493,13 +498,13 @@
},
{
"cell_type": "markdown",
- "source": [
- "![Session Drilldown](https://github.com/AgentOps-AI/agentops/assets/14807319/561d59f3-c441-4066-914b-f6cfe32a598c)"
- ],
+ "id": "a922a52ab5fce31",
"metadata": {
"collapsed": false
},
- "id": "a922a52ab5fce31"
+ "source": [
+ "![Session Drilldown](https://github.com/AgentOps-AI/agentops/blob/main/docs/images/external/app_screenshots/session-replay.png?raw=true)"
+ ]
}
],
"metadata": {
diff --git a/website/docs/ecosystem/agentops.md b/website/docs/ecosystem/agentops.md
index 76995b6eb5e4..581fb2671e97 100644
--- a/website/docs/ecosystem/agentops.md
+++ b/website/docs/ecosystem/agentops.md
@@ -1,29 +1,37 @@
-# AgentOps ποΈ
+# Agent Monitoring and Debugging with AgentOps
-![logo](https://raw.githubusercontent.com/AgentOps-AI/agentops/35d5682866921a9e28d8ef66ae3c3b3d92d8fa6b/img/logo.png)
+
-[AgentOps](https://agentops.ai/?=autogen) provides session replays, metrics, and monitoring for agents.
+[AgentOps](https://agentops.ai/?=autogen) provides session replays, metrics, and monitoring for AI agents.
At a high level, AgentOps gives you the ability to monitor LLM calls, costs, latency, agent failures, multi-agent interactions, tool usage, session-wide statistics, and more. For more info, check out the [AgentOps Repo](https://github.com/AgentOps-AI/agentops).
+| | |
+| ------------------------------------- | ------------------------------------------------------------- |
+| π **Replay Analytics and Debugging** | Step-by-step agent execution graphs |
+| πΈ **LLM Cost Management** | Track spend with LLM foundation model providers |
+| π§ͺ **Agent Benchmarking** | Test your agents against 1,000+ evals |
+| π **Compliance and Security** | Detect common prompt injection and data exfiltration exploits |
+| π€ **Framework Integrations** | Native Integrations with CrewAI, AutoGen, & LangChain |
+
- Agent Dashboard
+ Agent Dashboard
-
+
- Session Analytics
+ Session Analytics
-
+
- Session Replays
+ Session Replays
-
+
@@ -38,7 +46,7 @@ pip install agentops
```
2. **Create an API Key:**
-Create a user API key here: [Create API Key](https://app.agentops.ai/account)
+Create a user API key here: [Create API Key](https://app.agentops.ai/settings/projects)
3. **Configure Your Environment:**
Add your API key to your environment variables
diff --git a/website/docs/topics/llm-observability.md b/website/docs/topics/llm-observability.md
index 6a95d185f979..f80b55ea0982 100644
--- a/website/docs/topics/llm-observability.md
+++ b/website/docs/topics/llm-observability.md
@@ -1,42 +1,37 @@
-# LLM Observability
+# Agent Observability
-AutoGen supports advanced LLM observability and monitoring through built-in logging and partner providers.
+AutoGen supports advanced LLM agent observability and monitoring through built-in logging and partner providers.
-## What is LLM Observability
-AI agent observability is the ability to monitor, measure, and understand the internal states and behaviors of AI agent systems.
-Observability is crucial for ensuring transparency, reliability, and accountability in your agent systems.
+## AutoGen Observability Integrations
+### Built-In Logging
+AutoGen's SQLite and File Logger - [Tutorial Notebook](/docs/notebooks/agentchat_logging)
-## Development
+### Full-Service Partner Integrations
+AutoGen partners with [AgentOps](https://agentops.ai) to provide multi-agent tracking, metrics, and monitoring - [Tutorial Notebook](/docs/notebooks/agentchat_agentops)
-### Agent Development in Terminal is Limited
-- Lose track of what your agents did in between executions
-- Parsing through terminal output searching for LLM completions
-- Printing βtool calledβ
-### Agent Development Dashboards Enable More
-- Visual dashboard so you can see what your agents did in human-readable format
-- LLM calls are magically recorded - prompt, completion, timestamps for each - with one line of code
-- Agents and their events (including tool calls) are recorded with one more line of code
-- Errors are magically associated to its causal event
-- Record any other events to your session with two more lines of code
-- Tons of other useful data if youβre developing with supported agent frameworks: SDK version
+## What is Observability?
+Observability provides developers with the necessary insights to understand and improve the internal workings of their agents. Observability is necessary for maintaining reliability, tracking costs, and ensuring AI safety.
-## Compliance
+**Without observability tools, developers face significant hurdles:**
-Observability and monitoring is critical to ensure AI agent systems adhere to laws and regulations in industries like finance and healthcare, preventing violations such as data breaches and privacy issues.
+- Tracking agent activities across sessions becomes a complex, error-prone task.
+- Manually sifting through verbose terminal outputs to understand LLM interactions is inefficient.
+- Pinpointing the exact moments of tool invocations is often like finding a needle in a haystack.
-- Insights into AI decision-making, allowing organizations to explain outcomes and build trust with stakeholders.
-- Helps detect anomalies and unintended behaviors early, mitigating operational, financial, and reputational risks.
-- Ensures compliance with data privacy regulations, preventing unauthorized access and misuse of sensitive information.
-- Quick identification and response to compliance violations, supporting incident analysis and prevention.
-## Available Observability Integrations
+**Key Features of Observability Dashboards:**
+- Human-readable overview analytics and replays of agent activities.
+- LLM cost, prompt, completion, timestamp, and metadata tracking for performance monitoring.
+- Tool invocation, events, and agent-to-agent interactions for workflow monitoring.
+- Error flagging and notifications for faster debugging.
+- Access to a wealth of data for developers using supported agent frameworks, such as environments, SDK versions, and more.
-### Logging
-- Autogen SQLite and File Logger - [Tutorial](/docs/notebooks/agentchat_logging)
+### Compliance
-### Full-Service Partners
-Autogen is currently partnered with [AgentOps](https://agentops.ai) for seamless observability integration.
-
-[Learn how to install AgentOps](/docs/notebooks/agentchat_agentops)
+Observability is not just a development convenienceβit's a compliance necessity, especially in regulated industries:
+- It offers insights into AI decision-making processes, fostering trust and transparency.
+- Anomalies and unintended behaviors are detected promptly, reducing various risks.
+- Ensuring adherence to data privacy regulations, thereby safeguarding sensitive information.
+- Compliance violations are quickly identified and addressed, enhancing incident management.