-
Notifications
You must be signed in to change notification settings - Fork 166
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
fix: replace ndjson.org with ndjson spec (#3997)
the original website expired and it's currently serving malicious content. (cherry picked from commit 698d4e0) # Conflicts: # docs/en/serverless/apm/apm-server-api/api-events.mdx
- Loading branch information
1 parent
7739638
commit 215d5f8
Showing
2 changed files
with
139 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,138 @@ | ||
|
||
|
||
<div id="api-events"></div> | ||
|
||
<DocCallOut title="Note"> | ||
Most users do not need to interact directly with the events intake API. | ||
</DocCallOut> | ||
|
||
The events intake API is what we call the internal protocol that APM agents use to talk to the managed intake service. | ||
Agents communicate with the Server by sending events — captured pieces of information — in an HTTP request. | ||
Events can be: | ||
|
||
* Transactions | ||
* Spans | ||
* Errors | ||
* Metrics | ||
|
||
Each event is sent as its own line in the HTTP request body. | ||
This is known as [newline delimited JSON (NDJSON)](https://github.com/ndjson/ndjson-spec). | ||
|
||
With NDJSON, agents can open an HTTP POST request and use chunked encoding to stream events to the managed intake service | ||
as soon as they are recorded in the agent. | ||
This makes it simple for agents to serialize each event to a stream of newline delimited JSON. | ||
The managed intake service also treats the HTTP body as a compressed stream and thus reads and handles each event independently. | ||
|
||
Refer to <DocLink slug="/serverless/observability/apm-data-types" /> to learn more about the different types of events. | ||
|
||
<div id="api-events-endpoint"></div> | ||
|
||
### Endpoints | ||
|
||
The managed intake service exposes the following endpoints for Elastic APM agent data intake: | ||
|
||
| Name | Endpoint | | ||
|---|---| | ||
| APM agent event intake | `/intake/v2/events` | | ||
|
||
{/* | RUM event intake (v2) | `/intake/v2/rum/events` | | ||
| RUM event intake (v3) | `/intake/v3/rum/events` | */} | ||
|
||
<div id="api-events-example"></div> | ||
|
||
### Request | ||
|
||
Send an `HTTP POST` request to the managed intake service `intake/v2/events` endpoint: | ||
|
||
```bash | ||
https://{hostname}:{port}/intake/v2/events | ||
``` | ||
|
||
The managed intake service supports asynchronous processing of batches. | ||
To request asynchronous processing the `async` query parameter can be set in the POST request | ||
to the `intake/v2/events` endpoint: | ||
|
||
```bash | ||
https://{hostname}:{port}/intake/v2/events?async=true | ||
``` | ||
|
||
<DocCallOut title="Note"> | ||
Since asynchronous processing defers some of the event processing to the | ||
background and takes place after the client has closed the request, some errors | ||
can't be communicated back to the client and are logged by the managed intake service. | ||
Furthermore, asynchronous processing requests will only be scheduled if the managed intake service can | ||
service the incoming request, requests that cannot be serviced will receive an internal error | ||
`503` "queue is full" error. | ||
</DocCallOut> | ||
|
||
{/* For <DocLink id="enApmGuideApmRum">RUM</DocLink> send an `HTTP POST` request to the managed intake service `intake/v3/rum/events` endpoint instead: | ||
```bash | ||
http(s)://{hostname}:{port}/intake/v3/rum/events | ||
``` */} | ||
|
||
<div id="api-events-response"></div> | ||
|
||
### Response | ||
|
||
On success, the server will respond with a 202 Accepted status code and no body. | ||
|
||
Keep in mind that events can succeed and fail independently of each other. Only if all events succeed does the server respond with a 202. | ||
|
||
<div id="api-events-errors"></div> | ||
|
||
### API Errors | ||
|
||
There are two types of errors that the managed intake service may return to an agent: | ||
|
||
* Event related errors (typically validation errors) | ||
* Non-event related errors | ||
|
||
The managed intake service processes events one after the other. | ||
If an error is encountered while processing an event, | ||
the error encountered as well as the document causing the error are added to an internal array. | ||
The managed intake service will only save 5 event related errors. | ||
If it encounters more than 5 event related errors, | ||
the additional errors will not be returned to agent. | ||
Once all events have been processed, | ||
the error response is sent. | ||
|
||
Some errors, not relating to specific events, | ||
may terminate the request immediately. | ||
For example: IP rate limit reached, wrong metadata, etc. | ||
If at any point one of these errors is encountered, | ||
it is added to the internal array and immediately returned. | ||
|
||
An example error response might look something like this: | ||
|
||
```json | ||
{ | ||
"errors": [ | ||
{ | ||
"message": "<json-schema-err>", [^1] | ||
"document": "<ndjson-obj>" [^2] | ||
},{ | ||
"message": "<json-schema-err>", | ||
"document": "<ndjson-obj>" | ||
},{ | ||
"message": "<json-decoding-err>", | ||
"document": "<ndjson-obj>" | ||
},{ | ||
"message": "too many requests" [^3] | ||
}, | ||
], | ||
"accepted": 2320 [^4] | ||
} | ||
``` | ||
[^1]: An event related error | ||
[^2]: The document causing the error | ||
[^3]: An immediately returning non-event related error | ||
[^4]: The number of accepted events | ||
|
||
If you're developing an agent, these errors can be useful for debugging. | ||
|
||
<div id="api-events-schema-definition"></div> | ||
|
||
### Event API Schemas | ||
|
||
The managed intake service uses a collection of JSON Schemas for validating requests to the intake API. |