Skip to content

Commit

Permalink
Bug 1918443 - Add pings-as-events to "real-time" documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
travis79 committed Sep 27, 2024
1 parent adfdc11 commit 456139d
Showing 1 changed file with 33 additions and 16 deletions.
49 changes: 33 additions & 16 deletions docs/user/user/howto/real-time-events/real-time-events.md
Original file line number Diff line number Diff line change
@@ -1,36 +1,53 @@
# "Real-Time" Events

## Defining "Real-Time" Glean Events
## Defining "real-time" events within the Glean SDK

"Real-Time" in the context of [Glean events](../../../reference/metrics/event.md) directly relates to configuring the Glean SDK to send each event in an ["events" ping](../../pings/events.md) as soon as it is recorded.
This minimizes the time between instrumentation and reporting, but doesn't describe how quickly received data is made available for querying.
For the purposes of the Glean SDK and its capabilities, "real-time" is limited to: minimizing the time between instrumentation and reporting.
It does not imply or describe how quickly received data is made available for querying.

## Configuring Glean For "Real-Time" Events
## Methods to achieve this with Glean

Glean "events" ping submission can either be configured at [initialization](../../../reference/general/initializing.md) or through [Server Knobs](../../../user/server-knobs/other/max-events.md).
### Option 1: Configuring Glean to send all events as soon as they are recorded

Setting the maximum event threshold to a value of `1` will configure the Glean SDK to submit an "events" ping for every event recorded.
Glean ["events" ping]((../../pings/events.md)) submission can be configured either during [initialization](../../../reference/general/initializing.md) or through [Server Knobs](../../../user/server-knobs/other/max-events.md).

Setting the maximum event threshold to a value of `1` will configure the Glean SDK to submit an "events" ping for each and every [event](../../../reference/metrics/event.md) as they
are recorded. By default, the Glean SDK will batch 500 events per "events" ping.

### Option 2: Using a custom ping and submitting it immediately ("Pings-as-Events")

If it isn't necessary to receive all Glean SDK events that are instrumented in an application in "real-time", it may be preferable to create a
[custom ping](../../pings/custom.md) which contains the relevant information to capture the context around the event and submit it as soon as
the application event occurs.

This has some additional advantages over using just an event in that custom pings are less restrictive than the extras attached to the event
in what data and Glean SDK metric types can be used.

If it is important to see the event that is being represented as a custom ping in context with other application events, then you only need to
define an event metric and use the `send_in_pings` parameter to send it in both the custom ping and the Glean built-in "events" ping. It can
then be seen in sequence and within context of all of the application events, and still be sent in "real-time" as needed.

## Considerations

### What "Real-Time" Glean Events Are _Not_
### What "real-time" Glean events/pings are _not_

Configuring the Glean SDK to submit events as soon as they are recorded does not mean to imply that the event data is available for analysis in
real time. There are networks to traverse, ingestion pipelines, etl, etc. that are all factors to keep in mind when considering how soon the data
is available for analysis purposes. This documentation only purports to cover configuring the Glean SDK to send the data in a real-time fashion and
does not make any assumptions about the analysis of data in real-time.
Configuring the Glean SDK to submit events as soon as they are recorded or using custom pings to submit data immediately does not mean that the
data is available for analysis in real time. There are networks to traverse, ingestion pipelines, etl, etc. that are all factors to keep in
mind when considering how soon the data is available for analysis purposes. This documentation only purports to cover configuring the Glean SDK
to send the data in a real-time fashion and does not make any assumptions about the analysis of data in real-time.

### More Network Requests
### More network requests

For every event recorded, a network request will be generated when the event is submitted for ingestion. By default, the Glean SDK batches up to
500 events per "events" ping, so this has the potential to generate up to 500 times as many network requests than the current default.
For every event recorded or custom ping submitted, a network request will be generated as the ping is submitted for ingestion. By default, the
Glean SDK batches up to 500 events per "events" ping, so this has the potential to generate up to 500 times as many network requests than the
current defaults for the Glean SDK "events" ping.

### More Ingestion Endpoint Traffic
### More ingestion endpoint traffic

As a result of the increased network requests, the ingestion endpoint will need to handle this additional traffic. This increases the load
of all the processing steps that are involved with ingesting event data from an application.

### Storage Space
### Storage space requirements

Typically the raw dataset for Glean events contains 1-500 events in a single row of the database. This row also includes metadata such as
information about the client application and the ping itself. With only a single event per "events" ping, the replication of this metadata
Expand Down

0 comments on commit 456139d

Please sign in to comment.