Port PR 4488 to Serverless - adding k8s-otel quickstart guide (#4589)

* k8s-otel quickstart added to serverless
* minor fixes in stateful

(cherry picked from commit 58c98f8)

# Conflicts:
#	docs/en/serverless/index.asciidoc
#	docs/en/serverless/observability-get-started.asciidoc

docs/en/observability/observability-get-started.asciidoc

@@ -54,7 +54,8 @@ Follow the steps in these guides to get started quickly:
 
 * <<quickstart-monitor-hosts-with-elastic-agent>>
 * <<monitor-k8s-logs-metrics-with-elastic-agent>>
-
+* <<monitor-k8s-otel-edot>>
+* <<collect-data-with-aws-firehose>>
 
 [discrete]
 === Get started with other features

docs/en/observability/quickstarts/monitor-k8s-otel.asciidoc

@@ -60,7 +60,7 @@ image::images/quickstart-k8s-otel-dashboard.png[Kubernetes overview dashboard]
 [discrete]
 == Troubleshooting and more
 
-* For troubleshooting deployment and installation, refer to https://github.com/elastic/opentelemetry/tree/main/docs/kubernetes/operator#installation-verification[installation verification].
+* To troubleshoot deployment and installation, refer to https://github.com/elastic/opentelemetry/tree/main/docs/kubernetes/operator#installation-verification[installation verification].
 * For application instrumentation details, refer to https://github.com/elastic/opentelemetry/blob/main/docs/kubernetes/operator/instrumenting-applications.md[Instrumenting applications with EDOT SDKs on Kubernetes].
-* For customizing the configuration, refer to https://github.com/elastic/opentelemetry/tree/main/docs/kubernetes/operator#custom-configuration[custom configuration].
+* To customize the configuration, refer to https://github.com/elastic/opentelemetry/tree/main/docs/kubernetes/operator#custom-configuration[custom configuration].
 * Refer to <<observability-introduction>> for a description of other useful features.

docs/en/serverless/index.asciidoc

@@ -0,0 +1,226 @@
+include::{asciidoc-dir}/../../shared/versions/stack/current.asciidoc[]
+include::{asciidoc-dir}/../../shared/attributes.asciidoc[]
+
+[[what-is-observability-serverless]]
+== Elastic Observability serverless
+
+++++
+<titleabbrev>Elastic Observability</titleabbrev>
+++++
+
+include::./what-is-observability-serverless.asciidoc[leveloffset=+2]
+
+// Group: Get started with Elastic Observability
+include::observability-get-started.asciidoc[leveloffset=+2]
+
+// What is Observability?
+include::./observability-overview.asciidoc[leveloffset=+3]
+
+// Observability billing dimensions
+include::./projects/billing.asciidoc[leveloffset=+3]
+
+// Create an Elastic Observability Serverless project
+include::./projects/create-an-observability-project.asciidoc[leveloffset=+3]
+
+// Quickstarts
+include::./quickstarts/monitor-hosts-with-elastic-agent.asciidoc[leveloffset=+3]
+include::./quickstarts/k8s-logs-metrics.asciidoc[leveloffset=+3]
+include::./quickstarts/monitor-k8s-otel.asciidoc[leveloffset=+3]
+include::./quickstarts/collect-data-with-aws-firehose.asciidoc[leveloffset=+3]
+
+// Dashboards
+include::./dashboards/dashboards-and-visualizations.asciidoc[leveloffset=+3]
+
+// Group: Application and service monitoring
+include::./application-and-service-monitoring.asciidoc[leveloffset=+2]
+
+// APM
+include::./apm/apm.asciidoc[leveloffset=+3]
+include::./apm/apm-get-started.asciidoc[leveloffset=+4]
+include::./apm/apm-send-traces-to-elastic.asciidoc[leveloffset=+4]
+include::./apm-agents/apm-agents-elastic-apm-agents.asciidoc[leveloffset=+5]
+include::./apm-agents/apm-agents-opentelemetry.asciidoc[leveloffset=+5]
+include::./apm-agents/apm-agents-opentelemetry-opentelemetry-native-support.asciidoc[leveloffset=+6]
+include::./apm-agents/apm-agents-opentelemetry-collect-metrics.asciidoc[leveloffset=+6]
+include::./apm-agents/apm-agents-opentelemetry-limitations.asciidoc[leveloffset=+6]
+include::./apm-agents/apm-agents-opentelemetry-resource-attributes.asciidoc[leveloffset=+6]
+include::./apm-agents/apm-agents-aws-lambda-functions.asciidoc[leveloffset=+5]
+include::./apm/apm-view-and-analyze-traces.asciidoc[leveloffset=+4]
+include::./apm/apm-find-transaction-latency-and-failure-correlations.asciidoc[leveloffset=+5]
+include::./apm/apm-integrate-with-machine-learning.asciidoc[leveloffset=+5]
+include::./apm/apm-create-custom-links.asciidoc[leveloffset=+5]
+include::./apm/apm-track-deployments-with-annotations.asciidoc[leveloffset=+5]
+include::./apm/apm-query-your-data.asciidoc[leveloffset=+5]
+include::./apm/apm-filter-your-data.asciidoc[leveloffset=+5]
+include::./apm/apm-observe-lambda-functions.asciidoc[leveloffset=+5]
+include::./apm/apm-ui-overview.asciidoc[leveloffset=+5]
+include::./apm/apm-ui-services.asciidoc[leveloffset=+6]
+include::./apm/apm-ui-traces.asciidoc[leveloffset=+6]
+include::./apm/apm-ui-dependencies.asciidoc[leveloffset=+6]
+include::./apm/apm-ui-service-map.asciidoc[leveloffset=+6]
+include::./apm/apm-ui-service-overview.asciidoc[leveloffset=+6]
+include::./apm/apm-ui-transactions.asciidoc[leveloffset=+6]
+include::./apm/apm-ui-trace-sample-timeline.asciidoc[leveloffset=+6]
+include::./apm/apm-ui-errors.asciidoc[leveloffset=+6]
+include::./apm/apm-ui-metrics.asciidoc[leveloffset=+6]
+include::./apm/apm-ui-infrastructure.asciidoc[leveloffset=+6]
+include::./apm/apm-ui-logs.asciidoc[leveloffset=+6]
+include::./apm/apm-data-types.asciidoc[leveloffset=+4]
+include::./apm/apm-distributed-tracing.asciidoc[leveloffset=+4]
+include::./apm/apm-reduce-your-data-usage.asciidoc[leveloffset=+4]
+include::./apm/apm-transaction-sampling.asciidoc[leveloffset=+5]
+include::./apm/apm-compress-spans.asciidoc[leveloffset=+5]
+include::./apm/apm-stacktrace-collection.asciidoc[leveloffset=+5]
+include::./apm/apm-keep-data-secure.asciidoc[leveloffset=+4]
+include::./apm/apm-troubleshooting.asciidoc[leveloffset=+4]
+include::./apm/apm-reference.asciidoc[leveloffset=+4]
+include::./apm/apm-kibana-settings.asciidoc[leveloffset=+5]
+include::./apm/apm-server-api.asciidoc[leveloffset=+5]
+
+// Synthetics
+include::./synthetics/synthetics-intro.asciidoc[leveloffset=+3]
+
+include::./synthetics/synthetics-get-started.asciidoc[leveloffset=+4]
+include::./synthetics/synthetics-get-started-project.asciidoc[leveloffset=+5]
+include::./synthetics/synthetics-get-started-ui.asciidoc[leveloffset=+5]
+
+include::./synthetics/synthetics-journeys.asciidoc[leveloffset=+4]
+include::./synthetics/synthetics-create-test.asciidoc[leveloffset=+5]
+include::./synthetics/synthetics-monitor-use.asciidoc[leveloffset=+5]
+include::./synthetics/synthetics-recorder.asciidoc[leveloffset=+5]
+
+include::./synthetics/synthetics-lightweight.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-manage-monitors.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-params-secrets.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-analyze.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-private-location.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-command-reference.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-configuration.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-mfa.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-settings.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-feature-roles.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-manage-retention.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-scale-and-architect.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-security-encryption.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-troubleshooting.asciidoc[leveloffset=+4]
+
+// Group: Infrastructure and hosts
+include::./infrastructure-and-host-monitoring-intro.asciidoc[leveloffset=+2]
+
+include::./infra-monitoring/infra-monitoring.asciidoc[leveloffset=+3]
+include::./infra-monitoring/get-started-with-metrics.asciidoc[leveloffset=+4]
+include::./infra-monitoring/view-infrastructure-metrics.asciidoc[leveloffset=4]
+include::./infra-monitoring/analyze-hosts.asciidoc[leveloffset=+4]
+include::./infra-monitoring/detect-metric-anomalies.asciidoc[leveloffset=+4]
+include::./infra-monitoring/configure-infra-settings.asciidoc[leveloffset=+4]
+
+include::./infra-monitoring/troubleshooting-infra.asciidoc[leveloffset=+3]
+include::./infra-monitoring/handle-no-results-found-message.asciidoc[leveloffset=+4]
+
+include::./infra-monitoring/metrics-reference.asciidoc[leveloffset=+3]
+include::./infra-monitoring/host-metrics.asciidoc[leveloffset=+4]
+include::./infra-monitoring/container-metrics.asciidoc[leveloffset=+4]
+include::./infra-monitoring/kubernetes-pod-metrics.asciidoc[leveloffset=+4]
+include::./infra-monitoring/aws-metrics.asciidoc[leveloffset=+4]
+
+// Group: Logs
+// TODO: Check the diff to see which changes need to be applied to these files. Also ask Mike if it's expected for serverless and stateful to be out of sync.
+
+include::./logging/log-monitoring.asciidoc[leveloffset=+2]
+
+include::./logging/get-started-with-logs.asciidoc[leveloffset=+3]
+include::./logging/stream-log-files.asciidoc[leveloffset=+3]
+include::./logging/correlate-application-logs.asciidoc[leveloffset=+3]
+include::./logging/plaintext-application-logs.asciidoc[leveloffset=+4]
+include::./logging/ecs-application-logs.asciidoc[leveloffset=+4]
+include::./logging/send-application-logs.asciidoc[leveloffset=+4]
+include::./logging/parse-log-data.asciidoc[leveloffset=+3]
+include::./logging/filter-and-aggregate-logs.asciidoc[leveloffset=+3]
+include::./logging/view-and-monitor-logs.asciidoc[leveloffset=+3]
+include::./logging/add-logs-service-name.asciidoc[leveloffset=+3]
+include::./logging/run-log-pattern-analysis.asciidoc[leveloffset=+3]
+include::./logging/troubleshoot-logs.asciidoc[leveloffset=+3]
+
+//TODO: Figure out where to put this. It's under "view and analyze data" in stateful, but that category doesn't exist in serverless yet.
+include::./inventory.asciidoc[leveloffset=+2]
+
+// Group: Incident management
+include::./incident-management.asciidoc[leveloffset=+2]
+
+// Alerting
+include::./alerting/alerting.asciidoc[leveloffset=+3]
+include::./alerting/create-manage-rules.asciidoc[leveloffset=+4]
+include::./alerting/aiops-generate-anomaly-alerts.asciidoc[leveloffset=+5]
+include::./alerting/create-anomaly-alert-rule.asciidoc[leveloffset=+5]
+include::./alerting/create-custom-threshold-alert-rule.asciidoc[leveloffset=+5]
+include::./alerting/create-elasticsearch-query-alert-rule.asciidoc[leveloffset=+5]
+include::./alerting/create-error-count-threshold-alert-rule.asciidoc[leveloffset=+5]
+include::./alerting/create-failed-transaction-rate-threshold-alert-rule.asciidoc[leveloffset=+5]
+include::./alerting/create-inventory-threshold-alert-rule.asciidoc[leveloffset=+5]
+include::./alerting/create-latency-threshold-alert-rule.asciidoc[leveloffset=+5]
+include::./alerting/create-slo-burn-rate-alert-rule.asciidoc[leveloffset=+5]
+include::./alerting/synthetic-monitor-status-alert.asciidoc[leveloffset=+5]
+include::./alerting/aggregation-options.asciidoc[leveloffset=+4]
+include::./alerting/rate-aggregation.asciidoc[leveloffset=+5]
+include::./alerting/view-alerts.asciidoc[leveloffset=+4]
+include::./alerting/triage-slo-burn-rate-breaches.asciidoc[leveloffset=+5]
+include::./alerting/triage-threshold-breaches.asciidoc[leveloffset=+5]
+
+// Cases
+include::./cases/cases.asciidoc[leveloffset=+3]
+
+include::./cases/create-manage-cases.asciidoc[leveloffset=+4]
+
+include::./cases/manage-cases-settings.asciidoc[leveloffset=+4]
+
+//SLOs
+include::./slos/slos.asciidoc[leveloffset=+3]
+
+include::./slos/create-an-slo.asciidoc[leveloffset=+4]
+
+//Data Set Quality
+include::./monitor-datasets.asciidoc[leveloffset=+2]
+
+//Observability AI Assistant
+include::./ai-assistant/ai-assistant.asciidoc[leveloffset=+2]
+
+//Machine learning
+
+include::./machine-learning/machine-learning.asciidoc[leveloffset=+2]
+include::./machine-learning/aiops-detect-anomalies.asciidoc[leveloffset=+3]
+include::./machine-learning/aiops-tune-anomaly-detection-job.asciidoc[leveloffset=+4]
+include::./machine-learning/aiops-forecast-anomaly.asciidoc[leveloffset=+4]
+include::./machine-learning/aiops-analyze-spikes.asciidoc[leveloffset=+3]
+include::./machine-learning/aiops-detect-change-points.asciidoc[leveloffset=+3]
+
+// Reference group
+
+include::./reference.asciidoc[leveloffset=+2]
+
+// Fields
+
+include::./reference/metrics-app-fields.asciidoc[leveloffset=+3]
+
+// Elastic Entity Model
+
+include::./reference/elastic-entity-model.asciidoc[leveloffset=+3]
+
+// Technical preview limitations
+
+include::./limitations.asciidoc[leveloffset=+2]
+
+// add redirects file
+include::redirects.asciidoc[]

docs/en/serverless/observability-get-started.asciidoc

@@ -0,0 +1,79 @@
+[[observability-get-started]]
+= Get started with Elastic Observability
+
+++++
+<titleabbrev>Get started</titleabbrev>
+++++
+
+New to Elastic {observability}? Discover more about our observability features and how to get started.
+
+[discrete]
+== Learn about Elastic {observability}
+
+Learn about key features available to help you get value from your observability data and what it will cost you:
+
+* <<observability-serverless-observability-overview>>
+* <<observability-billing>>
+
+[discrete]
+[[get-started-with-use-case]]
+== Get started with your use case
+
+Learn how to create an Observability project and use Elastic
+Observability to gain deeper insight into the behavior of your applications and
+systems.
+
+image::images/get-started.svg[]
+
+1. **Choose your source.** Elastic integrates with hundreds of data sources for
+unified visibility across all your applications and systems.
+
+2. **Ingest your data.** Turn-key integrations provide a repeatable workflow to
+ingest data from all your sources: you install an integration, configure it, and
+deploy an agent to collect your data.
+
+3. **View your data.** Navigate seamlessly between Observabilty UIs and
+dashboards to identify and resolve problems quickly.
+
+4. **Customize.** Expand your deployment and add features like alerting and anomaly
+detection.
+
+To get started, <<observability-create-an-observability-project,create an Observability project>>,
+then follow one of our <<quickstarts-overview,quickstarts>> to learn how to ingest and visualize your observability data.
+
+[discrete]
+[[quickstarts-overview]]
+=== Quickstarts
+
+Our quickstarts dramatically reduce your time-to-value by offering a fast path to ingest and visualize your Observability data.
+Each quickstart provides:
+
+* A highly opinionated, fast path to data ingestion
+* Sensible configuration defaults with minimal configuration required
+* Auto-detection of logs and metrics for monitoring hosts
+* Quick access to related dashboards and visualizations
+
+Follow the steps in these guides to get started quickly:
+
+* <<observability-quickstarts-monitor-hosts-with-elastic-agent>>
+* <<observability-quickstarts-k8s-logs-metrics>>
+* <<monitor-k8s-otel-edot>>
+* <<collect-data-with-aws-firehose>>
+
+[discrete]
+=== Get started with other features
+
+Want to use {fleet} or some other feature not covered in the quickstarts?
+Follow the steps in these guides to get started:
+
+* <<observability-get-started-with-logs,Get started with system metrics>>
+* <<observability-apm-get-started,Get started with application traces and APM>>
+* <<observability-synthetics-get-started,Get started with synthetic monitoring>>
+
+[discrete]
+== Additional guides
+
+Ready to dig into more features of Elastic Observability? See these guides:
+
+* <<observability-alerting>>
+* <<observability-slos>>

docs/en/serverless/quickstarts/monitor-k8s-otel.asciidoc

@@ -0,0 +1,66 @@
+[[monitor-k8s-otel-edot]]
+= Quickstart: Unified Kubernetes Observability with Elastic Distributions of OpenTelemetry (EDOT)
+
+preview::[]
+
+In this quickstart guide, you'll learn how to send Kubernetes logs, metrics, and application traces to Elasticsearch, using the https://github.com/open-telemetry/opentelemetry-operator/[OpenTelemetry Operator] to orchestrate https://github.com/elastic/opentelemetry/tree/main[Elastic Distributions of OpenTelemetry] (EDOT) Collectors and SDK instances.
+
+All the components will be deployed through the https://github.com/open-telemetry/opentelemetry-helm-charts/tree/main/charts/opentelemetry-kube-stack[opentelemetry-kube-stack] helm chart. They include:
+
+* https://github.com/open-telemetry/opentelemetry-operator/[OpenTelemetry Operator].
+* `DaemonSet` EDOT Collector configured for node level metrics.
+* `Deployment` EDOT Collector configured for cluster level metrics.
+* `Instrumentation` object for applications https://opentelemetry.io/docs/kubernetes/operator/automatic/[auto-instrumentation].
+
+For a more detailed description of the components and advanced configuration, refer to the https://github.com/elastic/opentelemetry/blob/main/docs/kubernetes/operator/README.md[elastic/opentelemetry] GitHub repository.
+
+[discrete]
+== Prerequisites
+
+* An {obs-serverless} project. To learn more, refer to <<observability-create-an-observability-project>>.
+* A running Kubernetes cluster (v1.23 or newer).
+* https://kubernetes.io/docs/reference/kubectl/[Kubectl].
+* https://helm.sh/docs/intro/install/[Helm].
+* (optional) https://cert-manager.io/docs/installation/[Cert-manager], if you opt for automatic generation and renewal of TLS certificates.
+
+[discrete]
+== Collect your data
+
+. <<observability-create-an-observability-project,Create a new {obs-serverless} project>>, or open an existing one.
+. In your {obs-serverless} project, go to **Add Data**.
+. Under **What do you want to monitor?** select **Kubernetes**, and then select **OpenTelemetry: Full Observability**.
++
+[role="screenshot"]
+image::images/quickstart-k8s-otel-entry-point.png[Kubernetes-OTel entry point]
+
+. Follow the on-screen instructions to install all needed components.
++
+[NOTE]
+====
+The default installation deploys the OpenTelemetry Operator with a self-signed TLS certificate valid for 365 days. This certificate **won't be renewed** unless the Helm Chart release is manually updated. Refer to the https://github.com/elastic/opentelemetry/blob/main/docs/kubernetes/operator/README.md#cert-manager[cert-manager integrated installation] guide to enable automatic certificate generation and renewal using https://cert-manager.io/docs/installation/[cert-manager].
+====
++
+Deploy the OpenTelemetry Operator and EDOT Collectors using the kube-stack Helm chart with the provided `values.yaml` file. You will run a few commands to:
++ 
+ * Add the helm chart repository needed for the installation.
+ * Create a namespace.
+ * Create a secret with an API Key and the {es} endpoint to be used by the collectors.
+ * Install the `opentelemetry-kube-stack` helm chart with the provided `values.yaml`.
+ * Optionally, for instrumenting applications, apply the corresponding `annotations` as shown in {kib}.
+
+[discrete]
+== Visualize your data
+
+After installation is complete and all relevant data is flowing into Elastic,
+the **Visualize your data** section provides a link to the *[OTEL][Metrics Kubernetes]Cluster Overview* dashboard used to monitor the health of the cluster.
+
+[role="screenshot"]
+image::images/quickstart-k8s-otel-dashboard.png[Kubernetes overview dashboard]
+
+[discrete]
+== Troubleshooting and more
+
+* To troubleshoot deployment and installation, refer to https://github.com/elastic/opentelemetry/tree/main/docs/kubernetes/operator#installation-verification[installation verification].
+* For application instrumentation details, refer to https://github.com/elastic/opentelemetry/blob/main/docs/kubernetes/operator/instrumenting-applications.md[Instrumenting applications with EDOT SDKs on Kubernetes].
+* To customize the configuration, refer to https://github.com/elastic/opentelemetry/tree/main/docs/kubernetes/operator#custom-configuration[custom configuration].
+* Refer to <<observability-serverless-observability-overview>> for a description of other useful features.