[Security Solution][Endpoint] API emulator developer utility (elastic…

…#182990)

## Summary

- Adds a new script and utility to run a standalone HTTP web server for
emulating external API calls (ex. APIs called by Connectors in support
for Bi-Directional response actions)
- Includes two plugins: SentinelOne and Crowdstrike. Only SentinelOne
has some working APIs (APIs that respond with payloads)
- Has not been tested with Kibana yet, but should be able to start it
and then use the URL for the desired plugin (SentinelOne or Crowdstrike)
to setup a Connector in Kibana
- See `README` file for more on this utility and associated framework

x-pack/plugins/security_solution/scripts/endpoint/api_emulator/README.md

@@ -0,0 +1,67 @@
+# API Emulator
+
+API Emulator is a framework wrapped around [Hapi](https://hapi.dev/) that enables developer to quickly create API interfaces for development and testing purposes. Emulator plugins (a wrapper around [Hapi plugins](https://hapi.dev/api/?v=21.3.3#plugins)) is the mechanism used to create an given set of APIs for emulation, and these are then added to the server framework which makes them available via the server's routes.  
+
+The following script can be used to start the External EDR Server Emulator from the command line:
+
+```shell
+node x-pack/plugins/security_solution/scripts/endpoint/start_external_edr_server_emulator.js
+```
+
+Use the `--help` option to view what arguments can be used
+
+For usages other than the command line, see the Development section below.
+
+
+
+## Development
+
+### Adding an new Plugin
+
+Plugins are the mechanism for adding API emulators into this framework. Each plugin is defined via an object that includes at a minimum the `name` and a `register()` callback. This callback for registering the plugin will be provided with an interface that allows the plugin to interact with the HTTP server and provide access to "core" services available at the server level for use by all plugins. 
+
+Example: A method that returns the definition for a plugin
+
+```typescript
+
+export const getFooPluginRegistration = () => {
+  return {
+    name: 'foo',                // [1]
+    register(server) {
+      // register routes
+      server.router.route({
+        path: '/api/get',       // [2]
+        method: 'GET',
+        handler: async (req, h) => {
+          return 'alive!';
+        }
+      })
+    }
+  }
+}
+```
+
+In the above example:
+
+1. a plugin with the name `foo` [1] will be registered. The name of the plugin will also be the default `prefix` to all API routes (an optional attributed named `prefix` is also available if wanting to use a different value for the namespacing the routes).
+2. the `register()` callback will be given a `server` argument that provides access to server level services like the HTTP `router`
+3. a new route is registered [2], which will be mounted at `/foo/api/get` - note the use of the plugin name as the route prefix
+
+
+#### Plugin HTTP routes
+
+HTTP route handlers work very similar to the route handlers in Kibana today. You are given a `Request` and a Response Factory by the Hapi framework - see the [HAPI docs on Lifecycle Methods](https://hapi.dev/api/?v=21.3.3#lifecycle-methods) for more details.
+
+This emulator framework will expose the core services (ex. for the EDR server emulator, this would include Kibana and Elasticsearch clients) to each route under `request.pre` (pre-handler methods).
+
+Example: a route handler under the EDR server emulator that returns the version of kibana
+
+```typescript
+
+const handler = async (req, h) => {
+  const kbnStatus = await req.pre.services.kbnClient.status.get();
+
+  return kbnStatus.version.number;
+}
+
+```

x-pack/plugins/security_solution/scripts/endpoint/api_emulator/emulator_plugins/crowdstrike/index.ts

@@ -0,0 +1,25 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+import type { EmulatorServerPlugin } from '../../lib/emulator_server.types';
+
+export const getCrowdstrikeEmulator = () => {
+  const plugin: EmulatorServerPlugin = {
+    name: 'crowdstrike',
+    register({ router }) {
+      router.route({
+        path: '/',
+        method: 'GET',
+        handler: () => {
+          return { message: `Live! But not implemented` };
+        },
+      });
+    },
+  };
+
+  return plugin;
+};

x-pack/plugins/security_solution/scripts/endpoint/api_emulator/emulator_plugins/sentinelone/index.ts

@@ -0,0 +1,27 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+import type { ExternalEdrServerEmulatorCoreServices } from '../..';
+import { getSentinelOneRouteDefinitions } from './routes';
+import type { EmulatorServerPlugin } from '../../lib/emulator_server.types';
+
+export const getSentinelOneEmulator =
+  (): EmulatorServerPlugin<ExternalEdrServerEmulatorCoreServices> => {
+    const plugin: EmulatorServerPlugin<ExternalEdrServerEmulatorCoreServices> = {
+      name: 'sentinelone',
+      register({ router, expose, services }) {
+        router.route(getSentinelOneRouteDefinitions());
+
+        // TODO:PT define the interface for programmatically interact with sentinelone api emulator
+        expose('setResponse', () => {
+          services.logger.info('setResponse() is available');
+        });
+      },
+    };
+
+    return plugin;
+  };

x-pack/plugins/security_solution/scripts/endpoint/api_emulator/emulator_plugins/sentinelone/routes/activities_route.ts

@@ -0,0 +1,44 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+import type {
+  SentinelOneActivityRecord,
+  SentinelOneGetActivitiesParams,
+} from '@kbn/stack-connectors-plugin/common/sentinelone/types';
+import type { DeepPartial, Mutable } from 'utility-types';
+import { SentinelOneDataGenerator } from '../../../../../../common/endpoint/data_generators/sentinelone_data_generator';
+import { buildSentinelOneRoutePath } from './utils';
+import type { ExternalEdrServerEmulatorRouteHandlerMethod } from '../../../external_edr_server_emulator.types';
+import type { EmulatorServerRouteDefinition } from '../../../lib/emulator_server.types';
+
+const generator = new SentinelOneDataGenerator();
+
+export const getActivitiesRouteDefinition = (): EmulatorServerRouteDefinition => {
+  return {
+    path: buildSentinelOneRoutePath('/activities'),
+    method: 'GET',
+    handler: activitiesRouteHandler,
+  };
+};
+
+const activitiesRouteHandler: ExternalEdrServerEmulatorRouteHandlerMethod<
+  {},
+  NonNullable<SentinelOneGetActivitiesParams>
+> = async (request) => {
+  const queryParams = request.query;
+  const activityOverrides: DeepPartial<Mutable<SentinelOneActivityRecord>> = {};
+
+  if (queryParams?.activityTypes) {
+    activityOverrides.activityType = Number(queryParams.activityTypes.split(',').at(0));
+  }
+
+  if (queryParams?.agentIds) {
+    activityOverrides.agentId = queryParams.agentIds.split(',').at(0);
+  }
+
+  return generator.generateSentinelOneApiActivityResponse(activityOverrides);
+};

x-pack/plugins/security_solution/scripts/endpoint/api_emulator/emulator_plugins/sentinelone/routes/agent_action_connect_route.ts

@@ -0,0 +1,34 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+import type { ExternalEdrServerEmulatorRouteHandlerMethod } from '../../..';
+import { buildSentinelOneRoutePath } from './utils';
+import type { EmulatorServerRouteDefinition } from '../../../lib/emulator_server.types';
+
+export const getAgentActionConnectRouteDefinition = (): EmulatorServerRouteDefinition => {
+  return {
+    path: buildSentinelOneRoutePath('/agents/actions/connect'),
+    method: 'POST',
+    handler: connectActionRouteHandler,
+  };
+};
+
+const connectActionRouteHandler: ExternalEdrServerEmulatorRouteHandlerMethod<
+  {},
+  {},
+  {
+    filter: {
+      ids: string;
+    };
+  }
+> = async (request) => {
+  return {
+    data: {
+      affected: request.payload.filter.ids.split(',').length,
+    },
+  };
+};

x-pack/plugins/security_solution/scripts/endpoint/api_emulator/emulator_plugins/sentinelone/routes/agent_action_disconnect_route.ts

@@ -0,0 +1,34 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+import type { ExternalEdrServerEmulatorRouteHandlerMethod } from '../../..';
+import { buildSentinelOneRoutePath } from './utils';
+import type { EmulatorServerRouteDefinition } from '../../../lib/emulator_server.types';
+
+export const getAgentActionDisconnectRouteDefinition = (): EmulatorServerRouteDefinition => {
+  return {
+    path: buildSentinelOneRoutePath('/agents/actions/disconnect'),
+    method: 'POST',
+    handler: disconnectActionRouteHandler,
+  };
+};
+
+const disconnectActionRouteHandler: ExternalEdrServerEmulatorRouteHandlerMethod<
+  {},
+  {},
+  {
+    filter: {
+      ids: string;
+    };
+  }
+> = async (request) => {
+  return {
+    data: {
+      affected: request.payload.filter.ids.split(',').length,
+    },
+  };
+};

x-pack/plugins/security_solution/scripts/endpoint/api_emulator/emulator_plugins/sentinelone/routes/agents_route.ts

@@ -0,0 +1,44 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+import type {
+  SentinelOneGetAgentsParams,
+  SentinelOneGetAgentsResponse,
+} from '@kbn/stack-connectors-plugin/common/sentinelone/types';
+import type { DeepPartial, Mutable } from 'utility-types';
+import { SentinelOneDataGenerator } from '../../../../../../common/endpoint/data_generators/sentinelone_data_generator';
+import { buildSentinelOneRoutePath } from './utils';
+import type { ExternalEdrServerEmulatorRouteHandlerMethod } from '../../../external_edr_server_emulator.types';
+import type { EmulatorServerRouteDefinition } from '../../../lib/emulator_server.types';
+
+const generator = new SentinelOneDataGenerator();
+
+export const getAgentsRouteDefinition = (): EmulatorServerRouteDefinition => {
+  return {
+    path: buildSentinelOneRoutePath('/agents'),
+    method: 'GET',
+    handler: agentsRouteHandler,
+  };
+};
+
+const agentsRouteHandler: ExternalEdrServerEmulatorRouteHandlerMethod<
+  {},
+  SentinelOneGetAgentsParams
+> = async (request) => {
+  const queryParams = request.query;
+  const agent: Mutable<DeepPartial<SentinelOneGetAgentsResponse['data'][number]>> = {};
+
+  if (queryParams.uuid) {
+    agent.uuid = queryParams.uuid;
+  }
+
+  if (queryParams.ids) {
+    agent.id = queryParams.ids.split(',').at(0);
+  }
+
+  return generator.generateSentinelOneApiAgentsResponse(agent);
+};

x-pack/plugins/security_solution/scripts/endpoint/api_emulator/emulator_plugins/sentinelone/routes/index.ts

@@ -0,0 +1,21 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+import { getAgentActionConnectRouteDefinition } from './agent_action_connect_route';
+import { getAgentActionDisconnectRouteDefinition } from './agent_action_disconnect_route';
+import { getActivitiesRouteDefinition } from './activities_route';
+import { getAgentsRouteDefinition } from './agents_route';
+import type { EmulatorServerRouteDefinition } from '../../../lib/emulator_server.types';
+
+export const getSentinelOneRouteDefinitions = (): EmulatorServerRouteDefinition[] => {
+  return [
+    getAgentsRouteDefinition(),
+    getActivitiesRouteDefinition(),
+    getAgentActionConnectRouteDefinition(),
+    getAgentActionDisconnectRouteDefinition(),
+  ];
+};

x-pack/plugins/security_solution/scripts/endpoint/api_emulator/emulator_plugins/sentinelone/routes/utils.ts

@@ -0,0 +1,18 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+// The base API path for the API requests for sentinelone. Value is the same as the one defined here:
+// `x-pack/plugins/stack_connectors/server/connector_types/sentinelone/sentinelone.ts:50`
+const BASE_API_PATH = '/web/api/v2.1';
+
+export const buildSentinelOneRoutePath = (path: string): string => {
+  if (!path.startsWith('/')) {
+    throw new Error(`'path' must start with '/'!`);
+  }
+
+  return `${BASE_API_PATH}${path}`;
+};

x-pack/plugins/security_solution/scripts/endpoint/api_emulator/external_edr_server_emulator.ts

@@ -0,0 +1,65 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+import { getCrowdstrikeEmulator } from './emulator_plugins/crowdstrike';
+import { handleProcessInterruptions } from '../common/nodejs_utils';
+import { EmulatorServer } from './lib/emulator_server';
+import type { ExternalEdrServerEmulatorCoreServices } from './external_edr_server_emulator.types';
+import { getSentinelOneEmulator } from './emulator_plugins/sentinelone';
+
+export interface StartExternalEdrServerEmulatorOptions {
+  coreServices: ExternalEdrServerEmulatorCoreServices;
+  /**
+   * The port where the server should listen on. Default is `0` which means an available port is
+   * auto-assigned.
+   */
+  port?: number;
+}
+
+/**
+ * Starts a server that provides API emulator for external EDR systems in support of bi-directional
+ * response actions.
+ *
+ * After staring the server, the `emulatorServer.stopped` property provides a way to `await` until it
+ * is stopped
+ *
+ * @param options
+ */
+export const startExternalEdrServerEmulator = async ({
+  port,
+  coreServices,
+}: StartExternalEdrServerEmulatorOptions): Promise<EmulatorServer> => {
+  const emulator = new EmulatorServer<ExternalEdrServerEmulatorCoreServices>({
+    logger: coreServices.logger,
+    port: port ?? 0,
+    services: coreServices,
+  });
+
+  // Register all emulators
+  await emulator.register(getSentinelOneEmulator());
+  await emulator.register(getCrowdstrikeEmulator());
+
+  let wasStartedPromise: ReturnType<EmulatorServer['start']>;
+
+  handleProcessInterruptions(
+    async () => {
+      wasStartedPromise = emulator.start();
+      await wasStartedPromise;
+      await emulator.stopped;
+    },
+    () => {
+      coreServices.logger.warning(
+        `Process was interrupted. Shutting down External EDR Server Emulator`
+      );
+      emulator.stop();
+    }
+  );
+
+  // @ts-expect-error TS2454: Variable 'wasStartedPromise' is used before being assigned.
+  await wasStartedPromise;
+  return emulator;
+};