[Detection Engine][FTR] Add prebuilt role tests for rule execution

[yctercero]

Summary

Adds FTR for serverless prebuilt roles around rule execution. Uses the preview API to speed up the tests.

[yctercero]

@approksiu @caitlinbetz could you confirm the behaviors here? All but the Endpoint Operations Analyst seem to match. The sheet says they're read on rules, but seems they can execute them.

[elasticmachine]

Pinging @elastic/security-detection-engine (Team:Detection Engine)

[jbudz]

.buildkite/ftr_security_serverless_configs.yml

[yctercero]

/ci

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: 1289092

Failed CI Steps

• Jest Tests #16

Test Failures

• [job] [logs] Jest Tests #16 / FileDownloadButton not isIcon renders download button with correct href

Metrics [docs]

✅ unchanged

History

• 💔 Build #254757 failed d477f76
• 💔 Build #254749 failed d477f76
• 💛 Build #253821 was flaky a52a396

cc @yctercero

[dhurley14]

This looks good thank you for adding test suites for this functionality on serverless! This follows the similar pattern for the value list testing with roles. After thinking about this some more I am wondering if we can speed up the test suites a little. I think by iterating over the roles with the same outcome for a given test we remove having to do all the build up and tear down that we would have to do if we used individual test suites for each individual role.

So instead of having a test suite for each role that tests the preview api, we would instead have a singular test suite for the preview api and iterate over all the roles within a singular test item in that suite. For example the following could live within a preview_rule_api test file and look something like this:

/*
 * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
 * or more contributor license agreements. Licensed under the Elastic License
 * 2.0; you may not use this file except in compliance with the Elastic License
 * 2.0.
 */

import TestAgent from 'supertest/lib/agent';
import expect from '@kbn/expect';

import {
  QueryRuleCreateProps,
  RulePreviewRequestBody,
} from '@kbn/security-solution-plugin/common/api/detection_engine';
import { DETECTION_ENGINE_RULES_PREVIEW } from '@kbn/security-solution-plugin/common/constants';
import { FtrProviderContext } from '../../../../../../../ftr_provider_context';
import { EsArchivePathBuilder } from '../../../../../../../es_archive_path_builder';
import {
  deleteAllRules,
  deleteAllAlerts,
  getRuleForAlertTesting,
} from '../../../../../../../../common/utils/security_solution';

/**
 * Specific _id to use for some of the tests. If the archiver changes and you see errors
 * here, update this to a new value of a chosen auditbeat record and update the tests values.
 */
const ID = 'BhbXBmkBR346wHgn4PeZ';

export default ({ getService }: FtrProviderContext): void => {
  const log = getService('log');
  const utils = getService('securitySolutionUtils');
  const config = getService('config');
  const isServerless = config.get('serverless');
  const dataPathBuilder = new EsArchivePathBuilder(isServerless);
  const auditbeatPath = dataPathBuilder.getPath('auditbeat/hosts');
  const esArchiver = getService('esArchiver');
  const es = getService('es');
  const esDeleteAllIndices = getService('esDeleteAllIndices');

  let admin: TestAgent;
  let endpointPolicyManager: TestAgent;
  let socManager: TestAgent;
  let viewer: TestAgent;
  // ... add other roles as needed

  describe('@serverless @serverlessQA roles rule preview execution API behaviors', () => {
    before(async () => {
      await esArchiver.load(auditbeatPath);
      admin = await utils.createSuperTest('admin');
      endpointPolicyManager = await utils.createSuperTest('endpoint_policy_manager');
      socManager = await utils.createSuperTest('soc_manager');
      viewer = await utils.createSuperTest('viewer');
      // ... more roles to be tested
    });

    afterEach(async () => {
      await esDeleteAllIndices('.preview.alerts*');
    });

    after(async () => {
      await esArchiver.unload(auditbeatPath);
      await deleteAllAlerts(admin, log, es, [
        '.preview.alerts-security.alerts-*',
        '.alerts-security.alerts-*',
      ]);
      await deleteAllRules(admin, log);
    });

    // Here we want to test that these two roles can create and run a rule against the preview api
    // This will be faster than having to run a full test suite for each individual role.
    it('previews rule', async () => {
      await Promise.all(
        [endpointPolicyManager, socManager].map(async (role) => {
          const rule: QueryRuleCreateProps = {
            ...getRuleForAlertTesting(['auditbeat-*']),
            query: `_id:${ID}`,
          };
          const previewRequest: RulePreviewRequestBody = {
            ...rule,
            invocationCount: 1,
            timeframeEnd: new Date().toISOString(),
          };
          return role
            .post(DETECTION_ENGINE_RULES_PREVIEW)
            .query({ enable_logged_requests: true })
            .set('kbn-xsrf', 'true')
            .set('elastic-api-version', '2023-10-31')
            .send(previewRequest)
            .expect(200);
        })
      );
    });
  });
};

[approksiu]

@approksiu @caitlinbetz could you confirm the behaviors here? All but the Endpoint Operations Analyst seem to match. The sheet says they're read on rules, but seems they can execute them.

Looks like the configuration is indeed set to "all" here. @caitlinbetz could you check the intended definition and comment if it should be changed to read? Thanks!