From b31c429d193a0f1871fc2c62f8fd77329b4f8be9 Mon Sep 17 00:00:00 2001 From: Andrea Selva Date: Tue, 26 Sep 2023 09:13:50 +0200 Subject: [PATCH 01/11] Expanded the description how to size the memory of the JVM (#15210) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Expands the description of memory used by Logstash, dividing the heap and non-heap; describing in details which parts composes the non-heap, how to size it and list the JVM settings that can be used to properly define this space. Co-authored-by: Karen Metts <35154725+karenzone@users.noreply.github.com> Co-authored-by: João Duarte --- docs/static/config-details.asciidoc | 79 +++++++++++++++++++++++++++-- 1 file changed, 74 insertions(+), 5 deletions(-) diff --git a/docs/static/config-details.asciidoc b/docs/static/config-details.asciidoc index 0ff63d6da86..226807b4e1a 100644 --- a/docs/static/config-details.asciidoc +++ b/docs/static/config-details.asciidoc @@ -49,9 +49,19 @@ JVM falls in the inclusive range of the two numbers * all other lines are rejected +[[memory-size]] +==== Setting the memory size + +The memory of the JVM executing {ls} can be divided in two zones: heap and off-heap memory. +In the heap refers to Java heap, which contains all the Java objects created by {ls} during its operation, see <> for +description on how to size it. +What's not part of the heap is named off-heap and consists of memory that can be used and controlled by {ls}, generally +thread stacks, direct memory and memory mapped pages, check <> for comprehensive descriptions. +In off-heap space there is some space which is used by JVM and contains all the data structures functional to the execution +of the virtual machine. This memory can't be controlled by {ls} and the settings are rarely customized. [[heap-size]] -==== Setting the JVM heap size +===== Setting the JVM heap size Here are some tips for adjusting the JVM heap size: @@ -77,15 +87,74 @@ process. info, see <>. // end::heap-size-tips[] - [[off-heap-size]] -==== Setting the off-heap size +===== Setting the off-heap size The operating system, persistent queue mmap pages, direct memory, and other processes require memory in addition to memory allocated to heap size. -Keep the overall memory requirements in mind when you allocate memory. +Internal JVM data structures, thread stacks, memory mapped files and direct memory for input/output (IO) operations are all parts of the off-heap JVM memory. +Memory mapped files are not part of the Logstash's process off-heap memory, but consume RAM when paging files from disk. +These mapped files speed up the access to Persistent Queues pages, a performance improvement - or trade off - to reduce expensive disk operations such as read, write, and seek. +Some network I/O operations also resort to in-process direct memory usage to avoid, for example, copying of buffers between network sockets. Input plugins such as Elastic Agent, Beats, TCP, and HTTP inputs, use direct memory. +The zone for Thread stacks contains the list of stack frames for each Java thread created by the JVM; each frame keeps the local arguments passed during method calls. +Read on <> if the size needs to be adapted to the processing needs. + +Plugins, depending on their type (inputs, filters, and outputs), have different thread models. +Every input plugin runs in its own thread and can potentially spawn others. For example, each JDBC input +plugin launches a scheduler thread. Netty based plugins like TCP, Beats or HTTP input spawn a thread pool with 2 * number_of_cores threads. +Output plugins may also start helper threads, such as a connection management thread for each +{es} output instance. +Every pipeline, also, has its own thread responsible to manage the pipeline lifecycle. + +To summarize, we have 3 categories of memory usage, where 2 can be limited by the JVM and the other relies on available, free memory: + +[cols="<,<,<",options="header",] +|===== +| Memory Type | Configured using | Used by +| JVM Heap | -Xmx | any normal object allocation +| JVM direct memory | -XX:MaxDirectMemorySize | beats, tcp and http inputs +| Native memory | N/A | Persistent Queue Pages, Thread Stacks +|===== + +Keep these memory requirements in mind as you calculate your ideal memory allocation. + +[[memory-size-calculation]] +===== Memory sizing + +Total JVM memory allocation must be estimated and is controlled indirectly using Java heap and direct memory settings. By default, a JVM's off-heap direct memory limit is the same as the heap size. Check out <>. -Consider setting `-XX:MaxDirectMemorySize` to half of the heap size. +Consider setting `-XX:MaxDirectMemorySize` to half of the heap size or any value that can accommodate the load you expect these plugins to handle. + +As you make your capacity calculations, keep in mind that the JVM can't consume the total amount of the host's memory available, +as the Operating System and other processes will require memory too. + +For a {ls} instance with persistent queue (PQ) enabled on multiple pipelines, we could +estimate memory consumption using: + +[source,text] +----- +pipelines number * (pipeline threads * stack size + 2 * PQ page size) + direct memory + Java heap +----- + +NOTE: Each Persistent Queue requires that at least head and tail pages are present and accessible in memory. +The default page size is 64 MB so each PQ requires at least 128 MB of heap memory, which can be a significant source +of memory consumption per pipeline. Note that the size of memory mapped file can't be limited with an upper bound. + +NOTE: Stack size is a setting that depends on the JVM used, but could be customized with `-Xss` setting. + +NOTE: Direct memory space by default is big as much as Java heap, but can be customized with the `-XX:MaxDirectMemorySize` setting. + +**Example** + +Consider a {ls} instance running 10 pipelines, with simple input and output plugins that doesn't start additional threads, +it has 1 pipelines thread, 1 input plugin thread and 12 workers, summing up to 14. +Keep in mind that, by default, JVM allocates direct memory equal to memory allocated for Java heap. + +The calculation results in: + +* native memory: 1.4Gb [derived from 10 * (14 * 1Mb + 128Mb)] +* direct memory: 4Gb +* Java heap: 4Gb [[stacks-size]] From dd77c9f9278cd669c61de99ff59fd374db943154 Mon Sep 17 00:00:00 2001 From: Dimitrios Liappis Date: Tue, 26 Sep 2023 16:00:56 +0300 Subject: [PATCH 02/11] [ci] Commit DRA -SNAPSHOT buildkite pipeline (#15337) This commit adds support for building + publishing DRA (-SNAPSHOT for now) artifacts for Logstash. It builds on top of #15312 and therefore only targets the `main` branch and is intended to be run manually during a trial period before we retire the corresponding Jenkins job. The structure is similar to Jenkins: 1. Three steps runs in parallel to build packages, x86_64 docker and aarch64 docker artifacts. 2. Once 1. is successfully done, use release manager to publish the artifacts. We generate the pipeline steps for 1. and 2. dynamically (with a simple Python script) to avoid repetition for future PRs: we will add a new pipeline in a follow up PR for -STAGING. The actual shell scripts are simplified copies from the existing `dra*` scripts under https://github.com/elastic/logstash/tree/main/ci; the simplification comes from native support for copying artifacts between steps in Buildkite and not having to use an intermediate bucket. Relates: https://github.com/elastic/ingest-dev/issues/1720 Blocked by: https://github.com/elastic/ci/pull/2312/files --- .buildkite/dra_pipeline.yml | 42 ++++-- .buildkite/scripts/dra/build_docker.sh | 90 ++++++++++++ .buildkite/scripts/dra/build_packages.sh | 58 ++++++++ .buildkite/scripts/dra/common.sh | 47 ++++++ .buildkite/scripts/dra/docker-env-setup.sh | 23 +++ .buildkite/scripts/dra/docker-env-teardown.sh | 15 ++ .buildkite/scripts/dra/generatesteps.py | 134 ++++++++++++++++++ .buildkite/scripts/dra/publish.sh | 100 +++++++++++++ 8 files changed, 500 insertions(+), 9 deletions(-) create mode 100755 .buildkite/scripts/dra/build_docker.sh create mode 100755 .buildkite/scripts/dra/build_packages.sh create mode 100755 .buildkite/scripts/dra/common.sh create mode 100755 .buildkite/scripts/dra/docker-env-setup.sh create mode 100755 .buildkite/scripts/dra/docker-env-teardown.sh create mode 100644 .buildkite/scripts/dra/generatesteps.py create mode 100755 .buildkite/scripts/dra/publish.sh diff --git a/.buildkite/dra_pipeline.yml b/.buildkite/dra_pipeline.yml index 988848fe88e..a7821022d70 100644 --- a/.buildkite/dra_pipeline.yml +++ b/.buildkite/dra_pipeline.yml @@ -1,12 +1,36 @@ # yaml-language-server: $schema=https://raw.githubusercontent.com/buildkite/pipeline-schema/main/schema.json -agents: - cpu: "2" - memory: "4Gi" - ephemeralStorage: "20Gi" - +## TODO rename this file to dra_pipeline_snapshot (and change the respective definition in .pipelines.yaml) steps: - - label: ":wave: Greetings" - command: "echo 'TODO: run DRA Logstash'" - agents: - image: "docker.elastic.co/ci-agent-images/platform-ingest/buildkite-agent-logstash-ci:0.1" + - input: "Build parameters" + if: build.source != "schedule" + fields: + - text: "VERSION_QUALIFIER_OPT" + key: "VERSION_QUALIFIER_OPT" + default: "" + required: false + hint: "Optional version qualifier for built artifacts e.g.: alpha1,beta1" + - select: "DRA DRY-RUN" + key: "DRA_DRY_RUN" + required: false + default: "" + options: + - label: "True" + value: "--dry-run" + - label: "False" + value: "" + hint: "Whether the DRA release manager will actually publish artifacts, or run in dry-run mode." + + - wait: ~ + if: build.source != "schedule" + + - label: ":pipeline: Generate steps" + command: | + set -eo pipefail + + export WORKFLOW_TYPE="snapshot" + python3 -m pip install pyyaml + echo "--- Generating dynamic pipeline steps:" + python3 .buildkite/scripts/dra/generatesteps.py + + python3 .buildkite/scripts/dra/generatesteps.py | buildkite-agent pipeline upload diff --git a/.buildkite/scripts/dra/build_docker.sh b/.buildkite/scripts/dra/build_docker.sh new file mode 100755 index 00000000000..fee8cce0eea --- /dev/null +++ b/.buildkite/scripts/dra/build_docker.sh @@ -0,0 +1,90 @@ +#!/bin/bash -ie +#Note - ensure that the -e flag is set to properly set the $? status if any command fails +echo "####################################################################" +echo "##################### Starting $0" +echo "####################################################################" + +source ./$(dirname "$0")/common.sh + +# WORKFLOW_TYPE is a CI externally configured environment variable that could assume "snapshot" or "staging" values +case "$WORKFLOW_TYPE" in + snapshot) + info "Building artifacts for the $WORKFLOW_TYPE workflow..." + if [ -z "$VERSION_QUALIFIER_OPT" ]; then + rake artifact:docker || error "artifact:docker build failed." + rake artifact:docker_oss || error "artifact:docker_oss build failed." + rake artifact:dockerfiles || error "artifact:dockerfiles build failed." + if [ "$ARCH" != "aarch64" ]; then + rake artifact:docker_ubi8 || error "artifact:docker_ubi8 build failed." + fi + else + VERSION_QUALIFIER="$VERSION_QUALIFIER_OPT" rake artifact:docker || error "artifact:docker build failed." + VERSION_QUALIFIER="$VERSION_QUALIFIER_OPT" rake artifact:docker_oss || error "artifact:docker_oss build failed." + VERSION_QUALIFIER="$VERSION_QUALIFIER_OPT" rake artifact:dockerfiles || error "artifact:dockerfiles build failed." + if [ "$ARCH" != "aarch64" ]; then + VERSION_QUALIFIER="$VERSION_QUALIFIER_OPT" rake artifact:docker_ubi8 || error "artifact:docker_ubi8 build failed." + fi + # Qualifier is passed from CI as optional field and specify the version postfix + # in case of alpha or beta releases: + # e.g: 8.0.0-alpha1 + STACK_VERSION="${STACK_VERSION}-${VERSION_QUALIFIER_OPT}" + fi + STACK_VERSION=${STACK_VERSION}-SNAPSHOT + info "Build complete, setting STACK_VERSION to $STACK_VERSION." + ;; + staging) + info "Building artifacts for the $WORKFLOW_TYPE workflow..." + if [ -z "$VERSION_QUALIFIER_OPT" ]; then + RELEASE=1 rake artifact:docker || error "artifact:docker build failed." + RELEASE=1 rake artifact:docker_oss || error "artifact:docker_oss build failed." + RELEASE=1 rake artifact:dockerfiles || error "artifact:dockerfiles build failed." + if [ "$ARCH" != "aarch64" ]; then + RELEASE=1 rake artifact:docker_ubi8 || error "artifact:docker_ubi8 build failed." + fi + else + VERSION_QUALIFIER="$VERSION_QUALIFIER_OPT" RELEASE=1 rake artifact:docker || error "artifact:docker build failed." + VERSION_QUALIFIER="$VERSION_QUALIFIER_OPT" RELEASE=1 rake artifact:docker_oss || error "artifact:docker_oss build failed." + VERSION_QUALIFIER="$VERSION_QUALIFIER_OPT" RELEASE=1 rake artifact:dockerfiles || error "artifact:dockerfiles build failed." + if [ "$ARCH" != "aarch64" ]; then + VERSION_QUALIFIER="$VERSION_QUALIFIER_OPT" RELEASE=1 rake artifact:docker_ubi8 || error "artifact:docker_ubi8 build failed." + fi + # Qualifier is passed from CI as optional field and specify the version postfix + # in case of alpha or beta releases: + # e.g: 8.0.0-alpha1 + STACK_VERSION="${STACK_VERSION}-${VERSION_QUALIFIER_OPT}" + fi + info "Build complete, setting STACK_VERSION to $STACK_VERSION." + ;; + *) + error "Workflow (WORKFLOW_TYPE variable) is not set, exiting..." + ;; +esac + +info "Saving tar.gz for docker images" +save_docker_tarballs "${ARCH}" "${STACK_VERSION}" + +info "Generated Artifacts" +for file in build/logstash-*; do shasum $file;done + +info "Uploading DRA artifacts in buildkite's artifact store ..." +# Note the deb, rpm tar.gz AARCH64 files generated has already been loaded by the build_packages.sh +images="logstash logstash-oss" +if [ "$ARCH" != "aarch64" ]; then + # No logstash-ubi8 for AARCH64 + images="logstash logstash-oss logstash-ubi8" +fi +for image in ${images}; do + buildkite-agent artifact upload "build/$image-${STACK_VERSION}-docker-image-${ARCH}.tar.gz" +done + +# Upload 'docker-build-context.tar.gz' files only when build x86_64, otherwise they will be +# overwritten when building aarch64 (or viceversa). +if [ "$ARCH" != "aarch64" ]; then + for image in logstash logstash-oss logstash-ubi8 logstash-ironbank; do + buildkite-agent artifact upload "build/${image}-${STACK_VERSION}-docker-build-context.tar.gz" + done +fi + +echo "####################################################################" +echo "##################### Finishing $0" +echo "####################################################################" diff --git a/.buildkite/scripts/dra/build_packages.sh b/.buildkite/scripts/dra/build_packages.sh new file mode 100755 index 00000000000..2bd4ed1750a --- /dev/null +++ b/.buildkite/scripts/dra/build_packages.sh @@ -0,0 +1,58 @@ +#!/bin/bash -ie +#Note - ensure that the -e flag is set to properly set the $? status if any command fails +echo "####################################################################" +echo "##################### Starting $0" +echo "####################################################################" + +source ./$(dirname "$0")/common.sh + +# WORKFLOW_TYPE is a CI externally configured environment variable that could assume "snapshot" or "staging" values +case "$WORKFLOW_TYPE" in + snapshot) + info "Building artifacts for the $WORKFLOW_TYPE workflow..." + if [ -z "$VERSION_QUALIFIER_OPT" ]; then + SKIP_DOCKER=1 rake artifact:all || error "rake artifact:all build failed." + else + # Qualifier is passed from CI as optional field and specify the version postfix + # in case of alpha or beta releases: + # e.g: 8.0.0-alpha1 + VERSION_QUALIFIER="$VERSION_QUALIFIER_OPT" SKIP_DOCKER=1 rake artifact:all || error "rake artifact:all build failed." + STACK_VERSION="${STACK_VERSION}-${VERSION_QUALIFIER_OPT}" + fi + STACK_VERSION=${STACK_VERSION}-SNAPSHOT + info "Build complete, setting STACK_VERSION to $STACK_VERSION." + ;; + staging) + info "Building artifacts for the $WORKFLOW_TYPE workflow..." + if [ -z "$VERSION_QUALIFIER_OPT" ]; then + RELEASE=1 SKIP_DOCKER=1 rake artifact:all || error "rake artifact:all build failed." + else + # Qualifier is passed from CI as optional field and specify the version postfix + # in case of alpha or beta releases: + # e.g: 8.0.0-alpha1 + VERSION_QUALIFIER="$VERSION_QUALIFIER_OPT" RELEASE=1 SKIP_DOCKER=1 rake artifact:all || error "rake artifact:all build failed." + STACK_VERSION="${STACK_VERSION}-${VERSION_QUALIFIER_OPT}" + fi + info "Build complete, setting STACK_VERSION to $STACK_VERSION." + ;; + *) + error "Workflow (WORKFLOW_TYPE variable) is not set, exiting..." + ;; +esac + +info "Generated Artifacts" +for file in build/logstash-*; do shasum $file;done + +info "Creating dependencies report for ${STACK_VERSION}" +mkdir -p build/distributions/dependencies-reports/ +bin/dependencies-report --csv=build/distributions/dependencies-reports/logstash-${STACK_VERSION}.csv + +info "Generated dependencies report" +shasum build/distributions/dependencies-reports/logstash-${STACK_VERSION}.csv + +info "Uploading DRA artifacts in buildkite's artifact store ..." +buildkite-agent artifact upload "build/logstash*;build/distributions/dependencies-reports/logstash*" + +echo "####################################################################" +echo "##################### Finishing $0" +echo "####################################################################" diff --git a/.buildkite/scripts/dra/common.sh b/.buildkite/scripts/dra/common.sh new file mode 100755 index 00000000000..c398322a158 --- /dev/null +++ b/.buildkite/scripts/dra/common.sh @@ -0,0 +1,47 @@ +function info { + echo "--- INFO: $1" +} + +function error { + echo "--- ERROR: $1" + exit 1 +} + +function save_docker_tarballs { + local arch="${1:?architecture required}" + local version="${2:?stack-version required}" + local images="logstash logstash-oss" + if [ "${arch}" != "aarch64" ]; then + # No logstash-ubi8 for AARCH64 + images="logstash logstash-oss logstash-ubi8" + fi + + for image in ${images}; do + tar_file="${image}-${version}-docker-image-${arch}.tar" + docker save -o "build/${tar_file}" \ + "docker.elastic.co/logstash/${image}:${version}" || \ + error "Unable to save tar file ${tar_file} for ${image} image." + # NOTE: if docker save exited with non-zero the error log already exited the script + gzip "build/${tar_file}" + done +} + +# Since we are using the system jruby, we need to make sure our jvm process +# uses at least 1g of memory, If we don't do this we can get OOM issues when +# installing gems. See https://github.com/elastic/logstash/issues/5179 +export JRUBY_OPTS="-J-Xmx1g" + +# Extract the version number from the version.yml file +# e.g.: 8.6.0 +# The suffix part like alpha1 etc is managed by the optional VERSION_QUALIFIER_OPT environment variable +STACK_VERSION=`cat versions.yml | sed -n 's/^logstash\:[[:space:]]\([[:digit:]]*\.[[:digit:]]*\.[[:digit:]]*\)$/\1/p'` + +info "Agent is running on architecture [$(uname -i)]" + +export VERSION_QUALIFIER_OPT=$(buildkite-agent meta-data get VERSION_QUALIFIER_OPT --default "") +export DRA_DRY_RUN=$(buildkite-agent meta-data get DRA_DRY_RUN --default "") + +if [[ ! -z $DRA_DRY_RUN && $BUILDKITE_STEP_KEY == "logstash_publish_dra" ]]; then + info "Release manager will run in dry-run mode [$DRA_DRY_RUN]" +fi + diff --git a/.buildkite/scripts/dra/docker-env-setup.sh b/.buildkite/scripts/dra/docker-env-setup.sh new file mode 100755 index 00000000000..6787a4e9dbe --- /dev/null +++ b/.buildkite/scripts/dra/docker-env-setup.sh @@ -0,0 +1,23 @@ +#!/bin/bash + +set -euo pipefail + +DOCKER_REGISTRY="docker.elastic.co" +DOCKER_REGISTRY_SECRET_PATH="kv/ci-shared/platform-ingest/docker_registry_prod" +CI_DRA_ROLE_PATH="kv/ci-shared/release/dra-role" + + +function docker_login { + DOCKER_USERNAME_SECRET=$(retry -t 5 -- vault kv get -field user "${DOCKER_REGISTRY_SECRET_PATH}") + DOCKER_PASSWORD_SECRET=$(retry -t 5 -- vault kv get -field password "${DOCKER_REGISTRY_SECRET_PATH}") + docker login -u "${DOCKER_USERNAME_SECRET}" -p "${DOCKER_PASSWORD_SECRET}" "${DOCKER_REGISTRY}" 2>/dev/null + unset DOCKER_USERNAME_SECRET DOCKER_PASSWORD_SECRET +} + +function release_manager_login { + DRA_CREDS_SECRET=$(retry -t 5 -- vault kv get -field=data -format=json ${CI_DRA_ROLE_PATH}) + VAULT_ADDR_SECRET=$(echo ${DRA_CREDS_SECRET} | jq -r '.vault_addr') + VAULT_ROLE_ID=$(echo ${DRA_CREDS_SECRET} | jq -r '.role_id') + VAULT_SECRET_ID=$(echo ${DRA_CREDS_SECRET} | jq -r '.secret_id') + export VAULT_ADDR_SECRET VAULT_ROLE_ID VAULT_SECRET_ID +} diff --git a/.buildkite/scripts/dra/docker-env-teardown.sh b/.buildkite/scripts/dra/docker-env-teardown.sh new file mode 100755 index 00000000000..8e7fe0a9a64 --- /dev/null +++ b/.buildkite/scripts/dra/docker-env-teardown.sh @@ -0,0 +1,15 @@ +#!/bin/bash + +set -euo pipefail + +# Unset all variables ending with _SECRET or _TOKEN +for var in $(printenv | sed 's;=.*;;' | sort); do + if [[ $var != "VAULT_ADDR" && ("$var" == *_SECRET || "$var" == *_TOKEN || "$var" == *VAULT* ) ]]; then + unset "$var" + fi +done + +if command -v docker &>/dev/null; then + DOCKER_REGISTRY="docker.elastic.co" + docker logout $DOCKER_REGISTRY +fi diff --git a/.buildkite/scripts/dra/generatesteps.py b/.buildkite/scripts/dra/generatesteps.py new file mode 100644 index 00000000000..d59b1350102 --- /dev/null +++ b/.buildkite/scripts/dra/generatesteps.py @@ -0,0 +1,134 @@ +import os +import sys + +import yaml + +def to_bk_key_friendly_string(key): + """ + Convert and return key to an acceptable format for Buildkite's key: field + Only alphanumerics, dashes and underscores are allowed. + """ + + mapping_table = str.maketrans({'.': '_'}) + + return key.translate(mapping_table) + +def package_x86_step(branch, workflow_type): + step = f''' +- label: ":package: Build packages / {branch}-{workflow_type.upper()} DRA artifacts" + key: "logstash_build_packages_dra" + agents: + image: "docker.elastic.co/ci-agent-images/platform-ingest/buildkite-agent-logstash-ci:0.2" + cpu: "8" + memory: "16Gi" + ephemeralStorage: "200Gi" + command: | + export WORKFLOW_TYPE="{workflow_type}" + export PATH="/usr/local/rbenv/bin:$PATH" + eval "$(rbenv init -)" + .buildkite/scripts/dra/build_packages.sh +''' + + return step + +def package_x86_docker_step(branch, workflow_type): + step = f''' +- label: ":package: Build x86_64 Docker / {branch}-{workflow_type.upper()} DRA artifacts" + key: "logstash_build_x86_64_docker_dra" + agents: + provider: gcp + imageProject: elastic-images-qa + image: family/platform-ingest-logstash-ubuntu-2204 + machineType: "n2-standard-16" + diskSizeGb: 200 + command: | + export WORKFLOW_TYPE="{workflow_type}" + export PATH="/opt/buildkite-agent/.rbenv/bin:/opt/buildkite-agent/.pyenv/bin:$PATH" + export ARCH="x86_64" + eval "$(rbenv init -)" + .buildkite/scripts/dra/build_docker.sh +''' + + return step + +def package_aarch64_docker_step(branch, workflow_type): + step = f''' +- label: ":package: Build aarch64 Docker / {branch}-{workflow_type.upper()} DRA artifacts" + key: "logstash_build_aarch64_docker_dra" + agents: + provider: gcp + imageProject: elastic-images-qa + image: family/platform-ingest-logstash-ubuntu-2204-aarch64 + machineType: "t2a-standard-8" + diskSizeGb: 200 + region: 'us-central1' + # so far only these regions support t2a instance types + zones: "us-central1-a,us-central1-b,us-central1-f" + command: | + export WORKFLOW_TYPE="{workflow_type}" + export PATH="/opt/buildkite-agent/.rbenv/bin:/opt/buildkite-agent/.pyenv/bin:$PATH" + export ARCH="aarch64" + eval "$(rbenv init -)" + .buildkite/scripts/dra/build_docker.sh +''' + + return step + +def publish_dra_step(branch, workflow_type, depends_on): + step = f''' +- label: ":elastic-stack: Publish / {branch}-{workflow_type.upper()} DRA artifacts" + key: "logstash_publish_dra" + depends_on: "{depends_on}" + agents: + provider: gcp + imageProject: elastic-images-qa + image: family/platform-ingest-logstash-ubuntu-2204 + machineType: "n2-standard-16" + diskSizeGb: 200 + command: | + echo "+++ Restoring Artifacts" + buildkite-agent artifact download "build/logstash*" . + buildkite-agent artifact download "build/distributions/**/*" . + echo "+++ Changing permissions for the release manager" + sudo chown -R :1000 build + echo "+++ Running DRA publish step" + export WORKFLOW_TYPE="{workflow_type}" + .buildkite/scripts/dra/publish.sh + ''' + + return step + +def build_steps_to_yaml(branch, workflow_type): + steps = [] + steps.extend(yaml.safe_load(package_x86_step(branch, workflow_type))) + steps.extend(yaml.safe_load(package_x86_docker_step(branch, workflow_type))) + steps.extend(yaml.safe_load(package_aarch64_docker_step(branch, workflow_type))) + + return steps + +if __name__ == "__main__": + try: + workflow_type = os.environ["WORKFLOW_TYPE"] + except ImportError: + print(f"Missing env variable WORKFLOW_TYPE. Use export WORKFLOW_TYPE=\n.Exiting.") + exit(1) + + branch = os.environ["BUILDKITE_BRANCH"] + + structure = {"steps": []} + + # Group defining parallel steps that build and save artifacts + group_key = to_bk_key_friendly_string(f"logstash_dra_{workflow_type}") + + structure["steps"].append({ + "group": f":Build Artifacts - {workflow_type.upper()}", + "key": group_key, + "steps": build_steps_to_yaml(branch, workflow_type), + }) + + # Final step: pull artifacts built above and publish them via the release-manager + structure["steps"].extend( + yaml.safe_load(publish_dra_step(branch, workflow_type, depends_on=group_key)), + ) + + print(yaml.dump(structure, Dumper=yaml.Dumper, sort_keys=False)) diff --git a/.buildkite/scripts/dra/publish.sh b/.buildkite/scripts/dra/publish.sh new file mode 100755 index 00000000000..f501b05a1d3 --- /dev/null +++ b/.buildkite/scripts/dra/publish.sh @@ -0,0 +1,100 @@ +#!/bin/bash -i +set -euo pipefail + +echo "####################################################################" +echo "##################### Starting $0" +echo "####################################################################" + +source ./$(dirname "$0")/common.sh + +PLAIN_STACK_VERSION=$STACK_VERSION + +# This is the branch selector that needs to be passed to the release-manager +# It has to be the name of the branch which originates the artifacts. +RELEASE_VER=`cat versions.yml | sed -n 's/^logstash\:[[:space:]]\([[:digit:]]*\.[[:digit:]]*\)\.[[:digit:]]*$/\1/p'` +if [ -n "$(git ls-remote --heads origin $RELEASE_VER)" ] ; then + RELEASE_BRANCH=$RELEASE_VER +else + RELEASE_BRANCH=main +fi + +if [ -n "$VERSION_QUALIFIER_OPT" ]; then + # Qualifier is passed from CI as optional field and specify the version postfix + # in case of alpha or beta releases: + # e.g: 8.0.0-alpha1 + STACK_VERSION="${STACK_VERSION}-${VERSION_QUALIFIER_OPT}" + PLAIN_STACK_VERSION="${PLAIN_STACK_VERSION}-${VERSION_QUALIFIER_OPT}" +fi + +case "$WORKFLOW_TYPE" in + snapshot) + STACK_VERSION=${STACK_VERSION}-SNAPSHOT + ;; + staging) + ;; + *) + error "Worklflow (WORKFLOW_TYPE variable) is not set, exiting..." + ;; +esac + +info "Uploading artifacts for ${WORKFLOW_TYPE} workflow on branch: ${RELEASE_BRANCH}" + +if [ "$RELEASE_VER" != "7.17" ]; then + # Version 7.17.x doesn't generates ARM artifacts for Darwin + # TODO see if we need to do anything here + : +fi + +# Deleting ubi8 for aarch64 for the time being. This image itself is not being built, and it is not expected +# by the release manager. +# See https://github.com/elastic/infra/blob/master/cd/release/release-manager/project-configs/8.5/logstash.gradle +# for more details. +# TODO filter it out when uploading artifacts instead +rm -f build/logstash-ubi8-${STACK_VERSION}-docker-image-aarch64.tar.gz + +info "Downloaded ARTIFACTS sha report" +for file in build/logstash-*; do shasum $file;done + +mv build/distributions/dependencies-reports/logstash-${STACK_VERSION}.csv build/distributions/dependencies-${STACK_VERSION}.csv + +# set required permissions on artifacts and directory +chmod -R a+r build/* +chmod -R a+w build + +chmod -R a+r $PWD/* +chmod -R a+w $PWD + +info "Setup docker credentials" +# TODO disable tracing +# set +o xtrace +source ./$(dirname "$0")/docker-env-setup.sh +release_manager_login + +# ensure the latest image has been pulled +docker pull docker.elastic.co/infra/release-manager:latest + +info "Running the release manager ..." + +# collect the artifacts for use with the unified build +docker run --rm \ + --name release-manager \ + -e VAULT_ADDR="${VAULT_ADDR_SECRET}" \ + -e VAULT_ROLE_ID \ + -e VAULT_SECRET_ID \ + --mount type=bind,readonly=false,src="$PWD",target=/artifacts \ + docker.elastic.co/infra/release-manager:latest \ + cli collect \ + --project logstash \ + --branch ${RELEASE_BRANCH} \ + --commit "$(git rev-parse HEAD)" \ + --workflow "${WORKFLOW_TYPE}" \ + --version "${PLAIN_STACK_VERSION}" \ + --artifact-set main \ + ${DRA_DRY_RUN} + +info "Teardown logins" +$(dirname "$0")/docker-env-teardown.sh + +echo "####################################################################" +echo "##################### Finishing $0" +echo "####################################################################" From 12a98250cb4a3bc517ddfaa6042f90b05c10ca75 Mon Sep 17 00:00:00 2001 From: Dimitrios Liappis Date: Wed, 27 Sep 2023 16:41:51 +0300 Subject: [PATCH 03/11] [DRA,buildkite] Use AWS aarch64 instance type (#15352) We've spotted network flakiness downloading artifacts with gradle (connection resets) when using GCP/t2a on us-central1. This commit switches to AWS Graviton instance types for building the aarch64 artifacts. --- .buildkite/scripts/dra/generatesteps.py | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/.buildkite/scripts/dra/generatesteps.py b/.buildkite/scripts/dra/generatesteps.py index d59b1350102..1ded7b33f47 100644 --- a/.buildkite/scripts/dra/generatesteps.py +++ b/.buildkite/scripts/dra/generatesteps.py @@ -56,14 +56,10 @@ def package_aarch64_docker_step(branch, workflow_type): - label: ":package: Build aarch64 Docker / {branch}-{workflow_type.upper()} DRA artifacts" key: "logstash_build_aarch64_docker_dra" agents: - provider: gcp - imageProject: elastic-images-qa - image: family/platform-ingest-logstash-ubuntu-2204-aarch64 - machineType: "t2a-standard-8" + provider: aws + imagePrefix: platform-ingest-logstash-ubuntu-2204-aarch64 + instanceType: "m6g.4xlarge" diskSizeGb: 200 - region: 'us-central1' - # so far only these regions support t2a instance types - zones: "us-central1-a,us-central1-b,us-central1-f" command: | export WORKFLOW_TYPE="{workflow_type}" export PATH="/opt/buildkite-agent/.rbenv/bin:/opt/buildkite-agent/.pyenv/bin:$PATH" From ad3b89a2b45e91d047edccedd078a07afa4d1bec Mon Sep 17 00:00:00 2001 From: Andres Rodriguez Date: Wed, 27 Sep 2023 17:24:21 -0400 Subject: [PATCH 04/11] Clarify Env Variable usage for list-type URI params (#15345) Clarify Env Variable usage for list-type URI params --- docs/static/env-vars.asciidoc | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/static/env-vars.asciidoc b/docs/static/env-vars.asciidoc index 91604861c42..56c54e09edb 100644 --- a/docs/static/env-vars.asciidoc +++ b/docs/static/env-vars.asciidoc @@ -10,6 +10,7 @@ * You can give a default value by using the form `${var:default value}`. Logstash uses the default value if the environment variable is undefined. * You can add environment variable references in any plugin option type: string, number, boolean, array, or hash. +* Environment variables for list-type URI parameters can support lists of space-delimited values. Currently, other non-URI based options do not support lists of values. See <> * Environment variables are immutable. If you update the environment variable, you'll have to restart Logstash to pick up the updated value. ==== Examples From 05c262712326ccd61c7f45f8863cc8734cac5f6c Mon Sep 17 00:00:00 2001 From: Dimitrios Liappis Date: Thu, 28 Sep 2023 11:16:51 +0300 Subject: [PATCH 05/11] [DRA, buildkite] Use GCP vms for package build step (#15356) We seem to be hitting a weird issue with the psych gem when running in a container: ``` > Task :bootstrap 2023-09-27T15:13:23.844Z [Execution worker Thread 5] WARN FilenoUtil : Native subprocess control requires open access to the JDK IO subsystem Pass '--add-opens java.base/sun.nio.ch=ALL-UNNAMED --add-opens java.base/java.io=ALL-UNNAMED' to enable. Error Errno::ENOENT, retrying 1/10 No such file or directory - /buildkite/builds/bk-agent-prod-k8s-1695827479442731913/elastic/logstash-dra-pipeline-ci/vendor/bundle/jruby/3.1.0/gems/psych-5.1.0-java/deps.lst ``` This commit works around this problem by running inside a VM. The underlying root cause (and that gradle should be erroring in this case) will be investigated separately. --- .buildkite/scripts/dra/generatesteps.py | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/.buildkite/scripts/dra/generatesteps.py b/.buildkite/scripts/dra/generatesteps.py index 1ded7b33f47..64ce387d4d7 100644 --- a/.buildkite/scripts/dra/generatesteps.py +++ b/.buildkite/scripts/dra/generatesteps.py @@ -18,13 +18,14 @@ def package_x86_step(branch, workflow_type): - label: ":package: Build packages / {branch}-{workflow_type.upper()} DRA artifacts" key: "logstash_build_packages_dra" agents: - image: "docker.elastic.co/ci-agent-images/platform-ingest/buildkite-agent-logstash-ci:0.2" - cpu: "8" - memory: "16Gi" - ephemeralStorage: "200Gi" + provider: gcp + imageProject: elastic-images-qa + image: family/platform-ingest-logstash-ubuntu-2204 + machineType: "n2-standard-16" + diskSizeGb: 200 command: | export WORKFLOW_TYPE="{workflow_type}" - export PATH="/usr/local/rbenv/bin:$PATH" + export PATH="/opt/buildkite-agent/.rbenv/bin:/opt/buildkite-agent/.pyenv/bin:$PATH" eval "$(rbenv init -)" .buildkite/scripts/dra/build_packages.sh ''' From da51a6ac53296b0c200db22c84d046f002ee5570 Mon Sep 17 00:00:00 2001 From: Dimitrios Liappis Date: Mon, 2 Oct 2023 12:10:59 +0300 Subject: [PATCH 06/11] [buildkite] DRA staging + snapshot pipelines (#15344) This commit is the step after #15337 and switches away from the generically named [logstash-dra-pipeline-ci](https://buildkite.com/elastic/logstash-dra-pipeline-ci) pipeline to two separate pipelines for `snapshot` and `staging`, reflecting the current structure in [Jenkins](https://logstash-ci.elastic.co/view/DRA/). We define a daily schedule for `snapshot`, but it's commented out for now, until we ramp down the Jenkins job. Finally, as we've created a dedicated Buildkite team for `logstash`, we place both pipelines under this team. Relates: - https://github.com/elastic/ingest-dev/issues/1720 - https://github.com/elastic/logstash/pull/15337 --- .buildkite/dra_pipeline.yml | 9 +-- .buildkite/scripts/dra/generatesteps.py | 6 +- catalog-info.yaml | 86 ++++++++++++++++++++++--- 3 files changed, 82 insertions(+), 19 deletions(-) diff --git a/.buildkite/dra_pipeline.yml b/.buildkite/dra_pipeline.yml index a7821022d70..f0bb01ea94e 100644 --- a/.buildkite/dra_pipeline.yml +++ b/.buildkite/dra_pipeline.yml @@ -1,6 +1,5 @@ # yaml-language-server: $schema=https://raw.githubusercontent.com/buildkite/pipeline-schema/main/schema.json -## TODO rename this file to dra_pipeline_snapshot (and change the respective definition in .pipelines.yaml) steps: - input: "Build parameters" if: build.source != "schedule" @@ -26,11 +25,9 @@ steps: - label: ":pipeline: Generate steps" command: | - set -eo pipefail + set -euo pipefail - export WORKFLOW_TYPE="snapshot" + echo "--- Building [${WORKFLOW_TYPE}] artifacts" python3 -m pip install pyyaml - echo "--- Generating dynamic pipeline steps:" - python3 .buildkite/scripts/dra/generatesteps.py - + echo "--- Building dynamic pipeline steps" python3 .buildkite/scripts/dra/generatesteps.py | buildkite-agent pipeline upload diff --git a/.buildkite/scripts/dra/generatesteps.py b/.buildkite/scripts/dra/generatesteps.py index 64ce387d4d7..1cc05cffe0a 100644 --- a/.buildkite/scripts/dra/generatesteps.py +++ b/.buildkite/scripts/dra/generatesteps.py @@ -19,7 +19,7 @@ def package_x86_step(branch, workflow_type): key: "logstash_build_packages_dra" agents: provider: gcp - imageProject: elastic-images-qa + imageProject: elastic-images-prod image: family/platform-ingest-logstash-ubuntu-2204 machineType: "n2-standard-16" diskSizeGb: 200 @@ -38,7 +38,7 @@ def package_x86_docker_step(branch, workflow_type): key: "logstash_build_x86_64_docker_dra" agents: provider: gcp - imageProject: elastic-images-qa + imageProject: elastic-images-prod image: family/platform-ingest-logstash-ubuntu-2204 machineType: "n2-standard-16" diskSizeGb: 200 @@ -78,7 +78,7 @@ def publish_dra_step(branch, workflow_type, depends_on): depends_on: "{depends_on}" agents: provider: gcp - imageProject: elastic-images-qa + imageProject: elastic-images-prod image: family/platform-ingest-logstash-ubuntu-2204 machineType: "n2-standard-16" diskSizeGb: 200 diff --git a/catalog-info.yaml b/catalog-info.yaml index 452b60a3330..cb7e9d17fa9 100644 --- a/catalog-info.yaml +++ b/catalog-info.yaml @@ -26,8 +26,8 @@ spec: dependsOn: - resource:buildkite-logstash-serverless-integration-testing - resource:logstash-snyk-report - - resource:logstash-dra-pipeline - + - logstash-dra-snapshot-pipeline + - logstash-dra-staging-pipeline # *********************************** # Declare serverless IT pipeline @@ -110,39 +110,105 @@ spec: message: "Run the Logstash Snyk report every day." # *********************************** -# Declare DRA pipelines +# SECTION START: DRA pipelines # *********************************** + --- # yaml-language-server: $schema=https://gist.githubusercontent.com/elasticmachine/988b80dae436cafea07d9a4a460a011d/raw/e57ee3bed7a6f73077a3f55a38e76e40ec87a7cf/rre.schema.json apiVersion: backstage.io/v1alpha1 kind: Resource metadata: - name: logstash-dra-pipeline - description: 'The logstash DRA pipeline.' + name: logstash-dra-snapshot-pipeline + description: Logstash Snapshot DRA pipeline links: - - title: Pipeline - url: https://buildkite.com/elastic/logstash-dra-pipeline-ci + - title: Logstash Snapshot DRA pipeline + url: https://buildkite.com/elastic/logstash-dra-snapshot-pipeline spec: type: buildkite-pipeline - owner: group:ingest-fp + owner: group:logstash + system: buildkite + implementation: + apiVersion: buildkite.elastic.dev/v1 + kind: Pipeline + metadata: + name: logstash-dra-snapshot-pipeline + description: ':logstash: The DRA snapshot :pipeline:' + spec: + repository: elastic/logstash + pipeline_file: ".buildkite/dra_pipeline.yml" + # TODO: uncomment out the schedule after testing + disabling Jenkins Job + # schedules: + # Daily 7_17: + # branch: '7.17' + # cronline: 30 01 * * * + # message: Daily SNAPSHOT build for 7.17 + # Daily 8_10: + # branch: '8.10' + # cronline: 30 01 * * * + # message: Daily SNAPSHOT build for 8.10 + # Daily main: + # branch: main + # cronline: 30 01 * * * + # message: Daily SNAPSHOT build for main + skip_intermediate_builds: true + provider_settings: + trigger_mode: none + env: + WORKFLOW_TYPE: 'snapshot' + ELASTIC_SLACK_NOTIFICATIONS_ENABLED: 'false' # don't alert during development + SLACK_NOTIFICATIONS_CHANNEL: '#logstash-build' + SLACK_NOTIFICATIONS_ON_SUCCESS: 'false' + teams: + ingest-fp: + access_level: MANAGE_BUILD_AND_READ + logstash: + access_level: MANAGE_BUILD_AND_READ + ingest-eng-prod: + access_level: MANAGE_BUILD_AND_READ + everyone: + access_level: READ_ONLY + +--- +# yaml-language-server: $schema=https://gist.githubusercontent.com/elasticmachine/988b80dae436cafea07d9a4a460a011d/raw/e57ee3bed7a6f73077a3f55a38e76e40ec87a7cf/rre.schema.json +apiVersion: backstage.io/v1alpha1 +kind: Resource +metadata: + name: logstash-dra-staging-pipeline + description: 'The logstash DRA staging pipeline.' + links: + - title: Logstash Staging DRA pipeline + url: https://buildkite.com/elastic/logstash-dra-staging-pipeline +spec: + type: buildkite-pipeline + owner: group:logstash system: buildkite implementation: apiVersion: buildkite.elastic.dev/v1 kind: Pipeline metadata: - name: logstash-dra-pipeline-ci - description: ':logstash: The DRA :pipeline:' + name: logstash-dra-staging-pipeline + description: ':logstash: The DRA staging :pipeline:' spec: repository: elastic/logstash pipeline_file: ".buildkite/dra_pipeline.yml" + skip_intermediate_builds: true provider_settings: trigger_mode: none env: + WORKFLOW_TYPE: 'staging' ELASTIC_SLACK_NOTIFICATIONS_ENABLED: 'false' # don't alert during development SLACK_NOTIFICATIONS_CHANNEL: '#logstash-build' SLACK_NOTIFICATIONS_ON_SUCCESS: 'false' teams: ingest-fp: access_level: MANAGE_BUILD_AND_READ + logstash: + access_level: MANAGE_BUILD_AND_READ + ingest-eng-prod: + access_level: MANAGE_BUILD_AND_READ everyone: access_level: READ_ONLY + +# *********************************** +# SECTION END: DRA pipelines +# *********************************** From 7ed8d376301d06be4bc08f0e4fd72e2764c6df04 Mon Sep 17 00:00:00 2001 From: Mashhur <99575341+mashhurs@users.noreply.github.com> Date: Mon, 2 Oct 2023 21:36:05 -0700 Subject: [PATCH 07/11] Bundle plugin. (#15359) --- rakelib/plugins-metadata.json | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/rakelib/plugins-metadata.json b/rakelib/plugins-metadata.json index 9a72fc75614..158f4463481 100644 --- a/rakelib/plugins-metadata.json +++ b/rakelib/plugins-metadata.json @@ -420,6 +420,10 @@ "default-plugins": true, "skip-list": false }, + "logstash-integration-logstash": { + "default-plugins": true, + "skip-list": false + }, "logstash-integration-rabbitmq": { "default-plugins": true, "skip-list": false From 9307c4de1c4d40fcbe6bd6bb61b7de105a33cf10 Mon Sep 17 00:00:00 2001 From: Dimitrios Liappis Date: Tue, 3 Oct 2023 17:21:03 +0300 Subject: [PATCH 08/11] [DRA] Rename Buildkite metadata for DRA pipelines (#15365) To provide a smoother migration experience from Jenkins, this commit adds better information about the two types of DRA jobs. Relates https://github.com/elastic/ingest-dev/issues/1720 --- catalog-info.yaml | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/catalog-info.yaml b/catalog-info.yaml index cb7e9d17fa9..94180ba82ee 100644 --- a/catalog-info.yaml +++ b/catalog-info.yaml @@ -119,9 +119,9 @@ apiVersion: backstage.io/v1alpha1 kind: Resource metadata: name: logstash-dra-snapshot-pipeline - description: Logstash Snapshot DRA pipeline + description: 'Logstash DRA SNAPSHOT (Daily, Auto) pipeline' links: - - title: Logstash Snapshot DRA pipeline + - title: 'Logstash DRA SNAPSHOT (Daily, Auto) pipeline' url: https://buildkite.com/elastic/logstash-dra-snapshot-pipeline spec: type: buildkite-pipeline @@ -132,7 +132,7 @@ spec: kind: Pipeline metadata: name: logstash-dra-snapshot-pipeline - description: ':logstash: The DRA snapshot :pipeline:' + description: ':logstash: The DRA SNAPSHOT (Daily, Auto) pipeline' spec: repository: elastic/logstash pipeline_file: ".buildkite/dra_pipeline.yml" @@ -174,9 +174,9 @@ apiVersion: backstage.io/v1alpha1 kind: Resource metadata: name: logstash-dra-staging-pipeline - description: 'The logstash DRA staging pipeline.' + description: 'Logstash DRA RELEASE (Staging) pipeline' links: - - title: Logstash Staging DRA pipeline + - title: 'Logstash DRA RELEASE (Staging) pipeline' url: https://buildkite.com/elastic/logstash-dra-staging-pipeline spec: type: buildkite-pipeline @@ -187,7 +187,7 @@ spec: kind: Pipeline metadata: name: logstash-dra-staging-pipeline - description: ':logstash: The DRA staging :pipeline:' + description: ':logstash: The DRA RELEASE (Staging) pipeline' spec: repository: elastic/logstash pipeline_file: ".buildkite/dra_pipeline.yml" From 2b091432cbe323f51b5e0fa75ca3e7c5ed19a1b0 Mon Sep 17 00:00:00 2001 From: Andres Rodriguez Date: Tue, 3 Oct 2023 10:58:51 -0400 Subject: [PATCH 09/11] Add native Logstash to Logstash documentation (#15346) Add native Logstash to Logstash documentation, which replaces HTTP Input/Output --- docs/static/ls-ls-config.asciidoc | 31 ++++---- docs/static/ls-ls-http.asciidoc | 6 +- docs/static/ls-ls-native.asciidoc | 128 ++++++++++++++++++++++++++++++ 3 files changed, 144 insertions(+), 21 deletions(-) create mode 100644 docs/static/ls-ls-native.asciidoc diff --git a/docs/static/ls-ls-config.asciidoc b/docs/static/ls-ls-config.asciidoc index a7adc5b4784..e84a8c5f394 100644 --- a/docs/static/ls-ls-config.asciidoc +++ b/docs/static/ls-ls-config.asciidoc @@ -10,12 +10,23 @@ one Logstash instance, see <>. Logstash-to-Logstash communication can be achieved in one of two ways: +* <> * <> -* <> + +[[native-considerations]]*Logstash to Logstash considerations* + +This is the preferred method to implement Logstash-to-Logstash. It replaces <> and has these considerations: + +* It relies on HTTP as the communication protocol between the Input and Output. +* It does not provide built-in high availability. You will need to implement your own load balancer in between the Logstash output and the Logstash input. +* If you need a proxy between the Logstash instances, you can use any HTTP proxy. +* No connection information is added to events. + +Ready to see more configuration details? See <>. [[lumberjack-considerations]]*Lumberjack-Beats considerations* -Lumberjack output to Beats input has been our standard approach for {ls}-to-{ls} communication, and may still be the best option for more robust use cases. +Lumberjack output to Beats input has been our standard approach for {ls}-to-{ls} communication, but our recommended approach is now <>. Before you implement the Lumberjack to Beats configuration, keep these points in mind: * Lumberjack to Beats provides high availability, but does not provide load balancing. @@ -25,20 +36,6 @@ The Lumberjack output plugin allows defining multiple output hosts for high avai Ready to see more configuration details? See <>. -[[http-considerations]]*HTTP-HTTP considerations* - -This approach relies on the use of <> to <> plugins. -Take these considerations into account before you implement: - -* HTTP does not provide built-in high availability. You will need to implement your own load balancer in between the HTTP output and the HTTP input. -* If you need a proxy between the Logstash instances, you can use any HTTP proxy. -* The HTTP input adds connection information to events, and this may be data you don't want. - -For now, <> to <> with manual configuration may be the best path forward if these limitations don't apply to your use case. - -Ready to see more configuration details? See <>. - -NOTE: In the future, we may replace the implementation of Logstash-to-Logstash with a purpose-build HTTP implementation, which would deprecate the use of Lumberjack and Beats, or the use of the HTTP Input and Output plugins. - include::ls-ls-lumberjack.asciidoc[] include::ls-ls-http.asciidoc[] +include::ls-ls-native.asciidoc[] diff --git a/docs/static/ls-ls-http.asciidoc b/docs/static/ls-ls-http.asciidoc index 6af1af7f772..445623cdbd6 100644 --- a/docs/static/ls-ls-http.asciidoc +++ b/docs/static/ls-ls-http.asciidoc @@ -4,9 +4,7 @@ HTTP output to HTTP input is an alternative to the Lumberjack output to Beats input approach for Logstash-to-Logstash communication. This approach relies on the use of <> to <> plugins. -NOTE: Check out these <> before you implement {ls}-to-{ls} using HTTP. - -For now, <> to <> with manual configuration may be the best path forward if these limitations don't apply to your use case. +NOTE: {ls}-to-{ls} using HTTP input/output plugins is now being deprecated in favor of <>. [[overview-http-http]] ==== Configuration overview @@ -60,7 +58,7 @@ output { } ---- -[[securing-logstash-to-logstash]] +[[securing-logstash-to-logstash-http]] ===== Secure Logstash to Logstash It is important that you secure the communication between Logstash instances. diff --git a/docs/static/ls-ls-native.asciidoc b/docs/static/ls-ls-native.asciidoc new file mode 100644 index 00000000000..52f26b3e5db --- /dev/null +++ b/docs/static/ls-ls-native.asciidoc @@ -0,0 +1,128 @@ +[[ls-to-ls-native]] +=== Logstash-to-Logstash: Output to Input + +The Logstash output to Logstash input is the default approach for Logstash-to-Logstash communication. + +NOTE: Check out these <> before you implement {ls}-to-{ls}. + +[[overview-ls-ls]] +==== Configuration overview + +To connect two Logstash instances: + +. Configure the downstream (server) Logstash to use Logstash input +. Configure the upstream (client) Logstash to use Logstash output +. Secure the communication between Logstash input and Logstash output + +[[configure-downstream-logstash-input]] +===== Configure the downstream Logstash to use Logstash input + +Configure the Logstash input on the downstream (receiving) Logstash to receive connections. +The minimum configuration requires this option: + +* `port` - To set a custom port. The default is 9800 if none is provided. + +[source,json] +---- +input { + logstash { + port => 9800 + } +} +---- + +[[configure-upstream-logstash-output]] +===== Configure the upstream Logstash to use Logstash output + +In order to obtain the best performance when sending data from one Logstash to another, the data is batched and compressed. As such, the upstream Logstash (the sending Logstash) only needs to be concerned about configuring the receiving endpoint with these options: + +* `hosts` - The receiving Logstash and port. If no port specified, 9800 will be used. + +NOTE: In the future, {ls} will support multiple output hosts. + +[source,json] +---- +output { + logstash { + hosts => '10.0.0.123:9800' + } +} +---- + +[[securing-logstash-to-logstash]] +===== Secure Logstash to Logstash + +It is important that you secure the communication between Logstash instances. +Use SSL/TLS mutual authentication in order to ensure that the upstream Logstash instance sends encrypted data to a trusted downstream Logstash instance, and vice versa. + +. Create a certificate authority (CA) in order to sign the certificates that you plan to use between Logstash instances. Creating a correct SSL/TLS infrastructure is outside the scope of this document. ++ +TIP: We recommend you use the {ref}/certutil.html[elasticsearch-certutil] tool to generate your certificates. + +. Configure the downstream (receiving) Logstash to use SSL. +Add these settings to the Logstash input configuration: ++ + * `ssl_enabled`: When set to `true`, it enables Logstash use of SSL/TLS + * `ssl_key`: Specifies the key that Logstash uses to authenticate with the client. + * `ssl_certificate`: Specifies the certificate that Logstash uses to authenticate with the client. + * `ssl_certificate_authorities`: Configures Logstash to trust any certificates signed by the specified CA. + * `ssl_client_authentication`: Specifies whether Logstash server verifies the client certificate against the CA. ++ +For example: ++ +[source,json] +---- +input { + logstash { + ... + + ssl_enabled => true + ssl_key => "server.pkcs8.key" + ssl_certificate => "server.crt" + ssl_certificate_authorities => "ca.crt" + ssl_client_authentication => required + } +} +---- + +. Configure the upstream (sending) Logstash to use SSL. +Add these settings to the Logstash output configuration: ++ + * `ssl_key`: Specifies the key the Logstash client uses to authenticate with the Logstash server. + * `ssl_certificate`: Specifies the certificate that the Logstash client uses to authenticate to the Logstash server. + * `ssl_certificate_authorities`: Configures the Logstash client to trust any certificates signed by the specified CA. ++ +For example: ++ +[source,json] +---- +output { + logstash { + ... + + ssl_enabled => true + ssl_key => "client.pkcs8.key" + ssl_certificate => "client.crt" + ssl_certificate_authorities => "ca.crt" + } +} +---- + +. If you would like an additional authentication step, you can also use basic user/password authentication in both Logstash instances: ++ + * `username`: Sets the username to use for authentication. + * `password`: Sets the password to use for authentication. ++ +For example, you would need to add the following to both Logstash instances: ++ +[source,json] +---- +... + logstash { + ... + + username => "your-user" + password => "your-secret" + } +... +---- From 70081bbcac96da694867d806798955941c2e5fe3 Mon Sep 17 00:00:00 2001 From: Ry Biesemeyer Date: Tue, 3 Oct 2023 14:32:28 -0700 Subject: [PATCH 10/11] deps: downgrade jruby, keep updated default-gem dependencies (forward-port #15283) (#15369) * deps: downgrade jruby, keep updated default-gem dependencies (#15283) forward-ports non-release-branch components of #15283 to `main` * deps: downgrade jruby, keep updated default-gem dependencies By downgrading JRuby to 9.4.2.0 we avoid the silent global crash of the scheduler backing `Concurrent::TimerTask` that occurs when Jruby 9.4.3.0's invokedynamic promotes a method to run natively, incorrectly. Upstream bug: https://github.com/jruby/jruby/issues/7904 Along with the downgrade of JRuby itself to 9.4.2.0, we cherry-pick the updates to gems that were included in the latest JRuby 9.4.3.0 to ensure we don't back out relevant fixes to stdlib. We also remove a pinned-dependency on `racc` that is no longer relevant. Resolves: https://github.com/elastic/logstash/issues/15282 * Imported the licenses for some gems - cgi - date - ffi-binary-libfixposix - io-console - net-http - net-protocol - reline - time - timeout - uri * specs: avoid mocking global ::Gem::Dependency::new * build: remove redundanct dependsOn declaration * deps: notice use of ffi-binary-libfixposix via Ruby license this gem is tri-licensed `Ruby` / `EPL-2.0` / `LGPL-2.1-or-later` and the Ruby license is preferred to EPL when available --------- Co-authored-by: andsel * deps: add license notices for gems moved from default to bundled --------- Co-authored-by: andsel --- NOTICE.TXT | 756 +++++++++++++++--- lib/pluginmanager/util.rb | 8 +- logstash-core/logstash-core.gemspec | 16 +- rubyUtils.gradle | 2 +- spec/unit/plugin_manager/util_spec.rb | 2 +- .../src/main/resources/licenseMapping.csv | 15 + .../resources/notices/bigdecimal-NOTICE.txt | 58 ++ .../src/main/resources/notices/cgi-NOTICE.txt | 22 + .../main/resources/notices/date-NOTICE.txt | 22 + .../notices/ffi-binary-libfixposix-NOTICE.txt | 58 ++ .../resources/notices/io-console-NOTICE.txt | 22 + .../resources/notices/mini_mime-NOTICE.txt | 23 + .../resources/notices/net-http-NOTICE.txt | 22 + .../resources/notices/net-imap-NOTICE.txt | 86 ++ .../main/resources/notices/net-pop-NOTICE.txt | 24 + .../resources/notices/net-protocol-NOTICE.txt | 22 + .../resources/notices/net-smtp-NOTICE.txt | 24 + .../main/resources/notices/reline-NOTICE.txt | 22 + .../main/resources/notices/time-NOTICE.txt | 22 + .../main/resources/notices/timeout-NOTICE.txt | 22 + .../src/main/resources/notices/uri-NOTICE.txt | 22 + versions.yml | 4 +- 22 files changed, 1167 insertions(+), 107 deletions(-) create mode 100644 tools/dependencies-report/src/main/resources/notices/bigdecimal-NOTICE.txt create mode 100644 tools/dependencies-report/src/main/resources/notices/cgi-NOTICE.txt create mode 100644 tools/dependencies-report/src/main/resources/notices/date-NOTICE.txt create mode 100644 tools/dependencies-report/src/main/resources/notices/ffi-binary-libfixposix-NOTICE.txt create mode 100644 tools/dependencies-report/src/main/resources/notices/io-console-NOTICE.txt create mode 100644 tools/dependencies-report/src/main/resources/notices/mini_mime-NOTICE.txt create mode 100644 tools/dependencies-report/src/main/resources/notices/net-http-NOTICE.txt create mode 100644 tools/dependencies-report/src/main/resources/notices/net-imap-NOTICE.txt create mode 100644 tools/dependencies-report/src/main/resources/notices/net-pop-NOTICE.txt create mode 100644 tools/dependencies-report/src/main/resources/notices/net-protocol-NOTICE.txt create mode 100644 tools/dependencies-report/src/main/resources/notices/net-smtp-NOTICE.txt create mode 100644 tools/dependencies-report/src/main/resources/notices/reline-NOTICE.txt create mode 100644 tools/dependencies-report/src/main/resources/notices/time-NOTICE.txt create mode 100644 tools/dependencies-report/src/main/resources/notices/timeout-NOTICE.txt create mode 100644 tools/dependencies-report/src/main/resources/notices/uri-NOTICE.txt diff --git a/NOTICE.TXT b/NOTICE.TXT index 9134966a019..06e5a85491c 100644 --- a/NOTICE.TXT +++ b/NOTICE.TXT @@ -90,7 +90,7 @@ Copyright © 2019 Red Hat, Inc. All rights reserved. “Red Hat,” is a registe All other trademarks are the property of their respective owners. ========== -Notice for: addressable-2.8.0 +Notice for: addressable-2.8.5 ---------- Copyright © Bob Aman @@ -298,7 +298,7 @@ See the License for the specific language governing permissions and limitations under the License. ========== -Notice for: amazing_print-1.4.0 +Notice for: amazing_print-1.5.0 ---------- MIT License @@ -722,7 +722,7 @@ Notice for: aws-eventstream-1.2.0 limitations under the License. ========== -Notice for: aws-partitions-1.610.0 +Notice for: aws-partitions-1.804.0 ---------- @@ -929,7 +929,7 @@ Notice for: aws-partitions-1.610.0 limitations under the License. ========== -Notice for: aws-sdk-cloudfront-1.65.0 +Notice for: aws-sdk-cloudfront-1.82.0 ---------- @@ -1136,7 +1136,7 @@ Notice for: aws-sdk-cloudfront-1.65.0 limitations under the License. ========== -Notice for: aws-sdk-cloudwatch-1.65.0 +Notice for: aws-sdk-cloudwatch-1.78.0 ---------- @@ -1343,7 +1343,7 @@ Notice for: aws-sdk-cloudwatch-1.65.0 limitations under the License. ========== -Notice for: aws-sdk-core-3.131.3 +Notice for: aws-sdk-core-3.180.3 ---------- @@ -1550,7 +1550,7 @@ Notice for: aws-sdk-core-3.131.3 limitations under the License. ========== -Notice for: aws-sdk-kms-1.58.0 +Notice for: aws-sdk-kms-1.71.0 ---------- @@ -1757,7 +1757,7 @@ Notice for: aws-sdk-kms-1.58.0 limitations under the License. ========== -Notice for: aws-sdk-resourcegroups-1.45.0 +Notice for: aws-sdk-resourcegroups-1.53.0 ---------- @@ -1964,7 +1964,7 @@ Notice for: aws-sdk-resourcegroups-1.45.0 limitations under the License. ========== -Notice for: aws-sdk-s3-1.114.0 +Notice for: aws-sdk-s3-1.132.1 ---------- @@ -2171,7 +2171,7 @@ Notice for: aws-sdk-s3-1.114.0 limitations under the License. ========== -Notice for: aws-sdk-sns-1.53.0 +Notice for: aws-sdk-sns-1.65.0 ---------- @@ -2378,7 +2378,7 @@ Notice for: aws-sdk-sns-1.53.0 limitations under the License. ========== -Notice for: aws-sdk-sqs-1.51.1 +Notice for: aws-sdk-sqs-1.62.0 ---------- @@ -2585,7 +2585,7 @@ Notice for: aws-sdk-sqs-1.51.1 limitations under the License. ========== -Notice for: aws-sigv4-1.5.1 +Notice for: aws-sigv4-1.6.0 ---------- @@ -2810,7 +2810,7 @@ See the License for the specific language governing permissions and limitations under the License. ========== -Notice for: bindata-2.4.10 +Notice for: bindata-2.4.15 ---------- Copyright (C) 2007-2012 Dion Mendel. All rights reserved. @@ -2900,7 +2900,7 @@ You can redistribute it and/or modify it under either the terms of the PURPOSE. ========== -Notice for: bundler-2.3.18 +Notice for: bundler-2.3.25 ---------- Portions copyright (c) 2010 Andre Arko @@ -2945,6 +2945,32 @@ See the License for the specific language governing permissions and limitations under the License. +========== +Notice for: cgi-0.3.6 +---------- + +Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. ========== Notice for: clamp-1.0.1 ---------- @@ -2997,7 +3023,7 @@ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ========== -Notice for: com.fasterxml.jackson.core:jackson-annotations-2.13.3 +Notice for: com.fasterxml.jackson.core:jackson-annotations-2.15.2 ---------- This copy of Jackson JSON processor annotations is licensed under the @@ -3214,7 +3240,7 @@ http://www.apache.org/licenses/LICENSE-2.0 limitations under the License. ========== -Notice for: com.fasterxml.jackson.core:jackson-core-2.13.3 +Notice for: com.fasterxml.jackson.core:jackson-core-2.15.2 ---------- # Jackson JSON processor @@ -3441,7 +3467,7 @@ from the source code management (SCM) system project uses. See the License for the specific language governing permissions and limitations under the License. ========== -Notice for: com.fasterxml.jackson.core:jackson-databind-2.13.3 +Notice for: com.fasterxml.jackson.core:jackson-databind-2.15.2 ---------- # Jackson JSON processor @@ -3677,7 +3703,7 @@ http://www.apache.org/licenses/LICENSE-2.0 See the License for the specific language governing permissions and limitations under the License. ========== -Notice for: com.fasterxml.jackson.dataformat:jackson-dataformat-cbor-2.13.3 +Notice for: com.fasterxml.jackson.dataformat:jackson-dataformat-cbor-2.15.2 ---------- This copy of Jackson JSON processor databind module is licensed under the @@ -3892,7 +3918,7 @@ http://www.apache.org/licenses/LICENSE-2.0 See the License for the specific language governing permissions and limitations under the License. ========== -Notice for: com.fasterxml.jackson.dataformat:jackson-dataformat-yaml-2.13.3 +Notice for: com.fasterxml.jackson.dataformat:jackson-dataformat-yaml-2.15.2 ---------- This copy of Jackson JSON processor databind module is licensed under the @@ -4107,7 +4133,7 @@ http://www.apache.org/licenses/LICENSE-2.0 See the License for the specific language governing permissions and limitations under the License. ========== -Notice for: com.fasterxml.jackson.module:jackson-module-afterburner-2.13.3 +Notice for: com.fasterxml.jackson.module:jackson-module-afterburner-2.15.2 ---------- This copy of Jackson JSON processor databind module is licensed under the @@ -4322,7 +4348,7 @@ http://www.apache.org/licenses/LICENSE-2.0 See the License for the specific language governing permissions and limitations under the License. ========== -Notice for: com.google.googlejavaformat:google-java-format-1.13.0 +Notice for: com.google.googlejavaformat:google-java-format-1.15.0 ---------- Copyright 2015 Google Inc. @@ -4598,7 +4624,7 @@ Program Directory ========== -Notice for: com.google.guava:guava-31.0.1-jre +Notice for: com.google.guava:guava-31.1-jre ---------- @@ -4804,7 +4830,7 @@ Notice for: com.google.guava:guava-31.0.1-jre See the License for the specific language governing permissions and limitations under the License. ========== -Notice for: commons-codec:commons-codec-1.14 +Notice for: commons-codec:commons-codec-1.15 ---------- Apache Commons Codec @@ -5266,44 +5292,7 @@ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ========== -Notice for: csv-3.2.5 ----------- - -Copyright (C) 2005-2016 James Edward Gray II. All rights reserved. -Copyright (C) 2007-2017 Yukihiro Matsumoto. All rights reserved. -Copyright (C) 2017 SHIBATA Hiroshi. All rights reserved. -Copyright (C) 2017 Olivier Lacan. All rights reserved. -Copyright (C) 2017 Espartaco Palma. All rights reserved. -Copyright (C) 2017 Marcus Stollsteimer. All rights reserved. -Copyright (C) 2017 pavel. All rights reserved. -Copyright (C) 2017-2018 Steven Daniels. All rights reserved. -Copyright (C) 2018 Tomohiro Ogoke. All rights reserved. -Copyright (C) 2018 Kouhei Sutou. All rights reserved. -Copyright (C) 2018 Mitsutaka Mimura. All rights reserved. -Copyright (C) 2018 Vladislav. All rights reserved. - -Redistribution and use in source and binary forms, with or without -modification, are permitted provided that the following conditions -are met: -1. Redistributions of source code must retain the above copyright -notice, this list of conditions and the following disclaimer. -2. Redistributions in binary form must reproduce the above copyright -notice, this list of conditions and the following disclaimer in the -documentation and/or other materials provided with the distribution. - -THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND -ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE -FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS -OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) -HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT -LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY -OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF -SUCH DAMAGE. -========== -Notice for: dalli-2.7.11 +Notice for: dalli-3.2.5 ---------- Copyright (c) Peter M. Goldstein, Mike Perham @@ -5327,6 +5316,32 @@ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +========== +Notice for: date-3.3.3 +---------- + +Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. ========== Notice for: domain_name-0.5.20190701 ---------- @@ -5411,7 +5426,7 @@ This file is generated from the Public Suffix List https://mozilla.org/MPL/2.0/ ========== -Notice for: dotenv-2.7.6 +Notice for: dotenv-2.8.1 ---------- Copyright (c) 2012 Brandon Keepers @@ -5463,6 +5478,34 @@ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ========== +Notice for: e2mmap-0.1.0 +---------- + +source: https://github.com/ruby/e2mmap/blob/master/LICENSE.txt + +Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. +========== Notice for: edn-1.1.1 ---------- @@ -5716,7 +5759,7 @@ Elastic Workplace Search Ruby client. Copyright 2012-2020 Elasticsearch B.V. ========== -Notice for: elasticsearch-7.17.1 +Notice for: elasticsearch-7.17.7 ---------- source: https://github.com/elastic/elasticsearch-ruby/blob/v5.0.4/elasticsearch-api/LICENSE.txt @@ -5736,7 +5779,7 @@ See the License for the specific language governing permissions and limitations under the License. ========== -Notice for: elasticsearch-api-7.17.1 +Notice for: elasticsearch-api-7.17.7 ---------- source: https://github.com/elastic/elasticsearch-ruby/blob/v5.0.4/elasticsearch-transport/LICENSE.txt @@ -5756,7 +5799,7 @@ See the License for the specific language governing permissions and limitations under the License. ========== -Notice for: elasticsearch-transport-7.17.1 +Notice for: elasticsearch-transport-7.17.7 ---------- source: https://github.com/elastic/elasticsearch-ruby/blob/v5.0.4/elasticsearch/LICENSE.txt @@ -5832,7 +5875,7 @@ THE SOFTWARE. Made in Japan ========== -Notice for: faraday-1.10.0 +Notice for: faraday-1.10.3 ---------- source: https://github.com/lostisland/faraday/blob/v0.9.2/LICENSE.md @@ -6049,6 +6092,287 @@ ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +========== +Notice for: ffi-binary-libfixposix-0.5.1.1 +---------- + +Eclipse Public License - v 2.0 + + THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE + PUBLIC LICENSE ("AGREEMENT"). ANY USE, REPRODUCTION OR DISTRIBUTION + OF THE PROGRAM CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS AGREEMENT. + +1. DEFINITIONS + +"Contribution" means: + + a) in the case of the initial Contributor, the initial content + Distributed under this Agreement, and + + b) in the case of each subsequent Contributor: + i) changes to the Program, and + ii) additions to the Program; + where such changes and/or additions to the Program originate from + and are Distributed by that particular Contributor. A Contribution + "originates" from a Contributor if it was added to the Program by + such Contributor itself or anyone acting on such Contributor's behalf. + Contributions do not include changes or additions to the Program that + are not Modified Works. + +"Contributor" means any person or entity that Distributes the Program. + +"Licensed Patents" mean patent claims licensable by a Contributor which +are necessarily infringed by the use or sale of its Contribution alone +or when combined with the Program. + +"Program" means the Contributions Distributed in accordance with this +Agreement. + +"Recipient" means anyone who receives the Program under this Agreement +or any Secondary License (as applicable), including Contributors. + +"Derivative Works" shall mean any work, whether in Source Code or other +form, that is based on (or derived from) the Program and for which the +editorial revisions, annotations, elaborations, or other modifications +represent, as a whole, an original work of authorship. + +"Modified Works" shall mean any work in Source Code or other form that +results from an addition to, deletion from, or modification of the +contents of the Program, including, for purposes of clarity any new file +in Source Code form that contains any contents of the Program. Modified +Works shall not include works that contain only declarations, +interfaces, types, classes, structures, or files of the Program solely +in each case in order to link to, bind by name, or subclass the Program +or Modified Works thereof. + +"Distribute" means the acts of a) distributing or b) making available +in any manner that enables the transfer of a copy. + +"Source Code" means the form of a Program preferred for making +modifications, including but not limited to software source code, +documentation source, and configuration files. + +"Secondary License" means either the GNU General Public License, +Version 2.0, or any later versions of that license, including any +exceptions or additional permissions as identified by the initial +Contributor. + +2. GRANT OF RIGHTS + + a) Subject to the terms of this Agreement, each Contributor hereby + grants Recipient a non-exclusive, worldwide, royalty-free copyright + license to reproduce, prepare Derivative Works of, publicly display, + publicly perform, Distribute and sublicense the Contribution of such + Contributor, if any, and such Derivative Works. + + b) Subject to the terms of this Agreement, each Contributor hereby + grants Recipient a non-exclusive, worldwide, royalty-free patent + license under Licensed Patents to make, use, sell, offer to sell, + import and otherwise transfer the Contribution of such Contributor, + if any, in Source Code or other form. This patent license shall + apply to the combination of the Contribution and the Program if, at + the time the Contribution is added by the Contributor, such addition + of the Contribution causes such combination to be covered by the + Licensed Patents. The patent license shall not apply to any other + combinations which include the Contribution. No hardware per se is + licensed hereunder. + + c) Recipient understands that although each Contributor grants the + licenses to its Contributions set forth herein, no assurances are + provided by any Contributor that the Program does not infringe the + patent or other intellectual property rights of any other entity. + Each Contributor disclaims any liability to Recipient for claims + brought by any other entity based on infringement of intellectual + property rights or otherwise. As a condition to exercising the + rights and licenses granted hereunder, each Recipient hereby + assumes sole responsibility to secure any other intellectual + property rights needed, if any. For example, if a third party + patent license is required to allow Recipient to Distribute the + Program, it is Recipient's responsibility to acquire that license + before distributing the Program. + + d) Each Contributor represents that to its knowledge it has + sufficient copyright rights in its Contribution, if any, to grant + the copyright license set forth in this Agreement. + + e) Notwithstanding the terms of any Secondary License, no + Contributor makes additional grants to any Recipient (other than + those set forth in this Agreement) as a result of such Recipient's + receipt of the Program under the terms of a Secondary License + (if permitted under the terms of Section 3). + +3. REQUIREMENTS + +3.1 If a Contributor Distributes the Program in any form, then: + + a) the Program must also be made available as Source Code, in + accordance with section 3.2, and the Contributor must accompany + the Program with a statement that the Source Code for the Program + is available under this Agreement, and informs Recipients how to + obtain it in a reasonable manner on or through a medium customarily + used for software exchange; and + + b) the Contributor may Distribute the Program under a license + different than this Agreement, provided that such license: + i) effectively disclaims on behalf of all other Contributors all + warranties and conditions, express and implied, including + warranties or conditions of title and non-infringement, and + implied warranties or conditions of merchantability and fitness + for a particular purpose; + + ii) effectively excludes on behalf of all other Contributors all + liability for damages, including direct, indirect, special, + incidental and consequential damages, such as lost profits; + + iii) does not attempt to limit or alter the recipients' rights + in the Source Code under section 3.2; and + + iv) requires any subsequent distribution of the Program by any + party to be under a license that satisfies the requirements + of this section 3. + +3.2 When the Program is Distributed as Source Code: + + a) it must be made available under this Agreement, or if the + Program (i) is combined with other material in a separate file or + files made available under a Secondary License, and (ii) the initial + Contributor attached to the Source Code the notice described in + Exhibit A of this Agreement, then the Program may be made available + under the terms of such Secondary Licenses, and + + b) a copy of this Agreement must be included with each copy of + the Program. + +3.3 Contributors may not remove or alter any copyright, patent, +trademark, attribution notices, disclaimers of warranty, or limitations +of liability ("notices") contained within the Program from any copy of +the Program which they Distribute, provided that Contributors may add +their own appropriate notices. + +4. COMMERCIAL DISTRIBUTION + +Commercial distributors of software may accept certain responsibilities +with respect to end users, business partners and the like. While this +license is intended to facilitate the commercial use of the Program, +the Contributor who includes the Program in a commercial product +offering should do so in a manner which does not create potential +liability for other Contributors. Therefore, if a Contributor includes +the Program in a commercial product offering, such Contributor +("Commercial Contributor") hereby agrees to defend and indemnify every +other Contributor ("Indemnified Contributor") against any losses, +damages and costs (collectively "Losses") arising from claims, lawsuits +and other legal actions brought by a third party against the Indemnified +Contributor to the extent caused by the acts or omissions of such +Commercial Contributor in connection with its distribution of the Program +in a commercial product offering. The obligations in this section do not +apply to any claims or Losses relating to any actual or alleged +intellectual property infringement. In order to qualify, an Indemnified +Contributor must: a) promptly notify the Commercial Contributor in +writing of such claim, and b) allow the Commercial Contributor to control, +and cooperate with the Commercial Contributor in, the defense and any +related settlement negotiations. The Indemnified Contributor may +participate in any such claim at its own expense. + +For example, a Contributor might include the Program in a commercial +product offering, Product X. That Contributor is then a Commercial +Contributor. If that Commercial Contributor then makes performance +claims, or offers warranties related to Product X, those performance +claims and warranties are such Commercial Contributor's responsibility +alone. Under this section, the Commercial Contributor would have to +defend claims against the other Contributors related to those performance +claims and warranties, and if a court requires any other Contributor to +pay any damages as a result, the Commercial Contributor must pay +those damages. + +5. NO WARRANTY + +EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, AND TO THE EXTENT +PERMITTED BY APPLICABLE LAW, THE PROGRAM IS PROVIDED ON AN "AS IS" +BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, EITHER EXPRESS OR +IMPLIED INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OR CONDITIONS OF +TITLE, NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR +PURPOSE. Each Recipient is solely responsible for determining the +appropriateness of using and distributing the Program and assumes all +risks associated with its exercise of rights under this Agreement, +including but not limited to the risks and costs of program errors, +compliance with applicable laws, damage to or loss of data, programs +or equipment, and unavailability or interruption of operations. + +6. DISCLAIMER OF LIABILITY + +EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, AND TO THE EXTENT +PERMITTED BY APPLICABLE LAW, NEITHER RECIPIENT NOR ANY CONTRIBUTORS +SHALL HAVE ANY LIABILITY FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, +EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING WITHOUT LIMITATION LOST +PROFITS), HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +ARISING IN ANY WAY OUT OF THE USE OR DISTRIBUTION OF THE PROGRAM OR THE +EXERCISE OF ANY RIGHTS GRANTED HEREUNDER, EVEN IF ADVISED OF THE +POSSIBILITY OF SUCH DAMAGES. + +7. GENERAL + +If any provision of this Agreement is invalid or unenforceable under +applicable law, it shall not affect the validity or enforceability of +the remainder of the terms of this Agreement, and without further +action by the parties hereto, such provision shall be reformed to the +minimum extent necessary to make such provision valid and enforceable. + +If Recipient institutes patent litigation against any entity +(including a cross-claim or counterclaim in a lawsuit) alleging that the +Program itself (excluding combinations of the Program with other software +or hardware) infringes such Recipient's patent(s), then such Recipient's +rights granted under Section 2(b) shall terminate as of the date such +litigation is filed. + +All Recipient's rights under this Agreement shall terminate if it +fails to comply with any of the material terms or conditions of this +Agreement and does not cure such failure in a reasonable period of +time after becoming aware of such noncompliance. If all Recipient's +rights under this Agreement terminate, Recipient agrees to cease use +and distribution of the Program as soon as reasonably practicable. +However, Recipient's obligations under this Agreement and any licenses +granted by Recipient relating to the Program shall continue and survive. + +Everyone is permitted to copy and distribute copies of this Agreement, +but in order to avoid inconsistency the Agreement is copyrighted and +may only be modified in the following manner. The Agreement Steward +reserves the right to publish new versions (including revisions) of +this Agreement from time to time. No one other than the Agreement +Steward has the right to modify this Agreement. The Eclipse Foundation +is the initial Agreement Steward. The Eclipse Foundation may assign the +responsibility to serve as the Agreement Steward to a suitable separate +entity. Each new version of the Agreement will be given a distinguishing +version number. The Program (including Contributions) may always be +Distributed subject to the version of the Agreement under which it was +received. In addition, after a new version of the Agreement is published, +Contributor may elect to Distribute the Program (including its +Contributions) under the new version. + +Except as expressly stated in Sections 2(a) and 2(b) above, Recipient +receives no rights or licenses to the intellectual property of any +Contributor under this Agreement, whether expressly, by implication, +estoppel or otherwise. All rights in the Program not expressly granted +under this Agreement are reserved. Nothing in this Agreement is intended +to be enforceable by any entity that is not a Contributor or Recipient. +No third-party beneficiary rights are created under this Agreement. + +Exhibit A - Form of Secondary Licenses Notice + +"This Source Code may also be made available under the following +Secondary Licenses when the conditions for such availability set forth +in the Eclipse Public License, v. 2.0 are satisfied: {name license(s), +version(s), and exceptions or additional permissions here}." + + Simply including a copy of this Agreement, including this Exhibit A + is not sufficient to license the Source Code under Secondary Licenses. + + If it is not possible or desirable to put the notice in a particular + file, then You may include the notice in a location (such as a LICENSE + file in a relevant directory) where a recipient would be likely to + look for such a notice. + + You may add additional accurate notices of copyright ownership. ========== Notice for: filesize-0.2.0 ---------- @@ -6076,7 +6400,7 @@ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ========== -Notice for: fugit-1.5.3 +Notice for: fugit-1.8.1 ---------- source: https://github.com/floraison/fugit/blob/v1.5.2/LICENSE.txt @@ -6299,7 +6623,7 @@ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ========== -Notice for: i18n-1.12.0 +Notice for: i18n-1.14.1 ---------- source: https://github.com/svenfuchs/i18n/blob/v0.6.9/MIT-LICENSE @@ -6345,6 +6669,32 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. +========== +Notice for: io-console-0.6.0 +---------- + +Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. ========== Notice for: jar-dependencies-0.4.1 ---------- @@ -6603,7 +6953,7 @@ See the License for the specific language governing permissions and limitations under the License. ========== -Notice for: jmespath-1.6.1 +Notice for: jmespath-1.6.2 ---------- source: https://github.com/jmespath/jmespath.rb/blob/v1.4.0/LICENSE.txt @@ -6784,7 +7134,7 @@ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION of your accepting any such warranty or additional liability. ========== -Notice for: jrjackson-0.4.16 +Notice for: jrjackson-0.4.18 ---------- https://github.com/guyboertje/jrjackson/blob/v0.4.6/README.md @@ -9276,7 +9626,7 @@ See the License for the specific language governing permissions and limitations under the License. ========== -Notice for: jruby-openssl-0.14.0 +Notice for: jruby-openssl-0.14.2 ---------- source: https://github.com/jruby/jruby-openssl/blob/v0.9.21/LICENSE.txt @@ -9336,7 +9686,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. ========== -Notice for: json-2.6.2 +Notice for: json-2.6.3 ---------- source: https://github.com/tmattia/json-generator/blob/v0.1.0/LICENSE.txt @@ -9364,7 +9714,7 @@ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ========== -Notice for: jwt-2.4.1 +Notice for: jwt-2.7.1 ---------- source: https://github.com/jwt/ruby-jwt/blob/v2.2.2/LICENSE @@ -9459,7 +9809,7 @@ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ========== -Notice for: march_hare-4.4.0 +Notice for: march_hare-4.5.0 ---------- source: https://github.com/ruby-amqp/march_hare/blob/v3.1.1/LICENSE @@ -9606,7 +9956,7 @@ terms of Ruby’s licence or the Simplified BSD licence. * Portions copyright 2004 Mauricio Julio Fernández Pradier. ========== -Notice for: msgpack-1.5.4 +Notice for: msgpack-1.7.2 ---------- source: https://github.com/msgpack/msgpack-ruby/blob/v1.2.4/ext/msgpack/ @@ -9651,7 +10001,7 @@ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ========== -Notice for: multipart-post-2.2.3 +Notice for: multipart-post-2.3.0 ---------- source: https://github.com/nicksieger/multipart-post/blob/v2.0.0/README.md#license @@ -9734,7 +10084,7 @@ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ========== -Notice for: mustermann-1.0.3 +Notice for: mustermann-2.0.2 ---------- Copyright (c) 2013-2017 Konstantin Haase @@ -9791,7 +10141,59 @@ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ========== -Notice for: nio4r-2.5.8 +Notice for: net-http-0.3.2 +---------- + +Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. +========== +Notice for: net-protocol-0.1.3 +---------- + +Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. +========== +Notice for: nio4r-2.5.9 ---------- Released under the MIT license. @@ -9806,7 +10208,7 @@ The above copyright notice and this permission notice shall be included in all c THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ========== -Notice for: nokogiri-1.13.8 +Notice for: nokogiri-1.15.4 ---------- source: https://github.com/sparklemotion/nokogiri/blob/v1.8.2/LICENSE.md @@ -10044,7 +10446,7 @@ OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ========== -Notice for: org.javassist:javassist-3.26.0-GA +Notice for: org.javassist:javassist-3.29.0-GA ---------- Copyright (C) 1999-2019 by Shigeru Chiba, All rights reserved. @@ -10052,7 +10454,7 @@ Copyright (C) 1999-2019 by Shigeru Chiba, All rights reserved. This software is distributed under the Mozilla Public License Version 1.1, the GNU Lesser General Public License Version 2.1 or later, or the Apache License Version 2.0. ========== -Notice for: org.jruby:jruby-core-9.3.6.0 +Notice for: org.jruby:jruby-core-9.4.2.0 ---------- JRuby is Copyright (c) 2007-2018 The JRuby project @@ -10335,7 +10737,7 @@ Eclipse Public License - v 2.0 You may add additional accurate notices of copyright ownership. ========== -Notice for: org.logstash:jvm-options-parser-8.4.0 +Notice for: org.logstash:jvm-options-parser-8.10.0 ---------- Copyright (c) 2022 Elasticsearch B.V. @@ -10352,7 +10754,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. ========== -Notice for: org.reflections:reflections-0.9.12 +Notice for: org.reflections:reflections-0.10.2 ---------- DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE @@ -10456,7 +10858,7 @@ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ========== -Notice for: pry-0.14.1 +Notice for: pry-0.14.2 ---------- Copyright (c) 2016 John Mair (banisterfiend) @@ -10480,6 +10882,32 @@ CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +========== +Notice for: psych-5.1.0 +---------- + +MIT License + +Copyright (c) 2009 Aaron Patterson, et al. + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + ========== Notice for: public_suffix-3.1.1 ---------- @@ -10508,7 +10936,7 @@ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ========== -Notice for: puma-5.6.4 +Notice for: puma-6.3.1 ---------- Some code copyright (c) 2005, Zed Shaw @@ -10596,7 +11024,7 @@ OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ========== -Notice for: rack-2.2.4 +Notice for: rack-2.2.8 ---------- The MIT License (MIT) @@ -10620,7 +11048,7 @@ THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ========== -Notice for: rack-protection-2.1.0 +Notice for: rack-protection-2.2.4 ---------- The MIT License (MIT) @@ -10647,7 +11075,7 @@ CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ========== -Notice for: rake-12.3.3 +Notice for: rake-13.0.6 ---------- Copyright (c) Jim Weirich @@ -10673,7 +11101,7 @@ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ========== -Notice for: redis-4.7.1 +Notice for: redis-4.8.1 ---------- Copyright (c) 2009 Ezra Zygmuntowicz @@ -10697,7 +11125,33 @@ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ========== -Notice for: rexml-3.2.5 +Notice for: reline-0.3.8 +---------- + +Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. +========== +Notice for: rexml-3.2.6 ---------- source: https://github.com/ruby/rexml/blob/v3.2.5/LICENSE.txt @@ -10725,7 +11179,7 @@ LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ========== -Notice for: ruby-progressbar-1.11.0 +Notice for: ruby-progressbar-1.13.0 ---------- Copyright (c) 2010-2016 The Kompanee, Ltd @@ -10848,7 +11302,7 @@ You can redistribute it and/or modify it under either the terms of the PURPOSE. ========== -Notice for: rufus-scheduler-3.8.2 +Notice for: rufus-scheduler-3.9.1 ---------- @@ -10892,7 +11346,7 @@ See the License for the specific language governing permissions and limitations under the License. ========== -Notice for: sequel-5.58.0 +Notice for: sequel-5.71.0 ---------- Copyright (c) 2007-2008 Sharon Rosner @@ -10941,7 +11395,7 @@ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ========== -Notice for: sinatra-2.1.0 +Notice for: sinatra-2.2.4 ---------- Copyright (c) 2007, 2008, 2009 Blake Mizerany @@ -11356,7 +11810,35 @@ additional liability. END OF TERMS AND CONDITIONS ========== -Notice for: tilt-2.0.11 +Notice for: thwait-0.2.0 +---------- + +source: https://github.com/ruby/thwait/blob/master/LICENSE.txt + +Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. +========== +Notice for: tilt-2.2.0 ---------- Copyright (c) 2010-2016 Ryan Tomayko @@ -11379,7 +11861,59 @@ IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ========== -Notice for: treetop-1.6.11 +Notice for: time-0.2.2 +---------- + +Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. +========== +Notice for: timeout-0.3.2 +---------- + +Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. +========== +Notice for: treetop-1.6.12 ---------- Copyright (c) 2007 Nathan Sobo. @@ -11428,7 +11962,7 @@ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ========== -Notice for: tzinfo-2.0.5 +Notice for: tzinfo-2.0.6 ---------- Copyright (c) 2005-2018 Philip Ross @@ -11452,7 +11986,7 @@ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ========== -Notice for: tzinfo-data-1.2022.1 +Notice for: tzinfo-data-1.2023.3 ---------- Copyright (c) 2005-2018 Philip Ross @@ -11504,6 +12038,32 @@ LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +========== +Notice for: uri-0.12.2 +---------- + +Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. ========== Notice for: webhdfs-0.10.2 ---------- diff --git a/lib/pluginmanager/util.rb b/lib/pluginmanager/util.rb index af19178fd47..2d1e3a9173b 100644 --- a/lib/pluginmanager/util.rb +++ b/lib/pluginmanager/util.rb @@ -66,7 +66,7 @@ def self.logstash_plugin?(plugin, version = nil, options = {}) return false end else - dep = ::Gem::Dependency.new(plugin, version || ::Gem::Requirement.default) + dep = _gem_dependency(plugin, version) ::Gem.sources = ::Gem::SourceList.from(options[:rubygems_source]) if options[:rubygems_source] specs, errors = ::Gem::SpecFetcher.fetcher.spec_for_dependency(dep) @@ -87,6 +87,12 @@ def self.logstash_plugin?(plugin, version = nil, options = {}) end end + # This test injection point allows us to avoid mocking the ::Gem::Dependency + # that is used by ruby internals for finding dependencies on the load path + def self._gem_dependency(gem_name, version = nil) + ::Gem::Dependency.new(gem_name, version || ::Gem::Requirement.default) + end + # Fetch latest version information as in rubygems # @param [String] The plugin name # @param [Hash] Set of available options when fetching the information diff --git a/logstash-core/logstash-core.gemspec b/logstash-core/logstash-core.gemspec index 7c306ff5ca7..00aa9fb1a93 100644 --- a/logstash-core/logstash-core.gemspec +++ b/logstash-core/logstash-core.gemspec @@ -56,7 +56,7 @@ Gem::Specification.new do |gem| gem.add_runtime_dependency "rack", '~> 2' gem.add_runtime_dependency "sinatra", '~> 2' gem.add_runtime_dependency 'puma', '~> 6.3', '>= 6.3.1' - gem.add_runtime_dependency "jruby-openssl", "~> 0.11" + gem.add_runtime_dependency "jruby-openssl", "~> 0.14.1" gem.add_runtime_dependency "treetop", "~> 1" #(MIT license) @@ -86,7 +86,15 @@ Gem::Specification.new do |gem| # https://github.com/logstash-plugins/logstash-mixin-scheduler/blob/v1.0.1/lib/logstash/plugin_mixins/scheduler/rufus_impl.rb#L85= # and https://github.com/elastic/logstash/issues/13773 - # TEMPORARY: racc-1.6.0 doesn't have JAVA counterpart (yet) - # SEE: https://github.com/ruby/racc/issues/172 - gem.add_runtime_dependency "racc", "~> 1.5.2" #(Ruby license) + # TEMPORARY: delta between JRuby 9.4.2.0 and 9.4.3.0 + gem.add_runtime_dependency "cgi", "~> 0.3.6" + gem.add_runtime_dependency "date", "~> 3.3.3" + gem.add_runtime_dependency "ffi", "~> 1.15.5" + gem.add_runtime_dependency "net-http", "~> 0.3.0" + gem.add_runtime_dependency "net-protocol", "~> 0.1.2" + gem.add_runtime_dependency "reline", "~> 0.3.5" + gem.add_runtime_dependency "ffi-binary-libfixposix", "~> 0.5.1.1" + gem.add_runtime_dependency "time", "~> 0.2.2" + gem.add_runtime_dependency "timeout", "~> 0.3.2" + gem.add_runtime_dependency "uri", "~> 0.12.1" end diff --git a/rubyUtils.gradle b/rubyUtils.gradle index 85cc8de6ab8..67b3474763b 100644 --- a/rubyUtils.gradle +++ b/rubyUtils.gradle @@ -25,7 +25,7 @@ buildscript { dependencies { classpath "org.yaml:snakeyaml:${snakeYamlVersion}" classpath "de.undercouch:gradle-download-task:4.0.4" - classpath "org.jruby:jruby-core:9.4.3.0" + classpath "org.jruby:jruby-core:9.4.2.0" } } diff --git a/spec/unit/plugin_manager/util_spec.rb b/spec/unit/plugin_manager/util_spec.rb index 9d590a3394c..4240c1be482 100644 --- a/spec/unit/plugin_manager/util_spec.rb +++ b/spec/unit/plugin_manager/util_spec.rb @@ -75,7 +75,7 @@ it "should load all available sources" do expect(subject).to receive(:plugin_file?).and_return(false) - expect(Gem::Dependency).to receive(:new).and_return(dep) + expect(subject).to receive(:_gem_dependency).with(plugin, version).and_return(dep).once expect(Gem::SpecFetcher).to receive(:fetcher).and_return(fetcher) subject.logstash_plugin?(plugin, version, options) diff --git a/tools/dependencies-report/src/main/resources/licenseMapping.csv b/tools/dependencies-report/src/main/resources/licenseMapping.csv index 8a37f4fbd20..64294ca77f0 100644 --- a/tools/dependencies-report/src/main/resources/licenseMapping.csv +++ b/tools/dependencies-report/src/main/resources/licenseMapping.csv @@ -17,11 +17,13 @@ dependency,dependencyUrl,licenseOverride,copyright,sourceURL "aws-sdk-sqs:1.51.1",https://github.com/aws/aws-sdk-ruby/tree/version-3/gems/aws-sdk-sqs,Apache-2.0 "aws-sigv4:1.5.1",https://github.com/aws/aws-sdk-ruby/tree/version-3/gems/aws-sigv4,Apache-2.0 "back_pressure:",https://github.com/yaauie/ruby_back_pressure,Apache-2.0 +"bigdecimal:",https://github.com/ruby/bigdecimal,BSD-2-Clause "bindata:",http://github.com/dmendel/bindata,BSD-2-Clause "buftok:",https://github.com/sferik/buftok,MIT "builder:",http://onestepback.org,MIT "bundler:",https://bundler.io/,MIT "cabin:",https://github.com/jordansissel/ruby-cabin,Apache-2.0 +"cgi:",https://github.com/ruby/cgi,BSD-2-Clause "clamp:",http://github.com/mdub/clamp,MIT "coderay:",http://coderay.rubychan.de,MIT "com.fasterxml.jackson.core:jackson-annotations:",https://github.com/FasterXML/jackson-annotations,Apache-2.0 @@ -41,6 +43,7 @@ dependency,dependencyUrl,licenseOverride,copyright,sourceURL "concurrent-ruby:",http://www.concurrent-ruby.com,MIT "csv:",https://github.com/ruby/csv,BSD-2-Clause "dalli:",https://github.com/petergoldstein/dalli,MIT +"date:",https://github.com/ruby/date,BSD-2-Clause "domain_name:",https://github.com/knu/ruby-domain_name,BSD-2-Clause "dotenv:",https://github.com/bkeepers/dotenv,MIT "down",https://github.com/janko/down,MIT @@ -66,6 +69,7 @@ dependency,dependencyUrl,licenseOverride,copyright,sourceURL "faraday-rack",https://github.com/lostisland/faraday,MIT "faraday-retry",https://github.com/lostisland/faraday,MIT "ffi:",https://github.com/ffi/ffi,BSD-3-CLAUSE +"ffi-binary-libfixposix:",https://github.com/byteit101/subspawn,Ruby "filesize:",https://github.com/dominikh,MIT "fugit:",https://github.com/floraison/fugit,MIT "gelfd2:",https://github.com/ptqa/gelfd2,Apache-2.0 @@ -78,6 +82,7 @@ dependency,dependencyUrl,licenseOverride,copyright,sourceURL "http:",https://github.com/httprb/http,MIT "http_parser.rb:",https://github.com/tmm1/http_parser.rb,MIT "i18n:",https://github.com/svenfuchs/i18n,MIT +"io-console:",https://github.com/ruby/io-console,BSD-2-Clause "io.netty:netty-all:",https://github.com/netty/netty,Apache-2.0 "insist:",https://github.com/jordansissel/ruby-insist,Apache-2.0 "jar-dependencies:",https://github.com/mkristian/jar-dependencies,MIT @@ -99,6 +104,7 @@ dependency,dependencyUrl,licenseOverride,copyright,sourceURL "method_source:","https://github.com/banister/method_source/",MIT "metriks:","https://github.com/eric/metriks/",MIT "mime-types:",https://github.com/mime-types/ruby-mime-types/,MIT +"mini_mime:",https://github.com/discourse/mini_mime,MIT "minitar:",https://github.com/halostatue/minitar/,RUBY|BSD-2-Clause "msgpack:","https://github.com/msgpack/msgpack-ruby",Apache-2.0 "multi_json:","https://github.com/intridea/multi_json",MIT @@ -107,6 +113,11 @@ dependency,dependencyUrl,licenseOverride,copyright,sourceURL "mustermann:","https://github.com/sinatra/mustermann",MIT "mustache:","https://github.com/mustache/mustache",MIT "naught:","https://github.com/avdi/naught/",MIT +"net-http:",https://github.com/ruby/net-http,BSD-2-Clause +"net-imap:",https://github.com/ruby/net-imap,BSD-2-Clause +"net-pop:",https://github.com/ruby/net-pop,BSD-2-Clause +"net-protocol:",https://github.com/ruby/net-protocol,BSD-2-Clause +"net-smtp:",https://github.com/ruby/net-smtp,BSD-2-Clause "nio4r:","https://github.com/socketry/nio4r",MIT "nokogiri:","http://nokogiri.org/",MIT "openssl_pkcs8_pure:",http://github.com/cielavenir/openssl_pkcs8_pure,Ruby @@ -154,6 +165,7 @@ dependency,dependencyUrl,licenseOverride,copyright,sourceURL "rake:",https://github.com/ruby/rake,MIT "Red Hat Universal Base Image minimal:",https://catalog.redhat.com/software/containers/ubi8/ubi-minimal/5c359a62bed8bd75a2c3fba8,Custom;https://www.redhat.com/licenses/EULA_Red_Hat_Universal_Base_Image_English_20190422.pdf,,https://oss-dependencies.elastic.co/red-hat-universal-base-image-minimal/8/ubi-minimal-8-source.tar.gz "redis:",https://github.com/redis/redis-rb,MIT +"reline:",https://github.com/ruby/reline,BSD-2-Clause "rexml:",https://github.com/ruby/rexml,MIT "ruby-progressbar:",https://github.com/jfelchner/ruby-progressbar,MIT "ruby2_keywords",https://github.com/ruby/ruby2_keywords,BSD-2-Clause @@ -171,10 +183,13 @@ dependency,dependencyUrl,licenseOverride,copyright,sourceURL "thread_safe:",https://github.com/ruby-concurrency/thread_safe,Apache-2.0 "thwait:",https://github.com/ruby/thwait,BSD-2-Clause "tilt:",https://github.com/rtomayko/tilt,MIT +"time:",https://github.com/ruby/time,BSD-2-Clause +"timeout:",https://github.com/ruby/timeout,BSD-2-Clause "treetop:",https://github.com/cjheath/treetop,MIT "twitter:",https://github.com/sferik/twitter,MIT "tzinfo-data:",https://github.com/tzinfo/tzinfo-data,MIT "tzinfo:",https://github.com/tzinfo/tzinfo,MIT,Philip Ross "unf:",https://github.com/knu/ruby-unf,BSD-2-Clause +"uri:",https://github.com/ruby/uri,BSD-2-Clause "webhdfs:",https://github.com/kzk/webhdfs,Apache-2.0 "xml-simple:",https://github.com/maik/xml-simple,BSD-2-Clause diff --git a/tools/dependencies-report/src/main/resources/notices/bigdecimal-NOTICE.txt b/tools/dependencies-report/src/main/resources/notices/bigdecimal-NOTICE.txt new file mode 100644 index 00000000000..74f70c0c37f --- /dev/null +++ b/tools/dependencies-report/src/main/resources/notices/bigdecimal-NOTICE.txt @@ -0,0 +1,58 @@ +# source: https://github.com/ruby/bigdecimal/blob/v3.1.4/LICENSE + +Ruby is copyrighted free software by Yukihiro Matsumoto . +You can redistribute it and/or modify it under either the terms of the +2-clause BSDL (see the file BSDL), or the conditions below: + + 1. You may make and give away verbatim copies of the source form of the + software without restriction, provided that you duplicate all of the + original copyright notices and associated disclaimers. + + 2. You may modify your copy of the software in any way, provided that + you do at least ONE of the following: + + a) place your modifications in the Public Domain or otherwise + make them Freely Available, such as by posting said + modifications to Usenet or an equivalent medium, or by allowing + the author to include your modifications in the software. + + b) use the modified software only within your corporation or + organization. + + c) give non-standard binaries non-standard names, with + instructions on where to get the original software distribution. + + d) make other distribution arrangements with the author. + + 3. You may distribute the software in object code or binary form, + provided that you do at least ONE of the following: + + a) distribute the binaries and library files of the software, + together with instructions (in the manual page or equivalent) + on where to get the original distribution. + + b) accompany the distribution with the machine-readable source of + the software. + + c) give non-standard binaries non-standard names, with + instructions on where to get the original software distribution. + + d) make other distribution arrangements with the author. + + 4. You may modify and include the part of the software into any other + software (possibly commercial). But some files in the distribution + are not written by the author, so that they are not under these terms. + + For the list of those files and their copying conditions, see the + file LEGAL. + + 5. The scripts and library files supplied as input to or produced as + output from the software do not automatically fall under the + copyright of the software, but belong to whomever generated them, + and may be sold commercially, and may be aggregated with this + software. + + 6. THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR + IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED + WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + PURPOSE. \ No newline at end of file diff --git a/tools/dependencies-report/src/main/resources/notices/cgi-NOTICE.txt b/tools/dependencies-report/src/main/resources/notices/cgi-NOTICE.txt new file mode 100644 index 00000000000..44d33b0b5cd --- /dev/null +++ b/tools/dependencies-report/src/main/resources/notices/cgi-NOTICE.txt @@ -0,0 +1,22 @@ +Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. \ No newline at end of file diff --git a/tools/dependencies-report/src/main/resources/notices/date-NOTICE.txt b/tools/dependencies-report/src/main/resources/notices/date-NOTICE.txt new file mode 100644 index 00000000000..44d33b0b5cd --- /dev/null +++ b/tools/dependencies-report/src/main/resources/notices/date-NOTICE.txt @@ -0,0 +1,22 @@ +Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. \ No newline at end of file diff --git a/tools/dependencies-report/src/main/resources/notices/ffi-binary-libfixposix-NOTICE.txt b/tools/dependencies-report/src/main/resources/notices/ffi-binary-libfixposix-NOTICE.txt new file mode 100644 index 00000000000..4981c19c094 --- /dev/null +++ b/tools/dependencies-report/src/main/resources/notices/ffi-binary-libfixposix-NOTICE.txt @@ -0,0 +1,58 @@ +source: https://github.com/byteit101/subspawn/blob/lfp-0.5.1.1/LICENSE.RUBY + +Ruby is copyrighted free software by Yukihiro Matsumoto . +You can redistribute it and/or modify it under either the terms of the +2-clause BSDL (see the file BSDL), or the conditions below: + + 1. You may make and give away verbatim copies of the source form of the + software without restriction, provided that you duplicate all of the + original copyright notices and associated disclaimers. + + 2. You may modify your copy of the software in any way, provided that + you do at least ONE of the following: + + a) place your modifications in the Public Domain or otherwise + make them Freely Available, such as by posting said + modifications to Usenet or an equivalent medium, or by allowing + the author to include your modifications in the software. + + b) use the modified software only within your corporation or + organization. + + c) give non-standard binaries non-standard names, with + instructions on where to get the original software distribution. + + d) make other distribution arrangements with the author. + + 3. You may distribute the software in object code or binary form, + provided that you do at least ONE of the following: + + a) distribute the binaries and library files of the software, + together with instructions (in the manual page or equivalent) + on where to get the original distribution. + + b) accompany the distribution with the machine-readable source of + the software. + + c) give non-standard binaries non-standard names, with + instructions on where to get the original software distribution. + + d) make other distribution arrangements with the author. + + 4. You may modify and include the part of the software into any other + software (possibly commercial). But some files in the distribution + are not written by the author, so that they are not under these terms. + + For the list of those files and their copying conditions, see the + file LEGAL. + + 5. The scripts and library files supplied as input to or produced as + output from the software do not automatically fall under the + copyright of the software, but belong to whomever generated them, + and may be sold commercially, and may be aggregated with this + software. + + 6. THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR + IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED + WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + PURPOSE. \ No newline at end of file diff --git a/tools/dependencies-report/src/main/resources/notices/io-console-NOTICE.txt b/tools/dependencies-report/src/main/resources/notices/io-console-NOTICE.txt new file mode 100644 index 00000000000..44d33b0b5cd --- /dev/null +++ b/tools/dependencies-report/src/main/resources/notices/io-console-NOTICE.txt @@ -0,0 +1,22 @@ +Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. \ No newline at end of file diff --git a/tools/dependencies-report/src/main/resources/notices/mini_mime-NOTICE.txt b/tools/dependencies-report/src/main/resources/notices/mini_mime-NOTICE.txt new file mode 100644 index 00000000000..65b2b8a3a31 --- /dev/null +++ b/tools/dependencies-report/src/main/resources/notices/mini_mime-NOTICE.txt @@ -0,0 +1,23 @@ +# source: https://github.com/discourse/mini_mime/blob/v1.1.5/LICENSE.txt + +The MIT License (MIT) + +Copyright (c) 2016 Discourse Construction Kit, Inc. + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. \ No newline at end of file diff --git a/tools/dependencies-report/src/main/resources/notices/net-http-NOTICE.txt b/tools/dependencies-report/src/main/resources/notices/net-http-NOTICE.txt new file mode 100644 index 00000000000..44d33b0b5cd --- /dev/null +++ b/tools/dependencies-report/src/main/resources/notices/net-http-NOTICE.txt @@ -0,0 +1,22 @@ +Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. \ No newline at end of file diff --git a/tools/dependencies-report/src/main/resources/notices/net-imap-NOTICE.txt b/tools/dependencies-report/src/main/resources/notices/net-imap-NOTICE.txt new file mode 100644 index 00000000000..ccbc3033c7b --- /dev/null +++ b/tools/dependencies-report/src/main/resources/notices/net-imap-NOTICE.txt @@ -0,0 +1,86 @@ +# source: https://github.com/ruby/net-imap/blob/v0.3.7/LICENSE.txt + +Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. + +------------------------------------------------------------------------- + +This software includes documentation which has been copied from the relevant +RFCs. The copied documentation is covered by the following licenses: + +RFC 3501 (Editor: M. Crispin) +Full Copyright Statement + + Copyright (C) The Internet Society (2003). All Rights Reserved. + + This document and translations of it may be copied and furnished to + others, and derivative works that comment on or otherwise explain it + or assist in its implementation may be prepared, copied, published + and distributed, in whole or in part, without restriction of any + kind, provided that the above copyright notice and this paragraph are + included on all such copies and derivative works. However, this + document itself may not be modified in any way, such as by removing + the copyright notice or references to the Internet Society or other + Internet organizations, except as needed for the purpose of + developing Internet standards in which case the procedures for + copyrights defined in the Internet Standards process must be + followed, or as required to translate it into languages other than + English. + + The limited permissions granted above are perpetual and will not be + revoked by the Internet Society or its successors or assigns. v This + document and the information contained herein is provided on an "AS + IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK + FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT + LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL + NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY + OR FITNESS FOR A PARTICULAR PURPOSE. + + +RFC9051 (Editors: A. Melnikov, B. Leiba) +Copyright Notice + + Copyright (c) 2021 IETF Trust and the persons identified as the + document authors. All rights reserved. + + This document is subject to BCP 78 and the IETF Trust's Legal + Provisions Relating to IETF Documents + (https://trustee.ietf.org/license-info) in effect on the date of + publication of this document. Please review these documents + carefully, as they describe your rights and restrictions with respect + to this document. Code Components extracted from this document must + include Simplified BSD License text as described in Section 4.e of + the Trust Legal Provisions and are provided without warranty as + described in the Simplified BSD License. + + This document may contain material from IETF Documents or IETF + Contributions published or made publicly available before November + 10, 2008. The person(s) controlling the copyright in some of this + material may not have granted the IETF Trust the right to allow + modifications of such material outside the IETF Standards Process. + Without obtaining an adequate license from the person(s) controlling + the copyright in such materials, this document may not be modified + outside the IETF Standards Process, and derivative works of it may + not be created outside the IETF Standards Process, except to format + it for publication as an RFC or to translate it into languages other + than English. \ No newline at end of file diff --git a/tools/dependencies-report/src/main/resources/notices/net-pop-NOTICE.txt b/tools/dependencies-report/src/main/resources/notices/net-pop-NOTICE.txt new file mode 100644 index 00000000000..889bab1e52d --- /dev/null +++ b/tools/dependencies-report/src/main/resources/notices/net-pop-NOTICE.txt @@ -0,0 +1,24 @@ +source: https://github.com/ruby/net-pop/blob/v0.1.2/LICENSE.txt + +Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. \ No newline at end of file diff --git a/tools/dependencies-report/src/main/resources/notices/net-protocol-NOTICE.txt b/tools/dependencies-report/src/main/resources/notices/net-protocol-NOTICE.txt new file mode 100644 index 00000000000..44d33b0b5cd --- /dev/null +++ b/tools/dependencies-report/src/main/resources/notices/net-protocol-NOTICE.txt @@ -0,0 +1,22 @@ +Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. \ No newline at end of file diff --git a/tools/dependencies-report/src/main/resources/notices/net-smtp-NOTICE.txt b/tools/dependencies-report/src/main/resources/notices/net-smtp-NOTICE.txt new file mode 100644 index 00000000000..8ed718d8dcb --- /dev/null +++ b/tools/dependencies-report/src/main/resources/notices/net-smtp-NOTICE.txt @@ -0,0 +1,24 @@ +# source: https://github.com/ruby/net-smtp/blob/v0.4.0/LICENSE.txt + +Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. \ No newline at end of file diff --git a/tools/dependencies-report/src/main/resources/notices/reline-NOTICE.txt b/tools/dependencies-report/src/main/resources/notices/reline-NOTICE.txt new file mode 100644 index 00000000000..44d33b0b5cd --- /dev/null +++ b/tools/dependencies-report/src/main/resources/notices/reline-NOTICE.txt @@ -0,0 +1,22 @@ +Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. \ No newline at end of file diff --git a/tools/dependencies-report/src/main/resources/notices/time-NOTICE.txt b/tools/dependencies-report/src/main/resources/notices/time-NOTICE.txt new file mode 100644 index 00000000000..44d33b0b5cd --- /dev/null +++ b/tools/dependencies-report/src/main/resources/notices/time-NOTICE.txt @@ -0,0 +1,22 @@ +Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. \ No newline at end of file diff --git a/tools/dependencies-report/src/main/resources/notices/timeout-NOTICE.txt b/tools/dependencies-report/src/main/resources/notices/timeout-NOTICE.txt new file mode 100644 index 00000000000..44d33b0b5cd --- /dev/null +++ b/tools/dependencies-report/src/main/resources/notices/timeout-NOTICE.txt @@ -0,0 +1,22 @@ +Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. \ No newline at end of file diff --git a/tools/dependencies-report/src/main/resources/notices/uri-NOTICE.txt b/tools/dependencies-report/src/main/resources/notices/uri-NOTICE.txt new file mode 100644 index 00000000000..44d33b0b5cd --- /dev/null +++ b/tools/dependencies-report/src/main/resources/notices/uri-NOTICE.txt @@ -0,0 +1,22 @@ +Copyright (C) 1993-2013 Yukihiro Matsumoto. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. \ No newline at end of file diff --git a/versions.yml b/versions.yml index 07aedfd997b..cac20c6628e 100644 --- a/versions.yml +++ b/versions.yml @@ -13,8 +13,8 @@ bundled_jdk: # jruby must reference a *released* version of jruby which can be downloaded from the official download url # *and* for which jars artifacts are published for compile-time jruby: - version: 9.4.3.0 - sha1: 41c683cb5c493c7bda937527611a23a8e60677f0 + version: 9.4.2.0 + sha1: c338c1d3846e51b651e31e248097fdee4920056a # jruby-runtime-override, if specified, will override the jruby version installed in vendor/jruby #jruby-runtime-override: # url: https://oss.sonatype.org/content/repositories/snapshots/org/jruby/jruby-dist/9.3.0.0-SNAPSHOT/jruby-dist-9.3.0.0-20210723.214927-259-bin.tar.gz From 0adcc09bf646f9a34de2abcafb7ff5d0daeeeb72 Mon Sep 17 00:00:00 2001 From: Mashhur <99575341+mashhurs@users.noreply.github.com> Date: Tue, 3 Oct 2023 23:33:46 -0700 Subject: [PATCH 11/11] Bump core version to 8.12 (#15378) --- versions.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/versions.yml b/versions.yml index cac20c6628e..dff993425c5 100644 --- a/versions.yml +++ b/versions.yml @@ -1,7 +1,7 @@ --- # alpha and beta qualifiers are now added via VERSION_QUALIFIER environment var -logstash: 8.11.0 -logstash-core: 8.11.0 +logstash: 8.12.0 +logstash-core: 8.12.0 logstash-core-plugin-api: 2.1.16 bundled_jdk: