diff --git a/CHANGELOG.md b/CHANGELOG.md index 06c3a280..f3a52d57 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [2.17.0] - 2024-03-15 +### Dependencies +- Use appropriate version of `kubernetes` Python module to match CSM 1.6 Kubernetes version + +### Removed +- BOS v1 + ## [2.16.0] - 2024-03-15 ### Changed - Install uWSGI using `apk` instead of `pip`; update uWSGI config file to point to Python virtualenv diff --git a/Dockerfile b/Dockerfile index 8b606a1e..e5404020 100644 --- a/Dockerfile +++ b/Dockerfile @@ -62,18 +62,15 @@ RUN apk add --upgrade --no-cache apk-tools busybox && \ ENV VIRTUAL_ENV=/app/venv RUN python3 -m venv $VIRTUAL_ENV ENV PATH="$VIRTUAL_ENV/bin:$PATH" -RUN pip3 install --no-cache-dir -U pip +RUN pip3 install --no-cache-dir -U pip -c constraints.txt RUN --mount=type=secret,id=netrc,target=/root/.netrc pip3 install --no-cache-dir -r requirements.txt -RUN cd lib && pip3 install --no-cache-dir . +RUN cd lib && pip3 install --no-cache-dir . -c ../constraints.txt -# Testing image +# Base testing image FROM base as testing WORKDIR /app -COPY docker_test_entry.sh . COPY test-requirements.txt . -RUN apk add --no-cache --repository https://arti.hpc.amslabs.hpecorp.net/artifactory/mirror-alpine/edge/testing/ etcd etcd-ctl RUN --mount=type=secret,id=netrc,target=/root/.netrc cd /app && pip3 install --no-cache-dir -r test-requirements.txt -CMD [ "./docker_test_entry.sh" ] # Codestyle reporting FROM testing as codestyle @@ -101,7 +98,7 @@ FROM intermediate as debug ENV PYTHONPATH "/app/lib/server" WORKDIR /app RUN apk add --no-cache busybox-extras && \ - pip3 install --no-cache-dir rpdb + pip3 install --no-cache-dir rpdb -c constraints.txt # Application image FROM intermediate as application diff --git a/README.md b/README.md index 9f101428..df0fa1cb 100644 --- a/README.md +++ b/README.md @@ -27,7 +27,6 @@ nodes (Node Personalization) or Images (Image Customization). * Boot -- Boot nodes that are off * Reboot -- Gracefully power down nodes that are on and then power them back up * Shutdown -- Gracefully power down nodes that are on - * Reconfigure -- Reconfigure the nodes using the Configuration Framework Service (CFS) (Currently, not supported.) * Boot Set -- A collection of nodes that you want to perform an operation upon. It contains * A list of nodes * The following applies to booting or rebooting: @@ -40,9 +39,7 @@ nodes (Node Personalization) or Images (Image Customization). * `boot_sets`: One or more Boot Sets as described above * `enable_cfs`: Whether to enable the Configuration Framework Service (CFS); Choices: true/false * cfs: The configuration framework service configuration options to use for all boot sets that don't already define their own. -* Session -- Performs an Operation (action) on a Session Template. The creation of - a Session results in the creation of one or more Kubernetes BOA jobs which interact - with other Shasta subsystems to perform the requested operation. +* Session -- Performs an Operation (action) on a Session Template. ## Launching a Boot Session @@ -148,17 +145,6 @@ $ cd $REPO $ ./regenerate_server.sh ``` -## Dependency: cray-boa - -`cray-bos` uses the `cray-boa` image built by the [`boa`](https://github.com/Cray-HPE/boa) repository. -We specify which major and minor version of the image we want with the -[`update_external_versions.conf`](update_external_versions.conf) file. -At build time the [`runBuildPrep.sh`](runBuildPrep.sh) script calls a utility -which finds the latest version with that major and minor number. - -When creating a new release branch, be sure to update this file to specify the -desired major and minor number of the image for the new release. - ## Build Helpers This repository uses some build helper scripts from the diff --git a/api/openapi.yaml.in b/api/openapi.yaml.in index 53b3a1b2..f0b1ca98 100644 --- a/api/openapi.yaml.in +++ b/api/openapi.yaml.in @@ -43,11 +43,11 @@ info: a. List available Session Templates. - GET /v1/sessiontemplate or /v2/sessiontemplates + GET /v2/sessiontemplates b. Create a new Session Template if desired. - POST /v1/sessiontemplate or PUT /v2/sessiontemplate/{template_name} + PUT /v2/sessiontemplate/{template_name} If no Session Template exists that satisfies requirements, then create a new Session Template. @@ -55,7 +55,7 @@ info: 2. Create the Session. - POST /v1/session or /v2/sessions + POST /v2/sessions Specify template_name and an operation to create a new Session. The template_name corresponds to the Session Template *name*. @@ -76,7 +76,7 @@ info: 3. Get details on the Session. - GET /v1/session/{session_id} or /v2/sessions/{session_id} + GET /v2/sessions/{session_id} ## Interactions with Other APIs @@ -84,10 +84,7 @@ info: ### Configuration Framework Service (CFS) If *enable_cfs* is true in a Session Template, then BOS will invoke CFS to - configure the target nodes during *boot*, *reboot*, or *configure* - operations. The *configure* operation is only available in BOS v1 Sessions; - if desiring to only perform a CFS configuration on a set of nodes, it is - recommended to use CFS directly. + configure the target nodes during *boot* or *reboot* operations. ### Hardware State Manager (HSM) @@ -286,18 +283,6 @@ components: example: ["x3000c0s19b1n0", "x3000c0s19b2n0"] items: $ref: '#/components/schemas/HardwareComponentName' - NodeListEmptyOk: - type: array - description: | - A node list that is allowed to be empty. - - It is recommended that this list should be no more than 65535 items in length. - - This restriction is not enforced in this version of BOS, but it is - targeted to start being enforced in an upcoming BOS version. - example: ["x3000c0s19b1n0", "x3000c0s19b2n0"] - items: - $ref: '#/components/schemas/HardwareComponentName' NodeGroupList: type: array description: | @@ -432,226 +417,173 @@ components: $ref: '#/components/schemas/LinkList' additionalProperties: false - # V1 - V1CfsBranch: - type: string - deprecated: true - description: | - The name of the branch containing the configuration that you want to - apply to the nodes. Mutually exclusive with commit. (DEPRECATED) - - It is recommended that this should be 1-1023 characters in length. - - When upgrading to this version of BOS, all existing V1 session - templates will automatically have this deprecated field removed from them. - - When a V1 session template is created, this deprecated field is - automatically removed from it before storing it in BOS. - V1CfsUrl: + # V2 + V2TenantName: type: string - deprecated: true description: | - The clone URL for the repository providing the configuration. (DEPRECATED) - - It is recommended that this should be 1-4096 characters in length. - - When upgrading to this version of BOS, all existing V1 session - templates will automatically have this deprecated field removed from them. - - When a V1 session template is created, this deprecated field is - automatically removed from it before storing it in BOS. - V1CfsParameters: + Name of the tenant that owns this resource. Only used in environments + with multi-tenancy enabled. An empty string or null value means the resource + is not owned by a tenant. The absence of this field from a resource indicates + the same. + nullable: true + readOnly: true + V2CfsParameters: type: object description: | This is the collection of parameters that are passed to the Configuration - Framework Service when configuration is enabled. + Framework Service when configuration is enabled. Can be set as the global value for + a Session Template, or individually within a Boot Set. properties: - clone_url: - $ref: '#/components/schemas/V1CfsUrl' - branch: - $ref: '#/components/schemas/V1CfsBranch' - commit: - type: string - deprecated: true - description: | - The commit ID of the configuration that you want to - apply to the nodes. Mutually exclusive with branch. (DEPRECATED) - - git commit hashes are hexadecimal strings with a length of 40 characters (although - fewer characters may be sufficient to uniquely identify a commit in some cases). - - When upgrading to this version of BOS, all existing V1 session - templates will automatically have this deprecated field removed from them. - - When a V1 session template is created, this deprecated field is - automatically removed from it before storing it in BOS. - playbook: - type: string - deprecated: true - description: | - The name of the playbook to run for configuration. The file path must be specified - relative to the base directory of the config repository. (DEPRECATED) - - It is recommended that this should be 1-255 characters in length. - - When upgrading to this version of BOS, all existing V1 session - templates will automatically have this deprecated field removed from them. - - When a V1 session template is created, this deprecated field is - automatically removed from it before storing it in BOS. configuration: $ref: '#/components/schemas/CfsConfiguration' additionalProperties: false - V1CompleteMetadata: - type: boolean - description: Is the object's status complete - example: true - V1ErrorCountMetadata: - type: integer - description: How many errors were encountered - example: 0 - V1InProgressMetadata: - type: boolean - description: Is the object still doing something - example: false - V1StartTimeMetadata: - type: string - description: The start time - example: "2020-04-24T12:00" - V1StopTimeMetadata: - type: string - description: The stop time. In some contexts, the value may be null before the operation finishes. - example: "2020-04-24T12:00" - nullable: true - V1GenericMetadata: - type: object - description: | - The status metadata - properties: - complete: - $ref: '#/components/schemas/V1CompleteMetadata' - error_count: - $ref: '#/components/schemas/V1ErrorCountMetadata' - in_progress: - $ref: '#/components/schemas/V1InProgressMetadata' - start_time: - $ref: '#/components/schemas/V1StartTimeMetadata' - stop_time: - $ref: '#/components/schemas/V1StopTimeMetadata' - additionalProperties: false - V1PhaseCategoryName: - type: string - description: | - Name of the Phase Category - not_started, in_progress, succeeded, failed, or excluded - example: "Succeeded" - pattern: '^([nN][oO][tT]_[sS][tT][aA][rR][tT][eE][dD]|[iI][nN]_[pP][rR][oO][gG][rR][eE][sS][sS]|[sS][uU][cC][cC][eE][eE][dD][eE][dD]|[fF][aA][iI][lL][eE][dD]|[eE][xX][cC][lL][uU][dD][eE][dD])$' - V1PhaseCategoryStatus: + V2SessionTemplate: type: object description: | - A list of the nodes in a given category within a Phase. + A Session Template object represents a collection of resources and metadata. + A Session Template is used to create a Session which applies the data to + group of Components. ## Link Relationships - * self : The phase category status object - + * self : The Session Template object properties: name: - $ref: '#/components/schemas/V1PhaseCategoryName' - node_list: - $ref: '#/components/schemas/NodeListEmptyOk' - V1PhaseStatus: - type: object - description: | - The phase's status. It is a list of all of the nodes in the phase and - what category those nodes fall into within the phase. - - ## Link Relationships + type: string + minLength: 1 + readOnly: true + description: | + Name of the Session Template. - * self : The phase status object + It is recommended to use names which meet the following restrictions: + * Maximum length of 127 characters. + * Use only letters, digits, periods (.), dashes (-), and underscores (_). + * Begin and end with a letter or digit. - properties: - name: - type: string + These restrictions are not enforced in this version of BOS, but they are + targeted to start being enforced in an upcoming BOS version. + example: "cle-1.0.0" + tenant: + $ref: '#/components/schemas/V2TenantName' + description: + $ref: '#/components/schemas/SessionTemplateDescription' + enable_cfs: + $ref: '#/components/schemas/EnableCfs' + cfs: + $ref: '#/components/schemas/V2CfsParameters' + boot_sets: + type: object description: | - Name of the Phase - boot, configure, or shutdown - example: "Boot" - pattern: '^([bB][oO][oO][tT]|[cC][oO][nN][fF][iI][gG][uU][rR][eE]|[sS][hH][uU][tT][dD][oO][wW][nN])$' - metadata: - $ref: '#/components/schemas/V1GenericMetadata' - categories: - type: array - items: - $ref: '#/components/schemas/V1PhaseCategoryStatus' - errors: - $ref: '#/components/schemas/V1NodeErrorsList' - V1SessionId: - type: string - description: Unique BOS v1 Session identifier. - format: uuid - example: "8deb0746-b18c-427c-84a8-72ec6a28642c" - V1BootSetStatus: - type: object - description: | - The status for a Boot Set. It as a list of the phase statuses for the Boot Set. + Mapping from Boot Set names to Boot Sets. - ## Link Relationships + It is recommended that: + * At least one Boot Set should be defined, because a Session Template with no + Boot Sets is not functional. + * Boot Set names should be 1-127 characters in length. + * Boot Set names should use only letters, digits, periods (.), dashes (-), and underscores (_). + * Boot Set names should begin and end with a letter or digit. - * self : The Boot Set Status object - * phase : A phase of the Boot Set + These restrictions are not enforced in this version of BOS, but they are + targeted to start being enforced in an upcoming BOS version. + additionalProperties: + $ref: '#/components/schemas/V2BootSet' + links: + $ref: '#/components/schemas/LinkListReadOnly' + additionalProperties: false + V2SessionTemplateValidation: + description: | + Message describing errors or incompleteness in a Session Template. + type: string + V2SessionName: + type: string + description: Name of the Session. + example: "session-20190728032600" + # These validation parameters are restricted by Kubernetes naming conventions. + minLength: 1 + maxLength: 45 + pattern: '^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$' + V2SessionOperation: + type: string + enum: ['boot', 'reboot', 'shutdown'] + description: | + A Session represents a desired state that is being applied to a group + of Components. Sessions run until all Components it manages have + either been disabled due to completion, or until all Components are + managed by other newer Sessions. + Operation -- An operation to perform on Components in this Session. + Boot Applies the Template to the Components and boots/reboots if necessary. + Reboot Applies the Template to the Components; guarantees a reboot. + Shutdown Power down Components that are on. + V2SessionCreate: + description: | + A Session Creation object. A UUID name is generated if a name is not provided. + type: object properties: name: - $ref: '#/components/schemas/BootSetName' - session: - $ref: '#/components/schemas/V1SessionId' - metadata: - $ref: '#/components/schemas/V1GenericMetadata' - phases: - type: array - items: - $ref: '#/components/schemas/V1PhaseStatus' - links: - $ref: '#/components/schemas/LinkList' - V1SessionStatus: + $ref: '#/components/schemas/V2SessionName' + operation: + $ref: '#/components/schemas/V2SessionOperation' + template_name: + $ref: '#/components/schemas/SessionTemplateName' + limit: + $ref: '#/components/schemas/SessionLimit' + stage: + type: boolean + description: | + Set to stage a Session which will not immediately change the state of any Components. + The "applystaged" endpoint can be called at a later time to trigger the start of this Session. + default: false + include_disabled: + type: boolean + description: | + Set to include nodes that have been disabled as indicated in the Hardware State Manager (HSM). + default: false + required: [operation, template_name] + additionalProperties: false + V2SessionStatusLabel: + type: string + enum: ['pending', 'running', 'complete'] + description: The status of a Session. + V2SessionStartTime: + type: string + description: When the Session was created. + V2SessionEndTime: + type: string + nullable: true + description: When the Session was completed. A null value means the Session has not ended. + V2SessionStatus: type: object description: | - The status for a Session. It is a list of all of the Boot Set Statuses in the Session. - - ## Link Relationships - - * self : The Session status object - * boot sets: URL to access the Boot Set status - + Information on the status of a Session. properties: - metadata: - $ref: '#/components/schemas/V1GenericMetadata' - boot_sets: - description: | - The Boot Sets in the Session - type: array - items: - $ref: '#/components/schemas/BootSetName' - minItems: 1 - id: - $ref: '#/components/schemas/V1SessionId' - links: - $ref: '#/components/schemas/LinkList' - V1BootSet: + start_time: + $ref: '#/components/schemas/V2SessionStartTime' + end_time: + $ref: '#/components/schemas/V2SessionEndTime' + status: + $ref: '#/components/schemas/V2SessionStatusLabel' + error: + type: string + nullable: true + description: | + Error which prevented the Session from running. + A null value means the Session has not encountered an error. + additionalProperties: false + V2BootSet: description: | - A Boot Set defines a collection of nodes and the information about the - boot artifacts and parameters to be sent to each node over the specified - network to enable these nodes to boot. When multiple Boot Sets are used - in a Session Template, the boot_ordinal and shutdown_ordinal indicate - the order in which Boot Sets need to be acted upon. Boot Sets sharing - the same ordinal number will be addressed at the same time. + A Boot Set is a collection of nodes defined by an explicit list, their functional + role, and their logical groupings. This collection of nodes is associated with one + set of boot artifacts and optional additional records for configuration and root + filesystem provisioning. type: object properties: name: $ref: '#/components/schemas/BootSetName' path: $ref: '#/components/schemas/BootManifestPath' + cfs: + $ref: '#/components/schemas/V2CfsParameters' type: $ref: '#/components/schemas/BootSetType' etag: @@ -664,593 +596,41 @@ components: $ref: '#/components/schemas/NodeRoleList' node_groups: $ref: '#/components/schemas/NodeGroupList' + arch: + type: string + description: > + The node architecture to target. Filters nodes that are not part of matching architecture from being targeted by boot actions. This value + should correspond to HSM component 'Arch' field exactly. For reasons of backwards compatibility, all HSM nodes that are of type Unknown + are treated as being of type X86. + default: X86 + enum: [X86, ARM, Other, Unknown] rootfs_provider: $ref: '#/components/schemas/BootSetRootfsProvider' rootfs_provider_passthrough: $ref: '#/components/schemas/BootSetRootfsProviderPassthrough' - network: - type: string - deprecated: true - description: | - The network over which the node will boot. - Choices: NMN -- Node Management Network - - When upgrading to this version of BOS, all existing V1 session - templates will automatically have this deprecated field removed from them. - - When a V1 session template is created, this deprecated field is - automatically removed from it before storing it in BOS. - pattern: '^[nN][mM][nN]$' - boot_ordinal: - type: integer - deprecated: true - minimum: 0 - description: | - The boot ordinal. This will establish the order for Boot Set operations. - Boot Sets boot in order from the lowest to highest boot_ordinal. - - It is recommended that this should have a maximum value of 65535. - - When upgrading to this version of BOS, all existing V1 session - templates will automatically have this deprecated field removed from them. - - When a V1 session template is created, this deprecated field is - automatically removed from it before storing it in BOS. - shutdown_ordinal: - type: integer - deprecated: true - minimum: 0 - description: | - The shutdown ordinal. This will establish the order for Boot Set - shutdown operations. Sets shutdown from low to high shutdown_ordinal. - - It is recommended that this should have a maximum value of 65535. - - When upgrading to this version of BOS, all existing V1 session - templates will automatically have this deprecated field removed from them. - - When a V1 session template is created, this deprecated field is - automatically removed from it before storing it in BOS. additionalProperties: false required: [path, type] - V1SessionTemplateUuid: - type: string - description: | - DEPRECATED - use templateName. This field is ignored if templateName is also set. - - Name of the Session Template. - - It is recommended to use names which meet the following restrictions: - * 1-127 characters in length. - * Use only letters, digits, periods (.), dashes (-), and underscores (_). - * Begin and end with a letter or digit. - minLength: 1 - example: "my-session-template" - deprecated: true - V1SessionTemplate: - type: object + V2SessionTemplateArray: + description: An array of Session Templates. + type: array + items: + $ref: '#/components/schemas/V2SessionTemplate' + V2Session: description: | - A Session Template object represents a collection of resources and metadata. - A Session Template is used to create a Session which when combined with an - action (i.e. boot, configure, reboot, shutdown) will create a Kubernetes BOA job - to complete the required tasks for the operation. - - When upgrading to this version of BOS, all existing V1 session templates - will automatically have all deprecated fields removed from them. - - When a V1 session template is created, all deprecated fields are automatically - removed from it before storing it in BOS. + A Session object ## Link Relationships - * self : The Session Template object + * self : The Session object + type: object properties: name: - $ref: '#/components/schemas/SessionTemplateName' - description: - $ref: '#/components/schemas/SessionTemplateDescription' - cfs_url: - $ref: '#/components/schemas/V1CfsUrl' - cfs_branch: - $ref: '#/components/schemas/V1CfsBranch' - enable_cfs: - $ref: '#/components/schemas/EnableCfs' - cfs: - $ref: '#/components/schemas/V1CfsParameters' - partition: - type: string - deprecated: true - description: | - The machine partition to operate on. - - It is recommended that this should be 1-255 characters in length. - - When upgrading to this version of BOS, all existing V1 session - templates will automatically have this deprecated field removed from them. - - When a V1 session template is created, this deprecated field is - automatically removed from it before storing it in BOS. - boot_sets: - type: object - description: | - Mapping from Boot Set names to Boot Sets. - - It is recommended that: - * At least one Boot Set should be defined, because a Session Template with no - Boot Sets is not functional. - * Boot Set names should be 1-127 characters in length. - * Boot Set names should use only letters, digits, periods (.), dashes (-), and underscores (_). - * Boot Set names should begin and end with a letter or digit. - additionalProperties: - $ref: '#/components/schemas/V1BootSet' - required: [name] - additionalProperties: false - V1BoaKubernetesJob: - type: string - maxLength: 64 - readOnly: true - description: The identity of the Kubernetes job that is created to handle the Session. - example: "boa-07877de1-09bb-4ca8-a4e5-943b1262dbf0" - V1Operation: - type: string - description: | - A Session represents an operation on a Session Template. - The creation of a Session effectively results in the creation - of a Kubernetes Boot Orchestration Agent (BOA) job to perform the - duties required to complete the operation. - - Operation -- An operation to perform on nodes in this Session. - - Boot Boot nodes that are off. - - Configure Reconfigure the nodes using the Configuration Framework - Service (CFS). - - Reboot Gracefully power down nodes that are on and then power - them back up. - - Shutdown Gracefully power down nodes that are on. - - pattern: '^([bB][oO][oO][tT]|[cC][oO][nN][fF][iI][gG][uU][rR][eE]|[rR][eE][bB][oO][oO][tT]|[sS][hH][uU][tT][dD][oO][wW][nN])$' - example: "boot" - V1SessionLink: - description: Link to other resources - type: object - properties: - href: - type: string - jobId: - $ref: '#/components/schemas/V1BoaKubernetesJob' - rel: - type: string - enum: ['session', 'status'] - type: - type: string - enum: ['GET'] - additionalProperties: false - V1SessionStatusUri: - type: string - description: URI to the status for this Session - format: uri - example: "/v1/session/90730844-094d-45a5-9b90-d661d14d9444/status" - V1SessionDetails: - description: Details about a Session. - type: object - properties: - complete: - $ref: '#/components/schemas/V1CompleteMetadata' - error_count: - $ref: '#/components/schemas/V1ErrorCountMetadata' - in_progress: - $ref: '#/components/schemas/V1InProgressMetadata' - job: - $ref: '#/components/schemas/V1BoaKubernetesJob' - operation: - $ref: '#/components/schemas/V1Operation' - start_time: - $ref: '#/components/schemas/V1StartTimeMetadata' - status_link: - $ref: '#/components/schemas/V1SessionStatusUri' - stop_time: - $ref: '#/components/schemas/V1StopTimeMetadata' - templateName: - $ref: '#/components/schemas/SessionTemplateName' - V1SessionDetailsByTemplateUuid: - description: | - Details about a Session using templateUuid instead of templateName. - DEPRECATED -- these will only exist from Sessions created before templateUuid was deprecated. - deprecated: true - type: object - properties: - complete: - $ref: '#/components/schemas/V1CompleteMetadata' - error_count: - $ref: '#/components/schemas/V1ErrorCountMetadata' - in_progress: - $ref: '#/components/schemas/V1InProgressMetadata' - job: - $ref: '#/components/schemas/V1BoaKubernetesJob' - operation: - $ref: '#/components/schemas/V1Operation' - start_time: - $ref: '#/components/schemas/V1StartTimeMetadata' - status_link: - $ref: '#/components/schemas/V1SessionStatusUri' - stop_time: - $ref: '#/components/schemas/V1StopTimeMetadata' - templateName: - $ref: '#/components/schemas/SessionTemplateName' - V1SessionLinkList: - type: array - readOnly: true - items: - $ref: '#/components/schemas/V1SessionLink' - V1Session: - description: | - A Session object - - ## Link Relationships - - * self : The Session object - type: object - properties: - operation: - $ref: '#/components/schemas/V1Operation' - templateName: - $ref: '#/components/schemas/SessionTemplateName' - job: - $ref: '#/components/schemas/V1BoaKubernetesJob' - limit: - $ref: '#/components/schemas/SessionLimit' - links: - $ref: '#/components/schemas/V1SessionLinkList' - required: [operation, templateName] - additionalProperties: false - V1SessionByTemplateName: - description: | - A Session object specified by templateName - - ## Link Relationships - - * self : The Session object - type: object - properties: - operation: - $ref: '#/components/schemas/V1Operation' - templateUuid: - $ref: '#/components/schemas/V1SessionTemplateUuid' - templateName: - $ref: '#/components/schemas/SessionTemplateName' - job: - $ref: '#/components/schemas/V1BoaKubernetesJob' - limit: - $ref: '#/components/schemas/SessionLimit' - links: - $ref: '#/components/schemas/V1SessionLinkList' - required: [operation, templateName] - additionalProperties: false - V1SessionByTemplateUuid: - description: | - A Session object specified by templateUuid (DEPRECATED -- use templateName) - - ## Link Relationships - - * self : The Session object - deprecated: true - type: object - properties: - operation: - $ref: '#/components/schemas/V1Operation' - templateUuid: - $ref: '#/components/schemas/V1SessionTemplateUuid' - job: - $ref: '#/components/schemas/V1BoaKubernetesJob' - limit: - $ref: '#/components/schemas/SessionLimit' - links: - $ref: '#/components/schemas/V1SessionLinkList' - required: [operation, templateUuid] - additionalProperties: false - V1PhaseName: - type: string - description: | - The phase that this data belongs to (boot, shutdown, or configure). If blank, - it belongs to the Boot Set itself, which only applies to the GenericMetadata type. - example: "Boot" - pattern: '^($|[sS][hH][uU][tT][dD][oO][wW][nN]$|[bB][oO][oO][tT]$|[cC][oO][nN][fF][iI][gG][uU][rR][eE]$)' - V1NodeChangeList: - type: object - description: | - The information used to update the status of a node list. It moves nodes from - one category to another within a phase. - properties: - phase: - $ref: '#/components/schemas/V1PhaseName' - source: - $ref: '#/components/schemas/V1PhaseCategoryName' - destination: - $ref: '#/components/schemas/V1PhaseCategoryName' - node_list: - $ref: '#/components/schemas/NodeListEmptyOk' - additionalProperties: false - required: [phase, source, destination, node_list] - V1NodeErrorsList: - type: object - description: | - Categorizing nodes into failures by the type of error they have. - This is an additive characterization. Nodes will be added to existing errors. - This does not overwrite previously existing errors. - additionalProperties: - $ref: '#/components/schemas/NodeListEmptyOk' - V1UpdateRequestNodeChange: - description: | - This is an element of the payload sent during an update request. It contains - updates to which categories nodes are in. - type: object - properties: - update_type: - description: The type of update data - enum: [ 'NodeChangeList' ] - type: string - phase: - $ref: '#/components/schemas/V1PhaseName' - data: - $ref: '#/components/schemas/V1NodeChangeList' - required: [ 'data', 'phase', 'update_type' ] - V1UpdateRequestNodeErrors: - description: | - This is an element of the payload sent during an update request. It contains - updates to which errors have occurred and which nodes encountered those errors - type: object - properties: - update_type: - description: The type of update data - enum: [ 'NodeErrorsList' ] - type: string - phase: - $ref: '#/components/schemas/V1PhaseName' - data: - $ref: '#/components/schemas/V1NodeErrorsList' - required: [ 'data', 'phase', 'update_type' ] - V1UpdateRequestGenericMetadata: - description: | - This is an element of the payload sent during an update request. It contains - updates to metadata, specifically start and stop times - type: object - properties: - update_type: - description: The type of update data - enum: [ 'GenericMetadata' ] - type: string - phase: - $ref: '#/components/schemas/V1PhaseName' - data: - $ref: '#/components/schemas/V1GenericMetadata' - required: [ 'data', 'phase', 'update_type' ] - V1UpdateRequestList: - description: | - This is the payload sent during an update request. It contains a list of updates. - type: array - items: - # Each item in the list should match exactly one of these schemas (since they are mutually exclusive) - oneOf: - - $ref: '#/components/schemas/V1UpdateRequestNodeChange' - - $ref: '#/components/schemas/V1UpdateRequestNodeErrors' - - $ref: '#/components/schemas/V1UpdateRequestGenericMetadata' - # V2 - V2TenantName: - type: string - description: | - Name of the tenant that owns this resource. Only used in environments - with multi-tenancy enabled. An empty string or null value means the resource - is not owned by a tenant. The absence of this field from a resource indicates - the same. - nullable: true - readOnly: true - V2CfsParameters: - type: object - description: | - This is the collection of parameters that are passed to the Configuration - Framework Service when configuration is enabled. Can be set as the global value for - a Session Template, or individually within a Boot Set. - properties: - configuration: - $ref: '#/components/schemas/CfsConfiguration' - additionalProperties: false - V2SessionTemplate: - type: object - description: | - A Session Template object represents a collection of resources and metadata. - A Session Template is used to create a Session which applies the data to - group of Components. - - ## Link Relationships - - * self : The Session Template object - properties: - name: - type: string - minLength: 1 - readOnly: true - description: | - Name of the Session Template. - - It is recommended to use names which meet the following restrictions: - * Maximum length of 127 characters. - * Use only letters, digits, periods (.), dashes (-), and underscores (_). - * Begin and end with a letter or digit. - - These restrictions are not enforced in this version of BOS, but they are - targeted to start being enforced in an upcoming BOS version. - example: "cle-1.0.0" - tenant: - $ref: '#/components/schemas/V2TenantName' - description: - $ref: '#/components/schemas/SessionTemplateDescription' - enable_cfs: - $ref: '#/components/schemas/EnableCfs' - cfs: - $ref: '#/components/schemas/V2CfsParameters' - boot_sets: - type: object - description: | - Mapping from Boot Set names to Boot Sets. - - It is recommended that: - * At least one Boot Set should be defined, because a Session Template with no - Boot Sets is not functional. - * Boot Set names should be 1-127 characters in length. - * Boot Set names should use only letters, digits, periods (.), dashes (-), and underscores (_). - * Boot Set names should begin and end with a letter or digit. - - These restrictions are not enforced in this version of BOS, but they are - targeted to start being enforced in an upcoming BOS version. - additionalProperties: - $ref: '#/components/schemas/V2BootSet' - links: - $ref: '#/components/schemas/LinkListReadOnly' - additionalProperties: false - V2SessionTemplateValidation: - description: | - Message describing errors or incompleteness in a Session Template. - type: string - V2SessionName: - type: string - description: Name of the Session. - example: "session-20190728032600" - # These validation parameters are restricted by Kubernetes naming conventions. - minLength: 1 - maxLength: 45 - pattern: '^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$' - V2SessionOperation: - type: string - enum: ['boot', 'reboot', 'shutdown'] - description: | - A Session represents a desired state that is being applied to a group - of Components. Sessions run until all Components it manages have - either been disabled due to completion, or until all Components are - managed by other newer Sessions. - - Operation -- An operation to perform on Components in this Session. - Boot Applies the Template to the Components and boots/reboots if necessary. - Reboot Applies the Template to the Components; guarantees a reboot. - Shutdown Power down Components that are on. - V2SessionCreate: - description: | - A Session Creation object. A UUID name is generated if a name is not provided. - type: object - properties: - name: - $ref: '#/components/schemas/V2SessionName' - operation: - $ref: '#/components/schemas/V2SessionOperation' - template_name: - $ref: '#/components/schemas/SessionTemplateName' - limit: - $ref: '#/components/schemas/SessionLimit' - stage: - type: boolean - description: | - Set to stage a Session which will not immediately change the state of any Components. - The "applystaged" endpoint can be called at a later time to trigger the start of this Session. - default: false - include_disabled: - type: boolean - description: | - Set to include nodes that have been disabled as indicated in the Hardware State Manager (HSM). - default: false - required: [operation, template_name] - additionalProperties: false - V2SessionStatusLabel: - type: string - enum: ['pending', 'running', 'complete'] - description: The status of a Session. - V2SessionStartTime: - type: string - description: When the Session was created. - V2SessionEndTime: - type: string - nullable: true - description: When the Session was completed. A null value means the Session has not ended. - V2SessionStatus: - type: object - description: | - Information on the status of a Session. - properties: - start_time: - $ref: '#/components/schemas/V2SessionStartTime' - end_time: - $ref: '#/components/schemas/V2SessionEndTime' - status: - $ref: '#/components/schemas/V2SessionStatusLabel' - error: - type: string - nullable: true - description: | - Error which prevented the Session from running. - A null value means the Session has not encountered an error. - additionalProperties: false - V2BootSet: - description: | - A Boot Set is a collection of nodes defined by an explicit list, their functional - role, and their logical groupings. This collection of nodes is associated with one - set of boot artifacts and optional additional records for configuration and root - filesystem provisioning. - type: object - properties: - name: - $ref: '#/components/schemas/BootSetName' - path: - $ref: '#/components/schemas/BootManifestPath' - cfs: - $ref: '#/components/schemas/V2CfsParameters' - type: - $ref: '#/components/schemas/BootSetType' - etag: - $ref: '#/components/schemas/BootSetEtag' - kernel_parameters: - $ref: '#/components/schemas/BootKernelParameters' - node_list: - $ref: '#/components/schemas/NodeList' - node_roles_groups: - $ref: '#/components/schemas/NodeRoleList' - node_groups: - $ref: '#/components/schemas/NodeGroupList' - arch: - type: string - description: > - The node architecture to target. Filters nodes that are not part of matching architecture from being targeted by boot actions. This value - should correspond to HSM component 'Arch' field exactly. For reasons of backwards compatibility, all HSM nodes that are of type Unknown - are treated as being of type X86. - default: X86 - enum: [X86, ARM, Other, Unknown] - rootfs_provider: - $ref: '#/components/schemas/BootSetRootfsProvider' - rootfs_provider_passthrough: - $ref: '#/components/schemas/BootSetRootfsProviderPassthrough' - additionalProperties: false - required: [path, type] - V2SessionTemplateArray: - description: An array of Session Templates. - type: array - items: - $ref: '#/components/schemas/V2SessionTemplate' - V2Session: - description: | - A Session object - - ## Link Relationships - - * self : The Session object - type: object - properties: - name: - $ref: '#/components/schemas/V2SessionName' - tenant: - $ref: '#/components/schemas/V2TenantName' - operation: - $ref: '#/components/schemas/V2SessionOperation' - template_name: + $ref: '#/components/schemas/V2SessionName' + tenant: + $ref: '#/components/schemas/V2TenantName' + operation: + $ref: '#/components/schemas/V2SessionOperation' + template_name: $ref: '#/components/schemas/SessionTemplateName' limit: $ref: '#/components/schemas/SessionLimit' @@ -1748,27 +1128,6 @@ components: application/json: schema: $ref: '#/components/schemas/Version' - # V1 - V1Session: - description: Session - content: - application/json: - schema: - $ref: '#/components/schemas/V1Session' - V1SessionDetails: - description: Session details - content: - application/json: - schema: - oneOf: - - $ref: '#/components/schemas/V1SessionDetails' - - $ref: '#/components/schemas/V1SessionDetailsByTemplateUuid' - V1SessionStatus: - description: A list of Boot Set Statuses and metadata - content: - application/json: - schema: - $ref: '#/components/schemas/V1SessionStatus' # V2 V2SessionTemplateDetails: description: Session Template details @@ -1845,14 +1204,14 @@ components: $ref: '#/components/schemas/ProblemDetails' BadRequestOrMultiTenancyNotSupported: description: | - Multi-tenancy is not supported for this BOS v1 request. + Multi-tenancy is not supported for this request. If no tenant was specified, then the request was bad for another reason. content: application/problem+json: schema: $ref: '#/components/schemas/ProblemDetails' MultiTenancyNotSupported: - description: Multi-tenancy is not supported for this BOS v1 request. + description: Multi-tenancy is not supported for this BOS request. content: application/problem+json: schema: @@ -1892,41 +1251,11 @@ components: type: string TemplateIdPathParam: name: session_template_id - in: path - description: Session Template name - required: true - schema: - $ref: '#/components/schemas/SessionTemplateName' - V1CategoryNamePathParam: - name: category_name - in: path - description: The category name - required: true - schema: - type: string - V1PhaseNamePathParam: - name: phase_name - in: path - description: The phase name - required: true - schema: - type: string - V1SessionIdPathParam: - name: session_id - in: path - description: Session ID - required: true - schema: - type: string - V1TenantHeaderParam: - name: Cray-Tenant-Name - in: header - description: | - Tenant name. Multi-tenancy is not supported for most BOS v1 endpoints. - If this parameter is set to a non-empty string, the request will be rejected. - required: false + in: path + description: Session Template name + required: true schema: - $ref: '#/components/schemas/TenantName' + $ref: '#/components/schemas/SessionTemplateName' V2ComponentIdPathParam: name: component_id in: path @@ -1992,435 +1321,6 @@ paths: type: array items: $ref: '#/components/schemas/Version' -# See Version guidance is in -# https://connect.us.cray.com/confluence/display/SMA/Shasta+RESTful+Service+Design#ShastaRESTfulServiceDesign-Versioning -# -# Need standard fmt operationId getVersion, postSession, etc. - /v1: - get: - summary: Get API version - description: Return the API version - tags: - - version - x-openapi-router-controller: bos.server.controllers.v1.base - operationId: v1_get - responses: - 200: - $ref: '#/components/responses/Version' - 500: - $ref: '#/components/responses/BadRequest' - /v1/healthz: - get: - summary: Get service health details - description: Get BOS health details. - tags: - - healthz - x-openapi-router-controller: bos.server.controllers.v1.healthz - operationId: v1_get_healthz - responses: - 200: - $ref: '#/components/responses/ServiceHealth' - 500: - $ref: '#/components/responses/BadRequest' - 503: - $ref: '#/components/responses/ServiceUnavailable' - /v1/sessiontemplate: - parameters: - - $ref: '#/components/parameters/V1TenantHeaderParam' - post: - summary: Create Session Template - description: | - Create a new Session Template. - - The created template will be modified if necessary to follow the BOS v2 session template format. - tags: - - sessiontemplate - x-openapi-router-controller: bos.server.controllers.v1.sessiontemplate - operationId: create_v1_sessiontemplate - requestBody: - description: A JSON object for creating a Session Template - required: true - content: - application/json: - schema: - $ref: '#/components/schemas/V1SessionTemplate' - responses: - 201: - $ref: '#/components/responses/SessionTemplateName' - 400: - $ref: '#/components/responses/BadRequestOrMultiTenancyNotSupported' - get: - summary: List Session Templates - description: | - List all Session Templates. - tags: - - sessiontemplate - x-openapi-router-controller: bos.server.controllers.v1.sessiontemplate - operationId: get_v1_sessiontemplates - responses: - 200: - $ref: '#/components/responses/V2SessionTemplateDetailsArray' - 400: - $ref: '#/components/responses/MultiTenancyNotSupported' - /v1/sessiontemplate/{session_template_id}: - parameters: - - $ref: '#/components/parameters/TemplateIdPathParam' - - $ref: '#/components/parameters/V1TenantHeaderParam' - get: - summary: Get Session Template by ID - description: | - Get Session Template by Session Template ID. - The Session Template ID corresponds to the *name* - of the Session Template. - tags: - - sessiontemplate - x-openapi-router-controller: bos.server.controllers.v1.sessiontemplate - operationId: get_v1_sessiontemplate - responses: - 200: - $ref: '#/components/responses/V2SessionTemplateDetails' - 400: - $ref: '#/components/responses/MultiTenancyNotSupported' - 404: - $ref: '#/components/responses/ResourceNotFound' - delete: - summary: Delete a Session Template - description: Delete a Session Template. - tags: - - sessiontemplate - x-openapi-router-controller: bos.server.controllers.v1.sessiontemplate - operationId: delete_v1_sessiontemplate - responses: - 204: - $ref: '#/components/responses/ResourceDeleted' - 400: - $ref: '#/components/responses/MultiTenancyNotSupported' - 404: - $ref: '#/components/responses/ResourceNotFound' - /v1/sessiontemplatetemplate: - get: - summary: Get an example Session Template. - description: | - Returns a skeleton of a Session Template, which can be - used as a starting point for users creating their own - Session Templates. - tags: - - sessiontemplate - x-openapi-router-controller: bos.server.controllers.v1.sessiontemplate - operationId: get_v1_sessiontemplatetemplate - responses: - 200: - $ref: '#/components/responses/V2SessionTemplateDetails' - /v1/session: - parameters: - - $ref: '#/components/parameters/V1TenantHeaderParam' - post: - summary: Create a Session - description: | - The creation of a Session performs the operation - specified in the SessionCreateRequest - on the Boot Sets defined in the Session Template. - tags: - - session - x-openapi-router-controller: bos.server.controllers.v1.session - operationId: create_v1_session - requestBody: - description: A JSON object for creating a Session - required: true - content: - application/json: - schema: - oneOf: - - $ref: '#/components/schemas/V1SessionByTemplateName' - - $ref: '#/components/schemas/V1SessionByTemplateUuid' - responses: - 201: - $ref: '#/components/responses/V1Session' - 400: - $ref: '#/components/responses/BadRequestOrMultiTenancyNotSupported' - 404: - $ref: '#/components/responses/ResourceNotFound' - get: - summary: List Session IDs - description: | - List IDs of all Sessions, including those in progress and those complete. - tags: - - session - x-openapi-router-controller: bos.server.controllers.v1.session - operationId: get_v1_sessions - responses: - 200: - description: A collection of Session IDs - content: - application/json: - schema: - type: array - items: - $ref: '#/components/schemas/V1SessionId' - 400: - $ref: '#/components/responses/MultiTenancyNotSupported' - /v1/session/{session_id}: - parameters: - - $ref: '#/components/parameters/V1SessionIdPathParam' - - $ref: '#/components/parameters/V1TenantHeaderParam' - get: - summary: Get Session details by ID - description: Get Session details by Session ID. - tags: - - session - x-openapi-router-controller: bos.server.controllers.v1.session - operationId: get_v1_session - responses: - 200: - $ref: '#/components/responses/V1SessionDetails' - 400: - $ref: '#/components/responses/MultiTenancyNotSupported' - 404: - $ref: '#/components/responses/ResourceNotFound' - delete: - summary: Delete Session by ID - description: Delete Session by Session ID. - tags: - - session - x-openapi-router-controller: bos.server.controllers.v1.session - operationId: delete_v1_session - responses: - 204: - $ref: '#/components/responses/ResourceDeleted' - 400: - $ref: '#/components/responses/MultiTenancyNotSupported' - 404: - $ref: '#/components/responses/ResourceNotFound' - /v1/session/{session_id}/status: - parameters: - - $ref: '#/components/parameters/V1SessionIdPathParam' - - $ref: '#/components/parameters/V1TenantHeaderParam' - get: - summary: A list of the statuses for the different Boot Sets. - description: | - A list of the statuses for the different Boot Sets. - tags: - - session - x-openapi-router-controller: bos.server.controllers.v1.status - operationId: get_v1_session_status - responses: - 200: - $ref: '#/components/responses/V1SessionStatus' - 400: - $ref: '#/components/responses/MultiTenancyNotSupported' - 404: - $ref: '#/components/responses/ResourceNotFound' - post: - summary: Create the initial Session status - description: | - Creates the initial Session status. - tags: - - session - x-openapi-router-controller: bos.server.controllers.v1.status - operationId: create_v1_session_status - requestBody: - description: A JSON object for creating the status for a Session - required: true - content: - application/json: - schema: - $ref: '#/components/schemas/V1SessionStatus' - responses: - 200: - $ref: '#/components/responses/V1SessionStatus' - 400: - $ref: '#/components/responses/BadRequestOrMultiTenancyNotSupported' - 409: - $ref: '#/components/responses/AlreadyExists' - patch: - summary: Update the Session status - description: | - Update the Session status. You can update the start or stop times. - tags: - - session - x-openapi-router-controller: bos.server.controllers.v1.status - operationId: update_v1_session_status - requestBody: - description: A JSON object for updating the status for a Session - required: true - content: - application/json: - schema: - $ref: '#/components/schemas/V1GenericMetadata' - responses: - 200: - $ref: '#/components/responses/V1SessionStatus' - 400: - $ref: '#/components/responses/MultiTenancyNotSupported' - 404: - $ref: '#/components/responses/BadRequest' - delete: - summary: Delete the Session status - description: | - Deletes an existing Session status - tags: - - session - x-openapi-router-controller: bos.server.controllers.v1.status - operationId: delete_v1_session_status - responses: - 204: - $ref: '#/components/responses/ResourceDeleted' - 400: - $ref: '#/components/responses/BadRequestOrMultiTenancyNotSupported' - 404: - $ref: '#/components/responses/ResourceNotFound' - /v1/session/{session_id}/status/{boot_set_name}: - parameters: - - $ref: '#/components/parameters/V1SessionIdPathParam' - - $ref: '#/components/parameters/BootSetNamePathParam' - - $ref: '#/components/parameters/V1TenantHeaderParam' - get: - summary: Get the status for a Boot Set. - description: Get the status for a Boot Set. - tags: - - session - x-openapi-router-controller: bos.server.controllers.v1.status - operationId: get_v1_session_status_by_bootset - responses: - 200: - description: Metadata and a list of the Phase Statuses for the Boot Set - content: - application/json: - schema: - $ref: '#/components/schemas/V1BootSetStatus' - 400: - $ref: '#/components/responses/MultiTenancyNotSupported' - 404: - $ref: '#/components/responses/ResourceNotFound' - post: - summary: Create a Boot Set Status - description: | - Create a status for a Boot Set - tags: - - session - - status - x-openapi-router-controller: bos.server.controllers.v1.status - operationId: create_v1_boot_set_status - requestBody: - description: A JSON object for creating a status for a Boot Set - required: true - content: - application/json: - schema: - $ref: '#/components/schemas/V1BootSetStatus' - responses: - 201: - description: The created Boot Set status - content: - application/json: - schema: - $ref: '#/components/schemas/V1BootSetStatus' - 400: - $ref: '#/components/responses/MultiTenancyNotSupported' - 409: - $ref: '#/components/responses/AlreadyExists' - patch: - summary: Update the status. - description: | - This will change the status for one or more nodes within - the Boot Set. - tags: - - session - x-openapi-router-controller: bos.server.controllers.v1.status - operationId: update_v1_session_status_by_bootset - requestBody: - description: A JSON object for updating the status for a Session - required: true - content: - application/json: - schema: - $ref: '#/components/schemas/V1UpdateRequestList' - responses: - 200: - description: A list of Boot Set Statuses and metadata - content: - application/json: - schema: - $ref: '#/components/schemas/V1BootSetStatus' - 400: - $ref: '#/components/responses/MultiTenancyNotSupported' - 404: - $ref: '#/components/responses/ResourceNotFound' - delete: - summary: Delete the Boot Set status - description: | - Deletes an existing Boot Set status - tags: - - session - x-openapi-router-controller: bos.server.controllers.v1.status - operationId: delete_v1_boot_set_status - responses: - 204: - $ref: '#/components/responses/ResourceDeleted' - 400: - $ref: '#/components/responses/BadRequestOrMultiTenancyNotSupported' - /v1/session/{session_id}/status/{boot_set_name}/{phase_name}: - parameters: - - $ref: '#/components/parameters/V1SessionIdPathParam' - - $ref: '#/components/parameters/BootSetNamePathParam' - - $ref: '#/components/parameters/V1PhaseNamePathParam' - - $ref: '#/components/parameters/V1TenantHeaderParam' - get: - summary: Get the status for a specific Boot Set and phase. - description: Get the status for a specific Boot Set and phase. - tags: - - session - x-openapi-router-controller: bos.server.controllers.v1.status - operationId: get_v1_session_status_by_bootset_and_phase - responses: - 200: - description: A list of the nodes in the Phase and Category - content: - application/json: - schema: - $ref: '#/components/schemas/V1PhaseStatus' - 400: - $ref: '#/components/responses/MultiTenancyNotSupported' - 404: - $ref: '#/components/responses/ResourceNotFound' - /v1/session/{session_id}/status/{boot_set_name}/{phase_name}/{category_name}: - parameters: - - $ref: '#/components/parameters/V1SessionIdPathParam' - - $ref: '#/components/parameters/BootSetNamePathParam' - - $ref: '#/components/parameters/V1PhaseNamePathParam' - - $ref: '#/components/parameters/V1CategoryNamePathParam' - - $ref: '#/components/parameters/V1TenantHeaderParam' - get: - summary: Get the status for a specific Boot Set, phase, and category. - description: Get the status for a specific Boot Set, phase, and category. - tags: - - session - x-openapi-router-controller: bos.server.controllers.v1.status - operationId: get_v1_session_status_by_bootset_and_phase_and_category - responses: - 200: - description: A list of the nodes in the Phase and Category - content: - application/json: - schema: - $ref: '#/components/schemas/V1PhaseCategoryStatus' - 400: - $ref: '#/components/responses/MultiTenancyNotSupported' - 404: - $ref: '#/components/responses/ResourceNotFound' - /v1/version: - get: - summary: Get API version - description: Return the API version - tags: - - version - x-openapi-router-controller: bos.server.controllers.v1.base - operationId: v1_get_version - responses: - 200: - $ref: '#/components/responses/Version' - 500: - $ref: '#/components/responses/BadRequest' /v2: get: diff --git a/api_tests/test_version.py b/api_tests/test_version.py index d6b5ba3f..437929b2 100644 --- a/api_tests/test_version.py +++ b/api_tests/test_version.py @@ -1,7 +1,7 @@ # # MIT License # -# (C) Copyright 2019, 2021-2022 Hewlett Packard Enterprise Development LP +# (C) Copyright 2019, 2021-2022, 2024 Hewlett Packard Enterprise Development LP # # Permission is hereby granted, free of charge, to any person obtaining a # copy of this software and associated documentation files (the "Software"), @@ -32,7 +32,7 @@ @pytest.mark.smoke def test_get_version(): r = common.create_session().get( - common.get_service_url('v1')) + common.get_service_url('v2')) assert r.status_code == 200, \ "expected 200 received {} with data\n{}".format( r.status_code, r.text) @@ -40,7 +40,7 @@ def test_get_version(): def test_get_version_major_number(): r = common.create_session().get( - common.get_service_url('v1')) + common.get_service_url('v2')) assert (r.status_code, int(r.json()['major'])) == (200, 1), \ "expected (200 ,1) received ({},{})".format(r.status_code, int(r.json()['major'])) @@ -48,7 +48,7 @@ def test_get_version_major_number(): def test_get_version_minor_number(): r = common.create_session().get( - common.get_service_url('v1')) + common.get_service_url('v2')) assert (r.status_code, int(r.json()['minor'])) == (200, 0), \ "expected (200 ,0) received ({},{})".format(r.status_code, int(r.json()['minor'])) @@ -56,7 +56,7 @@ def test_get_version_minor_number(): def test_get_version_patch_number(): r = common.create_session().get( - common.get_service_url('v1')) + common.get_service_url('v2')) assert (r.status_code, int(r.json()['patch'])) == (200, 0), \ "expected (200 ,0) received ({},{})".format(r.status_code, int(r.json()['patch'])) diff --git a/constraints.txt.in b/constraints.txt.in index 894deede..1b2a58d2 100644 --- a/constraints.txt.in +++ b/constraints.txt.in @@ -1,12 +1,12 @@ -async-timeout==4.0.2 +async-timeout==4.0.3 attrs==22.2.0 bos.server==0.0.1 -boto3==1.26.92 +boto3==1.26.165 botocore==1.29.165 -cachetools==5.3.1 +cachetools==5.3.3 certifi==2022.12.7 charset-normalizer==3.1.0 -click==8.1.6 +click==8.1.7 clickclick==20.10.2 connexion==2.14.2 etcd3==0.12.0 @@ -19,9 +19,10 @@ itsdangerous==2.1.2 Jinja2==3.0.3 jmespath==1.0.1 jsonschema==4.17.3 -kubernetes==26.1.0 +# CSM 1.6 uses Kubernetes 1.22, so use client v22.x to ensure compatability +kubernetes==22.6.0 liveness==0.0.0-liveness -MarkupSafe==2.1.3 +MarkupSafe==2.1.5 oauthlib==3.2.2 packaging==21.3 protobuf==4.22.5 @@ -36,10 +37,10 @@ requests==2.28.2 requests-oauthlib==1.3.1 retrying==1.3.4 rsa==4.9 -s3transfer==0.6.1 +s3transfer==0.6.2 six==1.16.0 swagger-ui-bundle==0.0.9 -tenacity==8.2.2 -urllib3==1.26.16 +tenacity==8.2.3 +urllib3==1.26.18 websocket-client==1.5.3 Werkzeug==2.2.3 diff --git a/copyright_license_check.yaml b/copyright_license_check.yaml index 7f758a76..fe950ac8 100644 --- a/copyright_license_check.yaml +++ b/copyright_license_check.yaml @@ -1,7 +1,7 @@ # # MIT License # -# (C) Copyright 2021-2022 Hewlett Packard Enterprise Development LP +# (C) Copyright 2021-2022, 2024 Hewlett Packard Enterprise Development LP # # Permission is hereby granted, free of charge, to any person obtaining a # copy of this software and associated documentation files (the "Software"), @@ -30,6 +30,5 @@ # We want to make sure to check these files as well also_include_files: - - ".version" - "run_codestylecheck" - "run_unittests" diff --git a/docker_test_entry.sh b/docker_test_entry.sh deleted file mode 100755 index 60cbf493..00000000 --- a/docker_test_entry.sh +++ /dev/null @@ -1,60 +0,0 @@ -#!/usr/bin/env sh -# -# MIT License -# -# (C) Copyright 2019-2022 Hewlett Packard Enterprise Development LP -# -# Permission is hereby granted, free of charge, to any person obtaining a -# copy of this software and associated documentation files (the "Software"), -# to deal in the Software without restriction, including without limitation -# the rights to use, copy, modify, merge, publish, distribute, sublicense, -# and/or sell copies of the Software, and to permit persons to whom the -# Software is furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included -# in all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL -# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR -# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, -# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR -# OTHER DEALINGS IN THE SOFTWARE. -# -set -e -set -o pipefail - -mkdir -p /results - -# Fire up etcd because it's needed for the status tests -etcd & - -# Remove the nosetest invocation when CASMCMS-5335 is resolved. -# Nosetests -python3 -m pip freeze 2>&1 | tee /results/pip_freeze.out -nosetests -v \ - -w /app/lib/server/bos/test \ - --with-xunit \ - --xunit-file=/results/nosetests.xml \ - --with-coverage \ - --cover-erase \ - --cover-package=bos \ - --cover-branches \ - --cover-inclusive \ - --cover-html \ - --cover-html-dir=/results/coverage \ - --cover-xml \ - --cover-xml-file=/results/coverage.xml \ - 2>&1 | tee /results/nosetests.out - -# pytest equivalent of the above -# The above nosetests can be removed once pytest duplicates -# all of the needed functionality. -pytest --cov=/app/lib/server/bos -k "not Status" \ - 2>&1 | tee /results/pytests.out - -# Running this test suite separately because it hangs when run with the other tests -# Apparently it is antisocial. -pytest --cov=/app/lib/server/bos /app/lib/server/bos/test/test_status_controller.py \ - 2>&1 | tee -a /results/pytests.out \ No newline at end of file diff --git a/kubernetes/cray-bos/Chart.yaml.in b/kubernetes/cray-bos/Chart.yaml.in index 9f025f92..f1949e1c 100644 --- a/kubernetes/cray-bos/Chart.yaml.in +++ b/kubernetes/cray-bos/Chart.yaml.in @@ -1,7 +1,5 @@ annotations: artifacthub.io/images: | - - name: cray-boa - image: artifactory.algol60.net/csm-docker/stable/cray-boa:0.0.0-boa - name: cray-bos image: artifactory.algol60.net/csm-docker/S-T-A-B-L-E/cray-bos:0.0.0-bos - name: redis @@ -13,13 +11,9 @@ dependencies: - name: cray-service version: "~10.0" repository: https://artifactory.algol60.net/artifactory/csm-helm-charts -- name: cray-etcd-base - version: "~1.1.0" - repository: https://artifactory.algol60.net/artifactory/csm-helm-charts description: "Kubernetes resources for the Boot Orchestration Service (BOS)" home: "https://github.com/Cray-HPE/bos" keywords: - - boa - bos - cray-bos maintainers: diff --git a/kubernetes/cray-bos/files/boa-job-create.yaml.j2 b/kubernetes/cray-bos/files/boa-job-create.yaml.j2 deleted file mode 100644 index 9f3fe055..00000000 --- a/kubernetes/cray-bos/files/boa-job-create.yaml.j2 +++ /dev/null @@ -1,85 +0,0 @@ -apiVersion: batch/v1 -kind: Job -metadata: - name: {{ boa_job_name }} - namespace: services -spec: - template: - spec: - containers: - - name: boa - image: "{{ boa_image }}" - volumeMounts: - - name: boot-session - mountPath: /mnt/boot_session - - name: ca-pubkey - mountPath: /etc/cray/ca - readOnly: true - env: - - name: OPERATION - value: "{{ operation }}" - - name: SESSION_ID - value: "{{ session_id }}" - - name: SESSION_TEMPLATE_ID - value: "{{ session_template_id }}" - - name: SESSION_LIMIT - value: "{{ session_limit }}" - - name: DATABASE_NAME - value: "{{ DATABASE_NAME }}" - - name: DATABASE_PORT - value: "{{ DATABASE_PORT }}" - - name: LOG_LEVEL - value: "{{ log_level }}" - - name: S3_ACCESS_KEY - valueFrom: - secretKeyRef: - name: {{ s3_credentials }} - key: access_key - - name: S3_SECRET_KEY - valueFrom: - secretKeyRef: - name: {{ s3_credentials }} - key: secret_key - - name: GIT_SSL_CAINFO - value: /etc/cray/ca/certificate_authority.crt - - name: S3_PROTOCOL - value: "{{ S3_PROTOCOL }}" - - name: S3_GATEWAY - value: "{{ S3_GATEWAY }}" - - name: NODE_STATE_CHECK_NUMBER_OF_RETRIES - value: "{{ NODE_STATE_CHECK_NUMBER_OF_RETRIES }}" - - name: GRACEFUL_SHUTDOWN_TIMEOUT - value: "{{ GRACEFUL_SHUTDOWN_TIMEOUT }}" - - name: FORCEFUL_SHUTDOWN_TIMEOUT - value: "{{ FORCEFUL_SHUTDOWN_TIMEOUT }}" - - name: GRACEFUL_SHUTDOWN_PREWAIT - value: "{{ GRACEFUL_SHUTDOWN_PREWAIT }}" - - name: POWER_STATUS_FREQUENCY - value: "{{ POWER_STATUS_FREQUENCY }}" - - name: VCS_USERNAME - valueFrom: - secretKeyRef: - name: vcs-user-credentials - key: vcs_username - - name: VCS_PASSWORD - valueFrom: - secretKeyRef: - name: vcs-user-credentials - key: vcs_password - volumes: - - name: boot-session - configMap: - name: {{ session_id }} - - name: ca-pubkey - configMap: - defaultMode: 420 - items: - - key: certificate_authority.crt - path: certificate_authority.crt - name: cray-configmap-ca-public-key - restartPolicy: Never - security_context: - run_as_non_root: true - run_as_user: 65534 - run_as_group: 65534 - backoffLimit: 4 diff --git a/kubernetes/cray-bos/templates/boa-job-configmap.yaml b/kubernetes/cray-bos/templates/boa-job-configmap.yaml deleted file mode 100644 index 7e370677..00000000 --- a/kubernetes/cray-bos/templates/boa-job-configmap.yaml +++ /dev/null @@ -1,31 +0,0 @@ -{{/* -MIT License - -(C) Copyright 2021-2022 Hewlett Packard Enterprise Development LP - -Permission is hereby granted, free of charge, to any person obtaining a -copy of this software and associated documentation files (the "Software"), -to deal in the Software without restriction, including without limitation -the rights to use, copy, modify, merge, publish, distribute, sublicense, -and/or sell copies of the Software, and to permit persons to whom the -Software is furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included -in all copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL -THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR -OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, -ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR -OTHER DEALINGS IN THE SOFTWARE. -*/}} -apiVersion: v1 -kind: ConfigMap -metadata: - name: boa-job-template - namespace: services -data: - boa_job_create.yaml.j2: | - {{- .Files.Get "files/boa-job-create.yaml.j2" | nindent 4 }} diff --git a/kubernetes/cray-bos/templates/configmap.yaml b/kubernetes/cray-bos/templates/configmap.yaml deleted file mode 100644 index 5655ed72..00000000 --- a/kubernetes/cray-bos/templates/configmap.yaml +++ /dev/null @@ -1,32 +0,0 @@ -{{/* -MIT License - -(C) Copyright 2021-2022 Hewlett Packard Enterprise Development LP - -Permission is hereby granted, free of charge, to any person obtaining a -copy of this software and associated documentation files (the "Software"), -to deal in the Software without restriction, including without limitation -the rights to use, copy, modify, merge, publish, distribute, sublicense, -and/or sell copies of the Software, and to permit persons to whom the -Software is furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included -in all copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL -THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR -OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, -ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR -OTHER DEALINGS IN THE SOFTWARE. -*/}} -{{- $baseChartValues := index .Values "cray-service" -}} ---- -apiVersion: v1 -kind: ConfigMap -metadata: - name: boa-config - namespace: services -data: - boa_image: "{{ .Values.boa_image.repository }}:{{ .Values.boa_image.tag }}" diff --git a/kubernetes/cray-bos/templates/post-upgrade-job.yaml b/kubernetes/cray-bos/templates/post-upgrade-job.yaml index 5b0b12f0..f32776a6 100644 --- a/kubernetes/cray-bos/templates/post-upgrade-job.yaml +++ b/kubernetes/cray-bos/templates/post-upgrade-job.yaml @@ -1,7 +1,7 @@ {{/* MIT License -(C) Copyright 2021-2023 Hewlett Packard Enterprise Development LP +(C) Copyright 2021-2024 Hewlett Packard Enterprise Development LP Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), @@ -56,32 +56,7 @@ spec: runAsNonRoot: true runAsUser: 65534 runAsGroup: 65534 - - initContainers: - - name: cray-bos-wait-for-etcd - image: artifactory.algol60.net/csm-docker/stable/docker-kubectl:1.19.15 - command: - - /bin/sh - - -c - - | - while true; do - JOB_CONDITION="$(kubectl get jobs -n services -l app.kubernetes.io/name=cray-bos-wait-for-etcd -o jsonpath='{.items[0].status.conditions[0].type}')" - JOB_CONDITION_RC=$? - if [ $JOB_CONDITION_RC -eq 0 ]; then - if [ "$JOB_CONDITION" == 'Complete' ]; then - echo "Completed" - break - fi - echo "Waiting for the cray-bos-wait-for-etcd job in the services namespace to complete, current condition is $(kubectl get jobs -n services -l app.kubernetes.io/name=cray-bos-wait-for-etcd -o jsonpath='{.items[0].status}')" - sleep 3 - elif [ $JOB_CONDITION_RC -ne 1 ]; then - echo "'kubectl get jobs' failed with exit code $JOB_CONDITION_RC , failing" - exit 1 - else - echo "'kubectl get jobs' failed with exit code $JOB_CONDITION_RC , will retry" - sleep 3 - fi - done + initContainers: [] containers: - name: bos-migrations image: {{ index .Values "cray-service" "containers" "cray-bos" "image" "repository" }}:{{ .Chart.AppVersion}} diff --git a/kubernetes/cray-bos/templates/tests/test-trial.yaml b/kubernetes/cray-bos/templates/tests/test-trial.yaml index 93d6031d..64e2ea1a 100644 --- a/kubernetes/cray-bos/templates/tests/test-trial.yaml +++ b/kubernetes/cray-bos/templates/tests/test-trial.yaml @@ -1,7 +1,7 @@ {{/* MIT License -(C) Copyright 2021-2022 Hewlett Packard Enterprise Development LP +(C) Copyright 2021-2022, 2024 Hewlett Packard Enterprise Development LP Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), @@ -61,4 +61,4 @@ spec: - --retry-delay - "1" - --retry-connrefused - - http://cray-bos/v1/version + - http://cray-bos/v2/version diff --git a/kubernetes/cray-bos/values.yaml.in b/kubernetes/cray-bos/values.yaml.in index e27a76fb..30bab5cc 100644 --- a/kubernetes/cray-bos/values.yaml.in +++ b/kubernetes/cray-bos/values.yaml.in @@ -7,9 +7,6 @@ # tag: "" (default = "latest") # pullPolicy: "" (default = "IfNotPresent") -boa_image: - repository: artifactory.algol60.net/csm-docker/stable/cray-boa - tag: 0.0.0-boa database: image: repository: artifactory.algol60.net/csm-docker/stable/docker.io/library/redis @@ -55,11 +52,6 @@ cray-service: value: bos - name: PROXY_PATH value: /apis/bos - - name: BOS_BOA_IMAGE - valueFrom: - configMapKeyRef: - name: boa-config - key: boa_image - name: S3_CREDENTIALS value: "bos-s3-credentials" - name: S3_PROTOCOL @@ -76,34 +68,14 @@ cray-service: secretKeyRef: name: "bos-s3-credentials" key: secret_key - # Default configuration for the number of retries is - # 30 minutes (5s x 360 ~ 1800s or 30 minutes). - # This value is placed in the boa-job-template config map. - - name: NODE_STATE_CHECK_NUMBER_OF_RETRIES - value: "360" - # Time in seconds to wait before declaring a graceful shutdown a failure. - - name: GRACEFUL_SHUTDOWN_TIMEOUT - value: "300" - # Time in seconds to wait before declaring a forceful shutdown a failure. - - name: FORCEFUL_SHUTDOWN_TIMEOUT - value: "180" - # Time in seconds to wait before initially querying the component for its status. - # This gives the component time to 'settle'. - - name: GRACEFUL_SHUTDOWN_PREWAIT - value: "20" - # Time in seconds to wait before re-checking the power status. - - name: POWER_STATUS_FREQUENCY - value: "10" - name: PYTHONPATH value: "/app/lib/server" volumeMounts: - name: ca-vol mountPath: /mnt/ca-vol - - name: job-template - mountPath: /mnt/bos/job_templates/ livenessProbe: httpGet: - path: /v1/version + path: /v2/version port: 9000 initialDelaySeconds: 10 periodSeconds: 60 @@ -111,7 +83,7 @@ cray-service: failureThreshold: 5 readinessProbe: httpGet: - path: /v1/healthz + path: /v2/healthz port: 9000 periodSeconds: 60 timeoutSeconds: 10 @@ -128,10 +100,6 @@ cray-service: name: ca-vol configMap: name: cray-configmap-ca-public-key - job-template: - name: job-template - configMap: - name: boa-job-template affinity: podAntiAffinity: preferredDuringSchedulingIgnoredDuringExecution: @@ -149,50 +117,6 @@ cray-service: enabled: true prefix: /apis/bos uri: / - etcdWaitContainer: true - -cray-etcd-base: - nameOverride: cray-bos - etcd: - enabled: true - fullnameOverride: "cray-bos-bitnami-etcd" - persistence: - storage: 8Gi - extraEnvVars: - - name: ETCD_HEARTBEAT_INTERVAL - value: "4200" - - name: ETCD_ELECTION_TIMEOUT - value: "21000" - - name: ETCD_MAX_SNAPSHOTS - value: "1" - - name: ETCD_QUOTA_BACKEND_BYTES - value: "10737418240" - - name: ETCD_SNAPSHOT_COUNT - value: "5000000" - - name: ETCD_SNAPSHOT_HISTORY_LIMIT - value: "24" - - name: ETCD_DISABLE_PRESTOP - value: "yes" - autoCompactionMode: revision - autoCompactionRetention: "100000" - extraVolumes: - - configMap: - defaultMode: 420 - name: cray-bos-bitnami-etcd-config - name: etcd-config - resources: - limits: - cpu: 4 - memory: 8Gi - requests: - cpu: 10m - memory: 64Mi - disasterRecovery: - cronjob: - snapshotsDir: "/snapshots/cray-bos-bitnami-etcd" - schedule: "0 */1 * * *" - historyLimit: 1 - snapshotHistoryLimit: 24 deploymentDefaults: service: diff --git a/lib/bos_session_template_create.py b/lib/bos_session_template_create.py deleted file mode 100644 index 7da873af..00000000 --- a/lib/bos_session_template_create.py +++ /dev/null @@ -1,286 +0,0 @@ -#!/usr/bin/env python -# -# MIT License -# -# (C) Copyright 2019, 2021-2022 Hewlett Packard Enterprise Development LP -# -# Permission is hereby granted, free of charge, to any person obtaining a -# copy of this software and associated documentation files (the "Software"), -# to deal in the Software without restriction, including without limitation -# the rights to use, copy, modify, merge, publish, distribute, sublicense, -# and/or sell copies of the Software, and to permit persons to whom the -# Software is furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included -# in all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL -# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR -# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, -# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR -# OTHER DEALINGS IN THE SOFTWARE. -# -""" -The purpose of this ansible module is to assist in the creation of boss session templates. -""" - -from ansible.module_utils.basic import AnsibleModule -from base64 import decodestring -import bos_python_helper -import subprocess -import time -import json - -PROTOCOL = 'https' -API_GW_DNSNAME = 'api-gw-service-nmn.local' -TOKEN_URL_DEFAULT = "{}://{}/keycloak/realms/shasta/protocol/openid-connect/token".format(PROTOCOL, API_GW_DNSNAME) -BOS_URL_DEFAULT = "{}://{}/apis/bos/v1".format(PROTOCOL, API_GW_DNSNAME) -OAUTH_CLIENT_ID_DEFAULT = "admin-client" -CERT_PATH_DEFAULT = "/var/opt/cray/certificate_authority/certificate_authority.crt" - - -ANSIBLE_METADATA = { - 'metadata_version': '1.1', - 'status': ['preview', 'stableinterface'], - 'supported_by': 'community' -} - - -DOCUMENTATION = ''' ---- -module: bos_session_template_create - -short_description: This module creates session templates for BOS - -version_added: "2.7.10" - -description: - - Creates session templates for BOS - - -options: - name: - required: True - type: String - description: - required: True - type: String - cfs-url (deprecated): - required: False - type: String - cfs-branch (deprecated): - required: False - type: String - enable-cfs: - required: True - type: Boolean - cfs: - required: False - type: Json - suboptions: - commit: - type: String - branch: - type: String - clone-url: - type: String - playbook: - type: String - partition: - required: True - type: String - boot-sets: - required: True - type: Json - bos-url - required: False - type: String - default: {BOS_URL_DEFAULT} - token-url - required: False - type: String - default: {TOKEN_URL_DEFAULT} - oauth-client-id - required: False - type: String - default: {OAUTH_CLIENT_ID_DEFAULT} - oauth-client-secret - required: False - type: String - default: '' - certificate - required: False - type: String - default: {CERT_PATH_DEFAULT} - max-sleep - required: False - type: Integer - default: 300 - -author: - - rbak -'''.format(BOS_URL_DEFAULT=BOS_URL_DEFAULT, - TOKEN_URL_DEFAULT=TOKEN_URL_DEFAULT, - OAUTH_CLIENT_ID_DEFAULT=OAUTH_CLIENT_ID_DEFAULT, - CERT_PATH_DEFAULT=CERT_PATH_DEFAULT) - -EXAMPLES = ''' -# Create a new session template - - name: Invoke bos_session_template_create - bos_session_template_create: - name: test_template - description: A test session template - cfs-url: "https://api-gw-service-nmn.local/vcs/cray/config-management.git" - cfs-branch: master - enable-cfs: True - partition: p1 - boot-sets: - boot_set1: - boot_ordinal: 1 - ims_image_id: 06d37efc-7ba2-4ba9-8c22-073a641f2bf3 - kernel_parameters: console=ttyS0,115200n8 - rd.shell rd.retry=10 ip=dhcp rd.neednet=1 crashkernel=256M - hugepagelist=2m-2g intel_iommu=off bad_page=panic - iommu=pt ip=dhcp numa_interleave_omit=headless - numa_zonelist_order=node oops=panic pageblock_order=14 - pcie_ports=native printk.synchronous=y quiet turbo_boost_limit=999 - network: nmn - node_list: - - x0c0s28b0n0 - rootfs_provider: 'cps' - rootfs_provider_passthrough: 'dvs:api-gw-service-nmn.local:eth0' - boot_set2: - boot_ordinal: 1 - ims_image_id: 06d37efc-7ba2-4ba9-8c22-073a641f2bf3 - kernel_parameters: console=ttyS0,115200n8 - rd.shell rd.retry=10 ip=dhcp rd.neednet=1 crashkernel=256M - hugepagelist=2m-2g intel_iommu=off bad_page=panic - iommu=pt ip=dhcp numa_interleave_omit=headless - numa_zonelist_order=node oops=panic pageblock_order=14 - pcie_ports=native printk.synchronous=y quiet turbo_boost_limit=999 - network: nmn - node_roles_groups: - - Compute - rootfs_provider: 'cps' - rootfs_provider_passthrough: 'dvs:api-gw-service-nmn.local:eth0' -''' - - -RETURN = ''' -original_message: - description: The original name param that was passed in - type: str - returned: always -message: - description: The output message that the sample module generates - type: str - returned: always -''' - - -class BOSSessionTemplateCreate(AnsibleModule): - def __init__(self, *args, **kwargs): - super(BOSSessionTemplateCreate, self).__init__(*args, **kwargs) - self.populate_oauth_client_secret() - - #Create an BOS Helper Object - self.session = bos_python_helper.create_session(self.params['oauth-client-id'], - self.params['oauth-client-secret'], - self.params['certificate'], - self.params['token-url'], - 2000) - self.helper = bos_python_helper.BosHelper(self.params['bos-url'], - self.session) - - def populate_oauth_client_secret(self): - """ - Talk with kubernetes and obtain the client secret; this only works if the - remote execution target allows such interactions; otherwise specify the - oauth-client-secret value in the call to this module. - """ - if self.params['oauth-client-secret']: - return - stdout = subprocess.check_output(['kubectl', 'get', 'secrets', 'admin-client-auth', "-ojsonpath='{.data.client-secret}"]) - self.params['oauth-client-secret'] = decodestring(stdout.strip()) - - def api_health_checks(self): - """ - Blocks and waits for required API endpoints to respond with a known good - response; this ensures proper ordering of actions during install, which - can come online asynchronously. - """ - self.health_check_bos() - - def health_check_bos(self): - endpoint = '%s/sessiontemplate' % (self.params['bos-url']) - sleep_count = 0 - while sleep_count < self.params['max-sleep']: - response = self.session.get(endpoint) - if response.ok: - return - else: - time.sleep(1) - sleep_count += 1 - self.fail_json(msg='BOS endpoing {} was not available after {} seconds'.format( - endpoint, self.params['max-sleep'])) - - - def __call__(self): - # Check Health of APIs before proceeding - self.api_health_checks() - - try: - self.helper.bos_create_session_template( - name = self.params['name'], - description = self.params['description'], - cfs_url = self.params.get('cfs-url'), - cfs_branch = self.params.get('cfs-branch'), - enable_cfs = self.params['enable-cfs'], - cfs = self.params.get('cfs'), - partition = self.params.get('partition'), - boot_sets = json.loads(self.params['boot-sets']), - ) - except Exception as e: - self.fail_json(msg="Exception running module: %s" % e) - self.exit_json(changed=True) - - -def main(): - fields = {# Session Template Information - 'name': {'required': True, "type": "str"}, - 'description': {'required': True, "type": "str"}, - 'cfs-url': {'required': False, "type": "str"}, # DEPRECATED - 'cfs-branch': {'required': False, "type": "str"}, # DEPRECATED - 'enable-cfs': {'required': True, "type": "bool"}, - 'cfs': {'required': False, "type": "json", 'options': { - 'commit': {'required': False, "type": "str"}, - 'branch': {'required': False, "type": "str"}, - 'clone-url': {'required': False, "type": "str"}, - 'playbook': {'required': False, "type": "str"}}}, - 'partition': {'required': False, "type": "str"}, - 'boot-sets': {'required': True, "type": "json"}, - - # Endpoint Information - 'bos-url': {'required': False, "type": 'str', 'default': BOS_URL_DEFAULT}, - 'token-url': {'required': False, "type": 'str', 'default': TOKEN_URL_DEFAULT}, - - # Authentication Information - 'oauth-client-id': {'required': False, "type": "str", 'default': OAUTH_CLIENT_ID_DEFAULT}, - 'oauth-client-secret': {'required': False, "type": 'str', 'default': ''}, - 'certificate': {'required': False, "type": "str", "default": CERT_PATH_DEFAULT}, - - # Other - 'max-sleep': {'required': False, "type": "int", "default": 300}, - } - module = BOSSessionTemplateCreate(argument_spec=fields) - try: - module() - except Exception as e: - module.response['stderr'] = str(e) - module.fail_json(**module.response) - - -if __name__ == '__main__': - main() diff --git a/regenerate_server.sh b/regenerate_server.sh index 3d8f0378..2875fece 100755 --- a/regenerate_server.sh +++ b/regenerate_server.sh @@ -2,7 +2,7 @@ # # MIT License # -# (C) Copyright 2019-2022 Hewlett Packard Enterprise Development LP +# (C) Copyright 2019-2022, 2024 Hewlett Packard Enterprise Development LP # # Permission is hereby granted, free of charge, to any person obtaining a # copy of this software and associated documentation files (the "Software"), @@ -22,7 +22,7 @@ # ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR # OTHER DEALINGS IN THE SOFTWARE. # -CLI_VERSION="v5.3.0" +CLI_VERSION="v6.6.0" cp ./api/openapi.yaml.in ./api/openapi.yaml docker run --rm -v ${PWD}:/local -e PYTHON_POST_PROCESS_FILE="/usr/local/bin/yapf -i" openapitools/openapi-generator-cli:${CLI_VERSION} \ generate \ diff --git a/requirements.txt b/requirements.txt index 68d5cea0..9ca5244f 100644 --- a/requirements.txt +++ b/requirements.txt @@ -7,10 +7,8 @@ kubernetes redis requests -etcd3 boto3 liveness -Jinja2==3.0.3 # The purpose of this file is to contain python runtime requirements # for controller code, e.g., code authored by developers, as opposed to diff --git a/src/bos/common/tenant_utils.py b/src/bos/common/tenant_utils.py index 8d80c619..03498478 100644 --- a/src/bos/common/tenant_utils.py +++ b/src/bos/common/tenant_utils.py @@ -1,7 +1,7 @@ # # MIT License # -# (C) Copyright 2023 Hewlett Packard Enterprise Development LP +# (C) Copyright 2023-2024 Hewlett Packard Enterprise Development LP # # Permission is hereby granted, free of charge, to any person obtaining a # copy of this software and associated documentation files (the "Software"), @@ -34,7 +34,7 @@ TENANT_HEADER = "Cray-Tenant-Name" SERVICE_NAME = 'cray-tapms/v1alpha2' BASE_ENDPOINT = "%s://%s" % (PROTOCOL, SERVICE_NAME) -TENANT_ENDPOINT = "%s/tenants" % BASE_ENDPOINT ## CASMPET-6433 will change this from tenant to tenants +TENANT_ENDPOINT = "%s/tenants" % BASE_ENDPOINT ## CASMPET-6433 changed this from tenant to tenants class InvalidTenantException(Exception): @@ -127,15 +127,3 @@ def wrapper(*args, **kwargs): detail=str("The provided tenant does not exist")) return func(*args, **kwargs) return wrapper - - -def no_v1_multi_tenancy_support(func): - """Decorator for returning errors if the endpoint doesn't support multi-tenancy""" - @functools.wraps(func) - def wrapper(*args, **kwargs): - if get_tenant_from_header(): - return connexion.problem( - status=400, title="Multi-tenancy not supported", - detail=str("BOS v1 endpoints do not support multi-tenancy and a tenant was specified in the header")) - return func(*args, **kwargs) - return wrapper diff --git a/src/bos/server/__main__.py b/src/bos/server/__main__.py index db2742ee..8c934b81 100644 --- a/src/bos/server/__main__.py +++ b/src/bos/server/__main__.py @@ -2,7 +2,7 @@ # # MIT License # -# (C) Copyright 2019-2022 Hewlett Packard Enterprise Development LP +# (C) Copyright 2019-2022, 2024 Hewlett Packard Enterprise Development LP # # Permission is hereby granted, free of charge, to any person obtaining a # copy of this software and associated documentation files (the "Software"), @@ -29,20 +29,8 @@ import connexion -from bos.server import specialized_encoder from bos.server.controllers.v2 import options - -### Shim so that old pickled records still work -# Remove when v1 is removed -import sys -import bos.server.controllers as controllers -import bos.server.models as models - -models.v1_generic_metadata.GenericMetadata = models.v1_generic_metadata.V1GenericMetadata -sys.modules['bos.controllers'] = controllers -sys.modules['bos.models'] = models -sys.modules['bos.models.generic_metadata'] = models.v1_generic_metadata -### +from bos.server.encoder import JSONEncoder LOGGER = logging.getLogger('bos.__main__') @@ -56,7 +44,7 @@ def create_app(): options._init() app = connexion.App(__name__, specification_dir='./openapi/') - app.app.json_encoder = specialized_encoder.MetadataEncoder + app.app.json_encoder = JSONEncoder app.add_api('openapi.yaml', arguments={'title': 'Cray Boot Orchestration Service'}, diff --git a/src/bos/server/controllers/base.py b/src/bos/server/controllers/base.py index f37ea083..bf5e8274 100644 --- a/src/bos/server/controllers/base.py +++ b/src/bos/server/controllers/base.py @@ -1,7 +1,7 @@ # # MIT License # -# (C) Copyright 2019, 2021-2022 Hewlett Packard Enterprise Development LP +# (C) Copyright 2019, 2021-2022, 2024 Hewlett Packard Enterprise Development LP # # Permission is hereby granted, free of charge, to any person obtaining a # copy of this software and associated documentation files (the "Software"), @@ -24,7 +24,6 @@ # Cray-provided base controllers for the Boot Orchestration Service -from bos.server.controllers.v1 import base as v1_base from bos.server.controllers.v2 import base as v2_base import logging @@ -35,7 +34,6 @@ def root_get(): """ Get a list of supported versions """ LOGGER.info('in get_versions') versions = [ - v1_base.calc_version(details=False), - v2_base.calc_version(details=False), + v2_base.calc_version(details=False) ] return versions, 200 diff --git a/src/bos/server/controllers/v1/__init__.py b/src/bos/server/controllers/v1/__init__.py deleted file mode 100644 index 0f020c7d..00000000 --- a/src/bos/server/controllers/v1/__init__.py +++ /dev/null @@ -1,23 +0,0 @@ -# -# MIT License -# -# (C) Copyright 2019, 2021-2022 Hewlett Packard Enterprise Development LP -# -# Permission is hereby granted, free of charge, to any person obtaining a -# copy of this software and associated documentation files (the "Software"), -# to deal in the Software without restriction, including without limitation -# the rights to use, copy, modify, merge, publish, distribute, sublicense, -# and/or sell copies of the Software, and to permit persons to whom the -# Software is furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included -# in all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL -# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR -# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, -# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR -# OTHER DEALINGS IN THE SOFTWARE. -# diff --git a/src/bos/server/controllers/v1/base.py b/src/bos/server/controllers/v1/base.py deleted file mode 100644 index ef329cd9..00000000 --- a/src/bos/server/controllers/v1/base.py +++ /dev/null @@ -1,79 +0,0 @@ -# -# MIT License -# -# (C) Copyright 2019-2022 Hewlett Packard Enterprise Development LP -# -# Permission is hereby granted, free of charge, to any person obtaining a -# copy of this software and associated documentation files (the "Software"), -# to deal in the Software without restriction, including without limitation -# the rights to use, copy, modify, merge, publish, distribute, sublicense, -# and/or sell copies of the Software, and to permit persons to whom the -# Software is furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included -# in all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL -# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR -# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, -# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR -# OTHER DEALINGS IN THE SOFTWARE. -# -# Cray-provided base controllers for the Boot Orchestration Service - -import logging -import subprocess -import yaml - -from bos.server.controllers.utils import url_for -from bos.server.models import Version, Link -from os import path - -LOGGER = logging.getLogger('bos.server.controllers.v1.base') - - -def calc_version(details): - links = [ - Link( - rel='self', - href=url_for('.bos_server_controllers_base_root_get'), - ), - ] - - if details: - links.extend([ - Link( - rel='versions', - href=url_for('.bos_server_controllers_v1_base_v1_get'), - ), - ]) - - # parse open API spec file from docker image or local repository - openapispec_f = '/app/lib/bos/server/openapi/openapi.yaml' - f = None - try: - f = open(openapispec_f, 'r') - except IOError as e: - LOGGER.debug('error opening openapi.yaml file: %s' % e) - - openapispec_map = yaml.safe_load(f) - f.close() - major, minor, patch = openapispec_map['info']['version'].split('.') - return Version( - major=major, - minor=minor, - patch=patch, - links=links, - ) - - -def v1_get(): - LOGGER.info('in v1_get') - return calc_version(details=True), 200 - - -def v1_get_version(): - LOGGER.info('in v1_get_version') - return calc_version(details=True), 200 diff --git a/src/bos/server/controllers/v1/healthz.py b/src/bos/server/controllers/v1/healthz.py deleted file mode 100644 index ffac4e8a..00000000 --- a/src/bos/server/controllers/v1/healthz.py +++ /dev/null @@ -1,52 +0,0 @@ -# -# MIT License -# -# (C) Copyright 2019-2022 Hewlett Packard Enterprise Development LP -# -# Permission is hereby granted, free of charge, to any person obtaining a -# copy of this software and associated documentation files (the "Software"), -# to deal in the Software without restriction, including without limitation -# the rights to use, copy, modify, merge, publish, distribute, sublicense, -# and/or sell copies of the Software, and to permit persons to whom the -# Software is furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included -# in all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL -# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR -# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, -# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR -# OTHER DEALINGS IN THE SOFTWARE. -# -# Cray-provided base controllers for the Boot Orchestration Service - -import etcd3 -import logging - -from bos.server.models.healthz import Healthz as Healthz -from bos.server.dbclient import BosEtcdClient - -LOGGER = logging.getLogger('bos.server.controllers.v1.healthz') - - -def v1_get_healthz(): - """GET /v1/healthz - - Query BOS etcd for health status - - :rtype: Healthz - """ - # check etcd connectivity - with BosEtcdClient() as bec: - bec.put('health', 'ok') - value, _ = bec.get('health') - if value.decode('utf-8') != 'ok': - return Healthz(db_status='Failed to read from cluster', - api_status='Not Ready'), 503 - return Healthz( - db_status='ok', - api_status='ok', - ), 200 diff --git a/src/bos/server/controllers/v1/session.py b/src/bos/server/controllers/v1/session.py deleted file mode 100644 index 2bcec85f..00000000 --- a/src/bos/server/controllers/v1/session.py +++ /dev/null @@ -1,326 +0,0 @@ -# -# MIT License -# -# (C) Copyright 2019-2023 Hewlett Packard Enterprise Development LP -# -# Permission is hereby granted, free of charge, to any person obtaining a -# copy of this software and associated documentation files (the "Software"), -# to deal in the Software without restriction, including without limitation -# the rights to use, copy, modify, merge, publish, distribute, sublicense, -# and/or sell copies of the Software, and to permit persons to whom the -# Software is furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included -# in all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL -# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR -# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, -# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR -# OTHER DEALINGS IN THE SOFTWARE. -# -# Cray-provided controllers for the Boot Orchestration Service - -import connexion -import pickle -import json -import logging -import tempfile -import os -import uuid -from connexion.lifecycle import ConnexionResponse - -from jinja2 import Environment, FileSystemLoader -from kubernetes import client, config, utils -from kubernetes.config.config_exception import ConfigException -from kubernetes.client.rest import ApiException - -from bos.common.tenant_utils import no_v1_multi_tenancy_support -from bos.server.controllers.v1.sessiontemplate import get_v1_sessiontemplate -from bos.server.controllers.v1.status import BootSetDoesNotExist, create_v1_session_status -from bos.server.dbclient import BosEtcdClient, DB_HOST, DB_PORT -from bos.server.models.v1_session import V1Session as Session # noqa: E501 - -LOGGER = logging.getLogger('bos.server.controllers.v1.session') -BASEKEY = "/session" - - -@no_v1_multi_tenancy_support -def create_v1_session(): # noqa: E501 - """POST /v1/session - Creates a new boot session. # noqa: E501 - :param session: A JSON object for creating sessions - :type session: dict | bytes - - :rtype: Session - """ - if not connexion.request.is_json: - return "Post must be in JSON format", 400 - LOGGER.debug("connexion.request.is_json") - received_object = connexion.request.get_json() - LOGGER.debug("type=%s", type(received_object)) - LOGGER.debug("Received: %s", received_object) - # Check if the session is using a templateUuid - if "templateUuid" in received_object: - # templateUuid is only used if templateName is not specified. - # Either way, delete templateUuid from the session object, because we - # no longer include that field when creating V1Session objects. - template_uuid = received_object.pop("templateUuid") - if "templateName" not in received_object: - received_object["templateName"] = template_uuid - session = Session.from_dict(connexion.request.get_json()) # noqa: E501 - template_name = session.template_name - if not template_name: - msg = "templateName is a required parameter" - LOGGER.error(msg) - return msg, 400 - LOGGER.debug("Template Name: %s operation: %s", template_name, - session.operation) - # Check that the templateName exists. - session_template_response = get_v1_sessiontemplate(template_name) - if isinstance(session_template_response, ConnexionResponse): - msg = "Session Template ID invalid: {}".format(template_name) - LOGGER.error(msg) - return msg, 404 - else: - session_template, _ = session_template_response - # Validate health/validity of the sessiontemplate before creating a session - boot_sets = session_template['boot_sets'] - if not boot_sets: - msg = "Session template '%s' must have one or more defined boot sets for " \ - "creation of a session." % (template_name) - return msg, 400 - hardware_specifier_fields = ('node_roles_groups', 'node_list', 'node_groups') - for bs_name, bs in session_template['boot_sets'].items(): - specified = [bs.get(field, None) - for field in hardware_specifier_fields] - if not any(specified): - msg = "Session template '%s' boot set '%s' must have at least one " \ - "hardware specifier field provided (%s); None defined." \ - % (template_name, bs_name, - ', '.join(sorted(hardware_specifier_fields))) - return msg, 400 - # Handle empty limit so that the environment var is not set to "None" - if not session.limit: - session.limit = '' - # Kubernetes set-up - # Get the Kubernetes configuration - try: - config.load_incluster_config() - except ConfigException: # pragma: no cover - config.load_kube_config() # Development - # Create API endpoint instance and API resource instances - k8s_client = client.ApiClient() - try: - api_instance = client.CoreV1Api(k8s_client) - except ApiException as err: - LOGGER.error("Exception when calling CoreV1API to create an API instance: %s\n", err) - raise - # Create the configMap for the BOA - json_data = {'data.json': json.dumps(session_template)} - namespace = os.getenv("NAMESPACE", 'services') - # Determine the session ID and BOA K8s job name - session_id, boa_job_name = _get_boa_naming_for_session() - LOGGER.debug("session_id: %s", session_id) - LOGGER.debug("boa_job_name: %s", boa_job_name) - body = client.V1ConfigMap(data=json_data) - body.metadata = client.V1ObjectMeta(namespace=namespace, name=session_id) - try: - api_response = api_instance.create_namespaced_config_map(namespace, body) - except ApiException as err: - LOGGER.error("Exception when calling CoreV1API to create a configMap: %s\n", err) - raise - LOGGER.debug("ConfigMap: %s", api_response) - # Create the BOA master job - # Fill out the template first with the input parameters - env = Environment(loader=FileSystemLoader('/mnt/bos/job_templates/')) - template = env.get_template('boa_job_create.yaml.j2') - try: - log_level = os.getenv("LOG_LEVEL", "DEBUG") - bos_boa_image = os.getenv("BOS_BOA_IMAGE") - s3_credentials = os.getenv("S3_CREDENTIALS") - s3_protocol = os.getenv("S3_PROTOCOL") - s3_gateway = os.getenv("S3_GATEWAY") - nbr_retries = os.getenv("NODE_STATE_CHECK_NUMBER_OF_RETRIES") - graceful_shutdown_timeout = os.getenv("GRACEFUL_SHUTDOWN_TIMEOUT") - forceful_shutdown_timeout = os.getenv("FORCEFUL_SHUTDOWN_TIMEOUT") - graceful_shutdown_prewait = os.getenv("GRACEFUL_SHUTDOWN_PREWAIT") - power_status_frequency = os.getenv("POWER_STATUS_FREQUENCY") - except KeyError as error: - LOGGER.error("Missing information necessary to create session %s", error) - raise - # Render the job submission template - rendered_template = template.render(boa_job_name=boa_job_name, - boa_image=bos_boa_image, - session_id=session_id, - session_template_id=str(template_name), - session_limit=session.limit, - operation=session.operation, - DATABASE_NAME=str(DB_HOST), - DATABASE_PORT=str(DB_PORT), - log_level=log_level, - s3_credentials=s3_credentials, - S3_PROTOCOL=s3_protocol, - S3_GATEWAY=s3_gateway, - NODE_STATE_CHECK_NUMBER_OF_RETRIES=nbr_retries, - graceful_shutdown_timeout=graceful_shutdown_timeout, - forceful_shutdown_timeout=forceful_shutdown_timeout, - graceful_shutdown_prewait=graceful_shutdown_prewait, - power_status_frequency=power_status_frequency) - LOGGER.debug(rendered_template) - # Write the rendered job template - ntf = tempfile.NamedTemporaryFile(delete=False).name - with open(ntf, 'w') as outf: - outf.write(rendered_template) - try: - utils.create_from_yaml(k8s_client, ntf) - except utils.FailToCreateError as err: - LOGGER.error("Failed to create BOA Job: %s", err) - raise - with BosEtcdClient() as bec: - key = "{}/{}/templateName".format(BASEKEY, session_id) - bec.put(key=key, value=template_name) - key = "{}/{}/operation".format(BASEKEY, session_id) - bec.put(key=key, value=session.operation) - key = "{}/{}/status_link".format(BASEKEY, session_id) - bec.put(key=key, value="/v1/session/{}/status".format(session_id)) - key = "{}/{}/job".format(BASEKEY, session_id) - bec.put(key=key, value=boa_job_name) - return_json_data = { - "operation": session.operation, - "templateName": template_name, - "job": boa_job_name, - "links": - [ - { - "rel": "session", - "href": "/v1/session/{}".format(session_id), - "jobId": boa_job_name, - "type": "GET" - }, - { - "rel": "status", - "href": "/v1/session/{}/status".format(session_id), - "type": "GET" - } - ] - } - if session.limit: - return_json_data['limit'] = session.limit - # Create a Session Status Record whenever we create a session - create_v1_session_status(session_id) - return return_json_data, 201 - - -@no_v1_multi_tenancy_support -def get_v1_sessions(): # noqa: E501 - """GET /v1/session - - List all sessions - """ - LOGGER.info("Called get v1 sessions") - with BosEtcdClient() as bec: - key = "{}/".format(BASEKEY) - sessions = set() - for _, metadata in bec.get_prefix(key): - sessions.add(metadata.key.decode('utf-8').split('/')[2]) - if not sessions: - LOGGER.debug("No Sessions were found.") - return list(sessions), 200 - - -@no_v1_multi_tenancy_support -def get_v1_session(session_id): # noqa: E501 - """GET /v1/session - Get the session by session ID - Args: - session_id (str): Session ID - Return: - Session Dictionary, Status Code - The session dictionary contains keys which are attributes about the Session - and the values for those attributes. Additional values from session status - are appended regarding start/stop/completion time. - Example: - Key | Value - operation | boot, shutdown, reboot, configure - """ - with BosEtcdClient() as bec: - key = "{}/{}/".format(BASEKEY, session_id) - session = {} - for value, metadata in bec.get_prefix(key): - # The metadata split looks like this: - # ['', 'session', '', ''] - attribute = metadata.key.decode('utf-8').split('/')[3] - if attribute == 'status': - # Subset of status metadata information is handled separately below - continue - session[attribute] = value.decode('utf-8') - if not session: - return session, 404 - # CASMCMS-5128: cherry-pick forward metadata keys if they're set - metadata_defaults = {'start_time': '', - 'stop_time': '', - 'complete': '', - 'in_progress': '', - "error_count": ''} - session.update(metadata_defaults) - with BosEtcdClient() as bec: - status_key = "%sstatus" % (key) - try: - value, _ = bec.get(status_key) - except (ValueError, AttributeError): - # No status available - return session, 200 - if not value: - return session, 200 - status = pickle.loads(value) - - for key in metadata_defaults.keys(): - try: - session[key] = getattr(status.metadata, key) - except (AttributeError, BootSetDoesNotExist): - # The key doesn't exist (yet?) Use the default. - pass - return session, 200 - - -@no_v1_multi_tenancy_support -def delete_v1_session(session_id): # noqa: E501 - """DELETE /v1/session - - Delete the session by session id - """ - with BosEtcdClient() as bec: - key = "{}/{}/".format(BASEKEY, session_id) - resp = bec.delete_prefix(key) - if resp.deleted >= 1: - return '', 204 - else: - return 'Sesssion: {} not found'.format(session_id), 404 - - -def _get_boa_naming_for_session(): - """ Return the BOA session ID and Kubernetes - BOA job name for this session. - - This method consolidates the logic of session ID and - K8s job name creation into a single place. The BOA - job template will use the boa_k8s_job_name returned here - and will not prepend or make further changes. This job - name is also returned as part of the session creation - result for use in monitoring BOA job completion. - """ - # Generate the session name. - session_id = str(uuid.uuid4()) - - # Construct the K8s BOA job name. - # Any future changes here should ensure that the name does not exceed 63 - # characters (k8s job name max). It must also conform to Kubernetes - # naming standards. See CASMCMS-3638. - # For now this is fine as len('boa-'+str(uuid.uuid4())) = 40 - # and uuid4 is valid for use in the Kubernetes job name. - boa_k8s_job_name = 'boa-' + session_id - - return session_id, boa_k8s_job_name diff --git a/src/bos/server/controllers/v1/sessiontemplate.py b/src/bos/server/controllers/v1/sessiontemplate.py deleted file mode 100644 index b09f903f..00000000 --- a/src/bos/server/controllers/v1/sessiontemplate.py +++ /dev/null @@ -1,232 +0,0 @@ -# -# MIT License -# -# (C) Copyright 2019-2023 Hewlett Packard Enterprise Development LP -# -# Permission is hereby granted, free of charge, to any person obtaining a -# copy of this software and associated documentation files (the "Software"), -# to deal in the Software without restriction, including without limitation -# the rights to use, copy, modify, merge, publish, distribute, sublicense, -# and/or sell copies of the Software, and to permit persons to whom the -# Software is furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included -# in all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL -# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR -# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, -# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR -# OTHER DEALINGS IN THE SOFTWARE. -# -# Cray-provided controllers for the Boot Orchestration Service - -import logging -import connexion -import json -import os - -from bos.common.tenant_utils import no_v1_multi_tenancy_support -from bos.server import redis_db_utils as dbutils -from bos.server.models.v1_session_template import V1SessionTemplate # noqa: E501 -from bos.server.models.v2_session_template import V2SessionTemplate # noqa: E501 -from bos.server.utils import _canonize_xname -from bos.common.tenant_utils import get_tenant_aware_key -from ..v2.sessiontemplates import get_v2_sessiontemplate, get_v2_sessiontemplates, delete_v2_sessiontemplate - -LOGGER = logging.getLogger('bos.server.controllers.v1.sessiontemplate') -DB = dbutils.get_wrapper(db='session_templates') - -EXAMPLE_BOOT_SET = { - "type": "s3", - "etag": "boot-image-s3-etag", - "kernel_parameters": "your-kernel-parameters", - "node_list": [ - "xname1", "xname2", "xname3"], - "path": "s3://boot-images/boot-image-ims-id/manifest.json", - "rootfs_provider": "cpss3", - "rootfs_provider_passthrough": "dvs:api-gw-service-nmn.local:300:hsn0,nmn0:0"} - -EXAMPLE_SESSION_TEMPLATE = { - "boot_sets": { - "name_your_boot_set": EXAMPLE_BOOT_SET}, - "cfs": { - "configuration": "desired-cfs-config"}, - "enable_cfs": True} - -V1_SPECIFIC_ST_FIELDS = [ "cfs_branch", "cfs_url", "partition" ] -V1_SPECIFIC_CFS_FIELDS = [ "branch", "clone_url", "commit", "playbook" ] -V1_SPECIFIC_BOOTSET_FIELDS = [ "boot_ordinal", "network", "shutdown_ordinal" ] - -def sanitize_xnames(st_json): - """ - Sanitize xnames - Canonize the xnames - N.B. Because python passes object references by value you need to use - the return value. It will have no impact on the inputted object. - Args: - st_json (string): The Session Template as a JSON object - - Returns: - The Session Template with all of the xnames sanitized - """ - if 'boot_sets' in st_json: - for boot_set in st_json['boot_sets']: - if 'node_list' in st_json['boot_sets'][boot_set]: - clean_nl = [_canonize_xname(node) for node in - st_json['boot_sets'][boot_set]['node_list']] - st_json['boot_sets'][boot_set]['node_list'] = clean_nl - return st_json - -def strip_v1_only_fields(template_data): - """ - Edits in-place the template data, removing any fields which are specific to BOS v1. - Returns True if any changes were made. - Returns False if nothing was removed. - """ - changes_made=False - - # Strip out the v1-specific fields from the dictionary - for v1_field_name in V1_SPECIFIC_ST_FIELDS: - try: - del template_data[v1_field_name] - LOGGER.info("Stripped %s field from session template %s", v1_field_name, - template_data.get("name", "")) - changes_made=True - except KeyError: - pass - - # Do the same for each boot set - # Oddly, boot_sets is not a required field, so only do this if it is present - if "boot_sets" in template_data: - for bs in template_data["boot_sets"].values(): - for v1_bs_field_name in V1_SPECIFIC_BOOTSET_FIELDS: - try: - del bs[v1_bs_field_name] - LOGGER.info("Stripped %s field from a boot set in session template %s", - v1_bs_field_name, template_data.get("name", "")) - changes_made=True - except KeyError: - pass - - # Do the same for the cfs field, if present - if "cfs" in template_data: - cfs_data = template_data["cfs"] - for v1_cfs_field_name in V1_SPECIFIC_CFS_FIELDS: - try: - del cfs_data[v1_cfs_field_name] - LOGGER.info("Stripped cfs.%s field from session template %s", v1_cfs_field_name, - template_data.get("name", "")) - changes_made=True - except KeyError: - pass - - return changes_made - -@no_v1_multi_tenancy_support -@dbutils.redis_error_handler -def create_v1_sessiontemplate(): # noqa: E501 - """POST /v1/sessiontemplate - - Creates a new session template. # noqa: E501 - """ - LOGGER.debug("POST /v1/sessiontemplate invoked create_v1_sessiontemplate") - if connexion.request.is_json: - LOGGER.debug("connexion.request.is_json") - LOGGER.debug("type=%s", type(connexion.request.get_json())) - LOGGER.debug("Received: %s", connexion.request.get_json()) - else: - return "Post must be in JSON format", 400 - - sessiontemplate = None - - try: - """Verify that we can convert the JSON request data into a - V1SessionTemplate object. - Any exceptions caught here would be generated from the model - (i.e. bos.server.models.session_template). Examples are - an exception for a session template missing the required name - field, or an exception for a session template name that does not - confirm to Kubernetes naming convention. - In this case return 400 with a description of the specific error. - """ - template_data = connexion.request.get_json() - V1SessionTemplate.from_dict(template_data) - except Exception as err: - return connexion.problem( - status=400, title="The session template could not be created.", - detail=str(err)) - - strip_v1_only_fields(template_data) - - # BOS v2 doesn't want the session template name inside the dictionary itself - # name is a required v1 field, though, so we can safely pop it here - session_template_id = template_data.pop("name") - - # Now basically follow the same process as when creating a V2 session template (except in the end, - # if successful, we will return 201 status and the name of the template, to match the v1 API spec) - try: - """Verify that we can convert the JSON request data into a - V2SessionTemplate object. - Any exceptions caught here would be generated from the model - (i.e. bos.server.models.session_template). - An example is an exception for a session template name that - does not conform to Kubernetes naming convention. - In this case return 400 with a description of the specific error. - """ - V2SessionTemplate.from_dict(template_data) - except Exception as err: - return connexion.problem( - status=400, title="The session template could not be created as a v2 template.", - detail=str(err)) - - template_data = sanitize_xnames(template_data) - template_data['name'] = session_template_id - # Tenants are not used in v1, but v1 and v2 share template storage - template_data['tenant'] = "" - template_key = get_tenant_aware_key(session_template_id, "") - DB.put(template_key, template_data) - return session_template_id, 201 - - -@no_v1_multi_tenancy_support -def get_v1_sessiontemplates(): # noqa: E501 - """ - GET /v1/sessiontemplates - - List all sessiontemplates - """ - LOGGER.debug("get_v1_sessiontemplates: Fetching sessions.") - return get_v2_sessiontemplates() - - -@no_v1_multi_tenancy_support -def get_v1_sessiontemplate(session_template_id): - """ - GET /v1/sessiontemplate - - Get the session template by session template ID - """ - LOGGER.debug("get_v1_sessiontemplate by ID: %s", session_template_id) # noqa: E501 - return get_v2_sessiontemplate(session_template_id) - - -def get_v1_sessiontemplatetemplate(): - """ - GET /v1/sessiontemplatetemplate - - Get the example session template - """ - return EXAMPLE_SESSION_TEMPLATE, 200 - - -@no_v1_multi_tenancy_support -def delete_v1_sessiontemplate(session_template_id): - """ - DELETE /v1/sessiontemplate - - Delete the session template by session template ID - """ - LOGGER.debug("delete_v1_sessiontemplate by ID: %s", session_template_id) - return delete_v2_sessiontemplate(session_template_id) diff --git a/src/bos/server/controllers/v1/status.py b/src/bos/server/controllers/v1/status.py deleted file mode 100644 index db7fe342..00000000 --- a/src/bos/server/controllers/v1/status.py +++ /dev/null @@ -1,791 +0,0 @@ -# -# MIT License -# -# (C) Copyright 2020-2023 Hewlett Packard Enterprise Development LP -# -# Permission is hereby granted, free of charge, to any person obtaining a -# copy of this software and associated documentation files (the "Software"), -# to deal in the Software without restriction, including without limitation -# the rights to use, copy, modify, merge, publish, distribute, sublicense, -# and/or sell copies of the Software, and to permit persons to whom the -# Software is furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included -# in all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL -# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR -# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, -# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR -# OTHER DEALINGS IN THE SOFTWARE. -# -''' -@author: Jason Sollom -''' -import connexion -import datetime -import logging -import pickle -import flask - -from bos.common.tenant_utils import no_v1_multi_tenancy_support -from bos.server.dbclient import BosEtcdClient -from bos.server.models.v1_session_status import V1SessionStatus as SessionStatusModel -from bos.server.models.v1_boot_set_status import V1BootSetStatus as BootSetStatus -from bos.server.models.v1_node_change_list import V1NodeChangeList as NodeChangeList -from bos.server.models.v1_node_errors_list import V1NodeErrorsList as NodeErrorsList -from bos.server.models import Link -from bos.server.models.v1_generic_metadata import V1GenericMetadata as GenericMetadata - -LOGGER = logging.getLogger(__name__) -BASEKEY = "/session" - - -class BaseStatusException(BaseException): - """ - Base Exception for BOS Status - """ - pass - - -class BadPhase(BaseStatusException): - """ - The Phase is either invalid in this context or unknown. - """ - pass - - -class InvalidCategory(BaseStatusException): - """ - The Category Name is invalid. - """ - pass - - -class MissingCategory(BaseStatusException): - """ - The Category is missing. - """ - pass - - -class SessionStatusDoesNotExist(BaseStatusException): - """ - The Session Status does not exist. - """ - pass - - -class BootSetDoesNotExist(BaseStatusException): - """ - The Boot Set does not exist. - """ - pass - - -class BadSession(BaseStatusException): - """ - The Session is invalid. - """ - pass - - -class Metadata(object): - """ - Generic metadata - Start - Time when processing started. - Stop - Time when processing ended - In_Progress - processing is in progress - error_count - The number of errors encounter; Not finalized until complete is true - """ - - def __init__(self, start=None): - self.start_time = start - self.stop_time = None - - @property - def in_progress(self): - """ - Is the object associated with this Metadata in progress? - This method is expected to be overridden. - - Returns: - True or False - """ - return False - - @property - def error_count(self): - """ - Are there errors with the object associated with this Metadata complete - This method is expected to be overridden. - - Returns: - An error count (integer) - """ - return 0 - - -class MetadataSession(Metadata): - def __init__(self, session, start=None): - self.session = session - self._boot_sets = [] - super().__init__(start=start) - - @property - def boot_sets(self): - """ - Loads all of the Boot Sets in the session from the database. - This will only get the Boot Sets one time and cache them. - """ - if not self.session.boot_sets: - return [] - if not self._boot_sets: - for bs in self.session.boot_sets: - self._boot_sets.append(BootSet.load(self.session.id, bs)) - return self._boot_sets - - @property - def in_progress(self): - """ - The Session is in_progress if any Boot Set it contains is in_progress. - - Returns: - complete (bool): True if the Session is in progress. False if it is not. - """ - return not self.complete - - @property - def error_count(self): - """ - The number of nodes that have failed in a Session. This counts all - of the failed nodes from each Boot Set. - """ - count = 0 - for bs in self.boot_sets: - count += bs.metadata.error_count - return count - - - @property - def complete(self): - return bool(self.stop_time) - - -class MetadataBootSet(Metadata): - - def __init__(self, boot_set, start=None): - self.boot_set = boot_set # This is a reference to the boot set, not its name. - super().__init__(start=start) - - @property - def in_progress(self): - """ - The Boot Set is in_progress if any Phase it contains is in_progress. - - Returns: - complete (bool): True if the Boot Set is in progress. False if it is not. - """ - in_progress = False - for p in self.boot_set.phases: - if p.metadata.in_progress: - in_progress = True - break - return in_progress - - @property - def complete(self): - """ - The Boot Set is complete when each phase it contains is complete. - - Returns: - complete (bool): True if the Boot Set is complete. False if it is not. - """ - complete = True - for p in self.boot_set.phases: - if not p.metadata.complete: - complete = False - break - return complete - - @property - def error_count(self): - """ - The number of nodes that have failed in a Boot Set. This counts all - of the failed nodes from each phase. - """ - count = 0 - for p in self.boot_set.phases: - count += p.metadata.error_count - return count - - -class MetadataPhase(Metadata): - - def __init__(self, boot_set, phase_name, start): - self.boot_set = boot_set # This is a reference to the boot set, not its name. - self.phase = boot_set.get_phase(phase_name) - super().__init__(start=start) - - @property - def in_progress(self): - """ - The Phase is in_progress if there are any nodes in the in_progress category. - - Returns: - in_progress (bool): True if the Phase is in progress. False if it is not. - """ - return len(self.boot_set.get_category(self.phase.name, 'in_progress').node_list) != 0 - - @property - def complete(self): - """ - The Phase is complete if there are no nodes in the not_started or in_progress - categories. Is that true? What about in the failed state? - - Returns: - complete (bool): True if the Boot Set is complete. False if it is not. - """ - if (len(self.boot_set.get_category(self.phase.name, 'not_started').node_list) != 0 - or len(self.boot_set.get_category(self.phase.name, # noqa: W503 - 'in_progress').node_list) != 0): - return False - return True - - @property - def error_count(self): - """ - The number of nodes that have failed in a phase. - """ - return len(self.boot_set.get_category(self.phase.name, 'failed').node_list) - - -class BootSet(BootSetStatus): - """ - A Boot Set operates on one or more nodes. This operation will have one or more phases. - This records the status for a Boot Set. - - The BootSet is a composite of the BootSetStatus and various useful methods. - I did not make it a subclass of BootSetStatus because I need to convert it back to - the BootSetStatus when I return it. It is easier to return just an attribute of - this class rather than attempting to strip out the methods. - """ - - def _create_links(self): - self_url = flask.helpers.url_for( - '.bos_server_controllers_v1_status_get_v1_session_status_by_bootset', session_id=self.session, - boot_set_name=self.name) - self.links = [Link(rel='self', href=self_url)] - - for phase in self.phases: - self.links.append( - Link( - rel='Phase', - href="{}/{}".format(self_url, phase.name)) - ) - - def initialize(self): - """ - If no metadata has been specified, it initializes the metadata. - It also creates the links - """ - # TODO: Decide. I could have an additional property called - # nonjson_metadata = Metadata(start_time=self._metadata.start_time) - # if not self._metadata: - start_value = None - if hasattr(self.metadata, "start_time"): - start_value = self.metadata.start_time - self.metadata = MetadataBootSet(self, start=start_value) - - # Convert the metadata to MetadataPhase - for i, _ in enumerate(self.phases): - start_value = None - if hasattr(self.phases[i].metadata, "start_time"): - start_value = self.phases[i].metadata.start_time - self.phases[i].metadata = MetadataPhase(self, - self.phases[i].name, - start=start_value) - - self._create_links() - - def start(self, time=None): - if not self.metadata.start_time: - if not time: - self.metadata.start_time = datetime.datetime.now() - else: - self.metadata.start_time = time - - def stop(self, time=None): - if not self.metadata.stop_time: - if not time: - self._metadata.stop_time = datetime.datetime.now() - else: - self._metadata.stop_time = time - - def save(self): - """ - Save the Boot Set - """ - with BosEtcdClient() as bec: - key = "{}/{}/status/{}".format(BASEKEY, self.session, self.name) - bec.put(key=key, value=pickle.dumps(self)) - - @classmethod - def load(cls, session, name): - """ - Load the Boot Set. - - Returns: - A Boot Set instance - """ - with BosEtcdClient() as bec: - key = "{}/{}/status/{}".format(BASEKEY, session, name) - value, _ = bec.get(key) - if not value: - raise BootSetDoesNotExist("ERROR: Boot Set %s not found.", name) - return pickle.loads(value) - - @classmethod - def delete(cls, session, name): - """ - Delete the Boot Set - - Return: - status (int): Returns a True if it was deleted, False if it - did not exist - """ - try: - BootSet.load(session, name) - except BootSetDoesNotExist: - return False - except Exception: - # For unexpected exceptions, just delete the key. - pass - - with BosEtcdClient() as bec: - key = "{}/{}/status/{}".format(BASEKEY, session, name) - bec.delete(key) - return True - - def update_nodes(self, phase, source, destination, nodes): - """ - Update a phase's status. This involves moving a node from one category - to another. - - Args: - phase (string): The phase we are updating - source (string): The category the node started in - destination (string): The category the node moved to - nodes (set): The nodes moving - """ - try: - - if source == destination: - LOGGER.warning("The source and destination are the same. Doing nothing.") - return - - phase_index = self._get_phase_index(phase) - source_index = self._get_category_index(phase, source) - destination_index = self._get_category_index(phase, destination) - if source_index is None: - raise MissingCategory("The source category is missing.") - if destination_index is None: - raise MissingCategory("The destination category is missing.") - - set_nodes = set(nodes) - source_nodes = set(self.phases[phase_index].categories[source_index].node_list) - destination_nodes = set(self.phases[phase_index].categories[destination_index].node_list) - not_in_source = set_nodes - source_nodes - if not_in_source: - LOGGER.warning("The following nodes were supposed to move from %s to %s, " - "but they were never in %s: %s", - source, destination, source, not_in_source) - LOGGER.warning("Regardless, they are being moved to %s", destination) - source_nodes -= set_nodes - destination_nodes |= set_nodes - self.phases[phase_index].categories[source_index].node_list = list(source_nodes) - self.phases[phase_index].categories[destination_index].node_list = list(destination_nodes) - except KeyError: - raise - - def update_errors(self, phase_name, errors): - """ - Update a phase's list of errors - """ - phase = self.get_phase(phase_name) - phase_index = self._get_phase_index(phase_name) - if phase.errors is None: - phase.errors = {} - if not phase.errors: - phase.errors = errors - self.phases[phase_index] = phase - return - - for error, nodes in errors.items(): - if phase.errors: - if error in phase.errors: - old_nodes = set(phase.errors[error]) - new_nodes = set(nodes) - all_nodes = set.union(old_nodes, new_nodes) - phase.errors[error] = list(all_nodes) - break - else: - phase.errors.update({error: nodes}) - - self.phases[phase_index] = phase - - def update_metadata_phase(self, phase, start=None, stop=None): - """ - Update a phase's metadata - """ - phase = self.get_phase(phase) - if start: - phase.metadata.start_time = start - if stop: - phase.metadata.stop_time = stop - - def _get_phase_index(self, phase_name): - """ - Return the index of the phase - - Args: - phase_name (str): The phase's name - """ - pi = None - for phase_index, p in enumerate(self.phases): - if phase_name.lower() == p.name.lower(): - pi = phase_index - break - return pi - - def get_phase(self, phase): - """ - Return the phase - """ - pi = None - for phase_index, p in enumerate(self.phases): - if phase.lower() == p.name.lower(): - pi = phase_index - break - if pi is not None: - return self.phases[pi] - else: - raise BadPhase("Phase '{}' is not found.".format(phase)) - - def _get_category_index(self, phase, category): - """ - Return the index of the phase category - """ - phase_index = self._get_phase_index(phase) - ci = None - for category_index, c in enumerate(self.phases[phase_index].categories): - if category.lower() == c.name.lower(): - ci = category_index - break - return ci - - def get_category(self, phase, category): - """ - Args: - phase (string): The phase we are interested in - category (string): Category to retrieve - - Returns: - PhaseCategory object - """ - phase_index = self._get_phase_index(phase) - category_index = self._get_category_index(phase, category) - return self.phases[phase_index].categories[category_index] - - def set_category_nodes(self, phase, category, nodes): - """ - Args: - nodes (list): A list of nodes - phase (string): The phase we are interested in - category (string): Category to retrieve - """ - phase_index = self._get_phase_index(phase) - category_index = self._get_category_index(phase, category) - self.phases[phase_index].categories[category_index].node_list = list(nodes) - - def get_all_categories(self, phase): - """ - Returns: - A dictionary of all categories in the phase - Key is the category name. The value is a PhaseCategory object - """ - phase_index = self._get_phase_index(phase) - return self.phases[phase_index].categories - - -class SessionStatus(SessionStatusModel): - """ - A Session lists the boot sets it contains as well as some metadata. - - Args: - session (string): The Session ID - boot_sets (string): A list contains the names of the boot sets in the session - """ - - def _create_links(self): - self_url = flask.helpers.url_for('.bos_server_controllers_v1_status_get_v1_session_status', - session_id=self.id) - self.links = [Link(rel='self', href=self_url)] - if not self.boot_sets: - return - for bs in self.boot_sets: - self.links.append( - Link( - rel='Boot Set', - href="{}/{}".format(self_url, bs)) - ) - - def initialize(self): - """ - It converts any original metadata into the operational metadata. - It creates links. - """ - start_value = None - if hasattr(self.metadata, "start_time"): - start_value = self.metadata.start_time - self.metadata = MetadataSession(self, start=start_value) - self._create_links() - - def start(self, time=None): - if not self.metadata.start_time: - if not time: - self._metadata.start_time = datetime.datetime.now() - else: - self._metadata.start_time = time - - def stop(self, time=None): - if not self.metadata.stop_time: - if not time: - self._metadata.stop_time = datetime.datetime.now() - else: - self._metadata.stop_time = time - - def save(self): - """ - Save the Session - """ - with BosEtcdClient() as bec: - key = "{}/{}/status".format(BASEKEY, self.id) - bec.put(key=key, value=pickle.dumps(self)) - - @classmethod - def load(cls, session_id): - """ - Load the Session. - - Returns: - A Session instance - """ - with BosEtcdClient() as bec: - key = "{}/{}/status".format(BASEKEY, session_id) - value, _ = bec.get(key) - if not value: - raise SessionStatusDoesNotExist("ERROR: Session %s not found.", session_id) - return pickle.loads(value) - - @classmethod - def delete(cls, session_id): - """ - Delete the Session - - Return: - status (int): Returns a True if it was deleted, False if it - did not exist - """ - try: - SessionStatus.load(session_id) - except SessionStatusDoesNotExist: - return False - except Exception: - # For unexpected exceptions, just delete the key. - pass - with BosEtcdClient() as bec: - key = "{}/{}/status".format(BASEKEY, session_id) - bec.delete(key) - return True - - -class Session(SessionStatus): - # This is here for backwards compatibility with previous - # BOS statuses that used code where SessionStatus was named - # Session. - pass - - -@no_v1_multi_tenancy_support -def create_v1_session_status(session_id): - """ - Create a Status for the Session - - Args: - session (string): Session ID - """ - LOGGER.debug("create_v1_session_status: %s/status/", session_id) - # Look up the Session. If it already exists, do not create a new one, but - # return a 409. - try: - session_status = None - status = 409 - session_status = SessionStatus.load(session_id) - except SessionStatusDoesNotExist: - request_body = connexion.request.get_json() - LOGGER.debug("Request body: {}".format(request_body)) - request_body['id'] = session_id - session_status = SessionStatus.from_dict(request_body) - session_status.initialize() - session_status.start() - session_status.save() - status = 200 - return session_status, status - - -@no_v1_multi_tenancy_support -def create_v1_boot_set_status(session_id, boot_set_name): - """ - Create a Status for Boot Set - - Args: - session (string): Session ID - boot_set_name (string): Boot Set Name - phases (list): Phases in the Boot Set (strings) - nodes (list): Nodes in the Boot Set - """ - - LOGGER.debug("create_v1_boot_set_status: %s/status/%s", session_id, - boot_set_name) - try: - bs = BootSet.load(session_id, boot_set_name) - status = 409 - except BootSetDoesNotExist: - request_body = connexion.request.get_json() - LOGGER.debug("Request body: {}".format(request_body)) - request_body['name'] = boot_set_name - request_body['session'] = session_id - bs = BootSet.from_dict(request_body) - bs.initialize() - bs.start() - bs.save() - status = 200 - return bs, status - - -@no_v1_multi_tenancy_support -def update_v1_session_status(session_id): - """ - Update an existing session's status - """ - request_body = connexion.request.get_json() - try: - session_status = SessionStatus.load(session_id) - except SessionStatusDoesNotExist: - return None, 404 - if 'start_time' in request_body: - session_status.start(request_body['start_time']) - if 'stop_time' in request_body: - session_status.stop(request_body['stop_time']) - session_status.save() - return session_status, 200 - - -@no_v1_multi_tenancy_support -def update_v1_session_status_by_bootset(session_id, boot_set_name): - """ - Update an existing status for a Boot Set - """ - request_body = connexion.request.get_json() - try: - bs = BootSet.load(session_id, boot_set_name) - except BootSetDoesNotExist: - return None, 404 - - for update_item in request_body: - if update_item['update_type'] == 'NodeChangeList': - payload = NodeChangeList.from_dict(update_item['data']) - bs.update_nodes(update_item['phase'], - payload.source, - payload.destination, - payload.node_list) - elif update_item['update_type'] == 'NodeErrorsList': - payload = NodeErrorsList.from_dict(update_item['data']) - bs.update_errors(update_item['phase'], - payload) - elif update_item['update_type'] == 'GenericMetadata': - payload = GenericMetadata.from_dict(update_item['data']) - if update_item['phase'] == 'boot_set': - # If the 'phase' specified is boot_set, update the Boot Set's metadata. - if payload.start_time: - bs.metadata.start_time = payload.start_time - if payload.stop_time: - bs.metadata.stop_time = payload.stop_time - else: - bs.update_metadata_phase(update_item['phase'], - payload.start_time, - payload.stop_time) - bs.save() - - -@no_v1_multi_tenancy_support -def get_v1_session_status(session_id): - """ - List the Boot Sets in the Session, so they can be queried individually. - Provide some metadata - """ - try: - session = SessionStatus.load(session_id) - status = 200 - except SessionStatusDoesNotExist: - status = 404 - session = None - return session, status - - -@no_v1_multi_tenancy_support -def get_v1_session_status_by_bootset(session_id, - boot_set_name): - try: - bs = BootSet.load(session_id, boot_set_name) - status = 200 - except BootSetDoesNotExist: - bs = None - status = 404 - return bs, status - - -@no_v1_multi_tenancy_support -def get_v1_session_status_by_bootset_and_phase(session_id, - boot_set_name, - phase_name): - bs = BootSet.load(session_id, boot_set_name) - return bs.get_phase(phase_name) - - -@no_v1_multi_tenancy_support -def get_v1_session_status_by_bootset_and_phase_and_category(session_id, - boot_set_name, - phase_name, - category_name): - bs = BootSet.load(session_id, boot_set_name) - return bs.get_category(phase_name, category_name) - - -@no_v1_multi_tenancy_support -def delete_v1_session_status(session_id): - if SessionStatus.delete(session_id): - return 204 - else: - return 404 - - -@no_v1_multi_tenancy_support -def delete_v1_boot_set_status(session_id, boot_set_name): - if BootSet.delete(session_id, boot_set_name): - return 204 - else: - return 404 diff --git a/src/bos/server/dbclient.py b/src/bos/server/dbclient.py deleted file mode 100644 index 982fcd84..00000000 --- a/src/bos/server/dbclient.py +++ /dev/null @@ -1,121 +0,0 @@ -# -# MIT License -# -# (C) Copyright 2019-2023 Hewlett Packard Enterprise Development LP -# -# Permission is hereby granted, free of charge, to any person obtaining a -# copy of this software and associated documentation files (the "Software"), -# to deal in the Software without restriction, including without limitation -# the rights to use, copy, modify, merge, publish, distribute, sublicense, -# and/or sell copies of the Software, and to permit persons to whom the -# Software is furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included -# in all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL -# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR -# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, -# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR -# OTHER DEALINGS IN THE SOFTWARE. -# -import logging -import os -from time import sleep -from etcd3.client import Etcd3Client -from etcd3.exceptions import ConnectionFailedError, ConnectionTimeoutError - -LOGGER = logging.getLogger(__name__) -DB_HOST = os.getenv('ETCD_HOST', 'cray-bos-bitnami-etcd') -DB_PORT = int(os.getenv('ETCD_PORT', 2379)) - - -class BosEtcdClient(Etcd3Client): - """ - A BOS-specific client to its underlying etcd3 database. This class - extends the opensource Etcd3Client implementation by making it resilient to - initial database availability, as well, some nature of static read/get operations. - - Please note: Even though this extends the Etcd3Client implementation, the underlying - watch and lock implementation pieces that are provided by the underlying etcd client - code are not expected to be resilient to connection failures. We do not use these - in the BOS API, so it isn't an issue. - """ - - def __init__(self, host=DB_HOST, port=DB_PORT, timeout=2000): - self.host = host - self.port = port - self.timeout = timeout - self.initialize_connection() - - def initialize_connection(self): - """ - Creates a connection to the etcd cluster in question; blocks until - completion. - - This function may need to be called multiple times if the connection - is ever lost (like the etcd cluster being taken down temporarily). - """ - while True: - try: - super().__init__(self.host, self.port, timeout=self.timeout) - return - except Exception as e: - # CASMCMS-5513: The method above does not throw any - # exception if the k8s host (service object) is missing. - # The generic exception will catch anything thrown - # and the repr will ensure the exception type is printed - # in the event the message string is empty. - LOGGER.warn("Unable to establish connection to %s", - self.host) - LOGGER.warn("Exception was %s; retrying...", repr(e)) - sleep(2) - - def put(self, *args, **kwargs): - while True: - try: - return super().put(*args, **kwargs) - except Exception as e: - # Methods on the super class will throw an exception if the - # connection to etcd is not valid. - # Warn if the connection has failed, print the exception type - # with message (if any) and log out the host we are trying to - # contact (this will usually be the cray-bos-etcd-client - # k8s service). - LOGGER.warn("Connect failed to %s. Caught %s", self.host, repr(e)) - self.initialize_connection() - - def delete(self, *args, **kwargs): - while True: - try: - return super().delete(*args, **kwargs) - except Exception as e: - LOGGER.warn("Connect failed to %s. Caught %s", self.host, repr(e)) - self.initialize_connection() - - def delete_prefix(self, *args, **kwargs): - while True: - try: - return super().delete_prefix(*args, **kwargs) - except Exception as e: - LOGGER.warn("Connect failed to %s. Caught %s", self.host, repr(e)) - self.initialize_connection() - - def get(self, *args, **kwargs): - while True: - try: - return super().get(*args, **kwargs) - except Exception as e: - LOGGER.warn("Connect failed to %s. Caught %s", self.host, repr(e)) - self.initialize_connection() - - def get_prefix_response(self, *args, **kwargs): - while True: - try: - return super().get_prefix_response(*args, **kwargs) - except Exception as e: - LOGGER.warn("Connect failed to %s. Caught %s", self.host, repr(e)) - self.initialize_connection() - diff --git a/src/bos/server/migrations.py b/src/bos/server/migrations.py index 2aa978a0..73ffdfad 100644 --- a/src/bos/server/migrations.py +++ b/src/bos/server/migrations.py @@ -21,181 +21,16 @@ # ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR # OTHER DEALINGS IN THE SOFTWARE. # -import json -from kubernetes import client, config import logging -import os - -from bos.server.controllers.v1.sessiontemplate import strip_v1_only_fields -from bos.server.dbclient import BosEtcdClient -from bos.common.utils import requests_retry_session -from bos.common.tenant_utils import get_tenant_aware_key -import bos.server.redis_db_utils as dbutils - LOGGER = logging.getLogger('bos.server.migration') -BASEKEY = "/sessionTemplate" - -PROTOCOL = 'http' -SERVICE_NAME = 'cray-bos' - -def MissingName(): - """ - The session template's name is missing - """ - pass - - -def pod_ip(): - """ - Find the IP address for the pod that corresponds to the correct labels; - specifically 'cray-bos' and the version number of this version of BOS. - """ - pod_ip = None - config.load_incluster_config() - v1 = client.CoreV1Api() - # Find the correct version of the cray-bos pod - version = os.getenv('APP_VERSION') - if not version: - msg = "Could not determine application's version. Therefore could not contact the correct BOS pod. Aborting." - LOGGER.error(msg) - raise ValueError(msg) - pods = v1.list_namespaced_pod("services", - label_selector=f"app.kubernetes.io/name=cray-bos,app.kubernetes.io/version={version}") - # Get the pod's IP address - if pods and pods.items: - return pods.items[0].status.pod_ip - msg = "Could not determine BOS pod IP address. Aborting." - LOGGER.error(msg) - raise ValueError(msg) - - -def convert_v1_to_v2(v1_st): - """ - Convert a v1 session template to a v2 session template. - Prune extraneous v1 attributes. - - Input: - v1_st: A v1 session template - - Returns: - v2_st: A v2 session template - name: The name of the session template - - Raises: - MissingName: If the session template's name is missing, then raise this - exception. - """ - session_template_keys = ['name', 'description', - 'enable_cfs', 'cfs', 'boot_sets', 'links'] - boot_set_keys = ['name', 'path', 'type', 'etag', 'kernel_parameters', - 'node_list', 'node_roles_groups', 'node_groups', - 'rootfs_provider', 'rootfs_provider_passthrough'] - - v2_st = {'boot_sets': {}} - try: - name = v1_st['name'] - except KeyError: - raise MissingName() - for k, v in v1_st.items(): - if k in session_template_keys: - if k != "boot_sets" and k != "name" and k!= "links": - v2_st[k] = v - else: - LOGGER.warning("Discarding attribute: '{}' from session template: '{}'".format(k, v1_st['name'])) - - for boot_set, bs_values in v1_st['boot_sets'].items(): - v2_st['boot_sets'][boot_set] = {} - for k, v in bs_values.items(): - if k in boot_set_keys: - v2_st['boot_sets'][boot_set][k] = v - else: - LOGGER.warning("Discarding attribute: '{}' from boot set: '{}' from session template: '{}'".format(k, - boot_set, - v1_st['name'])) - return v2_st, name - - -def migrate_v1_etcd_to_v2_redis_session_templates(): - """ - Read the session templates out of the V1 etcd key/value store and - write them into the v2 Redis database. - Do not overwrite existing session templates. - Sanitize the V1 session templates so they conform to the V2 session - template standards. - """ - pod_ip_addr = pod_ip() - pod_port = os.getenv('BOS_CONTAINER_PORT') - endpoint = f"{PROTOCOL}://{pod_ip_addr}:{pod_port}" - st_v2_endpoint = f"{endpoint}/v2/sessiontemplates" - st_v1_endpoint = f"{endpoint}/v1/sessiontemplate" - session = requests_retry_session() - with BosEtcdClient() as bec: - for session_template_byte_str, _meta in bec.get_prefix('{}/'.format(BASEKEY)): - v1_st = json.loads(session_template_byte_str.decode("utf-8")) - response = session.get("{}/{}".format(st_v1_endpoint, v1_st['name'])) - if response.status_code == 200: - LOGGER.warning("Session template: '{}' already exists. Not " - "overwriting.".format(v1_st['name'])) - elif response.status_code == 404: - LOGGER.info("Migrating v1 session template: '{}' to v2 " - "database".format(v1_st['name'])) - try: - v2_st, name = convert_v1_to_v2(v1_st) - except MissingName as err: - if 'name' in v1_st: - # We should probably never get here. - LOGGER.error("Session template: '{}' was not migrated because it was missing its name.".format(v1_st['name'])) - else: - LOGGER.error("A session template: '{}' was not migrated because it was missing its name.".format(name)) - response = session.put("{}/{}".format(st_v2_endpoint, name), - json=v2_st) - if not response.ok: - LOGGER.error("Session template: '{}' was not migrated for v2 due " - "to error: {}".format(v1_st['name'], - response.reason)) - LOGGER.error("Error specifics: {}".format(response.text)) - else: - LOGGER.error("Session template: '{}' was not migrated due " - "to error: {}".format(v1_st['name'], - response.reason)) - LOGGER.error("Error specifics: {}".format(response.text)) - -# Convert existing v1 session templates to v2 format -def convert_v1_to_v2_session_templates(): - db=dbutils.get_wrapper(db='session_templates') - response = db.get_keys() - for st_key in response: - data = db.get(st_key) - if strip_v1_only_fields(data): - name = data.get("name") - LOGGER.info(f"Converting {name} to BOS v2") - db.put(st_key, data) - -# Multi-tenancy key migrations - -def migrate_database(db): - response = db.get_keys() - for old_key in response: - data = db.get(old_key) - name = data.get("name") - tenant = data.get("tenant") - new_key = get_tenant_aware_key(name, tenant).encode() - if new_key != old_key: - LOGGER.info(f"Migrating {name} to new database key structure") - db.put(new_key, data) - db.delete(old_key) - - -def migrate_to_tenant_aware_keys(): - migrate_database(dbutils.get_wrapper(db='session_templates')) - migrate_database(dbutils.get_wrapper(db='sessions')) def perform_migrations(): - migrate_v1_etcd_to_v2_redis_session_templates() - migrate_to_tenant_aware_keys() - convert_v1_to_v2_session_templates() + # Not removing this file entirely because we are going to be adding + # code here to migrate the previous BOS data to enforce API field + # restrictions + pass if __name__ == "__main__": diff --git a/src/bos/server/specialized_encoder.py b/src/bos/server/specialized_encoder.py deleted file mode 100644 index dd8e1fb2..00000000 --- a/src/bos/server/specialized_encoder.py +++ /dev/null @@ -1,62 +0,0 @@ -# -# MIT License -# -# (C) Copyright 2020-2022 Hewlett Packard Enterprise Development LP -# -# Permission is hereby granted, free of charge, to any person obtaining a -# copy of this software and associated documentation files (the "Software"), -# to deal in the Software without restriction, including without limitation -# the rights to use, copy, modify, merge, publish, distribute, sublicense, -# and/or sell copies of the Software, and to permit persons to whom the -# Software is furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included -# in all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL -# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR -# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, -# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR -# OTHER DEALINGS IN THE SOFTWARE. -# -import six - -from bos.server.models.base_model_ import Model -from bos.server.models.v1_generic_metadata import V1GenericMetadata as GenericMetadata -from bos.server.encoder import JSONEncoder - - -class MetadataEncoder(JSONEncoder): - include_nulls = False - - def converter_metadata(self, metadata): - dikt = {} - if hasattr(metadata, 'start_time'): - dikt['start_time'] = metadata.start_time - if hasattr(metadata, 'stop_time'): - dikt['stop_time'] = metadata.stop_time - if hasattr(metadata, 'complete'): - dikt['complete'] = metadata.complete - if hasattr(metadata, 'in_progress'): - dikt['in_progress'] = metadata.in_progress - if hasattr(metadata, 'error_count'): - dikt['error_count'] = metadata.error_count - if hasattr(metadata, 'errors'): - dikt['errors'] = metadata.errors - return GenericMetadata(**dikt) - - def default(self, o): - """ - This replaces the GenericMetadata class that is not JSON-serializable - with a dictionary that is. It takes properties and turns them - into values in the dictionary. - """ - if isinstance(o, Model): - for attr, _type in six.iteritems(o.openapi_types): - if _type == GenericMetadata: - value = getattr(o, attr) - attribute = o.attribute_map[attr] - setattr(o, attribute, self.converter_metadata(value)) - return super().default(o) diff --git a/src/bos/server/test/__init__.py b/src/bos/server/test/__init__.py deleted file mode 100644 index 926b14ad..00000000 --- a/src/bos/server/test/__init__.py +++ /dev/null @@ -1,39 +0,0 @@ -# -# MIT License -# -# (C) Copyright 2021-2022 Hewlett Packard Enterprise Development LP -# -# Permission is hereby granted, free of charge, to any person obtaining a -# copy of this software and associated documentation files (the "Software"), -# to deal in the Software without restriction, including without limitation -# the rights to use, copy, modify, merge, publish, distribute, sublicense, -# and/or sell copies of the Software, and to permit persons to whom the -# Software is furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included -# in all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL -# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR -# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, -# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR -# OTHER DEALINGS IN THE SOFTWARE. -# -import logging - -import connexion -from flask_testing import TestCase - -from bos.server.encoder import JSONEncoder - - -class BaseTestCase(TestCase): - - def create_app(self): - logging.getLogger('connexion.operation').setLevel('ERROR') - app = connexion.App(__name__, specification_dir='../openapi/') - app.app.json_encoder = JSONEncoder - app.add_api('openapi.yaml', pythonic_params=True) - return app.app diff --git a/src/bos/server/test/test_healthz_controller.py b/src/bos/server/test/test_healthz_controller.py deleted file mode 100755 index 58ba97fd..00000000 --- a/src/bos/server/test/test_healthz_controller.py +++ /dev/null @@ -1,62 +0,0 @@ -# -# MIT License -# -# (C) Copyright 2019, 2021-2022 Hewlett Packard Enterprise Development LP -# -# Permission is hereby granted, free of charge, to any person obtaining a -# copy of this software and associated documentation files (the "Software"), -# to deal in the Software without restriction, including without limitation -# the rights to use, copy, modify, merge, publish, distribute, sublicense, -# and/or sell copies of the Software, and to permit persons to whom the -# Software is furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included -# in all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL -# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR -# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, -# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR -# OTHER DEALINGS IN THE SOFTWARE. -# -# coding: utf-8 - -from __future__ import absolute_import -import unittest -from unittest import mock - -from bos.server.models.healthz import Healthz # noqa: E501 -from bos.server.models.problem_details import ProblemDetails # noqa: E501 -from bos.server.controllers.v1.healthz import v1_get_healthz - - -class TestHealthzController(unittest.TestCase): - """HealthzController integration test stubs""" - - def test_v1_get_healthz(self): - """Test case for v1_get_healthz - - Get API health - """ - with mock.patch('bos.server.controllers.v1.healthz.BosEtcdClient') as mocked_class: - mocked_class.return_value.__enter__.return_value.get.return_value = ('ok'.encode('utf-8'), 'metadata') # noqa: E501 - healthz, response_code = v1_get_healthz() - self.assertEqual(response_code, 200, "Response code tests as healthy") - self.assertIsInstance(healthz, Healthz, "Proper response verified") - - def test_v1_get_healthz_unhealthy(self): - """Test case for v1_get_healthz - - Get API health - """ - with mock.patch('bos.server.controllers.v1.healthz.BosEtcdClient') as mocked_class: - mocked_class.return_value.__enter__.return_value.get.return_value = ('so_sick'.encode('utf-8'), 'metadata') # noqa: E501 - healthz, response_code = v1_get_healthz() - self.assertEqual(response_code, 503, "Response code tests as unhealthy") - self.assertIsInstance(healthz, Healthz, "Proper response verified") - - -if __name__ == '__main__': - unittest.main() diff --git a/src/bos/server/test/test_session_controller.py b/src/bos/server/test/test_session_controller.py deleted file mode 100644 index 621332cb..00000000 --- a/src/bos/server/test/test_session_controller.py +++ /dev/null @@ -1,129 +0,0 @@ -# -# MIT License -# -# (C) Copyright 2019, 2021-2022 Hewlett Packard Enterprise Development LP -# -# Permission is hereby granted, free of charge, to any person obtaining a -# copy of this software and associated documentation files (the "Software"), -# to deal in the Software without restriction, including without limitation -# the rights to use, copy, modify, merge, publish, distribute, sublicense, -# and/or sell copies of the Software, and to permit persons to whom the -# Software is furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included -# in all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL -# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR -# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, -# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR -# OTHER DEALINGS IN THE SOFTWARE. -# -# coding: utf-8 - -from __future__ import absolute_import -from bos.server.controllers.v1.session import get_v1_session, get_v1_sessions, delete_v1_session -from bos.server.controllers.v1.session import _get_boa_naming_for_session - -from unittest import mock -import testtools -import unittest - - -class TestSessionController(testtools.TestCase): - """SessionController integration test stubs""" - - def test_get_v1_session(self): - with mock.patch('bos.server.controllers.v1.session.BosEtcdClient') as mocked_class: - session_name = 'foo' - db_response = [(session_name.encode('utf-8'), mock.MagicMock())] - db_response[0][1].key.return_value = '//%s' % (session_name).encode('utf-8') - mocked_class.return_value.__enter__.return_value.get_prefix.return_value = db_response - result, status = get_v1_session(session_name) - - # Check the expected result and status. - self.assertEqual(db_response[0][0].decode('utf-8'), list(result.values())[0]) - self.assertEqual(200, status) - self.assertTrue(list(result.values())[0] == 'foo', "Results is the name of the session created.") - - def test_get_v1_session_not_found(self): - """Test case for get_v1_session - - Check for the expected handling when the session is not found. - """ - with mock.patch('bos.server.controllers.v1.session.BosEtcdClient') as mocked_class: - session_name = 'foo' - db_response = [] - mocked_class.return_value.__enter__.return_value.get_prefix.return_value = db_response - result, status = get_v1_session(session_name) - - # Check the expected result and status. - self.assertEqual(404, status) - self.assertEqual(result, {}, "Nothing returned when not found.") - - def test_get_v1_sessions(self): - """Test case for get_v1_sessions - - List Sessions - """ - with mock.patch('bos.server.controllers.v1.session.BosEtcdClient') as mocked_class: - db_responses = [] - for session_name in 'abcdefg': - db_responses.append((session_name.encode('utf-8'), mock.MagicMock())) - db_responses[-1][1].key.return_value = '//%s' % (session_name).encode('utf-8') - mocked_class.return_value.__enter__.return_value.get_prefix.return_value = db_responses - result, status = get_v1_sessions() - self.assertEqual(len(result), len(db_responses)) - self.assertEqual(200, status) - mocked_class.return_value.__enter__.return_value.get_prefix.assert_called_once() - - def test_get_v1_sessions_none(self): - """Test case for get_v1_sessions - - List Sessions - """ - with mock.patch('bos.server.controllers.v1.session.BosEtcdClient') as mocked_class: - db_responses = [] - for session_name in '': - db_responses.append((session_name.encode('utf-8'), mock.MagicMock())) - db_responses[-1][1].key.return_value = '//%s' % (session_name).encode('utf-8') - mocked_class.return_value.__enter__.return_value.get_prefix.return_value = db_responses - result, status = get_v1_sessions() - self.assertEqual(len(result), len(db_responses)) - self.assertEqual(200, status) - mocked_class.return_value.__enter__.return_value.get_prefix.assert_called_once() - - def test_delete_v1_session(self): - """Test case for delete_v1_sessiontemplate - - Delete Session - """ - with mock.patch('bos.server.controllers.v1.session.BosEtcdClient') as mocked_class: - mocked_class.return_value.__enter__.return_value.delete_prefix.return_value.deleted = 1 - result, status = delete_v1_session('test') - self.assertEqual('', result) - self.assertEqual(status, 204) - - def test_delete_v1_session_not_found(self): - """Test case for delete_v1_sessiontemplate - - Delete a Session that does not exist - """ - with mock.patch('bos.server.controllers.v1.session.BosEtcdClient') as mocked_class: - mocked_class.return_value.__enter__.return_value.delete_prefix.return_value.deleted = 0 - result, status = delete_v1_session('test') - self.assertEqual(status, 404) - self.assertTrue('not found' in result) - - def test_boa_naming(self): - # Check length and prefix on the session ID and BOA job name. - session_id, boa_name = _get_boa_naming_for_session() - self.assertTrue(len(session_id) <= 63) - self.assertTrue(boa_name.startswith('boa-')) - self.assertTrue(len(boa_name) <= 63) - - -if __name__ == '__main__': - unittest.main() diff --git a/src/bos/server/test/test_sessiontemplate_controller.py b/src/bos/server/test/test_sessiontemplate_controller.py deleted file mode 100644 index 53a80a15..00000000 --- a/src/bos/server/test/test_sessiontemplate_controller.py +++ /dev/null @@ -1,196 +0,0 @@ -# -# MIT License -# -# (C) Copyright 2019-2022 Hewlett Packard Enterprise Development LP -# -# Permission is hereby granted, free of charge, to any person obtaining a -# copy of this software and associated documentation files (the "Software"), -# to deal in the Software without restriction, including without limitation -# the rights to use, copy, modify, merge, publish, distribute, sublicense, -# and/or sell copies of the Software, and to permit persons to whom the -# Software is furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included -# in all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL -# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR -# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, -# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR -# OTHER DEALINGS IN THE SOFTWARE. -# -# coding: utf-8 - -from __future__ import absolute_import - -from bos.server.controllers.v1.sessiontemplate import create_v1_sessiontemplate -from bos.server.controllers.v1.sessiontemplate import get_v1_sessiontemplate -from bos.server.controllers.v1.sessiontemplate import get_v1_sessiontemplates -from bos.server.controllers.v1.sessiontemplate import get_v1_sessiontemplatetemplate -from bos.server.controllers.v1.sessiontemplate import delete_v1_sessiontemplate - -import connexion -import testtools -from unittest import mock -from connexion import problem -from connexion.lifecycle import ConnexionResponse - - -class TestSessiontemplateController(testtools.TestCase): - """SessiontemplateController unit tests""" - def test_create_v1_sessiontemplate(self): - """Test case for create_v1_sessiontemplate - - Create a Session Template - """ - with mock.patch('bos.server.controllers.v1.sessiontemplate.BosEtcdClient') as mocked_class: - # with Flask(__name__).test_request_context(): - # Generate mocked request data that would normally come from - # the http request. - sample_template_create_req_data = { - 'name': 'foo', - } - with connexion.FlaskApp(__name__).app.test_request_context(json=sample_template_create_req_data): - response, status_code = create_v1_sessiontemplate() - mocked_class.return_value.__enter__.return_value.put.assert_called_once() - self.assertEqual(status_code, 201) - self.assertEqual(response.split('/')[2], sample_template_create_req_data['name']) - - def test_create_v1_sessiontemplate_with_data(self): - """Test case for create_v1_sessiontemplate - - Create a Session Template with actual data - Verify that the data returned is accurate - """ - with mock.patch('bos.server.controllers.v1.sessiontemplate.BosEtcdClient') as mocked_class: - # Generate mocked request data that would normally come from - # the http request. - sample_template_create_req_data = { - "name": "bos-test", - "boot_sets": { - "computes": { - "boot_ordinal": 1, - "ims_image_id": "88769f0a-f8fa-4253-846a-773c07e7e51f", - "kernel_parameters": "console=tty0 console=ttyS0,115200n8 selinux=0 rd.shell " - "rd.net.timeout.carrier=40 rd.retry=40 ip=dhcp rd.neednet=1 " - "crashkernel=256M " - "htburl=https://10.2.100.50/apis/hbtd/hmi/v1/heartbeat k8s_gw=10.2.100.50", # noqa: 501 - "node_list": [ - "x0c0s0b0n0" - ], - "rootfs_provider": "ars", - "rootfs_provider_passthrough": "" - } - }, - "cfs_branch": "master", - "cfs_url": "https://api-gw-service-nmn.local/vcs/cray/config-management.git", - "description": "Template for booting compute nodes, generated by the installation", - "enable_cfs": True, - } - with connexion.FlaskApp(__name__).app.test_request_context(json=sample_template_create_req_data): - # Call the controller method simulating a request - response, status_code = create_v1_sessiontemplate() - - mocked_class.return_value.__enter__.return_value.put.assert_called_once() - self.assertEqual(status_code, 201) - self.assertEqual(response.split('/')[2], sample_template_create_req_data['name']) - - def test_get_v1_sessiontemplate_exists(self): - """Test case for get_v1_sessiontemplate - - Get details for an existing session template - """ - with mock.patch('bos.server.controllers.v1.sessiontemplate.BosEtcdClient') as mocked_class: - db_response = '{"name": "test"}'.encode('utf-8'), None - mocked_class.return_value.__enter__.return_value.get.return_value = db_response - result, status = get_v1_sessiontemplate('test') - self.assertEqual(200, status) - self.assertEqual(result, {'name': 'test'}) - - def test_get_v1_sessiontemplate_not_found(self): - """Test case for get_v1_sessiontemplate - - Attempt to get details for a session template that does not exist - """ - with mock.patch('bos.server.controllers.v1.sessiontemplate.BosEtcdClient') as mocked_class: - db_response = ''.encode('utf-8'), None - mocked_class.return_value.__enter__.return_value.get.return_value = db_response - result = get_v1_sessiontemplate('test') - self.assertIsInstance(result, ConnexionResponse, "Must return a 404 response.") - self.assertEqual(result.status_code, 404) - - def test_get_v1_sessiontemplates(self): - """Test case for get_v1_sessiontemplates - - List Session Templates - """ - with mock.patch('bos.server.controllers.v1.sessiontemplate.BosEtcdClient') as mocked_class: - db_response = [] - for st in 'abcd': - json_string = '{"name": "st_%s", "boot_sets": {"set1": []}}' % (st) - db_response.append((json_string.encode('utf-8'), - 'bogus_meta')) - mocked_class.return_value.__enter__.return_value.get_prefix.return_value = db_response - result, status = get_v1_sessiontemplates() - self.assertEqual(200, status) - self.assertEqual(len(result), 4) - - def test_get_v1_sessiontemplates_none_found(self): - """Test case for get_v1_sessiontemplates - - List Session Templates - """ - with mock.patch('bos.server.controllers.v1.sessiontemplate.BosEtcdClient') as mocked_class: - db_response = [] - for st in '': - json_string = '{"name": "st_%s", "boot_sets": {"set1": []}}' % (st) - db_response.append((json_string.encode('utf-8'), - 'bogus_meta')) - mocked_class.return_value.__enter__.return_value.get_prefix.return_value = db_response - result, status = get_v1_sessiontemplates() - self.assertEqual(200, status) - self.assertEqual(len(result), 0) - - def test_delete_v1_sessiontemplate(self): - """Test case for delete_v1_sessiontemplate - - Delete Session Template - """ - template_to_delete = 'foo' - with mock.patch('bos.server.controllers.v1.sessiontemplate.get_v1_sessiontemplate') as mocked_st_func: - mocked_st_func.return_value = ({"name": template_to_delete, "boot_sets": {"set1": []}}, 200) - with mock.patch('bos.server.controllers.v1.sessiontemplate.BosEtcdClient') as mocked_class: - mocked_class.return_value.__enter__.return_value.delete.return_value = None - _, status = delete_v1_sessiontemplate(template_to_delete) - mocked_class.return_value.__enter__.return_value.delete.assert_called_once() - self.assertEqual(status, 204) - - def test_delete_v1_sessiontemplate_not_found(self): - """Test case for delete_v1_sessiontemplate - - Delete Session Template - """ - template_to_delete = 'foo' - with mock.patch('bos.server.controllers.v1.sessiontemplate.get_v1_sessiontemplate') as mocked_st_func: - mocked_st_func.return_value = problem(404, 'oops', 'more oops') - with mock.patch('bos.server.controllers.v1.sessiontemplate.BosEtcdClient') as mocked_class: - status = delete_v1_sessiontemplate(template_to_delete) - mocked_class.return_value.__enter__.return_value.delete.assert_not_called() - self.assertEqual(status.status_code, 404) - mocked_st_func.assert_called_once() - - def test_get_v1_sessiontemplatetemplate(self): - """Test case for get_v1_sessiontemplatetemplate - - Get the example Session Template - """ - result, status = get_v1_sessiontemplatetemplate() - self.assertEqual(200, status) - self.assertEqual(result["name"], 'name-your-template') - - -if __name__ == '__main__': - import unittest - unittest.main() diff --git a/src/bos/server/test/test_status_controller.py b/src/bos/server/test/test_status_controller.py deleted file mode 100755 index 2938a849..00000000 --- a/src/bos/server/test/test_status_controller.py +++ /dev/null @@ -1,650 +0,0 @@ -#!/usr/bin/env python3 -# -# MIT License -# -# (C) Copyright 2019-2022 Hewlett Packard Enterprise Development LP -# -# Permission is hereby granted, free of charge, to any person obtaining a -# copy of this software and associated documentation files (the "Software"), -# to deal in the Software without restriction, including without limitation -# the rights to use, copy, modify, merge, publish, distribute, sublicense, -# and/or sell copies of the Software, and to permit persons to whom the -# Software is furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included -# in all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL -# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR -# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, -# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR -# OTHER DEALINGS IN THE SOFTWARE. -# -import copy -import datetime -import logging -import os -import pytest -import random -import types -import unittest - -import connexion -import flask -from flask import url_for -from builtins import staticmethod - -os.environ["ETCD_HOST"] = 'localhost' -os.environ["ETCD_PORT"] = '2379' - -from bos.server.controllers.v1.status import InvalidCategory, BadPhase, BootSetDoesNotExist # noqa: E402 -from bos.server.controllers.v1.status import SessionStatus, BootSet, Metadata, get_v1_session_status # noqa: E402 -from bos.server.controllers.v1.status import update_v1_session_status_by_bootset # noqa: E402 -from bos.server.controllers.v1.status import create_v1_boot_set_status # noqa: E402 -from bos.server.controllers.v1.status import get_v1_session_status_by_bootset # noqa: E402 -from bos.server.controllers.v1.status import get_v1_session_status_by_bootset_and_phase_and_category # noqa: E402 -from bos.server.controllers.v1.status import MetadataPhase, get_v1_session_status_by_bootset_and_phase # noqa: E402 -from bos.server.controllers.v1.status import create_v1_session_status, update_v1_session_status # noqa: E402 -from bos.server.dbclient import BosEtcdClient # noqa: E402 -from bos.server.models.generic_metadata import GenericMetadata # noqa: E402 - -LOGGER = logging.getLogger(__name__) - - -class TestBosEtcdClient(): - - def testPut(self): - s = BosEtcdClient() - s.put('key', 'value') - value, metadata = s.get('key') - assert value.decode('utf-8') == 'value' - print("Metadata: %s", metadata) - - -def _compare_phasecategory(cat1, cat2): - assert cat1.name == cat2.name - assert cat1.nodes == cat2.nodes - - -def _compare_objects(obj1, obj2, attribute_list): - """ - Compare the specified attributes of two objects - """ - if not isinstance(attribute_list, list): - raise ValueError("attribute_list is not a list.") - for attribute in attribute_list: - assert getattr(obj1, attribute) == getattr(obj2, attribute) - - -def _compare_phases(phase1, phase2): - """ - Compare two phases, except for metadata - """ - for attrib in ['name', 'errors', 'categories']: - assert getattr(phase1, attrib) == getattr(phase2, attrib) - - -def _compare_categories(cat1, cat2): - """ - Compare two categories, except for metadata - """ - for attrib in ['name', 'node_list']: - assert getattr(cat1, attrib) == getattr(cat2, attrib) - - -def _compare_bootset(bootset1, bootset2): - for attribute in ['name', 'links', 'session']: - assert getattr(bootset1, attribute) == getattr(bootset2, attribute) - for i, phase1 in enumerate(bootset1.phases): - phase_comp = bootset2.get_phase(bootset1.phases[i].name) - _compare_phases(phase1, phase_comp) -# for attrib in ['name', 'errors', 'categories']: -# assert getattr(bootset1.phases[i], attrib) == getattr(bootset2.phases[j], attrib) - - -def _convert_metadata(metadata): - dikt = {} - if hasattr(metadata, 'start_time'): - dikt['start_time'] = metadata.start_time - if hasattr(metadata, 'stop_time'): - dikt['stop_time'] = metadata.stop_time - if hasattr(metadata, 'complete'): - dikt['complete'] = metadata.complete - if hasattr(metadata, 'in_progress'): - dikt['in_progress'] = metadata.in_progress - if hasattr(metadata, 'error_count'): - dikt['error_count'] = metadata.error_count - # return GenericMetadata(**dikt) - return dikt - - -example_boot_set = {'name': 'test-boot-set', - 'metadata': None, - 'phases': [{'name': 'shutdown', - 'metadata': {'complete': None, - 'error_count': None, - 'in_progress': None, - 'start_time': None, - 'stop_time': None}, - 'categories': [{'name': 'not_started', - 'node_list': ['x3000c0s19b1n0', - 'x3000c0s19b2n0', - 'x3000c0s19b3n0']}, - {'name': 'in_progress', - 'node_list': []}, - {'name': 'succeeded', - 'node_list': []}, - {'name': 'failed', - 'node_list': []}, - {'name': 'excluded', - 'node_list': []} - ], - 'errors': {'Sloth': ['x3000c0s19b1n0'], - 'Greed': ['x3000c0s19b2n0'], - 'Hubris': ['x3000c0s19b3n0']} - }, - {'name': 'boot', - 'metadata': {'complete': None, - 'error_count': None, - 'in_progress': None, - 'start_time': None, - 'stop_time': None}, - 'categories': [{'name': 'not_started', - 'node_list': ['x3000c0s19b1n0', - 'x3000c0s19b2n0', - 'x3000c0s19b3n0']}, - {'name': 'in_progress', - 'node_list': []}, - {'name': 'succeeded', - 'node_list': []}, - {'name': 'failed', - 'node_list': []}, - {'name': 'excluded', - 'node_list': []} - ], - 'errors': {'Sloth': ['x3000c0s19b1n0'], - 'Greed': ['x3000c0s19b2n0'], - 'Hubris': ['x3000c0s19b3n0']} - - }, - {'name': 'configure', - 'metadata': {'complete': None, - 'error_count': None, - 'in_progress': None, - 'start_time': None, - 'stop_time': None}, - 'categories': [{'name': 'not_started', - 'node_list': ['x3000c0s19b1n0', - 'x3000c0s19b2n0', - 'x3000c0s19b3n0']}, - {'name': 'in_progress', - 'node_list': []}, - {'name': 'succeeded', - 'node_list': []}, - {'name': 'failed', - 'node_list': []}, - {'name': 'excluded', - 'node_list': []} - ], - 'errors': {'Sloth': ['x3000c0s19b1n0'], - 'Greed': ['x3000c0s19b2n0'], - 'Hubris': ['x3000c0s19b3n0']} - } - ] - } - - -class MockResponseBootSetCreation(): - - @staticmethod - def get_json(): - return copy.deepcopy(example_boot_set) - - -class TestBootSet(object): - - valid_categories = ['excluded', 'failed', 'succeeded', 'not_started', 'in_progress'] - - @pytest.mark.parametrize("nodes", [["x3000c0s19b1n0"], - ["x3000c0s19b1n0", "x3000c0s19b2n0", - "x3000c0s19b3n0"]]) - @pytest.mark.parametrize("phases", [['shutdown', 'boot', 'configure'], - ['shutdown', 'boot'], - ['boot', 'configure'], - ['configure'], - ['boot'], - ['shutdown']]) - def testSaveLoadBootSet(self, phases, nodes): - session_id = "session-123" - boot_set_name = "boot-set1" - ebs = copy.deepcopy(example_boot_set) - ebs['session'] = session_id - ebs['name'] = boot_set_name - bs = BootSet.from_dict(ebs) - bs.save() - bs_comp = bs.load(session_id, boot_set_name) - assert bs == bs_comp - BootSet.delete(session_id, boot_set_name) - - def testLoadNoExistentBootSet(self): - session = "session-invalid" - bootset = "boot-set-invalid" - with pytest.raises(BootSetDoesNotExist): - BootSet.load(session, bootset) - - @pytest.mark.parametrize("nodes", [["x3000c0s19b1n0"], - ["x3000c0s19b1n0", "x3000c0s19b2n0", - "x3000c0s19b3n0"]]) - @pytest.mark.parametrize("phases", [['shutdown', 'boot', 'configure'], - ['shutdown', 'boot'], - ['boot', 'configure'], - ['configure'], - ['boot'], - ['shutdown']]) - @pytest.mark.parametrize("source_category", valid_categories) - @pytest.mark.parametrize("destination_category", valid_categories) - def testUpdateBootSet_nodes(self, phases, nodes, - source_category, - destination_category): - session_id = "session-123" - boot_set_name = "boot-set1" - ebs = copy.deepcopy(example_boot_set) - ebs['session'] = session_id - ebs['name'] = boot_set_name - bs = BootSet.from_dict(ebs) - bs_comp = BootSet.from_dict(ebs) - assert bs == bs_comp - moving_nodes = set(random.choices(nodes, k=random.randint(1, max([1, len(nodes) - 1])))) - update_phase = random.choices(phases, k=1)[0] - bs.update_nodes(update_phase, source_category, destination_category, moving_nodes) - # bs_comp.phases[update_phase].categories[source_category].nodes -= set(moving_nodes) - if source_category != destination_category: - source_nodes = set(bs_comp.get_category(update_phase, source_category).node_list) - source_nodes -= moving_nodes - destination_nodes = set(bs_comp.get_category(update_phase, destination_category).node_list) - destination_nodes |= moving_nodes - bs_comp.set_category_nodes(update_phase, source_category, list(source_nodes)) - bs_comp.set_category_nodes(update_phase, destination_category, list(destination_nodes)) - assert bs == bs_comp - BootSet.delete(session_id, boot_set_name) - # _compare_bootset(bs, bs_comp) - - @pytest.mark.parametrize("nodes", [["x3000c0s19b1n0"], - ["x3000c0s19b1n0", "x3000c0s19b2n0", - "x3000c0s19b3n0"]]) - @pytest.mark.parametrize("phases", [['shutdown', 'boot', 'configure'], - ['shutdown', 'boot'], - ['boot', 'configure'], - ['configure'], - ['boot'], - ['shutdown']]) - def testUpdateBootSet_errors(self, nodes, phases): - session_id = "session-123" - boot_set_name = "boot-set1" - ebs = copy.deepcopy(example_boot_set) - ebs['session'] = session_id - ebs['name'] = boot_set_name - bs = BootSet.from_dict(ebs) - errors = {'Sucked': nodes, - 'Dumb': nodes, - 'Too beautiful to live': nodes} - for phase in phases: - bs.update_errors(phase, errors) - for phase in phases: - p = bs.get_phase(phase) - for k, v in errors.items(): - assert v == p.errors[k] - - @pytest.mark.parametrize("phases", [['shutdown', 'boot', 'configure'], - ['shutdown', 'boot'], - ['boot', 'configure'], - ['configure'], - ['boot'], - ['shutdown']]) - def testUpdateBootSet_metadata(self, phases): - session_id = "session-123" - boot_set_name = "boot-set1" - ebs = copy.deepcopy(example_boot_set) - ebs['session'] = session_id - ebs['name'] = boot_set_name - bs = BootSet.from_dict(ebs) - start = datetime.datetime(1976, 9, 12, 16, 30, 0, 79043) - stop = datetime.datetime(1977, 12, 15, 18, 30, 0, 79043) - for phase in phases: - bs.update_metadata_phase(phase, start, stop) - for phase in phases: - p = bs.get_phase(phase) - assert start == p.metadata.start_time - assert stop == p.metadata.stop_time - - -def _compare_session(session1, session2): - assert session1.id == session2.id - assert session1.boot_sets == session2.boot_sets - - -class MockResponse(): - - @staticmethod - def get_json(): - return copy.deepcopy(example_boot_set) - - -class MockResponse1(): - - @staticmethod - def get_json(): - resp = [{'update_type': 'NodeChangeList', - 'phase': 'shutdown', - 'data': {'phase': 'shutdown', - 'source': 'not_started', - 'destination': 'succeeded', - 'node_list': ["x3000c0s19b1n0", "x3000c0s19b2n0", "x3000c0s19b3n0"] - } - } - ] - return resp - - -class MockResponse2(): - - @staticmethod - def get_json(): - return {'boot_sets': ['boot_set1', 'boot_set2', 'boot_set3']} - - -class MockResponse3(): - - @staticmethod - def get_json(): - resp = [{'update_type': 'NodeErrorsList', - 'phase': 'shutdown', - 'data': {'Dumb': ["x3000c0s19b1n0", "x3000c0s19b2n0"], - 'Too beautiful to live': ["x3000c0s19b3n0"] - } - } - ] - return resp - - -class MockResponse4(): - - @staticmethod - def get_json(): - resp = [ - {'update_type': 'GenericMetadata', - 'phase': 'shutdown', - 'data': { - "start_time": "2020-04-24T12:00", - "stop_time": "2020-04-24T12:00"} - } - ] - return resp - - -class MockResponse5(): - - @staticmethod - def get_json(): - resp = {'start_time': '1:00', - 'stop_time': '2:00'} - return resp - - -class TestAPIEndpoints(object): - - valid_categories = ['excluded', 'failed', 'succeeded', 'not_started', 'in_progress'] - - def mockresponse(self, *args, **kwargs): - session_id = "1234" - response = [ - { - "href": "/v1/session/{}".format(session_id), - "rel": "self" - }, - { - "href": "/v1/session/{}/status/boot_set1".format(session_id), - "rel": "Boot Set" - }, - { - "href": "/v1/session/{}/status/boot_set2".format(session_id), - "rel": "Boot Set" - }, - { - "href": "/v1/session/{}/status/boot_set3".format(session_id), - "rel": "Boot Set" - } - ] - return response - - def testCreateV1BootSetStatus(self, monkeypatch): - session_id = "session-123" - # import pdb;pdb.set_trace() - # monkeypatch.setattr("flask", "url_for", mockresponse(session_id)) - # monkeypatch.setattr("url_for", "__call__", mockresponse(session_id)) - # monkeypatch.setattr("", "url_for", mockresponse(session_id)) - # monkeypatch.setattr(".", "url_for", mockresponse(session_id)) - # import pdb;pdb.set_trace() - - monkeypatch.setattr("flask.helpers.url_for", self.mockresponse) - monkeypatch.setattr(connexion, "request", MockResponseBootSetCreation) - boot_set_name = "test-boot-set" - BootSet.delete(session_id, boot_set_name) - bs, _status = create_v1_boot_set_status(session_id, boot_set_name) - ebs = copy.deepcopy(example_boot_set) - ebs['session'] = session_id - ebs['name'] = boot_set_name - bs_comp = BootSet.from_dict(ebs) - bs_comp.initialize() - bs_comp.metadata.start_time = bs.metadata.start_time - _compare_bootset(bs, bs_comp) - BootSet.delete(session_id, boot_set_name) - - def testUpdateV1SessionStatusStatusByBootSetNodeChangeList(self, monkeypatch): - - monkeypatch.setattr("flask.helpers.url_for", self.mockresponse) - monkeypatch.setattr(connexion, "request", MockResponseBootSetCreation) - session_id = "session-123" - boot_set_name = "test-boot-set" - create_v1_boot_set_status(session_id, boot_set_name) - - monkeypatch.setattr(connexion, "request", MockResponse1) - update_v1_session_status_by_bootset(session_id, boot_set_name) - - bs = BootSet.load(session_id, boot_set_name) - # TODO Create a Boot Set to compare against the updated Boot Set - nodes = ["x3000c0s19b1n0", "x3000c0s19b2n0", "x3000c0s19b3n0"] - ebs = copy.deepcopy(example_boot_set) - ebs['session'] = session_id - ebs['name'] = boot_set_name - bs_comp = BootSet.from_dict(ebs) - bs_comp.initialize() - bs_comp.update_nodes('shutdown', 'not_started', 'succeeded', nodes) - _compare_bootset(bs, bs_comp) - BootSet.delete(session_id, boot_set_name) - - def testUpdateV1SessionStatusStatusByBootSetNodeErrorsList(self, monkeypatch): - - monkeypatch.setattr("flask.helpers.url_for", self.mockresponse) - monkeypatch.setattr(connexion, "request", MockResponseBootSetCreation) - session_id = "session-123" - boot_set_name = "test-boot-set" - create_v1_boot_set_status(session_id, boot_set_name) - - monkeypatch.setattr(connexion, "request", MockResponse3) - update_v1_session_status_by_bootset(session_id, boot_set_name) - - bs = BootSet.load(session_id, boot_set_name) - # TODO Create a Boot Set to compare against the updated Boot Set - ebs = copy.deepcopy(example_boot_set) - ebs['session'] = session_id - ebs['name'] = boot_set_name - bs_comp = BootSet.from_dict(ebs) - bs_comp.initialize() - bs_comp.update_errors('shutdown', {'Dumb': ["x3000c0s19b1n0", "x3000c0s19b2n0"], - 'Too beautiful to live': ["x3000c0s19b3n0"] - }) - _compare_bootset(bs, bs_comp) - BootSet.delete(session_id, boot_set_name) - - def testUpdateV1SessionStatusStatusByBootSetGenericMetadata(self, monkeypatch): - - monkeypatch.setattr("flask.helpers.url_for", self.mockresponse) - monkeypatch.setattr(connexion, "request", MockResponseBootSetCreation) - session_id = "session-123" - boot_set_name = "test-boot-set" - create_v1_boot_set_status(session_id, boot_set_name) - - monkeypatch.setattr(connexion, "request", MockResponse4) - update_v1_session_status_by_bootset(session_id, boot_set_name) - - bs = BootSet.load(session_id, boot_set_name) - # TODO Create a Boot Set to compare against the updated Boot Set - ebs = copy.deepcopy(example_boot_set) - ebs['session'] = session_id - ebs['name'] = boot_set_name - bs_comp = BootSet.from_dict(ebs) - bs_comp.initialize() - bs_comp.update_metadata_phase('shutdown', "2020-04-24T12:00", - "2020-04-24T12:00") - _compare_bootset(bs, bs_comp) - phase = bs.get_phase('shutdown') - phase_comp = bs_comp.get_phase('shutdown') - assert phase.metadata.start_time == phase_comp.metadata.start_time - assert phase.metadata.stop_time == phase_comp.metadata.stop_time - BootSet.delete(session_id, boot_set_name) - - def testGetV1SessionStatusStatusByBootSet(self, monkeypatch): - monkeypatch.setattr("flask.helpers.url_for", self.mockresponse) - monkeypatch.setattr(connexion, "request", MockResponseBootSetCreation) - session_id = "session-123" - boot_set_name = "test-boot-set" - create_v1_boot_set_status(session_id, boot_set_name) - monkeypatch.setattr(connexion, "request", MockResponse) - bs, status = get_v1_session_status_by_bootset(session_id, boot_set_name) - # TODO Find a way to pass these in as parameters - ebs = copy.deepcopy(example_boot_set) - ebs['session'] = session_id - ebs['name'] = boot_set_name - bs_comp = BootSet.from_dict(ebs) - bs_comp.initialize() - bs_comp.metadata.start_time = bs.metadata.start_time - _compare_bootset(bs, bs_comp) - assert status == 200 - BootSet.delete(session_id, boot_set_name) - - def testGetV1SessionStatusStatusByBootSetAndPhase(self, monkeypatch): - monkeypatch.setattr("flask.helpers.url_for", self.mockresponse) - monkeypatch.setattr(connexion, "request", MockResponseBootSetCreation) - session_id = "session-123" - boot_set_name = "test-boot-set" - create_v1_boot_set_status(session_id, boot_set_name) - monkeypatch.setattr(connexion, "request", MockResponse) - bs, _status = get_v1_session_status_by_bootset(session_id, boot_set_name) - ebs = copy.deepcopy(example_boot_set) - ebs['session'] = session_id - ebs['name'] = boot_set_name - bs_comp = BootSet.from_dict(ebs) - bs_comp.initialize() - bs_comp.metadata.start_time = bs.metadata.start_time - # TODO Find a way to pass these in as parameters - phases = ['shutdown', 'boot', 'configure'] - for phase in phases: - current_phase = get_v1_session_status_by_bootset_and_phase(session_id, boot_set_name, phase) - _compare_phases(current_phase, bs_comp.get_phase(phase)) - BootSet.delete(session_id, boot_set_name) - - def testGetV1SessionStatusStatusByBootSetAndPhaseAndCategory(self, monkeypatch): - monkeypatch.setattr("flask.helpers.url_for", self.mockresponse) - monkeypatch.setattr(connexion, "request", MockResponseBootSetCreation) - session_id = "session-123" - boot_set_name = "test-boot-set" - create_v1_boot_set_status(session_id, boot_set_name) - monkeypatch.setattr(connexion, "request", MockResponse) - bs, _status = get_v1_session_status_by_bootset(session_id, boot_set_name) - ebs = copy.deepcopy(example_boot_set) - ebs['session'] = session_id - ebs['name'] = boot_set_name - bs_comp = BootSet.from_dict(ebs) - bs_comp.initialize() - bs_comp.metadata.start_time = bs.metadata.start_time - # TODO Find a way to pass these in as parameters - phases = ['shutdown', 'boot', 'configure'] - for phase in phases: - for category in self.valid_categories: - current_category = get_v1_session_status_by_bootset_and_phase_and_category(session_id, - boot_set_name, - phase, - category) - category_comp = bs_comp.get_category(phase, category) - _compare_categories(current_category, category_comp) - BootSet.delete(session_id, boot_set_name) - - def testCreateV1SessionStatusStatus(self, monkeypatch): - - session_id = "session-123" - monkeypatch.setattr("flask.helpers.url_for", self.mockresponse) - monkeypatch.setattr(connexion, "request", MockResponse2) - session, _status = create_v1_session_status(session_id) - - monkeypatch.setattr(connexion, "request", MockResponseBootSetCreation) - boot_sets = ['boot_set1', 'boot_set2', 'boot_set3'] - for boot_set_name in boot_sets: - BootSet.delete(session_id, boot_set_name) - _bs, _status = create_v1_boot_set_status(session_id, boot_set_name) - - # TODO Find a way to pass in these parameters - # We cannot use the session.metadata because it is not JSON serializable - # and the 'from_dict' method will choke on it. - # That is why it must be converted. - feeder_dict = {'boot_sets': boot_sets, 'id': session_id, - 'metadata': _convert_metadata(session.metadata)} - session_from_dict = SessionStatus.from_dict(feeder_dict) - session_from_params = SessionStatus(boot_sets=boot_sets, metadata=session.metadata) - session_from_params.id = session_id - - _compare_session(session, session_from_dict) - _compare_session(session, session_from_params) - SessionStatus.delete(session_id) - - def testGetV1SessionStatusStatus(self, monkeypatch): - monkeypatch.setattr("flask.helpers.url_for", self.mockresponse) - monkeypatch.setattr(connexion, "request", MockResponse2) - session_id = "session-123" - create_v1_session_status(session_id) - session, _status = get_v1_session_status(session_id) - # TODO Find a way to pass in these parameters - boot_sets = ['boot_set1', 'boot_set2', 'boot_set3'] - session_comp = SessionStatus(boot_sets=boot_sets, metadata=session.metadata, id=session_id) - - _compare_session(session, session_comp) - SessionStatus.delete(session_id) - - def testUpdateV1SessionStatusStatus(self, monkeypatch): - monkeypatch.setattr("flask.helpers.url_for", self.mockresponse) - - session_id = "session-123" - # Create SessionStatus Status - monkeypatch.setattr(connexion, "request", MockResponse2) - session, _status = create_v1_session_status(session_id) - # TODO Find a way to pass in these parameters - boot_sets = ['boot_set1', 'boot_set2', 'boot_set3'] - # We cannot use the session.metadata because it is not JSON serializable - # and the 'from_dict' method will choke on it. - # That is why it must be converted. - feeder_dict = {'boot_sets': boot_sets, 'id': session_id, - 'metadata': _convert_metadata(session.metadata)} - feeder_dict['metadata']['start_time'] = '1:00' - feeder_dict['metadata']['stop_time'] = '2:00' - session_from_dict = SessionStatus.from_dict(feeder_dict) - - # Update SessionStatus's Status with start_time and stop_time - monkeypatch.setattr(connexion, "request", MockResponse5) - session, status = update_v1_session_status(session_id) - - assert status == 200 - _compare_session(session, session_from_dict) - - SessionStatus.delete(session_id) diff --git a/src/bos/server/test/test_version_controller.py b/src/bos/server/test/test_version_controller.py deleted file mode 100644 index 1497ea99..00000000 --- a/src/bos/server/test/test_version_controller.py +++ /dev/null @@ -1,69 +0,0 @@ -# -# MIT License -# -# (C) Copyright 2019, 2021-2022 Hewlett Packard Enterprise Development LP -# -# Permission is hereby granted, free of charge, to any person obtaining a -# copy of this software and associated documentation files (the "Software"), -# to deal in the Software without restriction, including without limitation -# the rights to use, copy, modify, merge, publish, distribute, sublicense, -# and/or sell copies of the Software, and to permit persons to whom the -# Software is furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included -# in all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL -# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR -# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, -# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR -# OTHER DEALINGS IN THE SOFTWARE. -# -# coding: utf-8 - -from __future__ import absolute_import - -from flask import json -from six import BytesIO - -from bos.server.models.problem_details import ProblemDetails # noqa: E501 -from bos.server.models.version import Version # noqa: E501 -from bos.server.test import BaseTestCase -from bos.server.controllers.base import root_get - -from nose.tools import nottest - - -class TestVersionController(BaseTestCase): - """VersionController integration test stubs""" - - @nottest - def test_get_version(self): - """Test case for get_version - - API version - """ - response = self.client.open( - '/apis/bos/v1', - method='GET') - self.assert200(response, - 'Response body is : ' + response.data.decode('utf-8')) - - @nottest - def test_get_versions(self): - """Test case for get_versions - - API versions - """ - response = self.client.open( - '/apis/bos/', - method='GET') - self.assert200(response, - 'Response body is : ' + response.data.decode('utf-8')) - - -if __name__ == '__main__': - import unittest - unittest.main() diff --git a/tools/README.md b/tools/README.md deleted file mode 100644 index fee96e98..00000000 --- a/tools/README.md +++ /dev/null @@ -1,9 +0,0 @@ -This script makes creating a BOS Session Templates easier. Its sole input is a JSON file containing the content for the Session Template. This enables a workflow where you copy the standard Session Template out of BOS, modify it, and then feed the modified file to this script to create a new Session Template. You would get the standard Session Template via the Cray CLI. - -Workflow: - -\# cray bos v1 sessiontemplate describe --format json > my-session-template - -\# vi my-session-template # Modify the session as desired - -\# bos_session_template my-session-template \ No newline at end of file diff --git a/tools/bos_session_template.py b/tools/bos_session_template.py deleted file mode 100755 index 2f8bc9b6..00000000 --- a/tools/bos_session_template.py +++ /dev/null @@ -1,82 +0,0 @@ -#!/usr/bin/env python3 -# -# MIT License -# -# (C) Copyright 2020-2022 Hewlett Packard Enterprise Development LP -# -# Permission is hereby granted, free of charge, to any person obtaining a -# copy of this software and associated documentation files (the "Software"), -# to deal in the Software without restriction, including without limitation -# the rights to use, copy, modify, merge, publish, distribute, sublicense, -# and/or sell copies of the Software, and to permit persons to whom the -# Software is furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included -# in all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL -# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR -# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, -# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR -# OTHER DEALINGS IN THE SOFTWARE. -# -import argparse -import base64 -import json -import subprocess -import sys - -import requests - -""" -This script accepts a JSON file containing the contents for a Session Template. -It then gets the needed credentials and contacts the Boot Orchestration Service -(BOS) to create a Session Template. -""" - -def get_token(): - """ - Returns the access token. - """ - KEYCLOAK_ENDPOINT = "https://api-gw-service-nmn.local/keycloak/realms/shasta/protocol/openid-connect/token" - - cmd = "kubectl get secrets admin-client-auth -ojsonpath='{.data.client-secret}'".split() - result = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) - admin_secret = base64.b64decode(result.stdout) - resp = requests.post(KEYCLOAK_ENDPOINT, data = {'grant_type':'client_credentials', 'client_id': 'admin-client', 'client_secret': admin_secret}) - return resp.json()["access_token"] - - -def main(): - parser = argparse.ArgumentParser(description='Create a BOS Session Template from a JSON file.') - parser.add_argument('fp', metavar='input_file', - type=argparse.FileType('r', encoding='UTF-8'), - help='A JSON file containing a BOS Session Template') - args = parser.parse_args() - - BOS_ENDPOINT = "https://api-gw-service-nmn.local/apis/bos/v1/sessiontemplate" - try: - body_check = json.load(args.fp) - if not isinstance(body_check, dict): - print("ERROR: Session Template must be formatted as a dictionary.") - sys.exit(1) - body = json.dumps(body_check) - except json.decoder.JSONDecodeError as err: - print("ERROR: File '%s' was not proper JSON: %s" % (args.fp.name, err)) - sys.exit(1) - try: - headers = {"Authorization": "Bearer %s" % get_token(), "Content-Type": "application/json"} - except Exception as err: - print("ERROR: Unable to get TOKEN to access the Boot Orchestration Service: %s" % err) - sys.exit(1) - try: - resp = requests.post(BOS_ENDPOINT, data=body, headers=headers) - resp.raise_for_status() - except requests.RequestException as err: - print("ERROR: Problem contacting the Boot Orchestration Service (BOS): %s" % err) - sys.exit(1) - -if __name__ == "__main__": - main() diff --git a/tools/requirements_bos_session_template.txt b/tools/requirements_bos_session_template.txt deleted file mode 100644 index f2293605..00000000 --- a/tools/requirements_bos_session_template.txt +++ /dev/null @@ -1 +0,0 @@ -requests diff --git a/tools/setup.py b/tools/setup.py deleted file mode 100644 index a8e1e436..00000000 --- a/tools/setup.py +++ /dev/null @@ -1,66 +0,0 @@ -# -# MIT License -# -# (C) Copyright 2020-2022 Hewlett Packard Enterprise Development LP -# -# Permission is hereby granted, free of charge, to any person obtaining a -# copy of this software and associated documentation files (the "Software"), -# to deal in the Software without restriction, including without limitation -# the rights to use, copy, modify, merge, publish, distribute, sublicense, -# and/or sell copies of the Software, and to permit persons to whom the -# Software is furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included -# in all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL -# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR -# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, -# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR -# OTHER DEALINGS IN THE SOFTWARE. -# -# setuptools-based installation module for bos_session_template - -from os import path -from setuptools import setup, find_packages - - -here = path.abspath(path.dirname(__file__)) - -with open(path.join(here, 'README.md'), encoding='utf-8') as f: - long_description = f.read() - -with open(path.join(here, 'requirements_bos_session_template.txt'), encoding='utf-8') as f: - install_requires = [] - for line in f.readlines(): - commentless_line = line.split('#', 1)[0].strip() - if commentless_line: - install_requires.append(commentless_line) - -version = 1.0 - -setup( - name='bos_session_template', - version=version, - description="BOS Session Template creation script", - long_description=long_description, - long_description_content_type='text/markdown', - url='https://github.com/Cray-HPE/bos', - author='Cray, Inc.', - license='Cray Proprietary', - package_dir = {'': 'tools'}, - py_modules=['bos_session_template'], - python_requires='>=3, <4', - # Top-level dependencies are parsed from requirements.txt - install_requires=install_requires, - #scripts=['tools/bos_session_template.py'], - # This makes setuptools generate our executable script automatically for us. - entry_points={ - 'console_scripts': [ -# 'bos_session_template = bos_session_template.py:main' - 'bos_session_template = bos_session_template:main' - ] - }, -) \ No newline at end of file diff --git a/update_versions.conf b/update_versions.conf index 57cbc614..bc13900a 100644 --- a/update_versions.conf +++ b/update_versions.conf @@ -21,63 +21,43 @@ # Some of the sourcefile and tag lines below are # superfluous, but are present for clarity +# Many of these sourcefiles do not exist in the repo as static files +# They are generated at build time + sourcefile: .version tag: @RPM_VERSION@ targetfile: bos-reporter.spec -sourcefile-novalidate: .rpm_release -tag: @RPM_RELEASE@ -targetfile: bos-reporter.spec - sourcefile: .version tag: @VERSION@ targetfile: src/setup.py -# The following file does not exist in the repo as a static file -# It is generated at build time +sourcefile-novalidate: .rpm_release +tag: @RPM_RELEASE@ +targetfile: bos-reporter.spec + sourcefile: .chart_version tag: 0.0.0-chart targetfile: kubernetes/cray-bos/Chart.yaml -sourcefile: cray-boa.version -tag: 0.0.0-boa -targetfile: kubernetes/cray-bos/Chart.yaml + sourcefile: .docker_version tag: 0.0.0-bos targetfile: kubernetes/cray-bos/Chart.yaml -# The following file does not exist in the repo as a static file -# It is generated at build time +sourcefile: .docker_version +tag: 0.0.0-app-version +targetfile: kubernetes/cray-bos/values.yaml +targetfile: kubernetes/cray-bos/templates/post-upgrade-job.yaml + sourcefile: openapi.version tag: 0[.]0[.]0-api targetfile: api/openapi.yaml -# The following file does not exist in the repo as a static file -# It is generated at build time sourcefile-novalidate: .stable tag: S-T-A-B-L-E targetfile: kubernetes/cray-bos/values.yaml targetfile: kubernetes/cray-bos/Chart.yaml -# The following file does not exist in the repo as a static file -# It is generated at build time by the runBuildPrep.sh script -sourcefile: cray-boa.version -tag: 0.0.0-boa -targetfile: kubernetes/cray-bos/values.yaml - -# The following file does not exist in the repo as a static file -# It is generated at build time by the runBuildPrep.sh script -sourcefile: .docker_version -tag: 0.0.0-app-version -targetfile: kubernetes/cray-bos/values.yaml - -# The following file does not exist in the repo as a static file -# It is generated at build time by the runBuildPrep.sh script -sourcefile: .docker_version -tag: 0.0.0-app-version -targetfile: kubernetes/cray-bos/templates/post-upgrade-job.yaml - -# The following file does not exist in the repo as a static file -# It is generated at build time by the runBuildPrep.sh script sourcefile: liveness.version tag: 0.0.0-liveness targetfile: constraints.txt