From 19fe79f65028010bd018422d5606596995932dc1 Mon Sep 17 00:00:00 2001 From: Jaime Frey Date: Tue, 23 Jan 2024 15:05:46 -0600 Subject: [PATCH 1/3] HTCONDOR-2169 Update old HTCondor manual links All links now point to the LTS manual on readthedocs. --- docs/v23/configuration/job-router-overview.md | 4 +-- .../configuration/optional-configuration.md | 2 +- docs/v23/configuration/writing-job-routes.md | 8 +++--- docs/v23/troubleshooting/common-issues.md | 6 ++--- docs/v23/troubleshooting/debugging-tools.md | 27 ++++++++++--------- docs/v23/troubleshooting/logs.md | 2 +- 6 files changed, 26 insertions(+), 23 deletions(-) diff --git a/docs/v23/configuration/job-router-overview.md b/docs/v23/configuration/job-router-overview.md index b87f9a19e..57ca15130 100644 --- a/docs/v23/configuration/job-router-overview.md +++ b/docs/v23/configuration/job-router-overview.md @@ -45,12 +45,12 @@ in the following order: deprecated and will be removed for *V24* of the HTCondor Software Suite. New configuration syntax for the job router is defined using `JOB_ROUTER_ROUTE_NAMES` and `JOB_ROUTER_ROUTE_[name]`. - For new syntax example vist: - [HTCondor Documentation - Job Router](https://htcondor.readthedocs.io/en/latest/grid-computing/job-router.html#an-example-configuration) + [HTCondor Documentation - Job Router](https://htcondor.readthedocs.io/en/lts/grid-computing/job-router.html#an-example-configuration) - **Note:** The removal will occur during the lifetime of the HTCondor *V23* feature series. Since the inception of HTCondor-CE, job routes have been written as a [list of ClassAds](https://htcondor.readthedocs.io/en/lts/grid-computing/job-router.html#deprecated-router-configuration). -Each job route’s [ClassAd](http://research.cs.wisc.edu/htcondor/manual/v8.6/4_1HTCondor_s_ClassAd.html) is constructed +Each job route’s [ClassAd](https://htcondor.readthedocs.io/en/lts/classads/classad-mechanism.html) is constructed by combining each entry from the `JOB_ROUTER_ENTRIES` with the `JOB_ROUTER_DEFAULTS`: - `JOB_ROUTER_ENTRIES` is a configuration variable whose default is set in `/etc/condor-ce/config.d/02-ce-*.conf` but diff --git a/docs/v23/configuration/optional-configuration.md b/docs/v23/configuration/optional-configuration.md index 6178516bf..147176386 100644 --- a/docs/v23/configuration/optional-configuration.md +++ b/docs/v23/configuration/optional-configuration.md @@ -60,7 +60,7 @@ Inserting IDTOKENs into the routed job's sandbox If you want to insert IDTOKENS into the routed job's sandbox you can use the `SendIDTokens` route command, or the `JOB_ROUTER_SEND_ROUTE_IDTOKENS` global configuration variable. Tokens sent using this mechanism must be named and declared using the `JOB_ROUTER_CREATE_IDTOKEN_NAMES` -and [`JOB_ROUTER_CREATE_IDTOKEN_`](https://htcondor.readthedocs.io/en/latest/admin-manual/configuration-macros.html#JOB_ROUTER_CREATE_IDTOKEN_%3CNAME%3E) configuration variables. Tokens whose names are declared in +and [`JOB_ROUTER_CREATE_IDTOKEN_`](https://htcondor.readthedocs.io/en/lts/admin-manual/configuration-macros.html#JOB_ROUTER_CREATE_IDTOKEN_%3CNAME%3E) configuration variables. Tokens whose names are declared in the `JOB_ROUTER_SEND_ROUTE_IDTOKENS` configuration variable are sent by default for each route that does not have a `SendIDTokens` command. diff --git a/docs/v23/configuration/writing-job-routes.md b/docs/v23/configuration/writing-job-routes.md index 0cccd9b69..946432a2e 100644 --- a/docs/v23/configuration/writing-job-routes.md +++ b/docs/v23/configuration/writing-job-routes.md @@ -29,7 +29,7 @@ Syntax Differences deprecated and will be removed for *V24* of the HTCondor Software Suite. New configuration syntax for the job router is defined using `JOB_ROUTER_ROUTE_NAMES` and `JOB_ROUTER_ROUTE_[name]`. - For new syntax example vist: - [HTCondor Documentation - Job Router](https://htcondor.readthedocs.io/en/latest/grid-computing/job-router.html#an-example-configuration) + [HTCondor Documentation - Job Router](https://htcondor.readthedocs.io/en/lts/grid-computing/job-router.html#an-example-configuration) - **Note:** The removal will occur during the lifetime of the HTCondor *V23* feature series. @@ -836,12 +836,12 @@ This section outlines how to limit the number of total or idle jobs in a specifi !!! note If you are using an HTCondor batch system, limiting the number of jobs is not the preferred solution: HTCondor manages fair share on its own via - [user priorities and group accounting](http://research.cs.wisc.edu/htcondor/manual/v8.6/3_6User_Priorities.html). + [user priorities and group accounting](https://htcondor.readthedocs.io/en/lts/admin-manual/user-priorities-negotiation.html). ### Total jobs To set a limit on the number of jobs for a specific route, -set the [MaxJobs](http://research.cs.wisc.edu/htcondor/manual/v8.6/5_4HTCondor_Job.html#57134) attribute: +set the [MaxJobs](https://htcondor.readthedocs.io/en/lts/grid-computing/job-router.html#index-6) attribute: === "ClassAd Transform" @@ -871,7 +871,7 @@ set the [MaxJobs](http://research.cs.wisc.edu/htcondor/manual/v8.6/5_4HTCondor_J ### Idle jobs To set a limit on the number of idle jobs for a specific route, -set the [MaxIdleJobs](http://research.cs.wisc.edu/htcondor/manual/v8.6/5_4HTCondor_Job.html#57135) attribute: +set the [MaxIdleJobs](https://htcondor.readthedocs.io/en/lts/grid-computing/job-router.html#index-7) attribute: === "ClassAd Transform" diff --git a/docs/v23/troubleshooting/common-issues.md b/docs/v23/troubleshooting/common-issues.md index 12e31e136..d818cb3d7 100644 --- a/docs/v23/troubleshooting/common-issues.md +++ b/docs/v23/troubleshooting/common-issues.md @@ -441,7 +441,7 @@ The most common cases for this behavior are as follows: - **The route(s) that the job matches to are full:** See [limiting the number of jobs](../configuration/writing-job-routes.md#limiting-the-number-of-jobs). - **The job router is throttling submission to your batch system due to submission failures:** - See the HTCondor manual for [FailureRateThreshold](http://research.cs.wisc.edu/htcondor/manual/v8.6/5_4HTCondor_Job.html#55958). + See the HTCondor manual for [FailureRateThreshold](https://htcondor.readthedocs.io/en/lts/grid-computing/job-router.html#index-8). Check for errors in the [JobRouterLog](logs.md#jobrouterlog) or [GridmanagerLog](logs.md#gridmanagerlog) for HTCondor and non-HTCondor batch systems, respectively. @@ -465,10 +465,10 @@ Ensure that the owner of the job generates their proxy with `voms-proxy-init`. #### Held jobs: Invalid job universe -The HTCondor-CE only accepts jobs that have `universe` in their submit files set to `vanilla`, `standard`, `local`, or +The HTCondor-CE only accepts jobs that have `universe` in their submit files set to `vanilla`, `local`, or `scheduler`. These universes also have corresponding integer values that can be found in the -[HTCondor manual](http://research.cs.wisc.edu/htcondor/manual/v8.6/12_Appendix_A.html#104736). +[HTCondor manual](https://htcondor.readthedocs.io/en/lts/codes-other-values/job-universe-numbers.html). **Next actions** diff --git a/docs/v23/troubleshooting/debugging-tools.md b/docs/v23/troubleshooting/debugging-tools.md index c0c051b8e..1c769093f 100644 --- a/docs/v23/troubleshooting/debugging-tools.md +++ b/docs/v23/troubleshooting/debugging-tools.md @@ -143,8 +143,9 @@ Authorized: TRUE !!! note If you run the `condor_ce_ping` command on the CE that you are testing, omit the `-name` and `-pool` - options. `condor_ce_ping` takes the same arguments as `condor_ping` and is documented in the - [HTCondor manual](http://research.cs.wisc.edu/htcondor/manual/v8.6/condor_ping.html). + options. `condor_ce_ping` takes the same arguments as + `condor_ping`, which is documented in the + [HTCondor manual](https://htcondor.readthedocs.io/en/lts/man-pages/condor_ping.html). ### Troubleshooting ### @@ -182,8 +183,8 @@ user@host $ condor_ce_q -name condorce.example.com -pool condorce.example.com:96 !!! note If you run the `condor_ce_q` command on the CE that you are testing, omit the `-name` and `-pool` options. - `condor_ce_q` takes the same arguments as `condor_q` and is documented in the - [HTCondor manual](http://research.cs.wisc.edu/htcondor/manual/v8.6/condor_q.html). + `condor_ce_q` takes the same arguments as `condor_q`, which is documented in the + [HTCondor manual](https://htcondor.readthedocs.io/en/lts/man-pages/condor_q.html). ### Troubleshooting ### @@ -248,8 +249,8 @@ user@host $ condor_ce_history -name condorce.example.com -pool condorce.example. !!! note If you run the `condor_ce_history` command on the CE that you are testing, omit the `-name` and `-pool` options. - `condor_ce_history` takes the same arguments as `condor_history` and is documented in the - [HTCondor manual](http://research.cs.wisc.edu/htcondor/manual/v8.6/condor_history.html). + `condor_ce_history` takes the same arguments as `condor_history`, which is documented in the + [HTCondor manual](https://htcondor.readthedocs.io/en/lts/man-pages/condor_history.html). condor_ce_job_router_info @@ -343,8 +344,9 @@ and their statuses: user@host $ condor_ce_router_q ``` -`condor_ce_router_q` takes the same options as `condor_router_q` and `condor_q` and is documented in the -[HTCondor manual](http://research.cs.wisc.edu/htcondor/manual/v8.6/condor_router_q.html) +`condor_ce_router_q` takes the same options as `condor_router_q`, +which is documented in the +[HTCondor manual](https://htcondor.readthedocs.io/en/lts/man-pages/condor_router_q.html) condor_ce_status @@ -358,8 +360,8 @@ To see the daemons running on a CE, run the following command: user@host $ condor_ce_status -any ``` -`condor_ce_status` takes the same arguments as `condor_status`, which are documented in the -[HTCondor manual](http://research.cs.wisc.edu/htcondor/manual/v8.6/condor_status.html). +`condor_ce_status` takes the same arguments as `condor_status`, which is documented in the +[HTCondor manual](https://htcondor.readthedocs.io/en/lts/man-pages/condor_status.html). !!! note ""Missing" Worker Nodes" An HTCondor-CE will not show any worker nodes (e.g. `Machine` entries in the `condor_ce_status -any` output) if @@ -404,8 +406,9 @@ following command: user@host $ condor_ce_config_val -config ``` -`condor_ce_config_val` takes the same arguments as `condor_config_val` and is documented in the -[HTCondor manual](http://research.cs.wisc.edu/htcondor/manual/v8.6/condor_config_val.html). +`condor_ce_config_val` takes the same arguments as +`condor_config_val`, which is documented in the +[HTCondor manual](https://htcondor.readthedocs.io/en/lts/man-pages/condor_config_val.html). condor_ce_reconfig diff --git a/docs/v23/troubleshooting/logs.md b/docs/v23/troubleshooting/logs.md index 2e15706e9..64b8c7a2b 100644 --- a/docs/v23/troubleshooting/logs.md +++ b/docs/v23/troubleshooting/logs.md @@ -253,7 +253,7 @@ SharedPortLog The HTCondor-CE shared port log keeps track of all connections to all of the HTCondor-CE daemons other than the collector. This log is a good place to check if experiencing connectivity issues with HTCondor-CE. More information can be found -[here](http://research.cs.wisc.edu/htcondor/manual/v8.6/3_9Networking_includes.html#SECTION00492000000000000000). +[here](https://htcondor.readthedocs.io/en/lts/admin-manual/networking.html#reducing-port-usage-with-the-condor-shared-port-daemon). - Location: `/var/log/condor-ce/SharedPortLog` - Key contents: Every attempt to connect to HTCondor-CE (except collector queries) From c5cb784d2f2ab2db859e10b54c610798455b3dc4 Mon Sep 17 00:00:00 2001 From: Jaime Frey Date: Tue, 23 Jan 2024 15:06:24 -0600 Subject: [PATCH 2/3] HTCONDOR-2169 Add debugging entry for condor_ce_test_token --- docs/v23/troubleshooting/debugging-tools.md | 29 +++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/docs/v23/troubleshooting/debugging-tools.md b/docs/v23/troubleshooting/debugging-tools.md index 1c769093f..56dff7f4c 100644 --- a/docs/v23/troubleshooting/debugging-tools.md +++ b/docs/v23/troubleshooting/debugging-tools.md @@ -349,6 +349,35 @@ which is documented in the [HTCondor manual](https://htcondor.readthedocs.io/en/lts/man-pages/condor_router_q.html) +condor_ce_test_token +-------------------- + +## Usage ### + +Use the `condor_ce_test_token` command to test SciTokens +authentication in the CE. +It will create a token with an issuer and subject that you specify and +configure the CE daemons to accept that token as if it had been +generated by the given issuer (for one hour). +The token is printed to stdout; use it with `condor_ce_submit` to test +that SciTokens authentication and user mapping operate correctly. + +To create a temporary SciToken that appears to be issued by the +SciTokens demo issuer: + +``` console +root@host # condor_ce_token_test --issuer https://demo.scitokens.org +--audience ANY --scope condor:/WRITE --subject alice@foo.edu +``` + +!!! note + You must run `condor_ce_test_token` on the CE that you are testing + as the root user. + `condor_ce_test_token` takes the same arguments as + `condor_test_token`, which is documented in the + [HTCondor manual](https://htcondor.readthedocs.io/en/lts/man-pages/condor_test_token.html). + + condor_ce_status ---------------- From ab1ccc0fea40def8dbde82865e1e9be2c4214a6d Mon Sep 17 00:00:00 2001 From: Jaime Frey Date: Tue, 23 Jan 2024 15:43:40 -0600 Subject: [PATCH 3/3] HTCONDOR-2169 Update HTCondor installation links --- docs/v23/installation/central-collector.md | 2 +- docs/v23/installation/htcondor-ce.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/v23/installation/central-collector.md b/docs/v23/installation/central-collector.md index f5bdce9ce..88b78299f 100644 --- a/docs/v23/installation/central-collector.md +++ b/docs/v23/installation/central-collector.md @@ -35,7 +35,7 @@ There are some one-time (per host) steps to prepare in advance: - Ensure the host has a supported operating system (Red Hat Enterprise Linux variant 7) - Obtain root access to the host -- Prepare the [EPEL](https://fedoraproject.org/wiki/EPEL) and [HTCondor](https://research.cs.wisc.edu/htcondor/yum/) Yum +- Prepare the [EPEL](https://fedoraproject.org/wiki/EPEL) and [HTCondor](https://htcondor.readthedocs.io/en/lts/getting-htcondor/from-our-repositories.html) Yum repositories - Install CA certificates and VO data into `/etc/grid-security/certificates` and `/etc/grid-security/vomsdir`, respectively diff --git a/docs/v23/installation/htcondor-ce.md b/docs/v23/installation/htcondor-ce.md index cdaf30583..c2e0b1b98 100644 --- a/docs/v23/installation/htcondor-ce.md +++ b/docs/v23/installation/htcondor-ce.md @@ -34,7 +34,7 @@ There are some one-time (per host) steps to prepare in advance: - Ensure the host has a supported operating system (Red Hat Enterprise Linux variant 7) - Obtain root access to the host -- Prepare the [EPEL](https://fedoraproject.org/wiki/EPEL) and [HTCondor Development](https://research.cs.wisc.edu/htcondor/yum/) Yum +- Prepare the [EPEL](https://fedoraproject.org/wiki/EPEL) and [HTCondor Development](https://htcondor.readthedocs.io/en/lts/getting-htcondor/from-our-repositories.html) Yum repositories - Install CA certificates and VO data into `/etc/grid-security/certificates` and `/etc/grid-security/vomsdir`, respectively