[YUNIKORN-2439] Deprecate state-aware sorting policy (#403)

Closes: #403

docs/user_guide/gang_scheduling.md

@@ -52,7 +52,7 @@ To configure queue sorting policy, please refer to doc: [app sorting policies](s
 #### Why the `FIFO` sorting policy
 
 When Gang Scheduling is enabled, the scheduler proactively reserves resources
-for each application. If the queue sorting policy is not FIFO based (StateAware is FIFO based sorting policy),
+for each application. If the queue sorting policy is not FIFO-based (StateAware is FIFO based sorting policy),
 the scheduler might reserve partial resources for each app and causing resource segmentation issues.
 
 #### Side effects of `StateAware` sorting policy
@@ -61,6 +61,7 @@ We do not recommend using `StateAware`, even-though it is a FIFO based policy. A
 This is specifically an issue with Spark jobs when the driver performs a lot of pre-processing before requesting the executors.
 The `StateAware` timeout in those cases would slow down processing to just one application per timeout.
 This in effect will overrule the gang reservation and cause slowdowns and excessive resource usage.
+`StateAware` sorting is **deprecated** in YuniKorn 1.5.0 and will be **removed** from YuniKorn 1.6.0.
 
 ### App Configuration
 

docs/user_guide/labels_and_annotations_in_yunikorn.md

@@ -25,13 +25,13 @@ under the License.
 YuniKorn utilizes several Kubernetes labels and annotations to support various features:
 
 ### Labels in YuniKorn
-| Name                 | Description                                                                                                                                             |
-|----------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `applicationId`      | Associates this pod with an application.                                                                                                                |
-| `queue`              | Selects the YuniKorn queue this application should be scheduled in. This may be ignored if a placement policy is in effect.                             |
-| `spark-app-selector` | Alternative method of specifying `applicationId` used by Spark Operator if the label `applicationId` and annotation `yunikorn.apache.org/app-id` unset. |
-| `disableStateAware`  | If present, disables the YuniKorn state-aware scheduling policy for this pod. Set internally by the YuniKorn admission controller.                      |
-| `placeholder`        | Set if this pod represents a placeholder for gang scheduling. Set internally by YuniKorn.                                                               |
+| Name                              | Description                                                                                                                                             |
+|-----------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `applicationId`                   | Associates this pod with an application.                                                                                                                |
+| `queue`                           | Selects the YuniKorn queue this application should be scheduled in. This may be ignored if a placement policy is in effect.                             |
+| `spark-app-selector`              | Alternative method of specifying `applicationId` used by Spark Operator if the label `applicationId` and annotation `yunikorn.apache.org/app-id` unset. |
+| [DEPRECATED] `disableStateAware`  | If present, disables the YuniKorn state-aware scheduling policy for this pod. Set internally by the YuniKorn admission controller.                      |
+| `placeholder`                     | Set if this pod represents a placeholder for gang scheduling. Set internally by YuniKorn.                                                               |
 
 ### Annotations in YuniKorn
 All annotations are under the namespace `yunikorn.apache.org`. For example `yunikorn.apache.org/app-id`.

docs/user_guide/queue_config.md

@@ -299,6 +299,10 @@ Default value: `fifo`
 Sets the policy to be used when sorting applications within a queue. This
 setting has no effect on a _parent_ queue.
 
+**NOTE:** The `stateaware` policy is **deprecated** in YuniKorn 1.5.0 and will
+be **removed** from YuniKorn 1.6.0, where it will be treated as an alias for
+`fifo`.
+
 See the documentation on [application sorting](sorting_policies.md#application-sorting)
 for more information.
 
@@ -436,7 +440,7 @@ As an example:
          childtemplate:
            maxapplications: 10
            properties:
-             application.sort.policy: stateaware
+             application.sort.policy: fifo
            resources:
              guaranteed:
                vcore: 1

docs/user_guide/resource_quota_mgmt.md

@@ -207,16 +207,15 @@ partitions:
       - name: root
         submitacl: '*'
         properties:
-          application.sort.policy: stateaware
+          application.sort.policy: fifo
 ```
 
 This configuration places an application based on the `tag` rule.
 The tag selected is the `namespace` tag which is automatically added by the k8s shim to all applications that get created.
 The `create` flag is set to true which will trigger the creation of the queue with the same name as the namespace if it does not exist. 
 
 Applications within the automatically created child queues will be sorted based sorting policy set on the parent queue.
-In this case the property `application.sort.policy` is in this configuration set to `stateaware`.
-This is a simple app sorting policy applicable for batch jobs, you can find more document [here](sorting_policies.md#stateawarepolicy).
+In this case the property `application.sort.policy` is in this configuration set to `fifo`.
 
 You can change the configuration using the helm charts during the installation by overwriting the configuration in the
 [helm chart template](https://github.com/apache/yunikorn-release/blob/master/helm-charts/yunikorn/values.yaml#L71-L81).

docs/user_guide/sorting_policies.md

@@ -91,6 +91,12 @@ Short description: limit of one (1) application in Starting or Accepted state
 
 Config value: `stateaware`
 
+**DEPRECATED:** The `stateaware` policy is **deprecated** in YuniKorn 1.5.0 and
+will be **removed** in YuniKorn 1.6.0. To preserve backwards compatibility,
+`stateaware` will become an alias for `fifo` in YuniKorn 1.6.0 and later.
+Users are encouraged to migrate to `fifo` and utilize either gang scheduling or
+`maxapplications` to limit concurrency instead.
+
 This sorting policy requires an understanding of the application states.
 Applications states are described in the [application states](developer_guide/scheduler_object_states.md#application-state) documentation.