PWX-35477 : Support Openshift Prometheus for portworx monitoring on OCP 4.14

[nikita-bhatia]

Issue : OCP upgrade from 4.13.x to 4.14.5 is blocked due to PX prometheus. From OCP 4.14, PX Prometheus stopped working.

Resolution : From OCP 4.14, Portworx deployment on OCP should should use Openshift prometheus stack for monitoring Portworx metrics instead of deploying PX-Prometheus. We need to expose Portworx metrics to Openshift.
To do this, user-workload-monitoring has to be enabled on OCP cluster to use openshift's monitoring and alerting for portworx.

What this PR does / why we need it:

1. Default autopilot URL for OCP 4.14 changed from http://px-prometheus:9090/ to https://$THANOS_QUERIER_HOST
2. Creating autopilot-auth-token-secret to store ocp token and ca.cert
3. Mount ocp token and ca.cert as file in autopilot pod with file names ca-certificates.crt and token.

Which issue(s) this PR fixes (optional)
Closes # https://portworx.atlassian.net/browse/PWX-35477

Testing done

1. Tested on OCP 4.14 cluster with fresh install.
2. Tested that secret is created in cluster namespace with correct data.
3. Verified that default url for autopilot is correct set in stc.
4. Verified that autopilod pod is mounted with certificate and token files

[codecov]

Codecov Report

Attention: 35 lines in your changes are missing coverage. Please review.

Comparison is base (956b239) 75.88% compared to head (38a498d) 75.87%.

Files	Patch %	Lines
drivers/storage/portworx/component/autopilot.go	73.01%	9 Missing and 8 partials ⚠️
drivers/storage/portworx/util/util.go	69.38%	11 Missing and 4 partials ⚠️
drivers/storage/portworx/portworx.go	80.00%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1410      +/-   ##
==========================================
- Coverage   75.88%   75.87%   -0.01%     
==========================================
  Files          66       66              
  Lines       18840    18965     +125     
==========================================
+ Hits        14296    14390      +94     
- Misses       3529     3549      +20     
- Partials     1015     1026      +11     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[zoxpx]

@piyush-nimbalkar -- would you mind reviewing this?

It is a fairly complicated PR, and you were the author of the original drivers/storage/portworx/component/autopilot.go where the bulk of the changes are from.

[piyush-nimbalkar]

LGTM after addressing the comments.

@nikita-bhatia can we add more UTs for all corner cases like if we don't find the ocp secret or if the right data is not present in it? What happens when we disable and enable autopilot again - it should have the same deployment and the secret should have been deleted when autopilot was disabled.

Let's merge this PR after addressing comments and then add just the UTs in a different PR.