Merge pull request #239 from Cray-HPE/CRAYSAT-1787-ceph-healthcheck-b…

…ypass-in-bootsys

CRAYSAT-1787: Adding ceph health check bypass in sat bootsys ncn-power
Reviewer:Ryan

CHANGELOG.md

@@ -52,6 +52,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   ignorable. 
 - Automate the procedure of setting next boot device to disk before the management nodes are
   powered off as part of the full-system shutdown.
+- Adding a ceph health check bypass prompt to take input from user and act accordingly.
+  unfreezing of ceph would be done, only the wait period will be skipped if user wishes to.
 
 ### Fixed
 - Updated `sat bootsys` to increase the default management NCN shutdown timeout

sat/cli/bootsys/mgmt_power.py

@@ -477,10 +477,19 @@ def do_power_on_ncns(args):
                     if ncn_group == included_ncn_groups['storage']:
                         try:
                             do_ceph_unfreeze(included_ncn_groups)
+                            LOGGER.info('Ceph unfreeze completed successfully on storage NCNs.')
+
                         except FatalPlatformError as err:
                             LOGGER.error(f'Failed to unfreeze Ceph on storage NCNs: {err}')
-                            sys.exit(1)
-                        LOGGER.info('Ceph unfreeze completed successfully on storage NCNs.')
+                            # Use pester_choices to prompt the user
+                            user_choice = pester_choices('Ceph is not healthy. Do you want to continue anyway?',
+                                                         ['yes', 'no'])
+                            if user_choice == 'no':
+                                LOGGER.info('Exiting as per user\'s decision.')
+                                sys.exit(1)
+                            else:
+                                LOGGER.info('Continuing despite Ceph not being healthy as per user\'s input, '
+                                            'make sure to verify it later.')
 
                         # Mount Ceph and S3FS filesystems on ncn-m001
                         try: