Remove verbose=True from getMSLogger calls

[vkuznet]

Fixes #10317

Status

need confirmation from @amaltaro

Description

remove verbose=True in getMSLogger, the level should be applied by passing proper logger instance.

Is it backward compatible (if not, which system it affects?)

MAYBE

Related PRs

dmwm/deployment#1124

External dependencies / deployment changes

[vkuznet]

The changes to gitlab services_configs repo can be applied once dmwm/deployment#1124 is approved.

[cmsdmwmbot]

Jenkins results:

• Python3 Unit tests: failed
  • 1 new failures
  • 2 changes in unstable tests
• Python3 Pylint check: succeeded
• Pylint py3k check: failed
  • 2 warnings
• Pycodestyle check: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/12777/artifact/artifacts/PullRequestReport.html

[vkuznet]

Here are some additional places within WMCore with explicit DEBUG level:

src/python/WMCore/Services/Service.py
164:            logging.basicConfig(level=logging.DEBUG

src/python/WMCore/Storage/SiteLocalConfig.py
35:            logging.log(logging.DEBUG, m)

src/python/WMCore/WebTools/Root.py
247:            cherrypy.log.error_log.setLevel(configDict.get("error_log_level", logging.DEBUG))
248:            cherrypy.log.access_log.setLevel(configDict.get("access_log_level", logging.DEBUG))

src/python/WMCore/WebTools/Page.py
61:        self.log(msg, logging.DEBUG)

src/python/WMCore/ReqMgr/Web/tools.py
76:            self.log(msg, logging.DEBUG)

src/python/WMCore/WMRuntime/Scripts/SetupCMSSWPset.py
212:            self.logger.info("DEBUG output module = %s", outMod)
718:            self.logger.info("DEBUG: Now in the none scenario to load PSET")

src/python/WMCore/Credential/SimpleMyProxy.py
251:            loglevel = logging.DEBUG

@amaltaro please advise which parts should be changed.

[amaltaro]

@vkuznet thanks for providing these options, but the last bullet in the expected outcome - from the GH ticket description - states:

review ALL microservices source code and move useful logging records from DEBUG to INFO level, leaving only very deep (and with large content) logging records in DEBUG mode.

and this is the hard part. We need to review every single log record from all the microservices, and decide which ones should be logged all the time (INFO) and which ones are really meant to be debug mode only. I am afraid to that that the best way to deal with those would be to have Todor and myself looking into those microservices and swapping logs, e.g.::

self.logger.debug()

to

self.logger.info()

when relevant.

[vkuznet]

Alan, I understand, and I only wanted to help with what I can. In fact, I want to suggest a better approach to logging rather then simply swap INFO/DEBUG. In DBS GO server I use verbosity level, e.g. 0,1,2,3 such that I can change it in configuration and at run time and get more verbose information. Here it is a similar use-case. You actually need flexibility, i.e. increase the logging level during debugging phase and reduce it in normal operation. As such for MS services (since they are web services) you may add an extra end-point where you can send verbosity level (via POST request) and it will dynamically change web server behavior. Think about it.

Meanwhile, I provided necessary PRs and relevant info which you can use.

[amaltaro]

Just to "silence" this review request.

As already discussed in this PR, we need to review all the log records throughout the microservices and decide which ones should be defined as logger.debug() and logger.info(). Such that when we move log level to INFO, we do not miss important information currently set as logger.debug.

I suggest to either Todor or myself to look into it, while looking into the production logs in parallel and assess what is the minimum log that we need to have in the service logs (INFO level). We might want to come up with an extra PR to complement this one.

[amaltaro]

test this please

[cmsdmwmbot]

Jenkins results:

• Python3 Unit tests: failed
  • 339 new failures
  • 14 tests deleted
  • 1 tests no longer failing
  • 16 changes in unstable tests
• Python3 Pylint check: failed
  • 3 warnings and errors that must be fixed
  • 6 warnings
  • 83 comments to review
• Pylint py3k check: failed
  • 2 warnings
• Pycodestyle check: succeeded
  • 45 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/13270/artifact/artifacts/PullRequestReport.html

[amaltaro]

test this please

[cmsdmwmbot]

Jenkins results:

• Python3 Unit tests: failed
  • 339 new failures
  • 14 tests deleted
  • 1 tests no longer failing
  • 16 changes in unstable tests
• Python3 Pylint check: failed
  • 3 warnings and errors that must be fixed
  • 6 warnings
  • 83 comments to review
• Pylint py3k check: failed
  • 2 warnings
• Pycodestyle check: succeeded
  • 45 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/13271/artifact/artifacts/PullRequestReport.html

[amaltaro]

test this please

[cmsdmwmbot]

Jenkins results:

• Python3 Unit tests: failed
  • 69 new failures
  • 2 tests deleted
  • 1 tests no longer failing
  • 4 changes in unstable tests
• Python3 Pylint check: failed
  • 3 warnings and errors that must be fixed
  • 6 warnings
  • 83 comments to review
• Pylint py3k check: failed
  • 2 warnings
• Pycodestyle check: succeeded
  • 45 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/13272/artifact/artifacts/PullRequestReport.html

[amaltaro]

test this please

[cmsdmwmbot]

Jenkins results:

• Python3 Unit tests: failed
  • 36 new failures
  • 1 tests deleted
  • 1 changes in unstable tests
• Python3 Pylint check: failed
  • 3 warnings and errors that must be fixed
  • 6 warnings
  • 83 comments to review
• Pylint py3k check: failed
  • 2 warnings
• Pycodestyle check: succeeded
  • 45 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/13273/artifact/artifacts/PullRequestReport.html

[amaltaro]

test this please

[cmsdmwmbot]

Jenkins results:

• Python3 Unit tests: failed
  • 3 new failures
  • 2 changes in unstable tests
• Python3 Pylint check: failed
  • 3 warnings and errors that must be fixed
  • 6 warnings
  • 83 comments to review
• Pylint py3k check: failed
  • 2 warnings
• Pycodestyle check: succeeded
  • 45 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/13274/artifact/artifacts/PullRequestReport.html

[cmsdmwmbot]

Can one of the admins verify this patch?