Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rewrite/cleanup for documentation builder #277

Merged
merged 15 commits into from
Feb 16, 2017

Conversation

wdpypere
Copy link
Contributor

Refactored quattorpoddoc.py script.

Mainly:

  • it now has it's own directory
  • added setup.py
  • added prospector tests
  • added unit tests most of the functions
  • switched to jinja2 templating for consistency
  • refactored in separate modules
  • simplified functions
  • added small fixes and error handling

This should make the whole thing more readable, maintainable.

Also, this makes it easier to replicate to another build system or run your local copy. The following commands should be enough to build the docs (requires that you can build components on the system).

python setup.py install --user
build-quattor-documentation.sh

fixes #256

As per usual, remarks are very welcome!


logger = fancylogger.getLogger()

REPOMAP = {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmmmm. either move this to a config file (ini-style) or better yet, add a .docbuilder.cfg in each repo with whatever data you need. then you only need to a list of repos here (or query all repos like this PR in release helper quattor/release-helper#1).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea, I'll give that a go!


import os
import sys
import jinja2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about TT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know it exists for python but it's own wesbite says it is not intended for production use yet. (version 0.1.post1 with no updates since 2015 on pypi). The server where the code should be is down (http://template-toolkit.org/svnweb/Template-Python)
So I opted for jinja2 which is very very similar and is widely used and maintained.

Unless I'm missing something off course.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, I'll rework that.


logger = fancylogger.getLogger()

MAILREGEX = re.compile(("([a-z0-9!#$%&'*+\/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'*+\/=?^_`"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no uppercase?

if email[0].startswith('//'):
replace = False
for ignoremail in EXAMPLEMAILS:
if ignoremail in email[0]:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix indentation

setup(
name = 'documentation-builder',
description = 'Documentation Builder for Quattor',
url='https://github.com/wdpypere/release/tree/doc_builder_2.0/src/documentation_builder',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is probably wrong

"""Check if a repository mapping is valid."""
logger.info("Checking repository map.")
if repository_map is None:
logger.error("Repository map is empty.")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Distinguish between the two possible error states so the person reading the error logs knows which case was hit: None or []

"""Run several cleaners on the content we get from perl files."""
for source, content in markdown.iteritems():
if not source.endswith('.pan'):
if cleanup_options['remove_emails']:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the functions and keys match, you can do

for fn in [...]:
   if cleanup_options[fn]:
       content = globals()[fn](content)

@wdpypere
Copy link
Contributor Author

wdpypere commented Jan 5, 2017

@stdweird all remarks adressed, this now works with a .docbuilder.cfg in the repositories. I'll open PR's to them to add these. Maven-tools now works as well.

Copy link
Member

@jrha jrha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@stdweird stdweird merged commit 92cca33 into quattor:master Feb 16, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

quattorpoddoc: next version
4 participants