Add generic output formatter interface. #388

tsroten · 2017-03-29T03:29:21Z

Description

Adds a generic OutputFormatter class with all of tabulate's output formats.
Moves expanded output to the output formatter.
Adds terminaltables formats.

Checklist

I've added this contribution to the changelog.md.
I've added my name to the AUTHORS file (or it's already there).

tsroten · 2017-03-29T03:30:58Z

@meeuw @amjith This includes tabulate's formats, but not terminaltables. It also doesn't include any of @meeuw's handling of decimals as strings.

This is just a start, please feel free to edit/push changes as needed. The tests need some work as well...

tsroten · 2017-03-29T04:39:43Z

I believe @meeuw's number-as-string is working now in this PR since tabulate has disable_numparse, which seems like what we want here.

amjith · 2017-03-29T14:48:01Z

This is an excellent refactor. I love seeing the progression of changes as I step through the commits.

I have a few questions that I'll sprinkle throughout the diffs. Nicely done! 👍

amjith · 2017-03-29T14:52:40Z

mycli/main.py

            for line in output:
                click.echo(line, nl=new_line)

+    def format_output(self, title, cur, headers, status, expanded=False,
+                      max_width=None):
+        table_format = 'expanded' if expanded else self.table_format


We used to store the table format on the Mycli instance as self.table_format. Now that we have OutputFormat, we should place the table_format on the instance of OutputFormat. MyCli class shouldn't be in the business of holding state for other objects.

When we want to override the table_format based on the width, we'll simply overwite the value in the OutputFormat instance.

That's a good point. What do you think of having an optional keyword override for table format as well. For instance:

formatter = OutputFormatter(format_name='simple') formatter.format_output(data, headers) # formats as simple formatter.format_output(data, headers, format_name='expanded') # formats as expanded

amjith · 2017-03-29T14:55:25Z

mycli/output_formatter.py

+        fkwargs.update(kwargs)
+        preprocessor = fkwargs.pop('preprocessor', None)
+        if preprocessor:
+            data = preprocessor(data, **fkwargs)


The preprocessor should get all the args passed into the format_output. Right now we're only passing in the data, but what if the preprocessor wanted to color the headers?

Great point! We should definitely include headers in the preprocessor since they are part of the output.

codecov-io · 2017-04-01T10:56:53Z

Codecov Report

❗ No coverage uploaded for pull request base (master@a380707). Click here to learn what that means.
The diff coverage is 78.28%.

@@            Coverage Diff            @@
##             master     #388   +/-   ##
=========================================
  Coverage          ?   72.68%           
=========================================
  Files             ?       29           
  Lines             ?     2544           
  Branches          ?        0           
=========================================
  Hits              ?     1849           
  Misses            ?      695           
  Partials          ?        0

Impacted Files	Coverage Δ
mycli/output_formatter/expanded.py	`100% <100%> (ø)`
mycli/completion_refresher.py	`94.73% <100%> (ø)`
mycli/sqlcompleter.py	`81.9% <100%> (ø)`
mycli/sqlexecute.py	`91.02% <100%> (ø)`
mycli/output_formatter/preprocessors.py	`100% <100%> (ø)`
mycli/packages/tabulate.py	`67.82% <65.97%> (ø)`
mycli/main.py	`61.43% <76.31%> (ø)`
mycli/output_formatter/terminaltables_adapter.py	`83.33% <83.33%> (ø)`
mycli/encodingutils.py	`76% <87.5%> (ø)`
mycli/output_formatter/tabulate_adapter.py	`91.66% <91.66%> (ø)`
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a380707...8fa96a2. Read the comment docs.

meeuw · 2017-04-01T12:27:31Z

mycli/output_formatter.py

+        self.register_output_format('expanded', expanded_table,
+                                    preprocessor=override_missing_value,
+                                    missing_value='<null>')
+


I think register_output_format should be called outside of OutputFormatter (from MyCli.__init__ ?)

Or else we could also just directly initialize the self._output_formats dict like:

self._output_formats = { 'csv' : csv_wrapper, {preprocessor=override_missing_value, missing_value='null') ... }

And remove register_output_format

I think that's a good point.

I favor the idea of OutputFormatter setting up the dict when it's initialized. My main reason for liking this is keeping all details of formatting away from MyCli (e.g. wrappers, format names, format packages, etc.).

Hmm yes, that's true. Let's keep it this way. I was thinking of letting the user configure more output formatters at runtime but we can always refactor it by then (now it's too far fetched).

I do think there is a lot of room for new features in the future. I think you're right that we should keep it simple for now.

amjith · 2017-04-12T23:11:12Z

mycli/output_formatter/output_formatter.py

+    _output_formats = {}
+
+    def __init__(self, format_name=None):
+        """Register the supported output formats."""


This doc string is outdated. Looks like I missed it in my refactor.

amjith · 2017-04-12T23:29:07Z

Looks like we lost the ability to auto-complete on the \T command.

tsroten · 2017-04-13T02:33:51Z

@amjith Good catches. Looks like the background refresher wasn't setting the table formats when it created the SQLCompleter object. Should be fixed now.

amjith · 2017-04-13T03:11:39Z

I'm on linux (Ubuntu). When I choose 'Single' or 'fancy_grid' as the table format the output is completely garbled. Is anyone else seeing this?

It works fine with other table formats.

tsroten · 2017-04-13T13:40:06Z

@amjith Yes, that happens to me as well on MacOS. I'll try to track down what's going on later today.

meeuw · 2017-04-13T14:28:19Z

@tsroten is has to do with the less pager and escape codes. If you use cat as a pager it works.

…rmatter

amjith · 2017-04-21T14:43:16Z

Are we ready to merge this or should we wait for cli_helpers to land?

tsroten · 2017-04-21T14:44:57Z

@amjith I'm up for it being merged now. Then, when CLI Helpers is released we can update mycli to use it instead.

Add basic output formatter class functionality.

6ada4d4

amjith added the in progress label Mar 29, 2017

Add missing class.

2bc4fac

tsroten added 10 commits March 28, 2017 22:34

Make logic more readable.

3c78321

Move CSV to output formatter.

26ff1f3

Simply format output.

5da4175

Fix tsv/table logic.

8918a51

Use missing_value keyword for csv wrapper.

7b6398a

Do not use tabulate for TSV formatting.

f5a305d

Add space before expanded.

93c6306

Add preprocessor.

3a88128

Simplify missing value.

cff2ddd

Disable number parsing.

5cbdf48

tsroten added 2 commits March 28, 2017 23:44

Remove unused imports.

cced490

Remove tabulate license note.

d0f1dbf

amjith reviewed Mar 29, 2017

View reviewed changes

meeuw force-pushed the feature/output_formatter branch from 041973f to 5fb6f2a Compare April 1, 2017 10:13

meeuw added 3 commits April 1, 2017 12:53

Fix tests/test_sqlexecute unit tests

369b5b0

fix test_main unit tests

ae3712f

use str for python2 delimiter

3dbe5c0

meeuw force-pushed the feature/output_formatter branch from e2aa7e0 to 3dbe5c0 Compare April 1, 2017 10:54

meeuw reviewed Apr 1, 2017

View reviewed changes

meeuw force-pushed the feature/output_formatter branch from 09cd34f to 152a16c Compare April 1, 2017 14:28

add docstring to bytes_to_unicode function

434aea0

meeuw force-pushed the feature/output_formatter branch from 152a16c to 434aea0 Compare April 1, 2017 14:37

Add headers to preprocessor.

be0e2ed

tsroten force-pushed the feature/output_formatter branch from 0ab3327 to 359044c Compare April 12, 2017 20:47

tsroten added 3 commits April 12, 2017 16:06

Add output format changes to changelog.

13991b1

Add additional table formats to myclirc.

33e5c26

Don't import quote_whitespaces since it's not used.

f06451c

amjith reviewed Apr 12, 2017

View reviewed changes

tsroten added 3 commits April 12, 2017 21:11

Fix outdated docstring.

8f70b12

Make SQLCompleter options pass through refresher.

fd861d4

Pep8radius fixes.

81c5496

Fix fancy_grid table format.

9a23584

tsroten and others added 5 commits April 13, 2017 13:40

Remove single terminaltable format since it's causing problems.

4b0de47

Use constant for missing value default.

cc76dee

Make registering a format not require kwargs.

39c4aec

don't import as

9c70ce1

Merge remote-tracking branch 'upstream/master' into feature/output_fo…

cdbc7cd

…rmatter

meeuw force-pushed the feature/output_formatter branch from 8696bf7 to cdbc7cd Compare April 15, 2017 15:23

meeuw added 2 commits April 15, 2017 18:33

rename all adapter functions to adapter

a5f11e6

don't import as (also for tests/test_output_formatter.py)

8fa96a2

meeuw force-pushed the feature/output_formatter branch from 48b1daa to 8fa96a2 Compare April 15, 2017 19:22

amjith merged commit 818f5cd into master Apr 21, 2017

amjith removed the in progress label Apr 21, 2017

amjith deleted the feature/output_formatter branch April 21, 2017 16:41

This was referenced May 1, 2017

python-tabulate out dated / forked #384

Closed

Fixed precision decimals are not outputted correctly. #375

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add generic output formatter interface. #388

Add generic output formatter interface. #388

tsroten commented Mar 29, 2017 •

edited

Loading

tsroten commented Mar 29, 2017 •

edited

Loading

tsroten commented Mar 29, 2017

amjith commented Mar 29, 2017

amjith Mar 29, 2017

tsroten Apr 1, 2017

amjith Mar 29, 2017

tsroten Apr 1, 2017

codecov-io commented Apr 1, 2017 •

edited

Loading

meeuw Apr 1, 2017

tsroten Apr 1, 2017

meeuw Apr 1, 2017

tsroten Apr 1, 2017

amjith Apr 12, 2017

amjith commented Apr 12, 2017

tsroten commented Apr 13, 2017

amjith commented Apr 13, 2017

tsroten commented Apr 13, 2017

meeuw commented Apr 13, 2017

amjith commented Apr 21, 2017

tsroten commented Apr 21, 2017

Add generic output formatter interface. #388

Add generic output formatter interface. #388

Conversation

tsroten commented Mar 29, 2017 • edited Loading

Description

Checklist

tsroten commented Mar 29, 2017 • edited Loading

tsroten commented Mar 29, 2017

amjith commented Mar 29, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-io commented Apr 1, 2017 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

amjith commented Apr 12, 2017

tsroten commented Apr 13, 2017

amjith commented Apr 13, 2017

tsroten commented Apr 13, 2017

meeuw commented Apr 13, 2017

amjith commented Apr 21, 2017

tsroten commented Apr 21, 2017

tsroten commented Mar 29, 2017 •

edited

Loading

tsroten commented Mar 29, 2017 •

edited

Loading

codecov-io commented Apr 1, 2017 •

edited

Loading