Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[v0.2.1] Sort direction, minor fixes #23

Merged
merged 2 commits into from
Nov 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 18 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ locc [OPTIONS...] <path>

## Output

Examples of output (may be out of date until release):
Examples of output:

Linux kernel (36 million lines counted)

Expand Down Expand Up @@ -54,34 +54,29 @@ sys 0m1.076s
This repo (v0.1)

```
$ ./locc-cli -vpe=ext .
$ out/ubsan/cli/Debug/locc-cli -vpe=ext --sort-column=Comments --sort-ascend
params:
sort by :
exclude : ext
grammars :
threads : 16
no progress : true
verbose : true
path : .
sort by : Comments
exclude : ext
grammars :
threads : 16
sort ascend : true
no progress : true
verbose : true
path : .


-----------------------------------------------------------
| File Type | Files | Lines | Code | Comments | Empty |
-----------------------------------------------------------
| C++ | 17 | 1,262 | 1,068 | 16 | 178 |
| JSON | 3 | 742 | 739 | 3 | 0 |
| C++ header | 19 | 447 | 344 | 5 | 98 |
| CMake script | 4 | 192 | 160 | 0 | 32 |
| Markdown | 1 | 128 | 96 | 0 | 32 |
| Plain Text | 1 | 2 | 0 | 0 | 0 |
| Total | 45 | 2,773 | 2,407 | 24 | 340 |
| Markdown | 1 | 126 | 93 | 0 | 33 |
| JSON | 3 | 634 | 631 | 3 | 0 |
| C++ header | 19 | 448 | 345 | 5 | 98 |
| C++ | 17 | 1,279 | 1,083 | 16 | 180 |
| Total | 44 | 2,679 | 2,312 | 24 | 343 |
-----------------------------------------------------------


real 0m0.013s
user 0m0.011s
sys 0m0.006s

```

## Building
Expand Down Expand Up @@ -116,7 +111,9 @@ locc-lib clap
ktask djson
```

`djson` deals with parsing JSONs and is used for grammar IO. `ktask` is a lightweight task execution queue, `locc-lib` enqueues a counter task for each file using it. This _counter_ task is internal to `locc-lib` and not exposed to users. Users utilize a single `locc::LineCounter` wrapper task, which reports progress / status, and supports waiting (blocking) until all individual counters have completed (or if any counter has been dropped). It can be used by direct ownership, or shared ownership through `locc::Instance`, which owns a `ktask::Queue`. Using the instance is convenient when the queue isn't needed for any other tasks, otherwise own the queue and line counters yourself. As is evident, multiple line counters can share the same task queue, which can also have other tasks interleaved.
`djson` deals with parsing JSONs and is used for grammar IO. `ktask` is a lightweight task execution queue, `locc-lib` enqueues a counter task for each file using it. This _counter_ task is internal to `locc-lib` and not exposed to users. Users utilize a single `locc::LineCounter` wrapper task, which reports progress / status, and supports waiting (blocking) until all individual counters have completed (or if any counter has been dropped). It can be used directly or through a `locc::Instance`. Using the instance is convenient when all line counters use the same grammars and file filters. As is evident, multiple line counters can share the same task queue, which can also have other tasks interleaved.

`clap` is a command line argument parser, which `locc-cli` uses.

* [clap](https://github.com/karnkaul/clap)
* [ktask](https://github.com/karnkaul/ktask)
Expand Down
5 changes: 3 additions & 2 deletions cli/src/app.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,11 @@ auto App::run(int const argc, char const* const* argv) -> int {
auto default_grammars = false;

auto const args = std::array{
clap::option(m_params.sort_by, "s,sort", "sort by COLUMN_NAME (default Lines)"),
clap::option(m_params.sort_column, "c,sort-column", "sort by COLUMN_NAME (default Lines)"),
clap::option(m_params.exclude_patterns, "e,exclude", "comma-separated exclude list (will not be counted)"),
clap::option(m_params.grammars_json, "g,grammars", "path to JSON containing an array of custom grammars"),
clap::option(m_params.thread_count, "t,threads", "number of threads to use"),
clap::flag(m_params.sort_ascending, "a,sort-ascend", "sort in ascending order"),
clap::flag(m_params.no_progress, "p,no-progress", "do not print progress while counting"),
clap::flag(default_grammars, "default-grammars", "output default grammars as JSON and exit"),
clap::flag(m_params.verbose, "v,verbose", "verbose output"),
Expand Down Expand Up @@ -71,7 +72,7 @@ auto App::execute() -> int {
}

auto csv = Csv{m_params.exclude_patterns};
for (auto pattern = std::string_view{}; csv(pattern);) { m_filter.skip_patterns.push_back(pattern); }
for (auto pattern = std::string_view{}; csv(pattern);) { m_filter.exclude_patterns.push_back(pattern); }

Counter{m_params, std::move(ici)}.run();
return EXIT_SUCCESS;
Expand Down
24 changes: 13 additions & 11 deletions cli/src/counter.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ void Counter::run() {
if (m_params.verbose) { print_params(); }

auto line_counter = m_locc.start_count(m_params.path);
if (!line_counter) { return; }
if (!line_counter || line_counter->get_status() == LineCounter::Status::None) { return; }

if (!m_params.no_progress) {
std::println();
Expand All @@ -47,11 +47,12 @@ void Counter::run() {
}

auto rows = line_counter->to_rows();
auto const sort_by = to_metric(m_params.sort_by);
auto const sort_by = to_metric(m_params.sort_column);
auto const sort_dir = m_params.sort_ascending ? SortDir::Ascending : SortDir::Descending;
if (sort_by == by_file_type) {
sort_by_file_type(rows);
sort_by_file_type(rows, sort_dir);
} else {
sort_by_metric(rows, sort_by);
sort_by_metric(rows, sort_by, sort_dir);
}
rows.push_back(Row::aggregate(rows));

Expand All @@ -61,13 +62,14 @@ void Counter::run() {
void Counter::print_params() const {
std::println("params:");
auto const print_param = [](std::string_view left, auto const& right) { std::println(" {}: {}", left, right); };
print_param("sort by\t\t", m_params.sort_by);
print_param("exclude\t\t", m_params.exclude_patterns);
print_param("grammars\t\t", m_params.grammars_json);
print_param("threads\t\t", m_params.thread_count);
print_param("no progress\t\t", m_params.no_progress);
print_param("verbose\t\t", m_params.verbose);
print_param("path\t\t\t", m_params.path);
print_param("sort by\t", m_params.sort_column);
print_param("exclude\t", m_params.exclude_patterns);
print_param("grammars\t", m_params.grammars_json);
print_param("threads\t", m_params.thread_count);
print_param("sort ascend\t", m_params.sort_ascending);
print_param("no progress\t", m_params.no_progress);
print_param("verbose\t", m_params.verbose);
print_param("path\t\t", m_params.path);
std::println();
}

Expand Down
3 changes: 2 additions & 1 deletion cli/src/params.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,12 @@

namespace locc::cli {
struct Params {
std::string_view sort_by{};
std::string_view sort_column{};
std::string_view exclude_patterns{};
std::string_view grammars_json{};
std::uint8_t thread_count{};

bool sort_ascending{};
bool no_progress{};
bool verbose{};

Expand Down
2 changes: 1 addition & 1 deletion lib/include/locc/file_filter.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ class IFileFilter {
};

struct DefaultFileFilter : IFileFilter {
std::vector<std::string_view> skip_patterns{"build/", "out/", ".cache/", ".git/"};
std::vector<std::string_view> exclude_patterns{"build/", "out/", ".cache/", ".git/"};

[[nodiscard]] auto should_count(std::string_view path) const -> bool override;

Expand Down
4 changes: 1 addition & 3 deletions lib/include/locc/instance.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,12 @@ class Instance {

explicit Instance(ktask::Queue& queue, CreateInfo create_info = {});

[[nodiscard]] auto start_count(std::string_view path) -> std::shared_ptr<LineCounter>;
[[nodiscard]] auto start_count(std::string_view path) -> std::unique_ptr<LineCounter>;

private:
ktask::Queue* m_queue;

std::vector<Grammar> m_grammars;
IFileFilter const* m_filter;

std::vector<std::shared_ptr<LineCounter>> m_line_counters{};
};
} // namespace locc
6 changes: 4 additions & 2 deletions lib/include/locc/row.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
#include <string>

namespace locc {
enum class SortDir : std::uint8_t { Descending, Ascending };

struct Row {
std::string file_type{};
LineCount line_count{};
Expand All @@ -13,6 +15,6 @@ struct Row {

[[nodiscard]] auto beautify(std::uint64_t num) -> std::string;

void sort_by_metric(std::span<Row> rows, LineCount::Metric metric);
void sort_by_file_type(std::span<Row> rows);
void sort_by_metric(std::span<Row> rows, LineCount::Metric metric, SortDir dir = SortDir::Descending);
void sort_by_file_type(std::span<Row> rows, SortDir dir = SortDir::Descending);
} // namespace locc
2 changes: 1 addition & 1 deletion lib/src/file_filter.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

namespace locc {
auto DefaultFileFilter::should_count(std::string_view const path) const -> bool {
return !std::ranges::any_of(skip_patterns, [path](std::string_view const s) { return path.contains(s); });
return !std::ranges::any_of(exclude_patterns, [path](std::string_view const s) { return path.contains(s); });
}

auto DefaultFileFilter::get_instance() -> DefaultFileFilter const& {
Expand Down
6 changes: 2 additions & 4 deletions lib/src/instance.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,15 @@ namespace locc {
Instance::Instance(ktask::Queue& queue, CreateInfo create_info)
: m_queue(&queue), m_grammars(std::move(create_info.grammars)), m_filter(create_info.file_filter) {}

auto Instance::start_count(std::string_view const path) -> std::shared_ptr<LineCounter> {
auto Instance::start_count(std::string_view const path) -> std::unique_ptr<LineCounter> {
if (m_grammars.empty()) { return {}; }
auto const query = Query{
.grammars = m_grammars,
.path = path,
.filter = m_filter,
};
auto ret = std::make_shared<LineCounter>(*m_queue, query);
m_line_counters.push_back(ret);
auto ret = std::make_unique<LineCounter>(*m_queue, query);
m_queue->enqueue(*ret);
std::erase_if(m_line_counters, [](auto const& counter) { return !counter->is_busy(); });
return ret;
}
} // namespace locc
32 changes: 23 additions & 9 deletions lib/src/row.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,23 @@
#include <charconv>

namespace locc {
namespace {
template <template <typename> typename Comp>
void do_sort_by_metric(std::span<Row> rows, LineCount::Metric metric) {
metric = std::clamp(metric, LineCount::Metric{}, LineCount::COUNT_);
auto const pred = [metric](Row const& a, Row const& b) {
// NOLINTNEXTLINE(cppcoreguidelines-pro-bounds-constant-array-index)
return Comp{}(a.line_count.metrics[metric], b.line_count.metrics[metric]);
};
std::ranges::sort(rows, pred);
}

template <template <typename> typename Comp>
void do_sort_by_file_type(std::span<Row> rows) {
std::ranges::sort(rows, [](Row const& a, Row const& b) { return Comp{}(a.file_type, b.file_type); });
}
} // namespace

auto Row::aggregate(std::span<Row const> rows, std::string file_type) -> Row {
auto ret = Row{.file_type = std::move(file_type)};
for (auto const& in : rows) { ret.line_count += in.line_count; }
Expand All @@ -27,15 +44,12 @@ auto locc::beautify(std::uint64_t const num) -> std::string {
return ret;
}

void locc::sort_by_metric(std::span<Row> rows, LineCount::Metric metric) {
metric = std::clamp(metric, LineCount::Metric{}, LineCount::COUNT_);
auto const pred = [metric](Row const& a, Row const& b) {
// NOLINTNEXTLINE(cppcoreguidelines-pro-bounds-constant-array-index)
return a.line_count.metrics[metric] > b.line_count.metrics[metric];
};
std::ranges::sort(rows, pred);
void locc::sort_by_metric(std::span<Row> rows, LineCount::Metric metric, SortDir const sort_dir) {
if (sort_dir == SortDir::Ascending) { return do_sort_by_metric<std::less>(rows, metric); }
return do_sort_by_metric<std::greater>(rows, metric);
}

void locc::sort_by_file_type(std::span<Row> rows) {
std::ranges::sort(rows, [](Row const& a, Row const& b) { return a.file_type < b.file_type; });
void locc::sort_by_file_type(std::span<Row> rows, SortDir const sort_dir) {
if (sort_dir == SortDir::Ascending) { return do_sort_by_file_type<std::less>(rows); }
return do_sort_by_file_type<std::greater>(rows);
}