Skip to content

Commit

Permalink
.
Browse files Browse the repository at this point in the history
  • Loading branch information
traversc committed Sep 3, 2024
1 parent 09b85cd commit bd10a79
Show file tree
Hide file tree
Showing 6 changed files with 18 additions and 11 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,8 @@ package. It is not compatible with the original `qs` format.
install.packages("qs2")
```

On Mac or Linux, you can enable multi-threading by compile from source.
It is enabled by default on Windows.
On Mac or Linux, you can enable multi-threading by compiling from
source. It is enabled by default on Windows.

``` r
remotes::install_cran("qs2", type = "source", configure.args = " --with-TBB --with-simd=AVX2")
Expand All @@ -42,7 +42,7 @@ framework via the `RcppParallel` package.

## Converting qs2 to RDS

Because the `qs2` format directly uses R serialization you can convert
Because the `qs2` format directly uses R serialization, you can convert
it to RDS and vice versa.

``` r
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ if(standalone) {
a(OUT) <- c("#include <Rcpp.h>", "using namespace Rcpp;")
} else {
a(OUT) <- c("#ifndef _QS2_XGBOOST_BLOCKSHUFFLE_MODEL_H", "#define _QS2_XGBOOST_BLOCKSHUFFLE_MODEL_H")
a(OUT) <- c("#include <array>", "#include <memory>", "#include <vector>")
a(OUT) <- c("#include <array>", "#include <memory>", "#include <vector>", '#include "io_common.h"')
}

a(OUT) <- '
Expand All @@ -65,7 +65,7 @@ struct XgTree {
const std::vector<uint8_t> & split_indices,
const std::vector<uint16_t> & left_children,
const std::vector<uint16_t> & right_children) :
nodes(std::make_unique<XgNode[]>(split_conditions.size())) {
nodes(MAKE_UNIQUE_BLOCK_CUSTOM(XgNode, split_conditions.size())) {
for(size_t i = 0; i < split_conditions.size(); ++i) {
nodes[i].split_idx = split_indices[i];
nodes[i].split_cond = split_conditions[i];
Expand Down
8 changes: 7 additions & 1 deletion src/io/io_common.h
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,12 @@ static constexpr uint32_t SHUFFLE_MASK = (1ULL << 31);
#define MAKE_UNIQUE_BLOCK(SIZE) new char[SIZE]
#endif

#if __cplusplus >= 201402L // Check for C++14 or above
#define MAKE_UNIQUE_BLOCK_CUSTOM(_TYPE_, SIZE) std::make_unique<_TYPE_[]>(SIZE)
#else
#define MAKE_UNIQUE_BLOCK_CUSTOM(_TYPE_, SIZE) new _TYPE_[SIZE]
#endif

// std::make_shared on an array requires c++20 (yes it is true)
// If you try to use make_shared on c++17 or lower it won't compile or will segfault.
#if __cplusplus >= 202002L // Check for C++20 or above
Expand Down Expand Up @@ -82,7 +88,7 @@ inline constexpr unsigned char operator "" _u8(unsigned long long arg) noexcept
return static_cast<uint8_t>(arg);
}

#define QS_MT_SERIALIZATION_DEBUG
// #define QS_MT_SERIALIZATION_DEBUG
#if defined(QS_MT_SERIALIZATION_DEBUG)
#include <iostream>
#include <sstream>
Expand Down
3 changes: 2 additions & 1 deletion src/io/xgboost_blockshuffle_model.h
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
#include <array>
#include <memory>
#include <vector>
#include "io_common.h"


namespace XgboostBlockshuffleModel {
Expand All @@ -24,7 +25,7 @@ struct XgTree {
const std::vector<uint8_t> & split_indices,
const std::vector<uint16_t> & left_children,
const std::vector<uint16_t> & right_children) :
nodes(std::make_unique<XgNode[]>(split_conditions.size())) {
nodes(MAKE_UNIQUE_BLOCK_CUSTOM(XgNode, split_conditions.size())) {
for(size_t i = 0; i < split_conditions.size(); ++i) {
nodes[i].split_idx = split_indices[i];
nodes[i].split_cond = split_conditions[i];
Expand Down
4 changes: 2 additions & 2 deletions vignettes/vignette.html
Original file line number Diff line number Diff line change
Expand Up @@ -335,13 +335,13 @@ <h1 class="title toc-ignore">qs2</h1>
<div id="installation" class="section level2">
<h2>Installation</h2>
<div class="sourceCode" id="cb2"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb2-1" title="1"><span class="kw">install.packages</span>(<span class="st">&quot;qs2&quot;</span>)</a></code></pre></div>
<p>On Mac or Linux, you can enable multi-threading by compile from source. It is enabled by default on Windows.</p>
<p>On Mac or Linux, you can enable multi-threading by compiling from source. It is enabled by default on Windows.</p>
<div class="sourceCode" id="cb3"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb3-1" title="1">remotes<span class="op">::</span><span class="kw">install_cran</span>(<span class="st">&quot;qs2&quot;</span>, <span class="dt">type =</span> <span class="st">&quot;source&quot;</span>, <span class="dt">configure.args =</span> <span class="st">&quot; --with-TBB --with-simd=AVX2&quot;</span>)</a></code></pre></div>
<p>Multi-threading in <code>qs2</code> uses the <code>Intel Thread Building Blocks</code> framework via the <code>RcppParallel</code> package.</p>
</div>
<div id="converting-qs2-to-rds" class="section level2">
<h2>Converting qs2 to RDS</h2>
<p>Because the <code>qs2</code> format directly uses R serialization you can convert it to RDS and vice versa.</p>
<p>Because the <code>qs2</code> format directly uses R serialization, you can convert it to RDS and vice versa.</p>
<div class="sourceCode" id="cb4"><pre class="sourceCode r"><code class="sourceCode r"><a class="sourceLine" id="cb4-1" title="1">file_qs2 &lt;-<span class="st"> </span><span class="kw">tempfile</span>(<span class="dt">fileext =</span> <span class="st">&quot;.qs2&quot;</span>)</a>
<a class="sourceLine" id="cb4-2" title="2">file_rds &lt;-<span class="st"> </span><span class="kw">tempfile</span>(<span class="dt">fileext =</span> <span class="st">&quot;.RDS&quot;</span>)</a>
<a class="sourceLine" id="cb4-3" title="3">x &lt;-<span class="st"> </span><span class="kw">runif</span>(<span class="fl">1e6</span>)</a>
Expand Down
4 changes: 2 additions & 2 deletions vignettes/vignette.rmd
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ Use the file extension `qs2` to distinguish it from the original `qs` package. I
install.packages("qs2")
```

On Mac or Linux, you can enable multi-threading by compile from source. It is enabled by default on Windows.
On Mac or Linux, you can enable multi-threading by compiling from source. It is enabled by default on Windows.

```{r eval=FALSE}
remotes::install_cran("qs2", type = "source", configure.args = " --with-TBB --with-simd=AVX2")
Expand All @@ -55,7 +55,7 @@ Multi-threading in `qs2` uses the `Intel Thread Building Blocks` framework via t

## Converting qs2 to RDS

Because the `qs2` format directly uses R serialization you can convert it to RDS and vice versa.
Because the `qs2` format directly uses R serialization, you can convert it to RDS and vice versa.

```{r eval=FALSE}
file_qs2 <- tempfile(fileext = ".qs2")
Expand Down

0 comments on commit bd10a79

Please sign in to comment.