01-chapter1.Rmd

# Rematching on-the-fly: Sequential Rematched Randomization and a case for covariate-adjusted randomization

```{r echo=FALSE,include=FALSE,fig.keep=FALSE,warning=FALSE,message=FALSE}
load("~/Dropbox/statistics/work/vandy/RandomizationMethods/journal-clinical trials/latex/results.RData")
library(rms)
library(xtable)
library(tableone)
library(kableExtra)
```

## Introduction
Randomization eliminates systematic confounding and has been considered the “gold standard” for clinical trials since World War II[@Bothwell:2016bc].  And yet, the most common approaches to randomization --- Block and Stratified Randomization --- are just as old [@Peirce:1884wra;@Hill:1952hc;@Armitage:1982ji].  They are limited in their ability to control the balance of baseline covariates.  Covariate-adjusted randomization, which includes Stratified Block Randomization, eliminates from the randomization space singular randomizations with poor baseline covariate balance and consequently reduces the uncertainty in permutation-based inference.  It also increases the efficiency of model-based inference by orthogonalizing baseline covariates with treatment assignment.

Balance is an important aim for clinical trials.  Poor chance imbalances on key baseline covariates can bring to question the face-validity of a randomized trial[@Rosenberger:2008cm;@LeylandJones:2003kt] especially when estimating the Population Average Treatment Effect from the Sample Average Treatment Effect. Also, greater covariate balance allows trialists to estimate heterogenous treatment effect across key subgroups. Trials with secondary aims for personalizing medicine to key subgroups rely upon overall covariate balance[@DienerWest:1989uq;@Fu:2016jy]. To reduce chance imbalances, the trialist may turn to covariate-adjusted randomization (the focus of this paper) and / or fit a model that adjusts for chance imbalances[@Berchialla:2018bk].  With model-based inference, the trialist must convince the clinical community their model is fully transparent and sufficiently adequate[@Freedman:2008eq; @Freedman:2008em; @Lin:2013jh].

In non-sequential trials, where all patients or clusters are known before randomization, covariate-adjusted randomization randomizes within optimal partitions of the patients.  Common and novel methods (and the imbalance measure they optimize) include: Stratified Block Randomization (exact covariate matches)[@fisher1935design], Matched Randomization (overall sum of paired distances from a distance matrix)[@Greevy:2004ke], Rerandomization (any imbalance measure though commonly overall difference in covariate distances)[@Morgan:2012iq], Propensity-Constrained Randomization (overall difference in propensity score variability)[@Loux:2014bu], and Kernel Randomization (a linear or non-linear function of covariate differences) [@Kallus:2018um].  Stratified Block Randomization randomizes within exact matches of categorized baseline covariate levels whereas the other methods randomize within near matches of continuous and categorical baseline covariates.  Exact matching is frequently limited in the number of matching baseline covariate levels before overwhelming the sample size.   Though still under development, some theoretical and empirical evidence points to Kernel Randomization as the preferred method for greatest balance[@Kallus:2018um].

With sequential enrollment, the full covariate pattern and optimal partitions are unknown until the end of the study.  Many of the non-sequential methods have extensions for sequential enrollment yet lose some optimization compared to non-sequential randomization.  Minimization handles the sequential optimization problem by directly allocating treatments to achieve optimal balance on specified imbalance criteria[@Taves:1974hn].  Randomization occurs when all treatment would have the same impact on minimizing the imbalance criteria.  Without randomization, Minimization schemes may remain subject to systematic confounding.  To relax deterministic allocation, treatment may instead be allocated with a biased coin favoring the optimal treatment per the imbalance criteria [@Pocock:1975wd; @Atkinson:1982kt].  A common covariate-adjusted biased coin method includes Urn Randomization[@Wei:1988if].  Of the remaining non-sequential randomization schemes, Sequential Matched Randomization and Sequential Rerandomization set a tuning parameter to sequentially optimize balance according to their imbalance measure[@Kapelner:2014cu;@Quan:SeqRerand].

This work achieves two purposes.  First, it extends Sequential Matched Randomization to recover some of the optimal balance achieved in single batch Matched Randomization (when all patients or clusters are known prior to randomization).  The extensions allow (1) matches to rematch if a better mate enters the study, (2) the tuning parameter for optimal matches to adjust dynamically according to the number of current matches and covariate distribution, and (3) patients to enroll in blocks. Second, through a two-arm case study (n = 512 patients) it re-emphasizes the value of sequential covariate-adjusted methods, as a whole, compared to Block Randomization.  Sequential allocation methods considered include Block Randomization, Stratified Block Randomization, Urn Randomization, Sequential Matched Randomization, Atkinson’s Minimization with and without a biased coin, and Begg and Iglewicz’s Minimization.  We do not include Sequential Rerandomization as it requires determining an optimal threshold.  

## Notation

We denote a study as having enrolled $i = 1, \dots,$ to $N$ patients throughout $b = 1,\dots,$ to $M$ batches of sequential enrollment. At the $b^{th}$ batch of enrolling patients, denote the set of unmatched patients as $U_b$ and the number of patients in the set as $||U_b||$.  Let $||R_b||$ be the number of expected remaining study entrants.  In general, covariate-adjusted randomizations are adjusted to $p$ baseline covariates in which a categorical covariate having $q$ levels is coded by $q-1$ dummy variables.

In the simulation section, we generate $j = 1, \dots,$ to 20,000 randomization schedules for each allocation scheme.

## Sequential Matching and Sequential Rematching

Sequential Matched Randomization uses the matching on-the-fly algorithm published elsewhere[@Kapelner:2014cu] yet is worth summarizing to then extend. When the first set of patients individually enroll, they are randomized to either treatment or control and form a “reservoir” of patients who have been randomized but, as of yet, have not matched with any other patient. Once a reservoir of pre-specified size has built up, subsequent enrolling patients are compared to reservoir members on baseline covariates using a distance matrix. The entering patient becomes mates with their best reservoir match if they meet a pre-specified distance threshold of similarity, and the entering patient receives the opposite treatment randomized to their mate. Both are excluded from the reservoir of potential mates. If, however, the best reservoir match is not similar enough, the entering patient is randomized and joins the reservoir awaiting a mate. By the end of the study, some patients may not have found a mate, and treatment allocation may be imbalanced. Hereafter, we’ll refer to the matching-on-the-fly algorithm as Sequential Matching.

  Common practice uses Mahalanobis Distance and builds the initial reservoir to p+2 patients. This distance matrix reduces to euclidean distances when covariates are independent, and p+2 observations are required before distances may be uniquely estimated. Current literature suggests setting the pre-specified threshold as a quantile from the F(p, i - p) distribution[@Kapelner:2014cu]; justification lies in the fact that distances of normal covariates, when appropriately scaled, asymptotically follow the F(p, i-p) distribution.

  Throughout Sequential Matching, a patient may match before their better matching mate enrolls. We allow mates to break and rematch so long as patients do not match within their treatment group. For example, in a two arm study, controls are not allowed to match with controls and treatments to treatments. We refer to this re-matching on-the-fly algorithm as Sequential Rematching and the randomization scheme as Sequential Rematched Randomization.

## Dynamic and Empirical Threshold
Sequential Matching, as is, uses a fixed quantile of the F-distribution; we formalize the use of a dynamic threshold from an empirically-estimated distribution of randomly matched distances. Lacking omniscience, an improperly selected fixed similarity threshold may be problematic. An overly strict fixed threshold would yield no matches and in turn may be no better than Complete Randomization. In contrast, an overly relaxed threshold would degenerate to a block-two randomization scheme vulnerable to subversion bias. Our proposed dynamic threshold reflects the chance of matching an existing reservoir member out of all potential mates including those yet to enroll. More formally, this is the proportion $Q_b$ where

$$
Q_b = \frac{||  U_b || - 1}{ ||U_b|| + ||R_b|| - 1}.
$$
Although the number of possible mates in Rematching is the whole set of entrants, we develop $Q_b$ for use with both Sequential Matching and Sequential Rematching.

Since not all covariates are normally distributed, we empirically estimate a reference null distribution, $F$, as suggested in the matching on-the-fly paper[@Kapelner:2014cu]. To estimate $F$ at the $b^{th}$ batch of enrolling patients ($F_b$), a random set of $i/2$ matches are bootstrap sampled from the upper- (or lower-) triangle of the distance matrix.  The dynamic threshold is the averaged $Q_b$ percentile across bootstrap samples of $F_b$. To achieve equal treatment allocation, the threshold for matches is removed when the reservoir size is the same or less than the number of patients left to enroll.

$$
Threshold_b = \begin{cases} \hat{F}_b^{-1} (Q_b) &  || U_b || < || R_b || \\
                    \text{best match(es)}      &  || U_b || \ge || R_b || \\
      \end{cases}
$$

The threshold may also be removed to keep the reservoir size within a specified maximum tolerated imbalance [@Berger:2003im].

## Batch Entry
As is, Sequential Matching requires matching patients one at a time.  However, trials will allocate treatments in batches of various enrollment sizes. As when all patients are known at the study outset, batch enrollment increases the chance of finding a better mate. We extend Sequential Matching to allow for batches of multiple patients using an “optimal” algorithm; mates are collectively determined based upon the set of matches that yield the smallest sum of distances. The R package nbpMatching easily finds optimal mates.


## Case Study: REACH Trial
### Context, Data Preparation, and Simulation Set up
  The Rapid Education/Encouragement And Communications for Health (REACH) randomized clinical trial[@Nelson:2018bw] provides text message-delivered diabetes support for 12 months to help diabetic patients manage glycemic control (as measured by Hemoglobin A1c) and adhere to treatment medication. Patients are randomized into one of three treatment arms with a 2:1:1 allocation ratio: no text message, text message, text message and monthly phone coaching.  Now with complete enrollment, 512 patients are currently being followed up through 3, 6, 12, and 15 months.

  At baseline, clinical and demographics covariates were collected that may be associated with 12 month Hemoglobin A1c, including baseline Hemoglobin A1c, age at enrollment, gender, years of education, years of diabetes, race / ethnicity, medication type, income, and type of insurance.  Refer to supplement for the overall distribution of each baseline covariate.
  
  Randomization often occurred before all baseline covariates could be collected. Most notably, baseline Hemoglobin A1c generally took additional time to process. In this study, clinical coordinators enrolled patients and sent baseline data to the Data Coordinating Center for weekly batch randomization. Though REACH follow-up continues, all baseline covariates have been obtained, and 418 patients have recorded three month Hemoglobin A1c.

### Simulations
With complete baseline covariates, we simulated the ability of various treatment allocation schemes to achieve balance among baseline covariates among treatment groups and compared the efficiency in estimating predicted three month Hemoglobin A1c. Contender allocation schemes focused on Block Randomization, Stratified Block Randomization, Urn$(0,\beta)$ Randomization, single batch Matched Randomization, Sequential Matched Randomization without the proposed extensions (similar to current literature), Sequential Matched Randomization with the dynamic threshold, Atkinson’s Minimization Algorithm with a 2/3 biased coin (Efron 1971) and with deterministic allocation, and Begg and Iglewicz Minimization.  The minimization schemes aim to reduce the standard error in estimating the treatment effect.

  We carried out Block and Stratified Block Randomization using a block size of two. In practice, blocks of size two are ill-advised for increasing the risk of subversion[@Berger:2003im] but provides in simulation the greatest chance of equal treatment allocation especially when stratifying with many levels. As a point of reference we included single batch Matched Randomization as the optimal matching when all patients are known prior to randomization; and, to compare against current Sequential Matched Randomization literature, we included Sequential Matched Randomization with a fixed $20^{th}$ percentile threshold of the F(p, i-p) distribution. Atkinson's Algorithm determines the most impactful treatment for reducing the estimated treatment variability of a pre-specified model, which we specified as conditioning on all categorical covariates and first through third degree polynomials of continuous covariates. When there were fewer observations than model degrees of freedom, we determined the optimal treatment for the pre-specified model using the generalized inverse[@Senn:2010bg].

For a realistic comparison to Stratified Block Randomization, each covariate-adjusted allocation was adjusted to site and the most predictive baseline covariates of three month Hemoglobin A1c --- baseline Hemoglobin A1c, medication type, and time since diabetes diagnosis (see supplement for how these were determined).  And, though impractical for Stratified Block Randomization, each covariate-adjusted allocation scheme conditioned on all baseline covariates. Stratified Randomization, Urn(0,$\beta$), and Begg and Iglewicz Minimization require categorized covariates; for these schemes we conditioned on categorized derivations of baseline Hemoglobin A1c $(<7, 7-8, 8+)$, age $(\leq 60, >60)$, years of education $(\leq 12, >12)$, and time since diabetes diagnosis $(<10, 10+)$.

Though REACH includes three treatment arms, we simplified to two equally allocated arms --- receiving no or any text-message intervention. Missing baseline covariates and three month Hemoglobin A1c were multiply imputed using predictive meaning matching via the aregImpute function in the R rms package; and single mean and mode imputation from the multiple imputations was carried out to obtain a single complete dataset. Twenty thousand treatment allocation schedules for each scheme were simulated on the single mean imputed dataset. For each scheme’s generated 20,000 treatment schedules, we summarize with boxplots the maximum (worst-case) and average absolute Standardized Mean Difference of all baseline covariates.  We also summarize end-of-study treatment allocation difference.

To compare the efficiency of different schemes, we generated outcomes assuming a potential outcomes framework.  Twenty thousand datasets were created where a patient’s outcome under the control arm equaled predicted three month Hemoglobin A1c plus a random residual.  Under the REACH intervention arm, the outcome decreased by 0.5 (a beneficial decrease in this population).  By using predicted three month Hemoglobin A1c, we fixed the effect of baseline covariates.  The only random components in this framework are the treatment assignment and a random residual of predicted three month Hemoglobin A1c.

For the j$^{th}$ generated potential outcomes dataset, we obtained an observed study for each allocation scheme using the scheme’s j$^{th}$ generated allocation schedule.  From each observed study, we calculated the permutation-based 95% Confidence Interval Width of the Sample Average Treatment Effect (difference in observed means) and model-based 95% Confidence Interval Width of the estimated treatment effect.  The pre-specified Ordinary Least Square Model adjusted for all baseline covariates with restricted cubic splines on each continuous covariate and accounted for multiply imputed baseline covariates.  On the j$^{th}$ potential outcomes dataset, we also calculated the relative width of each scheme’s Confidence Interval Width compared to the width from Block Randomization.  Block Randomization was carried out twice for each generated dataset to compare one instance of Block Randomization to another.  From this we calculated the difference in effective sample size of each scheme relative to Block Randomization.

In the supplement, we further calculate the average size of the reservoir throughout enrollment among matched randomization methods and the expected amount of randomization occurring under each minimization scheme.



### Simulation Results
By themselves, the set of most predictive covariates, including site, are highly predictive of three-month Hemoglobin A1c (R$^2$ = `r r2redCovs`); adjusting for all baseline covariates is slightly more predictive (R$^2$ = `r r2allCovs`).  Atkinson’s Minimization only randomized the first patient and is therefore excluded from comparisons regarding permutation-based inference.  For this reason, it is also difficult to draw conclusions regarding its average performance balancing baseline covariates.

Balance of Covariates: As a point of reference and across simulations, Block Randomization’s median worst-balanced baseline covariate had an absolute Standardized Mean Difference of `r RPTblockMaxSMD["m"]` (95% Percentile Confidence Interval: `r RPTblockMaxSMD["ci"]`; Figure 1) and average absolute Standardized Mean Difference of `r RPTblockAveSMD["mci"]` (Figure 2).

Adjusting randomization to the most predictive baseline covariates yielded only trivial gain in lessoning the worst-case imbalance (maximum absolute Standardized Mean Difference; Figure 1).  Begg and Iglewicz Minimization performed best in reducing the worst-case imbalance to `r RPTbiRedMaxSMD["mci"]`.  Income level and race / ethnicity remained difficult covariates to balance (supplement).

Worst-case imbalance was better controlled when adjusting to all baseline covariates (Figure 1).  Sequenital Matched Randomization, utilizing a dynamic and empirical threshold, reduced the absolute Standardized Mean Difference to `r RPTseqtim_eMaxSMD["mci"]`.  This slightly improved upon Sequential Matched Randomization without these extensions `r RPTseqtim_fix20MaxSMD["mci"]`.  Sequential Rematched Randomization had a worst-case imbalance of `r RPTseqtdmMaxSMD["mci"]`, which was an improvement in recovering the performance of a single batched Matched Randomization (`r RPTtimMaxSMD["mci"]`).  Begg and Iglewicz Minimization performed best among the sequential allocation methods (`r RPTbiMaxSMD["mci"]`).

Overall baseline covariate imbalance (average absolute Standardized Mean Difference) improved similarly and non-trivially across allocation schemes when adjusting randomization to most predictive baseline covariates (Figure 2).  Again, greater improvement occurred when adjusting to all baseline covariates.  Adding the dynamic and empirical threshold improved the performance of Sequential Matched Randomization, and Sequential Rematched Randomization recovered much of the optimal performance of single batch Matched Randomization.  Begg and Iglewicz performed essentially as well as single batch Matched Randomization.

Improvement Upon Precision: Block Randomization achieved a treatment effect permutation-based 95% Confidence Interval width of `r RPTblockPermCIw["mci"]` (Figure 3) and a fully-adjusted (all baseline covariates) ordinary least squares 95% Confidence Interval width of `r RPTblockAdjCIw["mci"]` (Figure 4).  Relative to itself, the 95% Percentile Confidence Interval of change in effective sample size was +/- 20 (Table 1).

The greatest precision gains to permutation-based inference came from schemes adjusting for only the most predictive baseline covariates (Figure 3). An exception was Begg and Iglewicz Minimization, which performed essentially the same on median when adjusting for the most predictive versus all baseline covariates.

Excepting Urn Randomization, all other schemes that adjusted allocation to the most predictive covariates achieved a Confidence Interval width of `r rd(max(RPTredMedpermCIw[grep("VarAtk|Urn",names(RPTredMedpermCIw),invert=TRUE)]),2)` or smaller, with single batch Matched Randomization achieving on median the same efficiency as the model-based efficiency under Block Randomization (Figure 3).  Sequential Matched Randomization with the dynamic and empirical threshold yielded a permutation-based Confidence Interval width of `r RPTseqtimRed_ePermCIw["mci"]`, an improvement on Sequential Matched Randomization without extensions `r RPTseqtimRed_fix20PermCIw["mci"]`. Sequential Rematched Randomization (`r RPTseqtdmRedPermCIw["mci"]`) further recovered some of the efficiency lost from single batch Matched Randomization (`r RPTtimRedPermCIw["mci"]`). On median, Atkinson’s Algorithm with a Biased Coin yielded the greatest permutation-based precision, nearly doubling the effective sample size compared to Block Randomization. That is, Block Randomization would have needed on median `r RPTnEffPermBcVarAtkRed_Efron` additional patients to achieve the same amount of precision. Sequential Rematched Randomization increased the effective sample size by `r RPTnEffPermSeqTDMRed`. 

When adjusting randomization to all baseline covariates, efficiency gains in permutation-based inference were not as pronounced but still substantial (Figure 3).  Sequential Rematched Randomization achieved a permutation-based Confidence Interval Width of `r RPTseqtdmPermCIw["mci"]`.  Atkinson's Minimization with a Biased Coin and Begg and Iglewicz Minimization achieved the greatest permutation-based Confidence Interval Width precision, `r RPTbcVarAtkPermCIw["mci"]` and `r RPTbiPermCIw["mci"]` respectively.  On median, these were both superior to single batch Matched Randomization.

In contrast to permutation-based inference, schemes that adjusted to all baseline covariates achieved the greatest gain in precision from a linear model adjusting to all baseline covariates (Figure 4). While non-trivial, the gains relative to Block Randomization were much less pronounced.  

When adjusting to all baseline covariates, the best performing sequential enrollment schemes included Sequential Rematched Randomization, Atkinson’s Algorithm with and without a Biased Coin, and Begg and Iglewicz Minimization with Confidence Interval widths of at most, on median, `r rd(max(RPTmedAdjCIw),3)`.  Sequential Rematched Randomization yielded an effective increase in sample size of `r RPTnEffAdjSeqTDM` above and beyond the pre-specified model-based inference under Block Randomization.

End-of-Study Allocation Differences: Block Randomization, Sequential Matched Randomization with a dynamic and empirical thresold, Sequential Rematched Randomization, and single batch Matched Randomization always achieved equal end-of-study allocation (Figure 5). Begg and Iglewicz Minimization was close to achieving equal end-of-study allocation. Atkinson’s Algorithm with a Biased Coin, which performed well in achieving balance and efficiency ran a greater risk of ending the study with a treatment imbalance `r RPTbcVarAtkTrtImbal["mci"]`.

Randomization within Minimization: When adjusting Begg and Iglewicz Minimzation to the most predictive covariates, randomization occurred for roughly 20\% of patients (7.8\% of patients when adjusting to all baseline covariates).  See supplement.

Reservoir Size: See supplement for the average reservoir size for Sequential Matched and Rematched Randomization.


## Conclusions
We’ve introduced extensions to Sequential Matched Randomization that recover much of the optimality lost from single batch Matched Randomization due to sequential entry of patients.  These extensions include Sequential Rematching with a dynamic and empirical threshold and the ability to randomize patients in enrollment blocks.  Further, our case study re-emphasizes [@Ciolino:2011ff] the value of covariate-adjusted randomization for increasing overall covariate balance and efficiency estimating the Population Average Treatment Effect.  The precision of the permutation-based Sample Average Treatment Effect estimator achieved nearly the same efficiency as model-based inference estimating the treatment effect.  

Our work is consistent with other findings that there is a balance / efficiency trade-off in choosing which covariates to adjust randomization[@Kallus:2018um; @Atkinson:1999va].  Arguments can be made to prioritize balance, especially when investigating important subgroups and when the strength of relationship between baseline covariates and the outcome is unknown.  And, arguments are made for prioritizing efficiency especially when chance imbalances may be adjusted with a model [@Atkinson:1999hq].  Weighted distance measures can allow an investigator to design the study prioritizing certain covariates over others[@GreevyJr:2012hp].

In introducing Sequential Rematching, we add to a class of look-back allocation schemes. Though patients have matched, their baseline covariates may yet be updated or used again to increase covariate balance and precision. Also in this class are Minimization schemes with and without a biased coin and Sequential Rerandomization. Such schemes are beneficial when some baseline covariates are not initially available at the time of randomization as was the case with Baseline Hemoglobin A1c in the REACH trial.

In this case study, Begg and Iglewicz Minimization performed frequently among the top performing Sequenital allocation schemes and provided equal randomization to, on average, roughly 20\% of patients (7\% when adjust randomization to all baseline covariates).  This provides some reassurance against systematic confounding.  Minimization is powerful from a study design perspective, and randomization is critical for eliminating systematic confounding.  This work begs the question, what is the extent of randomization necessary to sufficiently reduce systematic confounding?

Many studies set sample size based on the average performance of block randomization (for example by basing of a t-test).  This case study provides a cautionary reminder that half of such studies will have a decreased effective sample size. Though all assumptions and operations of a study may work perfectly, a non-adaptive trial may fail to find a treatment effect due to chance alone.  Covariate-adjusted randomization helps protect against poor covariate imbalances and increases the chances of a sufficient effective sample size all else the same.



```{r MaxSMD, echo=FALSE, fig.pos = "H",fig.cap="Boxplot of the maximum absolute Standardized Mean Difference among all baseline covariates for each of 20,000 simulated allocation sequences per allocation scheme.  Matched-based Randomization schemes are shaded in blue.  Seq Matched (20p) is Sequential Matching before applying proposed extensions; it uses a fixed 20th percentile of the F(p, i-p) distribution.  Min SE schemes are Minimization schemes proposed by Atkinson and Begg and Iglewicz to reduce the standard error of a pre-specified Ordinary Least Squares Model.  Min SE (Atk, 2/3 Prob) uses a 2/3 biased coin to randomize to the favorable arm under Atkinson's Minimization."}
knitr::include_graphics("~/Dropbox/statistics/work/vandy/RandomizationMethods/journal-clinical trials/latex/MaxSMD.jpeg")
```

```{r AveSMD, echo=FALSE, fig.pos = "H", fig.cap="Boxplot of the average absolute Standardized Mean Difference among all baseline covariates for each of 20,000 simulated allocation sequences per allocation scheme."}
knitr::include_graphics("~/Dropbox/statistics/work/vandy/RandomizationMethods/journal-clinical trials/latex/AveSMD.jpg")
```

```{r permCIwidth, echo=FALSE, fig.pos = "H", fig.cap="Boxplot of the permutation-based 95% Confidence Interval Width estimating the Sample Average Treatment Effect for each of 20,000 simulated observed datasets. Observed outcomes were generated as the predicted three month Hemoglobin A1c plus a treatment effect (if allocated to treatment) and a random residual.  The horizontal line in red is the median treatment effect Confidence Interval Width under the pre-specified Ordinary Least Square model and Block Randomization (See Figure 4)."}
knitr::include_graphics("~/Dropbox/statistics/work/vandy/RandomizationMethods/journal-clinical trials/latex/permutationCIwidth.jpg")
```

```{r adjCIwidth, echo=FALSE, fig.pos = "H", fig.cap="Boxplot of the model-based 95% Confidence Interval Width estimating the Sample Average Treatment Effect, adjusted for all baseline covariates, for each of 20,000 simulated observed datasets. The Ordinarly Least Squares (OLS) model was pre-specified to include all baseline covariates with restricted cubic splines on each continuous outcome and accounted for multiple imputations."}
knitr::include_graphics("~/Dropbox/statistics/work/vandy/RandomizationMethods/journal-clinical trials/latex/modelAdjCIwidth.jpg")
```


```{r trtDiff, echo=FALSE, fig.pos = "H", fig.cap="Boxplot of the difference in end-of-study treatment allocation for each of 20,000 simulated allocation sequences per allocation scheme."}
knitr::include_graphics("~/Dropbox/statistics/work/vandy/RandomizationMethods/journal-clinical trials/latex/trtBalance.jpg")
```




<!-- # ```{r tableOne, echo=FALSE} -->
<!-- # rowsIndent <- (1:length(rownames(RPTtab1_0)))[startsWith(rownames(RPTtab1_0),prefix = "  ")] -->
<!-- # kable(RPTtab1_0) -->
<!-- # #kable(RPTtab1_0, booktabs = TRUE, escape = TRUE, align=c('l','l','l')) %>% -->
<!-- # #  add_indent(rowsIndent) -->
<!-- # ``` -->



```{r relEff, echo=FALSE}
kable(RPTnEffTable, booktabs = TRUE, caption = "Median and 95 Percentile Confidence Interval of difference in effective sample size relative to Block Randomization for each of 20,000 generated observed datasets.  Two Block Randomization sequences per generated observed dataset to allow comparing one instance of Block Randomization to another.  Schemes are ordered by efficiency gains for permutation-based inference.", escape = FALSE, align=c('l','l','c','c','c','c')) %>%
  add_header_above(c(" "=2, "Permutation Estimate"=2, "Adjusted Model Estimate"=2)) %>%
  kable_styling(latex_options = c("hold_position","scale_down"), position="left", full_width = FALSE) %>%
  column_spec(1, bold=TRUE, width = "3cm") %>%
  column_spec(3:6, width = "2.3cm") %>%
  collapse_rows(columns = 1)
```