-
Notifications
You must be signed in to change notification settings - Fork 0
/
README.Rmd
130 lines (83 loc) · 7.38 KB
/
README.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
---
title: "`faceplyr`: Tools to work with faces in R"
output: github_document
---
```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE, eval = FALSE, warning = FALSE, error = FALSE, message = FALSE)
```
# `faceplyr` <img src="imgs/faceplyr_hex.png" align="right" />
# Motivation
My motivation is primarily selfish. I wanted to work mostly in the R environment and have R code that performed like I would expect R to perform, but with the performance and breadth of a few Python image functions I wish R had natively. In a similar vein, I found myself loading multiple R image libraries to perform a set of operations on one image. A single package that brought all of this together under one convenient roof was the main motivating factor for writing this package.
This project started as part of my first "real" R package many years ago, so it is messy. Really messy. I've learned a lot since I started writing this package, and I have hopes of cleaning it up and reducing dependencies in the future (see `TODOs`). However, as a scientist and not a developer, my main motivation was to get the code I wanted to work rather than preform elegantly.
# Disclaimer
This package is very much a work in progress and is by no means meant for heavy production use. It started as a private repository of functions that I used regularly in daily work and research. As such, there are many package dependencies that make `faceplyr` quite large and cumbersome. Hopefully in the future I will have to time to parse some of these dependencies. A lot of the "fluff" is because I wanted things to be easy and readable on my end (e.g., `tidyverse`), or look pretty (`cli`), both of which are not good reasons to include in a codebase. However, these don't influence functionality and can ultimately be stripped out in the future.
Be aware that there are also bits and pieces of unfinished code for functions/data that I am either currently working with, or for functions that are still being tested.
If a function currently will not run, there is a good chance that I missed importing a package when I built and evaluated `faceplyr`. It would be wise to pre-install following packages and pre-load a subset of them when using `faceplyr` just to be safe:
```{r pak-setup}
paks <- c("tidyverse", "fs", "imager", "magick", "reticulate", "raster", "cli", "glue")
install.packages(paks)
if (!requireNamespace("BiocManager", quietly = TRUE))
install.packages("BiocManager")
BiocManager::install("EBImage")
library(tidyverse)
```
Why are there multiple image handling libraries needed? Because they all do different things, and no one library has all the specific image handling functions needed.
This package has been tested and works on MacOS and Ubuntu Linux.
## Quick start
You need to have conda installed with a `faceplyr` environment for this package to work properly. `reticulate` provides a helper function to download and install miniconda, as well as setup an environment. `faceplyr` also has a `requirements.txt` file so that you can `pip install` the minimum python requirements.
If these are already satisfied (see below) using `faceplyr` should be as simple as:
```{r quick-start, eval = FALSE}
# install.packages("remotes")
# remotes::install_github("d-bohn/faceplyr")
library(faceplyr)
reticulate::use_condaenv("faceplyr", required = TRUE)
```
## Setup Conda
If you don't have Miniconda or Anaconda already on your machine, you will have to install if first in order to work with `faceplyr`. Conda is a package manager mostly used for Python, but it is extendable to other programs. `reticulate` has helper functions for doing this easily. You can name the environment whatever you want, but you just need to remember to activate that specific environment whenever you want to use the package.
```{r setup-conda1}
reticulate::install_miniconda()
reticulate::conda_create(envname = "faceplyr")
```
Next, or if you already have Conda on your machine, you need to setup an environment for `faceplyr`. The `requirements.txt` outlines which python packages are necessary for making the code in `faceplyr` perform. Several functions in `faceplyr` either call external Python code, or are wrappers around already established and vetted code.
```{r setup-conda2}
requirements <- read.system.file("extdata", "requirements.txt", package = "faceplyr")
reticulate::conda_install(
envname = "faceplyr",
pip = TRUE,
pip_options = glue::glue("install -r {requirements}")
)
```
# Examples
## Extract face metrics
Let's say we want to extract several metrics from this (included) image:
All of the `extract_*` functions require as input the interior cropped face as a PNG file (to maintain an alpha channel). You can manually do so with some helper functions and pass the saved cropped image to each function, but the variable names expect the *unaltered* image name as the `image` parameter. This was done to encourage running the `extract_*` functions in a certain order starting with `extract_structure`. After `extract_structure` is run, the other `extract_*` functions can be run in any order. The structure metric is the most important because it does the heavy lifting of locating the face in the image and cropping out the area of interest. If I didn't specify running this function first, every other function written (both currently in the future) would have to call `extract_structure` before it could continue. It is easier for the time being to just run `extract_structure` first.
```{r extract-structure}
image <- system.file("extdata", "queen.png", package = "faceplyr")
extract_structure(image = image, data_save_folder = "./output", return_results = FALSE)
# → Image complete (structure): ./output/queen_cropped.png
```
The above code should result in the image saved to `./output/queen_cropped.png`:
Notice that a `data_save_folder` was specified and `return_results` was set to `FALSE`. This combination will save a `RDS` file to the `./output` folder on your machine where the results from the structure metric function will be written. Alternatively, you can specify `return_results` and it will instead return the output to the console (or an object) and `data_save_folder` will be ignored for saving data, but still needed to save the cropped image.
We can see how well `dlib` did at finding the face metric point locations with the `plot_landmarks` helper function:
```{r plot-landmarks, eval = FALSE}
plot_landmarks(image = system.file("extdata", "queen.png", package = "faceplyr"))
```
After running `extract_structure` on an image all of the other `extract_*` functions should auto-*magically* work on the same image.
```{r extract-hist}
extract_hist_colors(image = image, data_save_folder = "./output")
# → Image complete (color hist): ./output/queen_cropped.png
```
# TODO List
There are a number of "wish list" items that I want to implement in this package. Here is a running list:
- Create `Dockerfile` for easy install on host machines
- Major priority
- Reduce overall dependencies
- Create "in house" facial landmark classifier to reduce dependencies and speedup landmarking
- Major priority
- Cleanup `.R` files
- Cleanup `legacy`, `devel`, and unused Python files in `inst`
- ~~Estimate facial roundness (e.g., `faceplyr::extract_roundness`, but see, `faceplyr::calc_roundness`)~~ Complete.
- ~~Estimate facial angularity (e.g., `faceplyr::extract_angularity`)~~ Complete.