Skip to content

Commit

Permalink
Update general rules
Browse files Browse the repository at this point in the history
  • Loading branch information
emmt committed Jun 10, 2024
1 parent 50e26c7 commit a96e4ff
Show file tree
Hide file tree
Showing 5 changed files with 245 additions and 97 deletions.
2 changes: 1 addition & 1 deletion docs/make.jl
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ DEPLOYDOCS = (get(ENV, "CI", nothing) == "true")
makedocs(
sitename = "Image Metrics",
authors = "Éric Thiébaut and contributors",
pages = ["index.md", "general.md", "bc2016.md"],
pages = ["index.md", "general.md", "bc2016.md", "ic2024.md"],
format = Documenter.HTML(
prettyurls = DEPLOYDOCS,
mathengine = Documenter.KaTeX(
Expand Down
126 changes: 62 additions & 64 deletions docs/src/bc2016.md
Original file line number Diff line number Diff line change
@@ -1,53 +1,54 @@
# The 2016' Interferometric Beauty Contest

For the _2016' Interferometric Imaging Beauty Contest_[^Sanchez2016] held in
Edinburgh, the objective of the contest was to reconstruct multi-wavelength
images. A metric was thus specifically designed to compare multi-channel images
so as to be insensitive to irrelevant differences due to:
For the _2016' Interferometric Imaging Beauty Contest_[^Sanchez2016] held in Edinburgh,
the objective of the contest was to reconstruct multi-wavelength images. A metric was thus
specifically designed to compare multi-channel images so as to be insensitive to
irrelevant differences due to:

* **orientation**: image axes may be inverted, notably the E-W axis;
* **translation**: there may be an arbitrary shift between images (possibly a
different shift in every spectral channel);
* **pixel scale**: the size of the pixel used for the restored image is
arbitrary even though it should be sufficiently small to account for the
highest measured frequencies;
* **brightness**: not all entries are normalized according to the channel-wise
flux given in `OI_FLUX` data-block.

In addition, the comparison must take into account that the measurements have
limited angular resolution. Thus the reference image ``\Vy`` is the ground
truth image ``\Vz`` convolved with an effective _Point Spread Function_ (PSF)
whose _Full Width at Half Maximum_ (FWHM) chosen to match the interferometric
resolution. In the comparison, the restored images are also convolved with a
PSF whose FWHM is tuned to best match the reference image ``\Vy``.

Let ``\MR(\rho,\Vt,\omega)`` be the linear operator used to resample an image
with a given magnification ``\rho``, translation ``\Vt``, and blur parameter
``\omega``. The magnification ``\rho`` is the ratio of the output image pixel
size over the input image pixel size. The translation ``\Vt`` may be specified
for the input or output pixel grids (as is the most convenient). The blur
parameter ``\omega`` can be specified as the FWHM of the PSF introduced to
control the effective resolution of the output image.
* **translation**: there may be an arbitrary shift between images (possibly a different
shift in every spectral channel);
* **pixel size**: the size of the pixels used for the restored image is arbitrary even
though it should be sufficiently small to account for the highest measured frequencies;
* **brightness**: not all entries are normalized according to the channel-wise flux given
in `OI_FLUX` data-block;
* **out-of-field values**: after all geometric transformations, the fields of view of the
reference and reconstructed images may be different, it is assumed that missing pixel
values are equal to zero. The same rationale leads to impose that ``β = 0`` in the
general rules.

In addition, the comparison must take into account that the measurements have limited
angular resolution. Thus the reference image ``\Vy`` is the ground truth image ``\Vz``
convolved with an effective _Point Spread Function_ (PSF) whose _Full Width at Half
Maximum_ (FWHM) chosen to match the interferometric resolution. In the comparison, the
restored images are also convolved with a PSF whose FWHM is tuned to best match the
reference image ``\Vy``.

Let ``\MR(\rho,\Vt,\omega)`` be the linear operator used to resample an image with a given
magnification ``\rho``, translation ``\Vt``, and blur parameter ``\omega``. The
magnification ``\rho`` is the ratio of the output image pixel size over the input image
pixel size. The translation ``\Vt`` may be specified for the input or output pixel grids
(as is the most convenient). The blur parameter ``\omega`` can be specified as the FWHM of
the PSF introduced to control the effective resolution of the output image.

Using the resampling operator, the **reference image** is:

``` math
\Vy = \MR(\rho_{\mathrm{ref}},\boldsymbol{0},\omega_{\mathrm{ref}}) \cdot \Vz,
```

where ``\Vz`` is the ground truth image. Note that there is no translation
between the ground truth image ``\Vz`` and the reference image ``\Vy``.
where ``\Vz`` is the ground truth image. Note that there is no translation between the
ground truth image ``\Vz`` and the reference image ``\Vy``.

``\delta_{\mathrm{ref}} = 3\,\mathrm{mas}/\mathrm{pixel}`` is the pixel size of
the image ``z`` and ``\omega_{\mathrm{ref}} \sim
\lambda_{\mathrm{min}}/(2\,B_{\mathrm{max}})`` is the FWHM of the objective
resolution.
``\delta_{\mathrm{ref}} = 3\,\mathrm{mas}/\mathrm{pixel}`` is the pixel size of the image
``z`` and ``\omega_{\mathrm{ref}} \sim \lambda_{\mathrm{min}}/(2\,B_{\mathrm{max}})`` is
the FWHM of the objective resolution.

The score for a given image ``x`` is the sum of the squared differences between
the ``\Gamma``-corrected images:
The score for a given image ``x`` is the sum of the squared differences between the
``\Gamma``-corrected images:

``` math
s(\Vx) = \frac{
\Score_{\Gamma,p}(\Vx) = \frac{
\sum\limits_{\lambda} \min\limits_{\alpha_\lambda,\Vt_\lambda,\omega_\lambda}
\sum\limits_{j} \left(
\Gamma\bigl(
Expand All @@ -63,35 +64,33 @@ s(\Vx) = \frac{

where ``\Vx_{\lambda}`` is the restored image in the spectral channel indexed by
``\lambda``, ``j`` is the pixel index and ``\Gamma: \Reals\to\Reals`` is a brightness
correction function to emphasizes the interesting parts of the images. We have
chosen:
correction function to emphasizes the interesting parts of the images. We have chosen:

``` math
\gdef\Sign{\mathrm{sign}} % trick
\Gamma(t) = \Sign(t)\,|t|^\gamma \, ,
\Gamma(x) = \Sign(x)\,|x|^\gamma \, ,
```

where ``\Sign(t)`` is the sign of ``t``:
where ``\Sign(x)`` is the sign of ``x``:

``` math
\Sign(t) = \begin{cases}
-1 & \text{if $t < 0$}\\
+1 & \text{if $t > 0$}\\
\phantom{+}0 & \text{if $t = 0$}\\
\Sign(x) = \begin{cases}
-1 & \text{if $x < 0$}\\
+1 & \text{if $x > 0$}\\
\phantom{+}0 & \text{if $x = 0$}\\
\end{cases}
```

The sign function is introduced for generality even though restored images are
usually nonnegative. The denominator is to normalize the score: 0 is the best
possible value and 1 is the score for a black image. The lower the score the
better.
The sign function is introduced for generality even though restored images are usually
nonnegative. The denominator is to normalize the score: 0 is the best possible value and 1
is the score for a black image. The lower the score the better.

Because ``\Gamma(\alpha\,t) = \Gamma(\alpha)\,\Gamma(t)`` (whatever ``t`` and
``\alpha``), minimizing with respect to ``\alpha_\lambda`` when ``p = 2`` has a
closed form solution and the score simplifies to:
Because ``\Gamma(\alpha\,x) = \Gamma(\alpha)\,\Gamma(x)`` (whatever ``x`` and ``\alpha``),
minimizing with respect to ``\alpha_\lambda`` when ``p = 2`` has a closed form solution
and the score simplifies to:

``` math
s(x) = 1 - \frac{
\Score_{\Gamma,2}(x) = 1 - \frac{
\sum\limits_{\lambda} \max\limits_{\Vt_\lambda,\omega_\lambda}
c_\lambda(\Vt_\lambda,\omega_\lambda)
}{
Expand All @@ -117,23 +116,22 @@ with:
which is a normalized correlation between the ``\Gamma``-corrected images.


To compensate for different pixel sizes, the image ``x`` or ``x`` which has the
larger pixel size is interpolated so that both images have the same (smallest)
pixel size.
To compensate for different pixel sizes, the image ``\Vx`` or ``\Vy`` which has the larger
pixel size is interpolated so that both images have the same (smallest) pixel size.

Separable linear interpolation with a triangle kernel is applied for magnifying
and fine shifting the images.
Separable linear interpolation with a triangle kernel is applied for magnifying and fine
shifting the images.

The criterion is minimized for a translation between each images (for each
spectral channel).
The criterion is minimized for a translation between each images (for each spectral
channel).

In every spectral channel, the brightness of the restored images is scaled
so that the total flux per channel is the same as in the reference image.
In every spectral channel, the brightness of the restored images is scaled so that the
total flux per channel is the same as in the reference image.

Because the resolution of the reference image may change (see above) the score
is the ratio between the sum for all spectral channels of the scores between
the restored images and the reference images divided by the sum for all
spectral channels of the scores between a zero image and the reference images.
Because the resolution of the reference image may change (see above) the score is the
ratio between the sum for all spectral channels of the scores between the restored images
and the reference images divided by the sum for all spectral channels of the scores
between a zero image and the reference images.


[^Sanchez2016]:
Expand Down
75 changes: 46 additions & 29 deletions docs/src/general.md
Original file line number Diff line number Diff line change
@@ -1,45 +1,62 @@
# General image metrics

A general discussion about measuring discrepancy between images with a specific
focus on optical interferometry is developed in[^Gomes2016]. A proper image
metric shall provide a quantitative score suitable to order restored images.
This score must reflect the perception by an expert of the fidelity of the
images with a given ground truth or reference image. The score shall be
insensitive to changes that may be considered as irrelevant for the considered
context. For example, image quality indices widely used in the signal
processing community are insensitive to an affine transform of the pixel
values. For optical interferometry, the pixel size of a restored image is, to
some extend, a free parameter and may thus be different from the pixel size of
the ground truth image. Moreover, when only power spectrum and phase closure
data (`vis2data` and `t3phi` in `OI_VIS2` and `OI_T3` data-blocks of OI-FITS
files) are used to reconstruct an image, the position of the object in the
field of view is not constrained by the data and the image metric should not
depend on this position.
A general discussion about measuring discrepancy between images with a specific focus on
optical interferometry is developed in[^Gomes2016]. A proper image metric shall provide a
quantitative score suitable to order restored images. This score must reflect the
perception by an expert of the fidelity of the images with a given ground truth or
reference image. The score shall be insensitive to changes that may be considered as
irrelevant for the considered context. For example, image quality indices widely used in
the signal processing community are insensitive to an affine transform of the pixel
values. For optical interferometry, the pixel size and the field of view of a restored
image are, to some extend, free parameters and may thus be different from the pixel size
of the ground truth image. Moreover, when only power spectrum and phase closure data
(`vis2data` and `t3phi` in `OI_VIS2` and `OI_T3` data-blocks of OI-FITS files) are used to
reconstruct an image, the position of the object in the field of view is not constrained
by the data and the image metric should not depend on this position.

Accounting for all these remarks, a possible definition of the distance between ``\Vy``, a
reference image, and ``\Vx``, a reconstructed image, is given by:

```math
\Dist(\Vx,\Vy) = \min_{α,β} \Vert \alpha\,\MR_{\boldsymbol{θ}}
\cdot\Vx + \beta\,\One - \Vy\Vert
\begin{align}
\Dist(\Vx,\Vy) = \min_{α,β,\boldsymbol{θ}} \Bigl\{
&\sum_{i \in |\MR_{\boldsymbol{θ}}\cdot\Vx| \cap |\Vy|}
d\!\left(α\,(\MR_{\boldsymbol{θ}}\cdot\Vx)_i + β, y_i\right)
\notag\\
&+ \sum_{i \in |\MR_{\boldsymbol{θ}}\cdot\Vx| \backslash
(|\MR_{\boldsymbol{θ}}\cdot\Vx| \cap |\Vy|)}
d\!\left(α\,(\MR_{\boldsymbol{θ}}\cdot\Vx)_i + β, v_{\mathrm{out}}\right)
\notag\\
&+ \sum_{i \in |\Vy| \backslash
(|\MR_{\boldsymbol{θ}}\cdot\Vx| \cap |\Vy|)}
d\!\left(v_{\mathrm{out}},y_i\right)
\Bigr\}
\end{align}
```

where ``\Vert\ldots\Vert`` is some norm, ``\One`` is an image of the same size
as ``\Vy`` but filled with ones, ``\MR_{\boldsymbol{θ}}`` is a linear operator
which implements resampling with a given magnification, translation, and
blurring. Here ``\alpha``, ``\beta``, and ``\boldsymbol{θ}`` are nuisance
parameters to reduce the mismatch between the images.
where ``d(x,y)`` is some pixel-wise distance, ``\MR_{\boldsymbol{θ}}`` is a linear
operator which implements resampling with a given magnification, translation, and
blurring, and ``v_{\mathrm{out}}`` is the assumed out-of-field pixel value. Here
``\alpha``, ``\beta``, and ``\boldsymbol{θ}`` are nuisance parameters to reduce the
mismatch between the images.

The score may be defined by normalizing the distance:

```math
\Score(\Vx)
= \frac{\Dist(\Vx,\Vy)}{\Dist(\Zero,\Vy)}
= \frac{
\min_{\alpha,\beta} \Vert \alpha\,\MR\cdot\Vx + \beta\,\One - \Vy\Vert
}{
\min_{\beta} \Vert \beta\,\One - \Vy \Vert
}
= \frac{\Dist(\Vx,\Vy)}{\Dist(v_{\mathrm{out}}\,\One,\Vy)}
```

where ``\Zero`` is an image of the same size as ``\Vy`` but filled with zeros.
where ``\One`` is an image of the same size as ``\Vy`` but filled with ones,
hence``v_{\mathrm{out}}\,\One`` is an image of the same size as ``\Vy`` but filled with
``v_{\mathrm{out}}`` the assumed out-of-field pixel value.

The following properties are assumed for the pixel-wise distance:

1. ``d(x,x) = 0`` for any ``x \in \mathbb{R}``;
2. ``d(x,y) > 0`` for any ``(x,y) \in \mathbb{R}^2`` such that ``x \not= y``;
3. ``d(y,x) = d(y,x)`` for any ``(x,y) \in \mathbb{R}^2``.


[^Gomes2016]:
> N. Gomes, P. J. V. Garcia & É. Thiébaut, *Assessing the quality of
Expand Down
Loading

0 comments on commit a96e4ff

Please sign in to comment.