VAE Metrics Lab (GUI)

SDXL Breakdown

VAE Metrics Lab

This tool is a VAE evaluation and reconstruction fidelity benchmarking system that compares original input images against VAE reconstructions using both perceptual and signal-level metrics.

It takes a folder of images, preprocesses them into a consistent format, and optionally embeds a small color calibration marker into the input images. A VAE then reconstructs these images, and the system compares the outputs back to the original ground truth.

---

Evaluation Metrics

The system evaluates reconstruction quality using:

• LPIPS perceptual distance – measures human-perceived visual similarity between images

• Gradient energy – evaluates edge and fine-detail preservation

• FFT-based structure analysis – measures frequency-domain similarity and global structure fidelity

• Color diversity metrics – estimates texture richness and reconstruction entropy

• RGB pixel error analysis – measures direct per-channel reconstruction accuracy

• Black/white/RGB marker analysis – detects brightness bias, contrast scaling, and per-channel color drift using a fixed calibration patch

---

Outputs

The tool produces:

• A per-image CSV report

• A summary JSON file with averaged metrics

These outputs allow direct comparison between different VAEs in terms of:

• reconstruction quality

• color fidelity

• structural accuracy

• calibration stability

---

Math for the Evaluation

Let x in R^(H x W x 3). Define reconstruction:

x_hat = R(x) = D(E(x))

---

Metrics

Perceptual loss:
L_perc = d_phi(x, x_hat)

Gradient energy:
G(x) = E[ |grad g(x)|^2 ]
rho_G = G(x_hat) / G(x)

Color support:
C(x) = number of unique RGB values in x
rho_C = C(x_hat) / C(x)

Brightness bias:
b = E[ m_black(x_hat) - m_black(x) ]

Contrast gain:
gamma =
E[ m_white(x_hat) - m_black(x_hat) ] /
E[ m_white(x) - m_black(x) ]

Channel drift:
delta_c = E[ c(x_hat) - c(x) ], c in {R,G,B}

---

Final score

J =
lambda1 * L_perc

• lambda2 * abs(1 - rho_G)

• lambda3 * abs(1 - rho_C)

• lambda4 * norm(delta_c)

• lambda5 * abs(b)

• lambda6 * abs(gamma - 1)

---

System model

R(x) approx x