Conventions¶
The rules that let datasets, models, losses and metrics be combined freely.
Notation¶
| Symbol | Meaning |
|---|---|
C |
Coordinate axes, the in_features of an INR |
D |
Output channels, the out_features of an INR |
N |
Points in a flat batch; (N, C) is such a batch and (..., C) allows any leading dimensions |
K_c, K |
Basis functions on axis c, or on every axis |
R |
CP rank |
H, W |
Image height and width |
channels, depth |
An image's channels and a volume's first grid axis, spelled out because C and D mean the above |
Coordinates¶
Coordinates live in \([-1, 1]\) on every axis, both endpoints included. inro.create_coordinates((H, W)) returns an (H, W, 2) grid whose column \(c\) spans axis \(c\) (indexing="ij"), and the datasets build their inputs with it.
FUTON's bases and the hash and feature grids are defined on this domain, so do not pre-normalize to \([0, 1]\). Models accept any leading shape; the taps of FUTON's local bases need a flat (N, C) batch, which FUTON.forward produces internally.
Values¶
Targets are in the model's output range, and the reference signal keeps its native form.
| Dataset | input |
target |
_original |
|---|---|---|---|
ImageCoordinateDataset |
(H, W, 2) |
(H, W, channels) float in \([-1, 1]\) |
(channels, H, W) uint8 |
OccupancyCoordinateDataset |
(depth, H, W, 3) |
(depth, H, W, 1) in \(\{-1, 1\}\) |
(1, depth, H, W) float in \(\{0, 1\}\) |
nerf.BlenderDataset |
rays (H, W, 3) origins and directions |
(H, W, 4) RGBA in \([0, 1]\) |
- Subsampling (
subsample,max_samples) flattensinputandtargetto(N, ...)with shared random indices. preprocessmaps native values to targets:uint8to \([-1, 1]\), occupancy \(\{0, 1\}\) to \(\{-1, 1\}\).postprocessmaps model output back: clipping to \([-1, 1]\) before theuint8cast, thresholding occupancy at zero.
Because targets are in \([-1, 1]\), the benchmark models end in torch.tanh, and the default training loss reports PSNR with a peak-to-peak range of 2.
Metrics¶
In inro.train and inro.evaluate, metrics are called as fn(reconstruction, original) after postprocess, so they see uint8 images and {0, 1} volumes.
| Metric | Input | Notes |
|---|---|---|
inro.psnr, inro.ssim, inro.ms_ssim |
two (channels, H, W) images |
data_range from the target: 255 for uint8, 1 for floating point |
inro.lpips |
two (3, H, W) images |
VGG by default, net="alex" for AlexNet; its weights are downloaded once |
inro.iou |
two volumes | Thresholds both at zero, so it works on the {0, 1} original and on raw model output alike |
Widths, depths and activations¶
hidden_layers=h means h hidden layers and h + 1 weight layers. output_activation is applied after the last layer of every model; inside inro.nerf.RadianceField the density network must have none, because the field applies trunc_exp to the density and a sigmoid to the color.
Checkpoints¶
inro.trainsaves the model's state dict ascheckpoint.ptinlog_dirand returns it asmodel_dict.inro.nerf.trainsaves the field's state dict, including theaabbbuffer.MLP.layersholds only weight layers (layers.0tolayers.h). FUTON keeps its head underhead.*(head.layers.*for an MLP), and the grid models their decoder underdecoder.layers.*.
Seeds and devices¶
seedininro.trainandinro.nerf.trainseeds PyTorch's global RNG before training, after the model has been built. Construct the model aftertorch.manual_seedto also fix its initialization, as the scripts do.- A dataset's
seedreseeds Python's RNG only; subsampling draws from PyTorch's RNG. device=Noneselects CUDA when available. The training loop moves the model and, through theirtomethod, the datasets to that device, and_originalstays on the CPU.
Experiment files¶
A benchmark run lives in logs/<task>/<signal>/<model>/ with log.txt, log.json, checkpoint.pt, the final reconstruction (or two rendered test views for NeRF), and results.json: the training configuration, the model's setup from the YAML, num_params, train_time, the final metrics, and the history. Failed runs leave error.txt. The reproduction page describes the scripts that write and read them.