Colorizing a B&W movie with a palette: guiding the CNN through salient colours
CNNs colorize skies, foliage and faces very well — because those objects have memorable colours learned from large training datasets. But they paint every sports car red. My idea: feed the network a palette of salient colours for the scene, computed once for an entire image collection.
The biases a CNN learns from colour images
The memorable-colour bias: a colorization CNN trained on ImageNet associates "sports car" with "red" because the majority of training photos are. For a yellow or green car in a B&W movie, the result fails systematically — the network falls back on its statistical mean.
The cost of manual scribbles: classical interactive methods (Levin et al. 2004, scribble-based) require the user to draw colour patches on every frame. For a 90-minute movie at 24 fps, that is on the order of a hundred thousand frames. Unworkable.
The question: can we guide the network with a global signal instead of a per-pixel one, and still get faithful, automated results?
From the B&W scene to a colorized frame
Global palette + pretrained network
Core idea: feed the CNN a palette of salient colours as an additional input, without touching the colorizer's architecture. Plug-and-play on any pretrained model.
- Palette computation: representative colours for a scene (or a whole movie) are derived from colour references — paintings, contemporary photos, storyboards.
- "Salient" vs "memorable" criterion: the palette emphasises scene-specific colours (the yellow car, the hero's red sweater) and ignores those the network already infers well (sky blue, leaf green).
- Injection into the CNN: the palette is concatenated with the B&W image at input. The pretrained network learns to use this global hint as a soft constraint on its colour predictions.
- No full retraining: only the injection layer is added, plus light fine-tuning. The original colorizer keeps everything it learned.
What the palette changes
Suited to movie colorization
One palette is enough for hundreds of consistent frames — no more per-frame scribbles. Annotation cost drops from the order of a whole movie to the order of a scene.
Free temporal consistency
Because the palette is shared across frames of the same scene, transitions stay stable: no colour jumps between consecutive frames, unlike frame-by-frame approaches.
Compatible with open-source models
Tested with public reference implementations (Zhang/Isola/Efros, Iizuka/Simo-Serra/Ishikawa). No proprietary model required.
Off-distribution colours preserved
Atypical objects (yellow car, green outfit) get their real colour through the palette instead of regressing to the training-set mean.
What this publication brings
- Conceptual distinction between memorable colours (already well predicted by the network) and salient ones (worth surfacing). Not trivial: it is what makes the palette useful instead of redundant with what the network already knows.
- Plug-and-play approach: no need to retrain a colorizer from scratch. We augment a pretrained model with a global hint that plugs into the input.
- Paved the way for IPAS 2020 (my next paper), which combines the global palette with targeted scribbles for hard cases (flags, monuments).
- Practical cost reduction for archival movie restoration: from scribble-per-frame work to palette-per-scene work.
Venue & context
- Published at IEEE MCNA 2020 (International Conference on Multimedia Computing, Networking and Applications).
- Work carried out during my research master's at MIA Laboratory · University of La Rochelle.
- Keywords: CNN, colorization, colour palette, movie colorization, interactive colorization.