14

The Temporal Consistency Challenge in Video Restoration

The format of the article comes across as AI-sloppy. Each section is filled with numbered lists and there are several AIsms, such as the omni-present "not-only-x-but-y".

5 hours agonnevatie
[deleted]
5 days ago

if you were talking about erasure in classic film, and not the constraints of non-linear editable data streams (I and P blocks, you-name-it), how much of this would remain true? Yes, its a temporal-spatial space. But, it consists of a sequence of static images. (in the case of film) and so erasure could be a 2 phase process 1) find the mask per image and apply it and 2) construct an infill which respects the rest of the images.

The choice of a dog running down a beach is quite smart: the background has a plane of movement which is mechanistically unrelated to the dog. thats part 2) reconstruct waves lapping on the seashore. Hard. you can't do this per-image. you have to do this across the entire sequence.

I would think, even in a film model, this is a really quite complicated problem because for each static image an infill is plausible, but to maintain consistency across the image series, it has to avoid uncanny valley for the specifics of wave motion up a beach.