Relatedly, Marcin Wichary wrote a nice post about using FFT to remove moiré and halftone effects when scanning images that were printed with halftones.
There have been some interesting advances in trying to add spectral information to the data that a learning architecture has at its disposal, but there are a couple roadblocks that I don’t think have been solved yet.
1. Complex-valued NNs are not an easy generalization of real ones.
2. A localization in one domain implies non-local behavior in the other (this is the Fourier uncertainty principle).
Fourier Neural Operators (FNOs) come close to what I want to see in this area but since they enforce sparsity in the spectral domain their application is necessarily limited.
I think one can do better with a wavelet, shearlet, or curvelet transform that is adapted to the problem domain at hand. But the uncertainty principle still haunts those transforms, and anyway the goal is to be domain-agile.
Relatedly, Marcin Wichary wrote a nice post about using FFT to remove moiré and halftone effects when scanning images that were printed with halftones.
It's from 2021: Moiré no More (https://newsletter.shifthappens.site/archive/moire-no-more/).
I'd like to see a sequel where the fractional fourier transform is used for image restoration
There have been some interesting advances in trying to add spectral information to the data that a learning architecture has at its disposal, but there are a couple roadblocks that I don’t think have been solved yet.
1. Complex-valued NNs are not an easy generalization of real ones.
2. A localization in one domain implies non-local behavior in the other (this is the Fourier uncertainty principle).
Fourier Neural Operators (FNOs) come close to what I want to see in this area but since they enforce sparsity in the spectral domain their application is necessarily limited.
I do wonder if a wavelet transform might be better.
I think one can do better with a wavelet, shearlet, or curvelet transform that is adapted to the problem domain at hand. But the uncertainty principle still haunts those transforms, and anyway the goal is to be domain-agile.
See also: CosAE: Learnable Fourier Series for Image Restoration (2024)
https://sifeiliu.net/CosAE-page/
Was there a conclusion?
[2024]