Color Deconvolution / "Depainting"

[oshfdk]

I've been working on a machine learning based solution that would separate the image into layers of "vellum/ink/dirt (stains that are hard to classify)/scratches and defects/red paint/green paint/etc...", based on sampling of 3x3 pixel areas of the visible light TIFFs. The results so far have not been as good as I expected, it would detect traces of ink and paint in many random places, not as some distinct shapes, but more like blobs of random noise. And if I set a high enough detection threshold, then quite visible portions of ink or paint won't be classified correctly. At one point I realized, that there could very well be ink/paint traces all over the place due to ink/paint/tinted vellum particles shedding from opposite pages. So it's possible that there is no solution that can reliably separate layers using only local pixel values.

One can train a model that would have spacial awareness (working on colors AND shapes) and it might fare much better, but I'm personally against this, since models with spacial awareness will most likely introduce in their results some shapes that are not present in the original MS, and this will make any analysis based on the results from these models hard to validate.