Inpainting

Original and restored image.

Inpainting is the process of reconstructing lost or deteriorated parts of images and videos. In the museum world, in the case of a valuable painting, this task would be carried out by a skilled art conservator or art restorer. In the digital world, inpainting (also known as image interpolation or video interpolation) refers to the application of sophisticated algorithms to replace lost or corrupted parts of the image data (mainly small regions or to remove small defects).

Applications

There are many objectives and applications of this technique.

In photography and cinema, is used for film restoration; to reverse the deterioration (e.g., cracks in photographs or scratches and dust spots in film; see infrared cleaning). It is also used for removing red-eye, the stamped date from photographs and removing objects to creative effect.

This technique can be used to replace the lost blocks in the coding and transmission of images, for example, in a streaming video. It can also be used to remove logos in videos.

Methods

In painting

Inpainting is rooted in the restoration of images. Traditionally, inpainting has been done by professional restorers. The underlying methodology of their work is as follows:

The global picture determines how to fill in the gap. The purpose of inpainting is to restore the unity of the work.
The structure of the gap surroundings is supposed to be continued into the gap. Contour lines that arrive at the gap boundary are prolonged into the gap.
The different regions inside a gap, as defined by the contour lines, are filled with colors matching for those of its boundary.
The small details are painted, i.e. “texture” is added.

Computerized

Since the wide applications of digital camera and the digitalization of old photos, inpainting has become an automatic process that is performed on digital images. More than scratch removing, the inpainting techniques are also applied to object removal, text removal and other automatic modifications of images and videos. Furthermore, they can also be observed in applications like image compression and super resolution. Mainly three groups of 2D image inpainting algorithms can be found in literature. The first one to be noted is structural inpainting, the second one is texture inpainting and the last one is a combination of these two techniques. All these inpainting methods have one thing in common - they use the information of the known or undestroyed image areas in order to fill the gap.

Structural inpainting

Structural inpainting uses geometric approaches for filling in the missing information in the region which should be inpainted. These algorithms focus on the consistency of the geometric structure.

Textural inpainting

Like everything else the structural inpainting methods have both advantages and disadvantages. The main problem is that all the structural inpainting methods are not able to restore texture. Texture has a repetitive pattern which means that a missing portion cannot be restored by continuing the level lines into the gap.

Combined Structural and Textural inpainting

Combined structural and textural inpainting approaches simultaneously try to perform texture and structure filling in regions of missing image information. Most parts of an image consist of texture and structure. The boundaries between image regions accumulate structural information which is a complex phenomenon. This is the result when blending different textures together. That is why, the state of the art inpainting method attempts to combine structural and textural inpainting.

A more traditional method is to use differential equations (such as the Laplace's equation) with Dirichlet boundary conditions for continuity (a seamless fit). This works well if missing information lies within the homogeneous portion of an object area.^[1]

Other methods follow isophote directions (in an image, a contour of equal luminance), to do the inpainting.^[2]

Model based inpainting follows the Bayesian approach for which missing information is best fitted or estimated from the combination of the models of the underlying images as well as the image data actually being observed. In deterministic language, this has led to various variational inpainting models.^[3]

Manual computer methods include using a clone tool or healing tool, to copy existing parts of the image to restore a damaged texture. Texture synthesis may also be used.^[4]

Exemplar-based image inpainting attempts to automate the clone tool process. It fills "holes" in the image by searching for similar patches in a nearby source region of the image, and copying the pixels from the most similar patch into the hole. By performing the fill at the patch level as opposed to the pixel level, the algorithm reduces blurring artifacts caused by prior techniques.^[5]^[6]

References

↑ Peterson, Ivars (11 May 2002). "Filling in Blanks". Science News. Society for Science &#38. 161 (19): 299–300. doi:10.2307/4013521. JSTOR 4013521. Retrieved 2008-05-11.
↑ M. Bertalmío, G. Sapiro, V. Caselles and C. Ballester., "Image Inpainting", Proceedings of SIGGRAPH 2000, New Orleans, USA, July 2000.
↑ T. F. Chan and J. Shen, "Mathematical Models for Local Nontexture Inpainting", SIAM J. Applied Math., 62(3), 2001, 1019-1043.
↑ Image Replacement through Texture Synthesis, Homan Igehy and Lucas Pereira, Stanford University, Appears in the Proceedings of the 1997 IEEE International Conference on Image Processing
↑ Object Removal by Exemplar-Based Inpainting, Criminisi, A, Perez, P., & Toyama, K., Appears in the Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition
↑ Inpainting Strategies for Reconstruction of Missing Data in VHR Images, L. Lorenzi, F. Melgani and G. Mercier, IEEE Geoscience and Remote Sensing Letters, Set. 2011

External links

Interactive Point-and-Click Segmentation for Object Removal in Digital Images, ICCV-HCI 2005
Inpainting and the Fundamental Problem of Image Processing, SIAM
Image Inpainting, by the Image Processing Group at UCLA.
Image and Video Inpainting (webarchive) by Guillermo Sapiro
Video Inpainting by Kedar Patwardhan.
Mobile application which implements image inpainting by Bozhidar Dimitrov.
Program which realizes image inpainting by SoftOrbits.
Resynthesizer. An open source Gimp plug-in for inpainting and texture expanding.
Photo restoration software which implements image inpainting by Maxim Gapchenko
Mathematica Inpaint function.
Online Photo restoration software which implements image inpainting by Teorex
Photo touch up software for removing objects from photos called Photoupz
Inpainting of images on meshes or point clouds, TIP 2014
Image completion using planar structure guidance, SIGGRAPH 2014

Conservation-restoration of cultural heritage

Collection care and management	Collecting Collection (artwork) Collections care Collection catalog Collections maintenance Collections management (museum) Collection Management Policy Collections Management System Cultural heritage management Cultural resources management Deaccessioning (museum) Digital repository audit method based on risk assessment Display case Documentation of cultural property Emergency response (museum) Exhibition of cultural heritage objects Found in collection Inherent vice Inventory (museum) Museum integrated pest management Preservation metadata Preservation Metadata: Implementation Strategies Preservation of meaning Preservation survey Provenance Repatriation Restoration (cultural heritage) Storage of cultural heritage objects

Expertise	Art handler Collection manager Conservator-restorer Conservation scientist Conservation technician Curator Exhibition designer Mount maker Objects conservator Paintings conservator Photograph conservator Preservationist Registrar (museum) Textile conservator

Area specialities	Antiques restoration Archaeology Bioarchaeology Building restoration Conservation science Digital photograph restoration Digital preservation Database preservation Film preservation Frame conservation Heritage science Historic preservation Media preservation Object conservation Optical media preservation Painting conservation Preservation (library and archival science) Restoration Sustainable preservation Web archiving

Techniques	Aging (artwork) Anastylosis Arrested decay Archaeological sites Architecture Automobiles Bone, horn, and antler objects Books, manuscripts, documents and ephemera Ceramic objects Clocks Copper-based objects Feathers Flags and banners Frescos Fur objects Glass objects Historic gardens Human remains Illuminated manuscripts Iron and steel objects Ivory objects Lacquerware Leather objects Lighthouses Metals Musical instruments Neon objects New media art Outdoor artworks Outdoor bronze objects Outdoor murals Paintings Painting frames Parchment Photographs Plastic objects Shipwreck artifacts Silver objects Stained glass Textiles Tibetan thangkas Time-based media art Totem poles Vehicles Vinyl discs Woodblock prints Wooden furniture Cradling (paintings) Detachment of wall paintings Desmet method Fabric restoration Historic paint analysis Inpainting Kintsugi Leafcasting Lining of paintings Mass deacidification Mold control and prevention in libraries Mold prevention Overpainting Paper splitting Radiography of cultural objects Reconstruction (architecture) Textile stabilization Transfer of panel paintings UVC-based preservation

Notable projects	Conservation issues of Pompeii and Herculaneum Conservation-restoration of Ecce Homo by Elías García Martínez Conservation-restoration of Leonardo da Vinci's The Last Supper Conservation-restoration of the Shroud of Turin Conservation-restoration of the Sistine Chapel frescoes Conservation-restoration of the Statue of Liberty

This article is issued from Wikipedia - version of the 11/25/2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.