Image Completion with Discriminator Guided Context Encoder

Fatih Altay, Senem Velipasalar

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Image completion or inpainting is a technique that is used for reconstruction of damaged or distorted regions in an image. In this paper, a new convolutional neural network model is presented for image completion. The proposed method is based on an auto-encoder and a Generative Adversarial Network structure. To succeed at this task and produce a plausible output for the damaged or distorted region(s), the auto-encoder part of the network needs to understand the content of the entire image. On the other hand, discriminators that are used in the proposed network are responsible for deciding whether or not the inpainted output has the expected quality. The general discriminator looks at the entire image to evaluate if it is consistent as a whole, while the local discriminator looks only at the completed region to ensure the local consistency of the generated patches. The image completion network is then trained so that the discriminator networks conclude that the inpainted image is as real as the original. This approach is also aimed to reconstruct images regardless of where the damaged regions are. We compared our proposed method with two other approaches, and the results show that it performs better especially on images with low-texture.

Original languageEnglish (US)
Title of host publicationConference Record of the 52nd Asilomar Conference on Signals, Systems and Computers, ACSSC 2018
EditorsMichael B. Matthews
PublisherIEEE Computer Society
Pages2220-2224
Number of pages5
ISBN (Electronic)9781538692189
DOIs
StatePublished - Feb 19 2019
Event52nd Asilomar Conference on Signals, Systems and Computers, ACSSC 2018 - Pacific Grove, United States
Duration: Oct 28 2018Oct 31 2018

Publication series

NameConference Record - Asilomar Conference on Signals, Systems and Computers
Volume2018-October
ISSN (Print)1058-6393

Conference

Conference52nd Asilomar Conference on Signals, Systems and Computers, ACSSC 2018
CountryUnited States
CityPacific Grove
Period10/28/1810/31/18

Fingerprint

Discriminators
Textures
Neural networks

Keywords

  • context encoder
  • discriminator
  • Image completion
  • inpainting

ASJC Scopus subject areas

  • Signal Processing
  • Computer Networks and Communications

Cite this

Altay, F., & Velipasalar, S. (2019). Image Completion with Discriminator Guided Context Encoder. In M. B. Matthews (Ed.), Conference Record of the 52nd Asilomar Conference on Signals, Systems and Computers, ACSSC 2018 (pp. 2220-2224). [8645434] (Conference Record - Asilomar Conference on Signals, Systems and Computers; Vol. 2018-October). IEEE Computer Society. https://doi.org/10.1109/ACSSC.2018.8645434

Image Completion with Discriminator Guided Context Encoder. / Altay, Fatih; Velipasalar, Senem.

Conference Record of the 52nd Asilomar Conference on Signals, Systems and Computers, ACSSC 2018. ed. / Michael B. Matthews. IEEE Computer Society, 2019. p. 2220-2224 8645434 (Conference Record - Asilomar Conference on Signals, Systems and Computers; Vol. 2018-October).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Altay, F & Velipasalar, S 2019, Image Completion with Discriminator Guided Context Encoder. in MB Matthews (ed.), Conference Record of the 52nd Asilomar Conference on Signals, Systems and Computers, ACSSC 2018., 8645434, Conference Record - Asilomar Conference on Signals, Systems and Computers, vol. 2018-October, IEEE Computer Society, pp. 2220-2224, 52nd Asilomar Conference on Signals, Systems and Computers, ACSSC 2018, Pacific Grove, United States, 10/28/18. https://doi.org/10.1109/ACSSC.2018.8645434
Altay F, Velipasalar S. Image Completion with Discriminator Guided Context Encoder. In Matthews MB, editor, Conference Record of the 52nd Asilomar Conference on Signals, Systems and Computers, ACSSC 2018. IEEE Computer Society. 2019. p. 2220-2224. 8645434. (Conference Record - Asilomar Conference on Signals, Systems and Computers). https://doi.org/10.1109/ACSSC.2018.8645434
Altay, Fatih ; Velipasalar, Senem. / Image Completion with Discriminator Guided Context Encoder. Conference Record of the 52nd Asilomar Conference on Signals, Systems and Computers, ACSSC 2018. editor / Michael B. Matthews. IEEE Computer Society, 2019. pp. 2220-2224 (Conference Record - Asilomar Conference on Signals, Systems and Computers).
@inproceedings{7e0216a9c88e4683aab88c2c10f09a38,
title = "Image Completion with Discriminator Guided Context Encoder",
abstract = "Image completion or inpainting is a technique that is used for reconstruction of damaged or distorted regions in an image. In this paper, a new convolutional neural network model is presented for image completion. The proposed method is based on an auto-encoder and a Generative Adversarial Network structure. To succeed at this task and produce a plausible output for the damaged or distorted region(s), the auto-encoder part of the network needs to understand the content of the entire image. On the other hand, discriminators that are used in the proposed network are responsible for deciding whether or not the inpainted output has the expected quality. The general discriminator looks at the entire image to evaluate if it is consistent as a whole, while the local discriminator looks only at the completed region to ensure the local consistency of the generated patches. The image completion network is then trained so that the discriminator networks conclude that the inpainted image is as real as the original. This approach is also aimed to reconstruct images regardless of where the damaged regions are. We compared our proposed method with two other approaches, and the results show that it performs better especially on images with low-texture.",
keywords = "context encoder, discriminator, Image completion, inpainting",
author = "Fatih Altay and Senem Velipasalar",
year = "2019",
month = "2",
day = "19",
doi = "10.1109/ACSSC.2018.8645434",
language = "English (US)",
series = "Conference Record - Asilomar Conference on Signals, Systems and Computers",
publisher = "IEEE Computer Society",
pages = "2220--2224",
editor = "Matthews, {Michael B.}",
booktitle = "Conference Record of the 52nd Asilomar Conference on Signals, Systems and Computers, ACSSC 2018",
address = "United States",

}

TY - GEN

T1 - Image Completion with Discriminator Guided Context Encoder

AU - Altay, Fatih

AU - Velipasalar, Senem

PY - 2019/2/19

Y1 - 2019/2/19

N2 - Image completion or inpainting is a technique that is used for reconstruction of damaged or distorted regions in an image. In this paper, a new convolutional neural network model is presented for image completion. The proposed method is based on an auto-encoder and a Generative Adversarial Network structure. To succeed at this task and produce a plausible output for the damaged or distorted region(s), the auto-encoder part of the network needs to understand the content of the entire image. On the other hand, discriminators that are used in the proposed network are responsible for deciding whether or not the inpainted output has the expected quality. The general discriminator looks at the entire image to evaluate if it is consistent as a whole, while the local discriminator looks only at the completed region to ensure the local consistency of the generated patches. The image completion network is then trained so that the discriminator networks conclude that the inpainted image is as real as the original. This approach is also aimed to reconstruct images regardless of where the damaged regions are. We compared our proposed method with two other approaches, and the results show that it performs better especially on images with low-texture.

AB - Image completion or inpainting is a technique that is used for reconstruction of damaged or distorted regions in an image. In this paper, a new convolutional neural network model is presented for image completion. The proposed method is based on an auto-encoder and a Generative Adversarial Network structure. To succeed at this task and produce a plausible output for the damaged or distorted region(s), the auto-encoder part of the network needs to understand the content of the entire image. On the other hand, discriminators that are used in the proposed network are responsible for deciding whether or not the inpainted output has the expected quality. The general discriminator looks at the entire image to evaluate if it is consistent as a whole, while the local discriminator looks only at the completed region to ensure the local consistency of the generated patches. The image completion network is then trained so that the discriminator networks conclude that the inpainted image is as real as the original. This approach is also aimed to reconstruct images regardless of where the damaged regions are. We compared our proposed method with two other approaches, and the results show that it performs better especially on images with low-texture.

KW - context encoder

KW - discriminator

KW - Image completion

KW - inpainting

UR - http://www.scopus.com/inward/record.url?scp=85062979548&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85062979548&partnerID=8YFLogxK

U2 - 10.1109/ACSSC.2018.8645434

DO - 10.1109/ACSSC.2018.8645434

M3 - Conference contribution

AN - SCOPUS:85062979548

T3 - Conference Record - Asilomar Conference on Signals, Systems and Computers

SP - 2220

EP - 2224

BT - Conference Record of the 52nd Asilomar Conference on Signals, Systems and Computers, ACSSC 2018

A2 - Matthews, Michael B.

PB - IEEE Computer Society

ER -