Google Arts and culture VR

A Virtual Reality app for Google’s VR headsets (Google Cardboard, then Google Daydream).

The app lets you explore the Google Arts and culture collection, zoom in to see brush strokes, view artworks in real size, and listen to audio guides by expert museum curators

Textured artworks

For this project, I also worked on a technique to retrieve the tiny bumps on paintings, to give them a more realistic texture.

open in new tab

How it works

The technique is based on differential lighting: Take a picture with light coming from the top, another with light coming from the bottom, subtract the two, add some filtering (to get rid of the lighting gradient across the painting), and you have your horizontal bumps.

Repeat with left / right lighting, you get your vertical “bumps”. Combine that with the horizontal bumps, add some processing (eg. normalisation) and you get your normal map


Now, all we need to do is apply the technique to all the artworks in the Google Arts and Culture collection… That will be challenging, so instead we decided to get the help of some machine learning model

Using the examples we did have (like the city painting used in our tests) to train a computer to “go from one to the other”. Here is an example of what it has learned to do

  • Antoine Guerchais Unity development, prototypes
  • Jonathan Thanant Unity development, prototypes
  • Bastien Girschig Early prototypes, Unity development, Painting texture detection
  • Google Arts and Culture, Google VR The project involved multiple talented people from the Google Arts and culture team, as well as the Google VR team