Learn

What is RTI?

What Reflectance Transformation Imaging is, how it is captured and computed, where it comes from, and what it cannot do.

Definition

Reflectance Transformation Imaging (RTI) is a computational photography method for documenting and examining the surfaces of cultural heritage objects, among them inscriptions, reliefs, coins, cuneiform tablets, papyri and manuscripts. By letting the viewer control the lighting interactively and enhance the surface computationally, it reveals details that are difficult to distinguish in conventional photographs, particularly worn lettering, shallow decoration and traces of manufacture and use.

During capture, the camera and the documented surface stay fixed; only the direction of the light changes across a series of photographs. From these photographs a mathematical reflectance function is computed for every pixel, describing how that point's brightness changes with the light direction.

The result is not a single photograph but an interactive model: the light can be moved to any direction on screen, including directions never used during capture, which are reconstructed from the model.

Its main contribution to analysis lies in what can be computed from that model, beyond any single photograph. Diffuse Gain exaggerates how each point's brightness changes with the light, so that fine relief stands out more strongly than under any real light. Specular Enhancement keeps the relit image and adds a synthetic glossy reflection computed from the surface orientation estimated at each point, so the shape reads through the highlights. Shaded removes the colour and texture altogether and lights that orientation alone: worn lettering, tool and use marks and the texture of the surface are read by their shape, no longer hidden by colour, staining or weathering. Radiance Scaling (Vergne et al. 2010) goes one step further: it brightens raised and darkens hollow parts by the curvature of the surface, so that carved strokes read as dark lines, and can tint them warm and cool. Light at a low angle (raking light) is one way of reading the model; these enhancements are where RTI goes beyond it.

Moulded decoration of a terracotta ampulla fragment in the Natural view: a beaded border, a rosette and a running animal on orange clay, crossed by a break.Natural
The same decoration in Diffuse Gain: every bead, the coils of the rosette and the animal's body stand out strongly.Diffuse Gain
The same decoration in the Shaded view: without the colour of the clay, the moulding reads by its shape, down to the herringbone border and the animal's fur.Shaded
The same decoration as a normal map: colours encode the estimated direction each point of the surface faces.Normal Map
The same detail under one light (azimuth 135°, elevation 45°) in the Natural, Diffuse Gain (gain 2), Shaded and Normal Map views. The normal map encodes the direction each point is estimated to face (red right, green up, blue towards the viewer); it is not depth or a measurement. Open in the viewer: Natural, Diffuse Gain, Shaded, Normal Map, Specular Enhancement. RTI data: Late Roman Terracotta Ampulla Fragment (rti-0004), CC BY-NC 4.0; held by Antalya Museum. Computed from the published model with the viewer's formulas.

Capture methods

The light directions can be produced in three ways. All three end in the same kind of data: a set of images of one fixed view, each lit from one known direction, from which the RTI model is computed.

H-RTI (Highlight RTI)

A hand-held flash is moved around the object, kept at the same distance by a cord or a marked rod. One or two glossy black spheres in the frame record each light direction through their highlight. Light and portable, it works in the field, on large or fixed surfaces such as rock art, inscriptions and architecture (Mudge et al. 2006).

Equipment
Camera, hand-held flash, glossy sphere
Suited to
Any size; large and fixed surfaces, fieldwork
Limitation
Light positions less regular than in a dome
Highlight RTI: a fixed camera above a fixed object; a hand-held light is moved to many directions on an imaginary dome, one photograph per direction, and a glossy sphere records each direction. 1 2 3 4 5
  1. Camera: fixed
  2. Light: one photograph per direction
  3. Object: fixed
  4. Glossy sphere: its highlight shows the light direction
  5. Raking light: reaches the surface at 15° and shows relief through its shadows
H-RTI: camera and object stay still while a hand-held light is placed at many directions at the same distance from the object. The glossy sphere in the frame shows, through its highlight, where the light came from in each photograph.

Dome RTI

The object is placed under a hemispherical dome whose inside carries dozens of fixed lights (typically 40–100) in rings at known angles. The camera looks down through the top; the lights fire one after another, automatically. Every capture uses the same calibrated directions, so a full set is taken in minutes and is easy to repeat; the size of the dome limits the size of the object (Earl et al. 2011).

Equipment
Dome of fixed lights, camera
Suited to
Small and portable objects; repeatable captures
Limitation
The object must fit under the dome
Dome RTI: the object sits under a hemispherical dome whose inside carries many fixed lights at known positions; a camera looks down through an opening at the top and the lights fire one at a time.

Virtual RTI (V-RTI)

No lights are used on site: a 3D model of the surface, made by photogrammetry or structured-light scanning, is lit in rendering software by a virtual dome of lights, and one image is rendered per light. Useful where a physical light cannot be moved around the surface, such as narrow rock-cut chambers or tall walls. The result depends on the quality of the 3D model: detail finer than the mesh cannot appear (Greaves et al. 2020; Di Iorio et al. 2024).

Equipment
3D model, rendering software
Suited to
Surfaces where no light can be moved
Limitation
No detail finer than the 3D mesh
Virtual RTI: a 3D model of the surface, made by photogrammetry or scanning, is placed in a virtual dome of lights in rendering software; a virtual camera renders one image per light.

Scientific background

Photometric stereo. The theoretical roots of RTI lie in photometric stereo, a computer vision method. Robert J. Woodham (1980) showed that a few images taken from a fixed viewpoint under different lighting directions are enough to determine the surface orientation (normal) and reflectance at each point. RTI applies the same principle, but to examine a surface visually rather than to measure it.

Image-based relighting. Reflectance as a function of light direction is represented by a small number of coefficients per pixel. New lighting conditions can therefore be computed from photographs alone, without a detailed 3D model of the surface or knowledge of its material properties. The main mathematical models are:

  • PTM (Polynomial Texture Map): a second-order polynomial in the two components of the light direction for each pixel (six coefficients). Defined by Malzbender, Gelb and Wolters (2001); the same paper introduced image enhancement operations such as diffuse gain and specular enhancement.
  • HSH (Hemispherical Harmonics): a basis developed to represent functions defined only on the hemisphere (Gautron, Křivánek, Pattanaik and Bouatouch 2004). In RTI it is used as an alternative to PTM that can give more accurate results, particularly on shiny surfaces (Mudge et al. 2008).
  • PCA + RBF: the photograph stack is compressed with principal component analysis and interpolated between light directions with radial basis functions. Ponchio, Corsini and Scopigno (2018) showed that with this approach high-quality RTI can be delivered over the web at the size of a few JPEG images.

History and developers

1980
Photometric stereo. Robert J. Woodham publishes a method for determining surface orientation under varying illumination.
2001
Polynomial Texture Maps (PTM). Tom Malzbender, Dan Gelb and Hans Wolters, research scientists at Hewlett-Packard Laboratories, present the method they developed there in a landmark paper at SIGGRAPH 2001 (Polynomial texture maps). It is the direct starting point of RTI.
2002
Cultural Heritage Imaging (CHI). Mark Mudge and Carla Schroer found the non-profit CHI in San Francisco. They had begun working with Malzbender shortly after seeing the PTM work at SIGGRAPH 2001; CHI takes a leading role in applying the method to cultural heritage and spreading its use.
2004
Hemispherical harmonics. Gautron and colleagues define the HSH basis for reflectance functions on the hemisphere; RTI software later adopts it as an alternative to PTM.
2006
Highlight-RTI (H-RTI). Mudge, Malzbender, Schroer and Lum publish the method in which the light direction is computed from the highlight on a glossy black sphere placed in the frame. It makes RTI capture possible in the field with a camera and a hand-held flash, without a fixed rig. CHI develops the open-source RTIBuilder software supporting the method with a team at the University of Minho, Portugal.
2008
RTIViewer. An open-source viewer developed principally by the Visual Computing Laboratory of ISTI, part of the Italian National Research Council (CNR), with contributions from the University of California, Santa Cruz, the University of Minho, Tom Malzbender and CHI. It makes PTM and HSH files widely accessible to researchers.
2011
Dome systems and ancient documents. The AHRC-funded RTISAD project (Earl et al. 2011) develops and tests a range of capture and processing techniques, including light domes, for ancient documents and archaeological surfaces; its datasets include examples from Herculaneum, Abydos and Çatalhöyük.
2018
Compressed RTI for the web. Federico Ponchio, Massimiliano Corsini and Roberto Scopigno (ISTI-CNR) present a new PCA- and RBF-based representation at Web3D 2018 (best paper award). The software from this work, relight, allows RTI to be shown in the browser at high quality with little data; an extended version is published in 2019.
Today
New models and applications. Approaches such as multispectral RTI, capture at microscopic scale and neural relighting models are being researched (for an overview, Pintus et al. 2019; neural relighting, Dulecha et al. 2020). Guidance for heritage institutions has also been published (e.g. by Historic England).

OpenRTI was developed independently from the public descriptions of these methods so that it can read data produced with relight; it contains no code from the software named above and implies no partnership with the people or institutions mentioned.

Scientific limits

An RTI is not a 3D model and gives no depth measurement; enhanced views support, but do not replace, expert interpretation.

  • The Normal Map view shows estimated directions from the model as colour; it is not depth or measurement.
  • Diffuse Gain shows relief more strongly than any real light would, and the gloss of Specular Enhancement and the grey of Shaded and Radiance Scaling and the latter's warm and cool tints are synthetic: they show the estimated orientation of the surface, not the material's real gloss or colour.
  • An enhanced view is not raw data. On OpenRTI every mode other than Natural, and any sharpening, is flagged on screen as “Enhanced view”, and a shared link carries the settings used.
  • The capture setup, light calibration, model fit and image compression all affect the result. When interpreting a faint trace, compare several light directions.
  • Models are most reliable within the range of light directions used during capture; interpolation at very low light angles can produce artificial shadows and highlights.
  • Unless a scale calibration is provided, on-screen zoom gives no physical measurement.
  • OpenRTI does not add detail to images with generative AI; everything on screen is computed from the recorded RTI data.
  • “Published” status does not mean peer review or institutional endorsement.

References

Works cited

  1. Di Iorio, F., Es Sebar, L., Croci, S., Taverni, F., Auenmüller, J., Pozzi, F. and Grassini, S. (2024). The use of Virtual Reflectance Transformation Imaging (V-RTI) in the field of cultural heritage: approaching the materiality of an ancient Egyptian rock-cut chapel. Applied Sciences, 14(11), 4768. doi:10.3390/app14114768
  2. Dulecha, T. G., Fanni, F. A., Ponchio, F., Pellacini, F. and Giachetti, A. (2020). Neural reflectance transformation imaging. The Visual Computer, 36(10–12), 2161–2174. doi:10.1007/s00371-020-01910-9
  3. Earl, G. et al. (2011). Reflectance transformation imaging systems for ancient documentary artefacts. EVA London 2011: Electronic Visualisation and the Arts, 147–154. doi:10.14236/ewic/EVA2011.27
  4. Gautron, P., Křivánek, J., Pattanaik, S. and Bouatouch, K. (2004). A novel hemispherical basis for accurate and efficient rendering. Eurographics Symposium on Rendering 2004, 321–330. Eurographics. doi:10.2312/EGWR/EGSR04/321-330
  5. Greaves, A. M., Duffy, S., Peterson, J. R., Tekoğlu, Ş. R. and Hirt, A. (2020). Carved in stone: field trials of Virtual Reflectance Transformation Imaging (V-RTI) in Classical Telmessos (Fethiye). Journal of Field Archaeology, 45(7), 542–555. doi:10.1080/00934690.2020.1804135
  6. Malzbender, T., Gelb, D. and Wolters, H. (2001). Polynomial texture maps. Proceedings of SIGGRAPH 2001, 519–528. ACM. doi:10.1145/383259.383320
  7. Mudge, M., Malzbender, T., Chalmers, A., Scopigno, R., Davis, J., Wang, O., Gunawardane, P., Ashley, M., Doerr, M., Proenca, A. and Barbosa, J. (2008). Image-based empirical information acquisition, scientific reliability, and long-term digital preservation for the natural sciences and cultural heritage. Eurographics 2008 Tutorials. Eurographics. doi:10.2312/egt.20081050
  8. Mudge, M., Malzbender, T., Schroer, C. and Lum, M. (2006). New reflection transformation imaging methods for rock art and multiple-viewpoint display. VAST 2006: 7th International Symposium on Virtual Reality, Archaeology and Intelligent Cultural Heritage, 195–202. Eurographics. doi:10.2312/VAST/VAST06/195-202
  9. Pintus, R., Dulecha, T. G., Ciortan, I., Gobbetti, E. and Giachetti, A. (2019). State-of-the-art in multi-light image collections for surface visualization and analysis. Computer Graphics Forum, 38(3), 909–934. doi:10.1111/cgf.13732
  10. Ponchio, F., Corsini, M. and Scopigno, R. (2018). A compact representation of relightable images for the web. Proceedings of the 23rd International ACM Conference on 3D Web Technology (Web3D '18). doi:10.1145/3208806.3208820
  11. Ponchio, F., Corsini, M. and Scopigno, R. (2019). RELIGHT: A compact and accurate RTI representation for the web. Graphical Models, 105. doi:10.1016/j.gmod.2019.101040
  12. Woodham, R. J. (1980). Photometric method for determining surface orientation from multiple images. Optical Engineering, 19(1), 139–144. doi:10.1117/12.7972479

Further reading

  1. Akçay, A. (2016). Epigrafi araştırmalarında yeni bir belgeleme ve analiz metodu olarak RTI [RTI as a new documentation and analysis method in epigraphical studies]. Mediterranean Journal of Humanities, 6(2), 1–16. doi:10.13114/MJH.2016.285
  2. Earl, G., Martinez, K. and Malzbender, T. (2010). Archaeological applications of polynomial texture mapping: analysis, conservation and representation. Journal of Archaeological Science, 37(8), 2040–2050. doi:10.1016/j.jas.2010.03.009

Guidance and format records