Encode your digital data into DNA, then keep it on ice

The Global Seed Vault on the Arctic island of Svalbard. The vault presently stores almost half a million seed samples. Image source: Gizmodo.

Recent articles by Gizmodo and New Scientist follow the publication of an interesting paper by a Swiss team from Zurich’s Swiss Federal Institute of Technology. The paper highlights the challenges of physically preserving digital data for periods of greater than 50 years. It suggests, as an alternative to current methods, sequencing digital data as DNA specifying “we translated 83 kB of information to 4991 DNA segments, each 158 nucleotides long, which were encapsulated in silica”.

The 83kB in question were from the Swiss federal charter from the 13th century and the 10th century Archimedes Palimpsest – both items of heritage deemed worthy of being incorporated into the experiment (and unlikely to present copyright issues). In terms of performance, the article states that accelerated aging was undertaken at 70°c for a week and showed considerable promises of longevity, “thermally equivalent to storing information on DNA in central Europe for 2000 years”. The data was recovered without error, and the team even integrated error-correcting codes similar to traditional approaches to the archiving of digital data. With this experimental evidence in hand, the team’s paper notes that “The corresponding experiments show that only by the combination of the two concepts, could digital information be recovered from DNA stored at the Global Seed Vault (at -18°C) after over 1 million years”.

CyArk uses a bespoke LTFS setup as a practical stable-media solution in the archiving of its terabytes of data, so while side-by-side 83kB may not sound like a lot for our purposes, potentially “just 1 gram of DNA is theoretically capable of holding 455 exabytes”. The economics of this may not add-up yet, as it is indicated that it costed around £1000 to encode this small excerpt of data. Ultimately, if we want to see this level of DNA data storage and access become feasible or an everyday reality, it will need among other things strong market forces behind its adoption and development. I am reminded of this picture of the 1956 IBM RAMDAC computer which included the IBM Model 350 disk storage system (seen below), storing 5MB of data and hired at $3200/month, equivalent to the purchase price of $160,000.

1956 IBM RAMDAC computer included the IBM Model 350 disk storage system. Image source: Faber.se

What would a laser scanner look like at 20 billion frames per second?

To help answer the perennial question ‘What does a laser path look like in slow motion?’ a team of researchers (published Open Access in Nature in August 2014) undertook an experiment at Heriot-Watt University that used a 2D silicon CMOS array of single-photon avalanche diodes (‘SPAD’) to essentially construct a 0.1 megapixel resolution camera capable of recording the path of laser pulses through a given area. While the article acknowledges that light has been captured in flight as early as 1978, the challenge addressed by the team is one of simplifying data acquisition and reducing acquisition times “by achieving full imaging capability and low-light sensitivity while maintaining temporal resolution in the picosecond regime” (Gariepy et al, 2014: 2). To produce an image (or rather a video) from the experiment, the raw sensor data was put through a sequence of processing steps categorised into 3 stages: noise removal, temporal deconvolution and re-interpolation – which is illustrated in the graphic below:

Creating an image from the 32 x 32 array of sensors. Image from article (linked).

Creating an image from the 32 x 32 array of sensors. Image from article (linked).

The video produced by the team (GIF excerpt below) is an overlay of the 500 picosecond pulse of laser light on top of a DSLR photograph of the experimental setup. The scattering of light that makes the beam visible is remarkably only through interaction with ambient gas molecules (Gariepy et al, 2014: 4), versus a more ‘dense’ medium that is traditionally required to highlight laser light (e.g. fog, participating media such as airborne dust, etc).

GIF of laser path. Image created from video (linked)

GIF of laser path. Image created from video (linked)

This laser path in flight is the missing step from the following video produced as part of the Scottish Ten project: we see the Leica C10 scanner laser spot reflecting from the surface of a bridge at the Eastern Qing Tomb, China. If we applied the same methodology as the research team in the article to the scanner, we might see the same phenomenon repeated at incremental spatial locations to record the environment around the scanner – perhaps the ultimate LiDAR demo?

Microsoft HoloLens mixes the real and the virtual

The HoloLens headset. Image source: Engadget (linked).

With the announcement of Windows 10 in the live event, Microsoft also recently presented the HoloLens. This is a ‘mixed reality’ headset that overlays ‘holographic’ imagery over your day-to-day vision, allowing you to interact virtually – make Skype calls, build 3D objects in their HoloStudio software, play HoloBuilder (essentially MineCraft), and so on – untethered & without markers/tracking. The specification of it are unclear at this point, described variously as ‘sharp’ and having ‘HD lenses’, and in the presentation it is explained that a traditional CPU/GPU combination were not enough and that the answer was in inventing a ‘HPU’ (‘Holographic Processing Unit’), which deals with the input from various sensors detecting sound, our gestures and ‘spatially map the world around us’ in real-time.

It requires little imagination to visualise how units like this could integrate with archaeological excavation, training, virtual access/reconstruction, etc, in a similar vein to how it has already been employed by NASA. To quote Dave Lavery, program executive for the Mars Science Laboratory mission at NASA Headquarters in Washington: “OnSight gives our rover scientists the ability to walk around and explore Mars right from their offices [...] It fundamentally changes our perception of Mars, and how we understand the Mars environment surrounding the rover.”

Exploring Mars - Microsoft worked with NASA's Jet Propulsion Laboratory team and the Curiosity Mars rover

Exploring Mars – Microsoft worked with NASA’s Jet Propulsion Laboratory team and the Curiosity Mars rover to explore how engineers, geologists, etc could use the technology. Image source: Frame from video (linked).

We’ve all long been aware of the development of consumer VR headsets (e.g. Oculus Rift) – which can immerse us in entirely 3D environments, and Google’s Glass (Prototype production has now ended). HoloLens is an interesting move from Microsoft, and somewhat curiously there is also an absence of reference to ‘augmented reality’ (see Microsoft’s FAQ), which has been suggested may be for marketing purposes. In terms of availability, we are told the HoloLens will be made available within the timeframe of Windows 10. Going forwards it will be a question of which applications befit VR/AR – especially for heritage (and conservation), where virtual access to the present or the past in 2D and 3D form is a central requirement.

Faro’s new handheld scanner (‘Freestyle 3D’)

Faro Freestyle 3D handheld scanner. Image source http://www.faro.com/

Faro Freestyle 3D handheld scanner. Image source http://www.faro.com/

Faro surprised us at the start of this year with the release of a new handscanner, the Freestyle 3D. It is being marketed as a lightweight, mobile tool that can rapidly enable the capture of accurate full-colour pointcloud data on its own, or in tandem with Faro’s product line of terrestrial laser scanners such as the Focus and X330, where its direct integration with Faro’s Scene package gives it a workflow advantage. The hardware specification puts the point accuracy at less than 1.5mm under certain conditions, measuring at a distance between 0.5m and 3m from the subject.

It has already made a splash in the scanning community, with comparisons and early tasters of the handscanner being used in real-world scenarios. Being marketed at the AEC industries, it will be interesting to see if or how it is employed within the context of digital documentation of cultural heritage, which frequently puts us in situations with irregular, hard-to-reach built and natural environments.

Pointillism (Pic of the week | 20141212)

The Banquet Scene

The Banquet Scene – Captured with the Artec Eva at .6mm resolution (normal and diffuse maps transition image)

Alabaster wall panel relief fragment; garden scene; birds and a locust in the trees; the king has left his weapons on a table to the right; he reclines on a sofa beneath a vine and his queen sits opposite him; they are drinking and refreshments are on the table; maids fan the royal couple; others bring food or play music; suspended from a tree behind the queen is the head of Teumman, king of Elam; the furniture is very elaborate. © The Trustees of the British Museum

The scanned material is greater than the sum of its maps

Link to video

Quixel Megascans test scene. Top: Material mapped render; Bottom: Rendered scene with no materials. Image source: http://quixel.se/megascansintro

In seeking to model and render CGI environments, material shaders and their respective maps have always sat as the foundation of making the sterile virtual environment appear realistic. Data capture methods that bring 3D objects into virtual environments go some way in bringing a facsimile of the real into an entirely virtual place, and typically a mesh that was generated by Structure from Motion photogrammetry or structured light scanning for example will feature a diffuse texture map. Taking these principles, Quixel Megascans is a service and suite that allows artists/modellers alike to take advantage of a library of scanned-in material maps with a range of acquired parameters. They even built their own material scanner to generate these maps.

Breakdown of maps that constitute material parameters

Breakdown of maps that constitute material parameters

This is big, because it hasn’t previously been so accessible to take advantage of captured material data and integrate it into a 3D workflow with your everyday BRDF shader in 3DS Max. It means 3D reconstructions of cultural heritage sites, for example, don’t have to be accurate 3D data punctuated with environments of artistic representations of foliage, generic mudbrick, masonry, etc, but are physically based representations of those materials from colour to translucency and specular reflections (Quixel list captured maps as PBR Calibrated Albedo, Gloss, Normal, Translucency, Cavity, Displacement, AO, Bump and Alpha). This is exciting because, alongside a trend of movement from biased towards unbiased rendering algorithms and the continual advance of computational power, these richer environments aren’t just increasingly ‘realistic looking’ but actually become more accurate representations of natural and built environments.

3D scanning the US President: The new portrait

Link to video

3D SLS print of President Obama’s captured portrait. Image source: Smithsonian Institute DPO

The White House recently announced its most cutting edge use of 3D data capture technology for presidential portraiture with the release of a video examining the project and processes behind it. Initially proposed two years ago by the Smithsonian Institution as a way of documenting the president, the project used a 3D methodology that evoked the prior scanning of the Lincoln life masks: presidential casts originally taken in 1860 and 1865 respectively.

Over a century and a half later, this project saw the Smithsonian Institution use the handheld structured-light scanner Artec EVA (up to 0.5mm spatial resolution), combined with data captured in a Mobile Light Stage setup by partners at the University of Southern California’s Institute for Creative Technologies. With the Artec data and over 80 photographs captured via the MLS, the dataset was augmented with handheld photography, prior to quality assessment and dispatching the data to Autodesk for processing. 72 hours later having registered, unified and normalised spatial and photographic data, plus the addition of a modelled plinth, the produced mesh output consisted of 15 million triangles.

The final step to transition the digital mesh to a physical bust saw the transfer of the data to 3D Systems, and utilised 3D printing using SLS (Selective Laser Sintering) to create an accurate representation of the dataset, standing 19in (48cm) tall and weighing around 13lbs (5.8kg). The prints will be entered into the Smithsonian’s National Portrait Gallery collection alongside the raw data from the scanning.

For further information see the original Smithsonian blog from the Digitization Program Office detailing the project.

Data Management on the Scottish Ten Project

To echo a couple of the previous blogs, this is not my first Scottish Ten digital documentation project. While I was a part of the survey undertaken in November 2012 at the Eastern Qing Tombs in China, my focus was solely on-site data capture.

However, Nagasaki is my first Scottish Ten project where my role has been primarily data management and processing, which means assembling the data captured by the team, and registering and ensuring updated quadruplicate backups of over a terabyte of crucial survey information. I had arrived with a pre-prepared spreadsheet to help manage the multiple scanning and photography inputs and backup completion, though this quickly saw some amendments to fit the job and equipment at hand.

Some key equipment every office should have (Taken at Kosuge)

I have been well accommodated at the different sites so far with temporary office space: at the Kosuge slip-dock I was located in a nearby community hall, furnished with traditional Japanese tatami mats. At the working Mitsubishi dock my office was located within the footprint of the Cantilever crane itself, giving the team easy access to offload data during the day and preview areas that have been captured.

Reviewing data with Mr Hachiya of Mitsubishi Heavy Industries at the Giant Cantilever Crane office.

The crane in particular presented a real challenge for capturing data. Data coverage of steel beams with flanges and self-occluding structural elements could only be checked from different positions with other registered data. This meant tying together and examining data sets from the two jib teams and the ground team’s laser scanners to ensure there were no shadows or voids in the overall 3D point cloud.

To add to the difficulty, as the majority of the crane was scanned in a single static position, capturing all sides of the movable jib (overhanging the water, it was later rotated 180 degrees) meant delving into the pointclouds, cutting out the desired components then registering. With 5 scanners on the go from a variety of manufacturers (4 plus a scanning-capable ‘multi station’, the Leica MS50) it was easy to see the number of points captured for the crane registration alone soar to over a billion in a couple of days.

A perspective view of the laser scan data up the Kosuge slipway.

The same data management challenges are true of the Kosuge slip-dock and the No. 3 dry dock, with the primary focus of Hashima Island (aka ‘Gunkanjima’) being the vast amount of data from the various types of photography (including time-lapse, panoramic, HDR, 2D and 3D video). This will serve as a rich high resolution record of areas of the island in their current state, which are usually inaccessible to the public.

-Adam Frost