Visible Embryo Project

The Visible Embryo Project (VEP) is a multi-institutional, multidisciplinary research project originally created in the early 1990s as a collaboration between the Developmental Anatomy Center at the National Museum of Health and Medicine and the Biomedical Visualization Laboratory (BVL) at the University of Illinois at Chicago, "to develop software strategies for the development of distributed biostructural databases using cutting-edge technologies for high-performance computing and communications (HPCC), and to implement these tools in the creation of a large-scale digital archive of multidimensional data on normal and abnormal human development."^[1] This project related to BVL's other research in the areas of health informatics, educational multimedia, and biomedical imaging science.^[2][3][4] Over the following decades, the list of VEP collaborators grew to include over a dozen universities, national laboratories, and companies around the world.

An early (1993) goal of the project was to enable what it called "Spatial Genomics," to create tools and systems for three-dimensional morphological mapping of gene expression, to correlate data from the Human Genome Project with the multidimensional location of genomic expression activity within the morphological context of organisms. This led to the invention in the late 1990s by VEP collaborators of the first system for Spatial transcriptomics.^[5][6] Other areas that VEP researchers pioneered include early web technologies, cloud computing, blockchain, and virtual assistant technology.

The VEP was created in 1992 as a collaboration between the UIC Biomedical Visualization Laboratory, directed by Michael Doyle, and the Human Developmental Anatomy Center at the National Museum of Health and Medicine (NMHM), directed by Adrianne Noe. Doyle had been appointed to the oversight committee of the Visible Human Project at the National Library of Medicine, but it would be several years before that data would become available. Looking for other sources of high-resolution volume data on the human anatomical structure, he came across the Carnegie Collection of Human Embryology,^[7] housed at the NMHM. During a sabbatical working on methods for magnetic resonance microscopy (MRM) in the laboratory of Paul Lauterbur, Doyle created a plan for the VEP and worked with Noe to recruit a large group of prominent researchers to join as initial collaborators.^[8][9]

A primary goal of the project was to provide a testbed for the development of new technologies, and the refinement of existing ones, for the application of high-speed, high-performance computing and communications to current problems in biomedical science.^{[10][11][12][13][14][15]}

Much of the early work involved creating serial section reconstructions from microscope slides and extracting volumetric data from the NMHM specimens, rather than just surface data. Sets of serial microscopic cross-sections through human embryos (prepared by Carnegie Collection contributors between the 1890s and 1970s) were used as sample image data around which to design and implement various components of the system. These images were digitized and processed to create 3D voxel datasets representing embryonic anatomy. Standard techniques for 3D volume visualization could then be applied to these data.^[1][3] Image processing of these data was required to correct for certain artifacts that were found in the original microscope sections from routine histological techniques of the tissue preparation.

Later activities of the project would make use of MRM datasets acquired from the NMHM collection, ultra-high resolution histology images,^[16] and three-dimensional adult image data acquired via the Visible Human Project, in addition to embryo data.

The VEP became a far-reaching collaborative research program involving a large number of eminent scientists across the nation and around the world, including, among many others, Michael Doyle, of UIC, then UCSF, and project founder, Adrianne Noe, Director of the National Museum of Health and Medicine, George Washington University's Robert Ledley, inventor of the Full-body CT scanner, UIUC's Paul Lauterbur, MRI pioneer and Nobel laureate, LSU's Ray Gasser, eminent embryologist, Oregon Health & Science University's Kent Thornburg, internationally renown developmental biologist, Regan Moore, Director of the DICE group at the San Diego Supercomputer Center, William Lennon of Lawrence Livermore National Laboratory, Ingrid Carlbom of Digital Equipment Corporation's Cambridge Research Lab, and Demetri Terzopoulos of the University of Toronto.

Some notable Visible Embryo Project collaborations include:

In the mid-1990s, Michael Doyle collaborated with Harvard's Betsey Williams to create an internet atlas of mouse development, in a project named "Muritech." A prototype two- and three-dimensional color atlas of mouse development was developed, using two embryos, a 13.5 d normal mouse embryo and a PATCH mutant embryo of the same age. Serial sections of the embryos, with an external registration marker system, introduced into the paraffin embedding process, were prepared by standard histological methods. For the 2D atlas, color images were digitized from 100 consecutive sections of the normal embryo. For the 3D atlas, 300 gray-scale images digitized from the mutant embryo were conformally warped and reconstructed into a 3D volume dataset. The external fiducial system facilitated the three-dimensional reconstruction by providing accurate registration of consecutive images and also allowed for precise spatial calibration and the correction of warping artifacts. The atlases, with their associated anatomical knowledge base, were then integrated into a multimedia online information resource via the VEP's Web technology to provide research biologists with a set of advanced tools to analyze normal and abnormal murine embryonic development.^[17]

The Human Embryology Digital Library and Collaboratory Support Tools project was begun in 1999 as a demonstration of the biomedical application potential of the Next Generation Internet (NGI). The collaborators included eight organizations at sites around the continental USA, a mix of medical and information technology organizations, including George Mason University, Eolas, the Armed Forces Institute of Pathology, Johns Hopkins University, Lawrence Livermore National Laboratory, the Oregon Health & Science University, the San Diego Supercomputer Center, and the University of Illinois at Chicago. The project undertook three major applications, based on the Carnegie Collection of Embryos at the AFIP's National Museum of Health and Medicine Human Development Anatomy Center (HDAC), a collection of cellular-level tissue slides that is one of the world's largest repositories of human embryos.

These applications included:

1. Digitization, curation, and annotation of embryo data: The VEP team created a production digitization capability, using automated digital microscopy, with data automatically registered for tiling and transmitted to the repository at the San Diego Supercomputer Center, and annotated by teams of biomedical volunteers with expert-level quality control. 2. Distributed embryology education using materials derived from the Carnegie Collection to create animations of embryo development and recorded master classes that can be streamed over the Internet or downloaded to create a portable electronic classroom. 3. Clinical management planning where medical professionals and expectant parent patients can review normal and abnormal development patterns with collaborative consultation from distant experts.^[9]

To enable new ways to interactively explore the VEP's massive volume datasets, Michael Doyle created the zMap system, using the Visible Human Project image data for the first prototype. In 2011, Doyle collaborated with Steven Landers, Maurice Pescitelli, and others to use zMap to create an interactive tool that allows the user to select desired sets of anatomical structures for the automated generation of 3D Quicktime VR visualizations. The system used the resources available in the Eolas AnatLab knowledgebase, which has over 2200 structures identified involving a total of over 4600 sections and 700,000 annotations overall, to access the anatomical structure surface information for individual structures. This surface information was then used to automatically extract the contained volumetric image data and convert the data into a format compatible with the Osirix volume imaging system. Automated scripts then controlled Osirix in the creation of a 3D visualization of the group of selected anatomical structures. Photorealistic results were obtained by using the original color voxel information from the original Visible Human cryosection images to color the surface of the 3D reconstruction. The system then automatically progressed through a pre-defined set of rotations to generate the set of image frames required to create a Quicktime VR (QTVR) interactive movie. This system thereby allowed an anatomy instructor to quickly and easily generate customized interactive 3D reconstructions for use in the classroom.^[18]

Over the decades since it was begun, the work done in the Visible Embryo Project has led to the development of several important technological breakthroughs that have had a worldwide impact:

Even though spatial mapping of Omics data had been described as an initial goal of the VEP, it wasn't until 1999 that four VEP collaborators, Michael Doyle, George Michaels, Maurice Pescitelli, and Betsey Williams worked together to create a system for what they called "spatial genomics."^[5] Today, this technology is known as Spatial transcriptomics. As their 2001 U.S. patent application states,^[5] their system solved the need "to gather gene expression data in a manner that supports the type of exploratory research that can take advantage of the broad-spectrum types of biologic activity analysis enabled by today's microarray tools," as well as the need for "technology to allow the collection of large volumes of these types of data, to enable exploratory investigations into patterns of biologic activity ... to correlate gene expression data with morphological structure in a useful and easy to understand manner, such as in a volume visualization environment ... to allow the collection of larger volumes of gene expression data across a wider spectrum of gene types than ever before."

They named their system SAGA, short for Spatial Analysis of Genomic Activity. As described in the related U.S. patents,^[5][19][20] the SAGA system enabled the multidimensional morphological reconstruction of tissue biologic activity and "makes it possible for biological tissue specimens to be imaged in multiple dimensions to allow morphological reconstruction. The same tissue specimen is physically sampled in a regular raster array, so that tissue samples are taken in a regular multidimensional matrix pattern across each of the dimensions of the tissue specimen. Each sample is isolated and coded so it can be later correlated to the specific multidimensional raster array coordinates, thereby providing a correlation with the sample's original pre-sampling morphological location in the tissue specimen. Each tissue sample is then analyzed with broad-spectrum biological activity methods, providing information on a multitude of biologic functional characteristics [mRNA, etc.] for that sample. The resultant raster-based biological characteristic data may then be spatially mapped into the original multidimensional morphological matrix of image data. ... various types of analysis may then be performed on the resultant correlated multidimensional spatial datasets."

Spatial transcriptomics was named the "Method of the Year for 2020" by Nature, in January 2021.^[21]

In 1993, Doyle became the Director of the UCSF Center for Knowledge Management (CKM). To create the underlying software and hardware that would provide the needed computational power for the VEP, Doyle's CKM group designed a new paradigm for performing remote client-server volume visualization over the Internet.^[10] This involved creating a system for remotely computing visualizations through a networked cluster, or cloud, of distributed heterogeneous computational engines, and coordinating the computations to pass user interface control messages to those engines, causing the cloud computers to generate new rendered visualizations and stream the resulting views to the users' desktops, while delta-encoding and compressing the streamed data to optimize performance over low-bandwidth connections.^[22]

To hide the complexity of the system from the user, they modified one of the earliest versions of the NCSA Mosaic Web browser^[23] to allow their interactive cloud-computing applications to be automatically launched and run embedded within Web pages, so any user would need only to load a Web document from the VEP and would be able to immediately interactively explore the project's multidimensional datasets, rather than static representations of those datasets.^[24]

In November 1993, the CKM's VEP research group demonstrated this system, the first Web-based Cloud application platform, on-stage to a meeting of approximately 300 Bay Area SIGWEB members at Xerox PARC.^[25]

Today, this capability is called "the Cloud." The VEP team's work opened the door to the potential of the Web to provide rich information resources to users, regardless of where they were located and spawned a multi-trillion-dollar industry as a result.^[26]

Doyle then began to focus more directly on the problem of how to navigate within these complex biomedical volume datasets and developed a system for mapping the semantic identity of morphological structures within the datasets and integrating those mappings with the hypermedia linking mechanism of the Web. This led to the creation of the first three-dimensional Web image map system and was used to create a variety of online interactive reference systems for biomedical education and research throughout the 90s and beyond.^[27][28]

One of the challenges for large collaborative knowledge bases is how to assure the integrity of data over a long period of time. Standard cryptographic methods that depend upon trusting a central validation authority are vulnerable to a variety of factors that can lead to data corruption, including hacking, data breaches, insider fraud, and the possible disappearance of the validating authority itself. To solve this problem for the VEP data collections, Doyle created a novel type of cryptographic system, called Transient-key cryptography. This system allows the validation of data integrity without requiring users of the system to trust any central authority, and also represented the first decentralized blockchain system, enabling the later creation of the Bitcoin system. In the mid-2000s, this technology was adopted as a national standard in the ANSI ASC X9.95 Standard for trusted timestamps.^[29][30]

Since the mid-2000s, the VEP team has made great use of digital voice and text communications systems, to facilitate communications among geographically-distributed team members. To increase the efficiency of these communications, Michael Doyle and Steve Landers collaborated to create the Skybot system, the first AI-based mobile virtual assistant system. Skybot used the power and flexibility of AI to dramatically expand the use of messaging systems. Using Skybot, one could create a variety of programmable responses to incoming calls and chat messages. The system incorporated a state machine that could be configured to automatically trigger automated responses to various communication and user-context events. This provided the user with a surprisingly broad and powerful set of capabilities for automating mobile communication operations and pioneered the mobile intelligent-assistant product category that is now ubiquitous worldwide. ^[31][32]

Plans are underway to secure the funding necessary to expand the Visible Embryo Project to create a national resource that combines large-scale knowledgebase with advanced analytical tools in an innovative online collaborative environment to support and continue to advance the art and science of Spatial Omics.^[33]

• Blockchain
• Spatial transcriptomics
• Transient-key cryptography
• Virtual assistant
• Visible Human Project

• Home page of the Visible Embryo Project 2000-2004 NIH/NLM Next Generation Internet contract, hosted at George Mason University
• Pre-proposal white paper for the follow-on project to the Visible Embryo Project 2000-2004 NIH/NLM Next Generation Internet contract