While TV shows and movies often portray the isolated genius as someone who can save the world by working alone, collaboration is necessary to accomplish the most extraordinary projects. Conferences are a great way to discover these collaborations. One notable example is actually shown by two of the keynote speakers at AMUG 2023, Nicholas Jacobson, a professor at the University of Colorado’s Anschutz Medical Campus, and Rob Ducey, a technical supervisor at animation specialist Laika Studios. At AMUG 2019, they met and fell in love with bitmap printing, a relatively new 3D modeling and printing technique. They are now working together to develop innovative designs for pediatric epilepsy and cardiology. We spoke with them to find out more.
3DN: Please introduce yourself.
Nick: Hi, I’m Nick Jacobson and I’ve been involved in 3D printing since 2004. My background is in architecture and I started 3D printing at an architecture school. I used binder jetting with refined cornstarch on a Zcorp printer. We were making architectural and landscape models at the height of the computational design boom. As students, we were just beginning to learn to program and code. We became interested in using digital fabrication to create uncommon forms.
My love for form generation was a constant throughout my career as an architect, working with greats such as Zaha Hadid, Renzo Piano and Harry Teague. After years of practice, I was accepted to Harvard Graduate School of Design (HGBS) and Harvard Business School. I was part of an early research team that looked into Voxel Printing using what was then Connex, and now Stratasys. Voxel technology was used to develop software that could model and 3D print. Initial work was focused on structural engineering, design, and other aspects.
At the University of Colorado, on the Anschutz Medical Campus, I am currently a laboratory focused computer and 3D printer for medical applications. Our Harvard software has been useful in the medical field. All it takes is to modify the input data. It may seem like a stretch but it’s very similar to working with structural analysis data. Currently, we are focused in 3 areas: Surgical guidance and education models, Prosthetics, and Implants… all revolving around soft tissue applications.
Rob Ducey (left), and Nick Jacobson (right).
Rob: Hello, I’m Rob Ducey and I’m currently the technical supervisor for what we call rapid prototype at LAIKA. Our group focuses on modeling, facial animation and fabrication of complex puppets, which are the stars in our films. The moniker “rapid prototype” is a legacy of the innovations we pioneered on our first film Coraline back in 2009 using 3D printing for facial animation. This film won a Sci-Tech Oscar in 2016. In 2006, Brian Mclean (and Martin Munier) came to the studio to discuss 3D printing and their bold plan to use Objet Polyjet printers to create the faces for the movie’s facial performances. The task of building a pipeline to model and animate these faces, as well as print them, was my responsibility. Since that first production, we have innovated our processes to take full advantage of 3D printing with multiple materials. As a humble major in fine art, I get to play with the most advanced technology in computer-aided fabrication.
3DN: You come from two very different fields. How did your paths cross?
Nick: When LAIKA was the keynote speaker, we met at AMUG years ago. Their presentation was incredible and demonstrated how bitmap printing technology allowed them to create much their work. It was uplifting to see others using this esoteric technology. I was in awe of them and didn’t have the guts to chat with them eighth away. But, Brian and Rob Ducey later came to my talk about bitmap printing and voxel modeling for surgery. After the talk, they were the ones who came up to me first and talked about bitmap printing. This was a mutual love that we shared for bitmap printing. Next, we connected online and discussed our individual approaches. It was obvious immediately that we had simple solutions for many of the issues the other wanted to tackle. We had been struggling for years with the problem of tractography, so it was obvious that the special effects community had already developed solutions for dealing with thin hair structures. We threw out tons of ideas from the special effects community that have been around for years. These ideas allow for medical advancements that no one has ever seen before. It was a simple connection that allowed for these conversations.
Rob: We are constantly on the lookout for technical innovations and new ideas to enhance our abilities and redefine stop motion animation. In our use of multi-material printing, we discovered a small community of like-minded tinkerers who had access to features still considered “in development”. We were able to visit universities such as Virginia Tech and MIT, and have discussions and visits with them. They were interested in what a group of filmmakers and artists working in a warehouse in the rain-soaked northwest was doing with these machines. We were introduced to AMUG in 2015, and again in 2019, to deliver a keynote. We are laser-focused on learning from others and sharing our enthusiasm for 3D printing in multi-material materials when we attend AMUG. I saw the description of Nick’s presentation and made sure we were there. We approached Nick excited to find someone in what was then a very small club of “voxel printers”. It was partly to share what we had learned but also to discover his secrets. It was like meeting someone from a lost tribe.
Laika used 3D printing over a number years to make its films. Photo credit: Laika
3DN: Please tell us about your project. Also, could you tell us more about bitmap printing?
Nick: The following areas have been the focus of our projects: Advancement of bitmap printing techniques. Multimodal data fusion in medical applications that focus on soft tissue Parametric design of prosthetics.
Bitmap printing allows the creation of printed images directly from a stack. It’s very much like how an inkjet printer works, except in this case the pages are layers. It’s very simple and leverages how a polyjet printer natively works, we are are essentially bypassing the internal slicer of the printer and sending our own slices. These slices determine the exact location of each droplet. However, this means that we need to define each droplet and that’s harder. The majority of 3D printing software uses a surface mesh-based model paradigm. This allows objects to be represented by their boundaries and ignores internal volumetric data. Volumetric information is important to be able capture in many applications such as medical or structural. So, in order to bitmap print, we use a different paradigm for modeling called ‘voxel modeling’ which models volumetric data. With little or no data loss, we can convert image-based data like DICOM data directly into a 3D printed object. Voxel modeling and bitmap printers also allow us to make a gradient by manipulating droplet mixing ratios using dithering.
- Bitmap printing has a major drawback. You have to create the software necessary to create the files. There are no automated workflows that allow you to simply create bitmap prints. We’ve been trying to merge our workflows and develop new workflows that will help us improve our ability to make bitmap prints. There are many things that need to be considered and those that still need to been worked out. It is still difficult to predict the material mixing of mechanical and colors properties and often we get unexpected results. Because of the number of calculations required to digitally preview and forecast how a print will look, this is due to the complexity of computers. Our primary focus is to quantify color-based mixing ratios that include translucency and other mechanical properties.
- Multimodal data Fusion is an integral part of voxel based modeling. Voxel modeling uses compositing rather than booleaning to combine complex data sources. This makes it simple and computationally affordable. Voxel modeling can volumetrically alter the color of the form without changing it. These factors enable us to combine various data sources, both vector based as well as bitmap, into one model. This includes MRI, fMRI (percutaneous magnetic resonance imaging), PET, MEG and Tractography. Our work has concentrated on creating workflows that allow us control each data source individually and combine all the sources into a single holistic model.
- Parametric design is a very common modeling technique in both architecture and is the ‘bread and butter’ of stop motion animation. We have been working to create ‘rigged’ modeling software which allows for the facile manipulation of complex forms over time. This work was geared towards designing prosthetics for patients that can be modified over time. This work combines medical data with stop motion animation software to create a series stop motion models of human prosthetics.
The project could aid in the treatment of pediatric epilepsy (photo credit to Nicholas Jacobson and Robert Ducey).
Rob: It was clear from our first meeting that we had a lot in common about volumetric design for printing. We identified some inherent characteristics of medical imaging that could be used to generate volumetric data. The studio was working on ways to take volumetric data from a visual effects workflow, and convert it to multi-material print. The key to that process was “bitmap printing” or, as it has now come to be known as “voxel printing”. We were only concerned with the external appearance of the printed object, and the layering that affected it. Fraunhofer’s product Cuttlefish was a key partner in this. This allowed us to be familiar with the process of printing from slices. It is basically what bitmap printing and voxel print are. We devised a way to generate slices from our volumetric data. Nick’s work was especially interesting in the various modalities of medical imaging he was working with in DICOM format. These data were very similar to slices, and they were amazing to see printed. It was obvious that there were ways to incorporate this data into a VFX process and produce some beautiful prints. The next step was to start discussing how we could bring that data back into Nick’s research at the University of Colorado. The discussion led to further discussions about how we could leverage existing animation and visual effects techniques that might be applied to our ongoing medical research. LAIKA’s uniqueness is that we demand the same precision, scales, levels of detail in all parts that we create and engineer as we would in medical applications. We also have some insight into how to make things visually appealing.
3DN: What other applications can you see for it?
Nick: There are many possible applications, and we’re just scratching the surface. Cleft Palate and Pediatric Epilepsy Surgery are our main focus right now. Pediatric Epilepsy Surgery may be the most important application of bitmap printing to presurgical planning and intraoperative guidance. Because epilepsy is primarily physiologically-based, surgeons must review multiple 2D volumetric data sources individually and sequentially to build a mental understanding of each patient. We plan to combine the many, growing, datasets into a holistic 3D model for planning and guidance. To evaluate the efficacy of such a model in treatment planning and post-operative outcomes, we plan to begin a prospective trial by the end of this year.
Presurgical Naso Alveolar Molding is the focus of our work with cleft palate and lip. This is a common procedure for infants with cleft palate and lip. It can be done almost immediately after birth. This treatment involves attaching an acrylic device to the palate that directs growth towards closing the cleft. This eliminates the need for surgery and makes subsequent surgeries easier. The clinic must make adjustments weekly. This means that families will need to come to the clinic every week for 3-9months. Even if you live in close proximity, it is difficult. Unfortunately, access to this treatment is not available to many patients and there are not many providers. We are working to develop a 3D printed solution that allows patients to receive a set via the mail. Anyone with the ability take a tooth impression would be able to get this treatment.
Rob: Nick and his team are driving the research about how to apply what we’re learning in the field of medicine. The studio has gained clarity about technical design using voxels, as well how biology functions volumetrically from medical experts’ perspective. This is a unique opportunity for additive processes to take advantage of the biology concepts in design.
This procedure could aid in Naso Alveolar Molding (for cleft palate patients), which is a condition in which there are holes present at birth in either the palate (soft-hard) or the lips (photo credits to Sunrise Orthodontics).
3DN: Last words for our readers?
BothAMUG 2023 – If you want to find out more about the project, or just about us in general then come to AMUG 2023. We will be giving our keynote speech. “Collaborations Between an Animator, an Architect, and a Surgeon: The Keys to Impactful Innovation in Medicine” Tuesday, 21 March. If you haven’t already, you can register for the conference HERE.
What do you think Rob and Nick have accomplished? Are 3D printing conferences a key tool for encouraging collaboration within the industry? We’d love to hear from you via a comment below, or by emailing us at LinkedIn, Facebook?, and Twitter pages! Don’t forget to sign up for our free weekly Subscribe to our NewsletterReceive the latest news on 3D printing directly to your inbox All of our videos are also available on our. YouTube channel.
*Cover Photo Credits: Nicholas Jacobson and Robert Ducey
0
0
