How the Need To Print COVID-19 Test Swabs Led to an Innovative Contribution to the Intel OneTBB Parallel-Computing Library

At the onset of the COVID-19 pandemic, nasal swabs for testing COVID-19 were identified as being in high demand and extremely limited in supply. A team from the USF Health’s 3D Clinical Applications Division created an initial design, working with Northwell Health and collaborating with Formlabs to develop prototypes and secure materials for a 3D printed alternative. Before long, the design was available for 3D printing, with Formlabs printers able to create 324 swabs per print. But prior to the pandemic (PreForm 3.4), printing hundreds of identical parts was not a common use case. At that time, loading and analyzing the printability of the scene took about 80 seconds (~17 CPU-minutes) on a 2.4 GHz 8-Core Intel Core i9.

Working remotely with his team, Formlabs Technical Lead Ben FrantzDale asked the question: Can we speed the process by analyzing just one model and apply the same result to the 323 other identical swab models? One option was directly checking the scene for duplicates, but that would add unnecessary complexity. FrantzDale and intern Henry Heffan set out to engineer a better solution, but found that both the C++ standard library and other libraries lacked the core functionality necessary: There are ways to compute many things at the same times and ways to ensure something isn't calculated repeatedly, but these didn't operate well together, making it possible for the program to freeze or to do all the work on one CPU, leaving most of the computer idle. (See Technical Details section below for details.)

After prototyping a solution, the Formlabs team reached out to Intel to refine their design and to offer it for inclusion in their oneAPI Threading Building Blocks (oneTBB) library. The collaboration resulted in further improvements to the design and ultimately inclusion of the feature in the official shipping release of oneTBB as of November 2021.

With this solution, and various other performance improvements, the Formlabs team worked on over the pandemic, the same 324-swab file loads and processes printability in under two seconds (~15 CPU-seconds).

At Formlabs, we are committed to continuously improving your 3D printers through software updates. We believe the best hardware products are those that grow better over time, and we’re committed to continually evolving our hardware through software improvements to serve users across a range of industries. During the COVID-19 pandemic, Formlabs rapidly organized to help the medical community use 3D printing to address the pandemic and associated supply chain shortages. You can read more about those efforts here.

PreForm is written in C++17, a programming language widely used in performance-critical applications. Taking the mantra "no raw synchronization primitives" to heart, the team prototyped a cache built around std::call_once, provided by the C++ standard library, which implements the following:

• if there's a result: use it
• if not: compute it
• if it's already being computed: wait until it's done
• if there is an exception or cancelation: reset and let another thread try.

However, with hundreds of the same result to calculate, that causes one thread to do all of the work while the others just wait. With that approach, printiability would only get run only once, but it would only be processed using a single CPU core, leaving an additional potential 16x speedup unrealized. While std::call_once provides the right abstraction, the team realized the blocked threads need to collaborate.

Functionality like this, in which multiple threads interact, is notoriously difficult to write correctly, and the team ran into subtleties. PreForm uses Intel's oneTBB library for scalable parallelism. One of the many tricks oneTBB uses to keep all CPU cores busy, is a feature called "moonlighting" in which a thread that is out of work can go off looking for more work to do. This can be very effective at fully utilizing the computer, but can lead to deadlock wherein a thread blocks, waiting for itself to finish: like an ouroboros, it's eating its own tail. oneTBB provides a way to prevent this, isolating tasks to their own "task arena" to prevent deadlock. Therefore, for a call_once implementation to interact smoothly with oneTBB, it needs to provide work isolation.

In addition to being tricky to design correctly, the team realized that this feature has broad utility: There are many high-performance applications in which a program may need to compute something expensive, but don't know up-front when or even if it will need to be computed. For both of those reasons, the Formlabs team reached out to Intel, offering it for inclusion in oneTBB. In typical 2020 style, after a few international video calls among engineers, each working from home-offices, living rooms, and kitchens, the Formlabs team submitted the proposed design to Intel. Intel engineers then further refined it. For example, the initial implementation required memory allocation, which takes time. That could be avoided by sharing a stack-allocated block between threads and using a spinlock to ensure the owning thread's stack unwinds last. After those and other refinements, the Formlabs and Intel teams and we decided on the name tbb::collaborative_call_once. Ultimately the addition was merged into oneTBB and is included as of oneTBB 2021.4.0.

To learn more:

• Download the latest version of PreForm to try it out.
• Try out tbb::collabroative_call_once on Compiler Explorer.
• Download oneTBB at oneTBB 2021.4.0.

If you want to join the team on the leading edge of high-performance C++ and the future of 3D printing to help anyone make anything, Formlabs is hiring full-time and internships in Boston and Budapest.