Posts By: Benjamin Nelms

Opportunity for AI Autosegmentation Vendors

Posted on

Over the past four years, I have had the pleasure to get to know and collaborate with TROG, an organization based in Australia and New Zealand that conducts clinical trial research involving radiotherapy. TROG have used ProKnow software to run plan studies for SRS brain treatment as well as SBRT spine, pancreas, and liver cases. They have also done contouring workshops across multiple body sites.

For the 2022 TROG meeting –  and in conjunction with the annual ASMIRT meeting – TROG is taking it up a notch! For one, they are planning an exciting and important treatment plan study using an unprecedented experimental design that will focus on optimizing a plan based on lung function. Also, and as the main topic of this post, they will be doing a very interesting contouring workshop.

The 2022 contouring workshop is particularly exciting to me because it will do the following: (1) explore method(s) to build consensus across a group of expert physicians, (2) measure and visualize the variation in contouring across a large population of professional treatment planners and anatomists, (3) study population consensus vs. expert consensus, and (4) collect results and measure the accuracy of artificial intelligence (AI) based auto-segmentation engines vs. the gold standard and population on the whole.

TROG is inviting all AI/auto-segmentation vendors to participate. This is a great way to test your engines and showcase your results. Whether you are an established vendor, startup company with works in progress, or an academic software research group, I encourage you to participate. You can contact TROG’s Radiation Therapy Manager Alisha Moore ( to get involved.

This is not my project specifically (other than helping them with design and implementation), but I think studies like this are of utmost importance so I wanted to help TROG cast a wide net and maximize involvement. There’s nothing to lose and everything to gain. After all, we cannot prove (or improve) what we do not measure!


Launch of C3RO Program

Posted on

A group of us — including Eric Gillespie, M.D. (Memorial Sloan Kettering) and Dave Fuller, M.D. (MD Anderson) — are launching an international research and learning group called the “Contouring Collaborative for Consensus in Radiation Oncology,” or C3RO.

We will be doing a series of monthly exercises with a large team of volunteers, collecting contouring data and then going over results with experts. Results and discussion will have live Q&A with the C3RO team members, but will also be recorded and published for all to see! Think of them as episodes, or “talk shows,” covering everything we see and search for each body site and its critical anatomy.

See the intro video below. To register, please visit this registration page.


Top 10 Lessons Learned after 10 Years of Plan Studies

Posted on

I couple weeks ago, I gave a webinar for the “Best of QADS” series put on by Sun Nuclear Corporation. The webinar was called “The Top 10 Lessons Learned after 10 Years of International Plan Studies.” The presentation (including the live Q&A session) was recorded. You can watch it by visiting the following link.

Click here to register and watch that presentation. (It’s free of course)....

Invariably Variable: The Art of Contouring

Posted on


An object at rest tends to stay at rest…
Newton’s First Law

I’ll never forget the reactions of those two physicians, those many years ago. Or, at least I won’t let myself. Not yet at least.

Allow me a moment to retell.

It was about seven years ago and I had just given a talk at a regional meeting in the Midwest. This particular audience was made up of medical dosimetrists and radiation therapists, with a smattering of medical physicists and radiation oncologists. My topic: “Variation in Anatomical Contouring.”

One of my first slides was a clumsy cartoon I had sketched together in PowerPoint. It showed a horse (labelled “treatment planning”) pulling a train of carts, each labelled with a specific technology dependent on the preceding one. And under the horse, representing the road on which the horse and all carts depended, was written one big, bold word: CONTOURING. I found that old cartoon and I’ve reproduced it in Figure 1, below.

Figure 1. My slide (circa ~2010) used to say, essentially, “We can talk about the cart and the horse all we want, but let’s not forget the condition of the road…”

My simple argument was that if you don’t get your anatomy volumes defined correctly – both for targets and critical organs – then everything else downstream suffers. Or, following the horse-and-cart metaphor, inaccurate contours make for a really bumpy ride. All the benefits of the elegant technology of radiation therapy – inverse planning and dose optimization, dose calculation, DVH and other plan metrics, image-guidance, and precision delivery – don’t even matter if your patient anatomy blueprint is wrong in the first place. The anatomy contours are the original “design input” to the personalized medicine that is radiation oncology. Get that wrong, and you’re in trouble.

For the talk, I showed some preliminary data on inter-observer anatomical contours over a range of critical organs. These were controlled experiments where all clinicians were given the same CT images, and the variation I was seeing in some organs was shocking. While there was not much variation for some organs like the brain or lung which are easily seen as clearly defined pixel regions, there was very large variation for many other organs like the parotid, sub-mandibular glands, brainstem, larynx, and even the spinal cord (!)....

Choose Your Own Adventure

Posted on

Here is talk I did for the annual AAMD meeting (Atlanta, GA) on June 12, 2016. The response I got and the line of people inspired to tell me “their stories” was a big lift.

Sometimes you just have to let it all hang out. Enjoy.


2016 AAMD Contouring Workshop

Posted on

Dr. Aaron Kusano and I will be leading this year’s AAMD Contouring Workshop. This will be our third workshop of this kind in a row, and we believe this year’s agenda is the best. We have incorporated the most frequent request (namely, for more “hands on” contouring time) while keeping the instruction and testing methods that have been successful the past two years.

Below is a short conversation with both of us that provides a nice summary of what we’re doing this year and why. We hope you sign up and join the workshop! It’s a volunteer effort on our part, and our only motivation is because of our passion and yours.


It’s Here: ProKnow

Posted on

I am proud to be collaborating with a multi-institutional team to build a system called “ProKnow.” ProKnow stands for “Profound Knowledge,” a term that many of you will recognize from Deming’s famous “System of Profound Knowledge” to improve quality and pride in workmanship across any company, team, or project.

ProKnow will allow the worldwide community to study, and ultimately improve, the standard of care in radiation oncology. We have powerful analytical modules to help you: ensure accurate anatomy contouring, quantify and study plan quality metrics, identify best practices, and ultimately correlate your methods and modalities with patient outcomes.

Here is the link to our cloud-based system: Take a look!...

2016’s AAMD Contouring Workshop to be the best yet

Posted on

Pretty soon it will be June and the annual AAMD meeting in Atlanta will be one hot ticket!

We’re pretty proud of the educational agenda we’ve put together for this year’s AAMD Contouring Workshop. It includes elements and strategies that have succeeded in past workshops, combined with pretty much all the suggestions for improvement we have received from the past two years. Firstly, we’re allowing more time so that we maximize the “hands on” skills practice and interaction with each other and the experts. Secondly, the workflow of the workshop is tailor made to optimize learning. You will be contouring critical organs (thanks to MIM to donating workstations and their impressive contouring software), then hearing the incomparable Dr. Aaron Kusano give interactive lectures on each organ, then you will receive immediate feedback of your individual contours vs. the “gold standard” contours as scored by the StructSure software (thanks to Standard Imaging), and finally you will go back to the workstations to directly compare your contours to the gold standards and ask Dr. Kusano questions in real time during your review.

That’s the gist of it. It is going to be great and will set the bar even higher on the value of the workshop.

Here is a link to the AAMD page: AAMD 2016 Workshops...

Project Icarus Alive and Well In “PlanIQ” Product

Posted on

For closure on the “Project Icarus” thread, I’d like to make it clear the the project is alive and well, but is now living and breathing in the real world as the “Feasibility Analysis” (pre-planning and post-planning) tools in the PlanIQ software product now owned by Sun Nuclear Corporation.

For more information or to request a demo or research license, just contact your Sun Nuclear sales representative....

Testing the Accuracy of Your DVH Calculations

Posted on

As an industry, we’ve spent decades talking about testing and improving the accuracy of 3D dose calculations. We’ve invented countless products – both hardware (e.g. dosimeters) and software – to help us along the way, and many AAPM Task Groups have published reports of various flavors and angles on this topic. None of this is too surprising because accurate dose calcs, especially in and around complex tissue shapes and varying densities, are not an easy thing.

3D dose calculation results are primarily used – in conjunction with 3D anatomy contouring – to produce dose volume histograms (DVH) for each target and organ-at-risk. The irony is that for a long time (again, decades), as an industry we’ve forgotten that assembling DVH curves requires calculations itself, and not all of these are created equal. How are anatomy volumes simulated and at what resolution? How are anatomy superior and inferior “end caps” modeled and what is assumed to happen in between axial slices? How does anatomy interplay with an orthogonal 3D dose grid, and what are the effects of that dose grid’s spatial resolution?

DVH calculations are not standard, folks. Not by a far cry.

Together with Vladimir Feygelman, Ph.D. and other scientists at Moffitt Cancer Center (Tampa, FL), we’ve done some really important work studying DVH accuracy and recommending standardized datasets and methods to validate your TPS software. We’ve published our first work in Medical Physics (August, 2015), and you can link to it here.

I hope you can take some time to read our new publication. If you’re a physician, physicist, or dosimetrist (or, just as important, if you’re a TPS software vendor) it will be well worth your time....