Posts Categorized: ProKnow

AI Autosegmentation Podcasts: Invitation and Info

Posted on

Dear AI Autosegmentation Vendors,

I am reaching out to any/all vendors to invite you to take part in periodic, non-funded podcasts. The forum will allow you to introduce yourselves, show your wares, and share your thoughts about how the radiation therapy industry can best validate AI autosegmentation engines/models.

The conversations will center around the imaging datasets and population of radiation oncologists’ manual segmentations that are generated by the non-commercial, not-for-profit project called “C3RO” (Contouring Collaborative for Consensus in Radiation Oncology).

Please see the details below. Email me (Ben Nelms) directly at canislupusllc@gmail.com if you are interested and would like to get on the schedule.

Thanks,

Ben Nelms

What

An unbiased podcast/interview series focusing on AI autosegmentation of human anatomy, specifically to use as an input to radiation treatment planning.

Why

Goal 1. Elevate the conversation about how best to validate AI outputs, both in the short and long term.

Goal 2. Get to know AI vendors in a casual, scientific, and “non-salesy” forum.

Goal 3. Show cool, real-time results from (1) populations of human experts and (2) AI engines.

Goal 4. Generate some pretty great ideas about how to build gold standards of human anatomy segmentation to use for both education of clinicians as well as validation of AI software.

Who

Host. Ben Nelms, Ph.D. (Canis Lupus LLC)

Guests. Representative(s) from any/all willing AI vendors and research groups who specialize in anatomy autosegmentation

Note: Conversations (likely all of them, or at least the initial ones) will be one vendor at a time. This helps ensure equal airtime and no distractions.

How

The conversation flow will be casual, with an underlying structure to cover some, if not all, of the following topics.

Intro. Your background, and what brought you to this field? (Optional) How does your group currently do validation of your AI outputs? Is it quantitative, qualitative, or both?(Required)

Data / Results. Generate contours – ideally in real time right before or in the early minutes of the meeting – for the imageset in question. Compare your automated outputs to (1) the population Isoagreement clouds and (2) various expert contours (if available), per structure....

Come Wonk with Me: Digging into C3RO Data

Posted on

The Contouring Collaborative for Consensus in Radiation Oncology (C3RO) is picking up steam. We recently reached a checkpoint after our first ~quarter year – sessions on three different body sites – and asked ourselves, how is it going?

Are our goals clear? Will achieving our goals be impactful in a tangible way? Are our methods sound? Are we providing value to the radiation oncologists participating in the program as well as the industry as a whole? Are we squeezing out the goodness of knowledge out of this mysterious fruit? And are we having any fun?

Then we asked our participants a bunch of questions, too, as an electronic survey. Two of the main messages that came out of the survey were these:

[ 1 ]  People really hunger for detailed “How I created by anatomical contours and why” explanations by invited expert panelists. Rather than try to go fast and cover lots of material and many regions of interest, slow down and talk about them, and debate them, in greater detail.

[ 2 ]  People also are interested in the population statistics, the performance of the “wisdom of the crowd,” and how it relates to potentially deriving or vetting gold standards based on consensus calculations.

Well, you spoke, and we listened! So, we are going to start doing two parallel and complementary tracks in terms of podcasts.

The first track will be hosted by radiation oncologists with radiation oncologists as panelists. The main focus here will be the “how” and “why” of experts’ contours, and hopefully some healthy discussion and negotiation of observed differences.

The second track will be led by yours truly, and we’re going to get unabashedly wonky and nerdy about it. I’m going to try to get interested AI vendors, researchers, and other people who deep-think about these things as my panelists. We’ll talk a lot about statistical variation, what we can learn from it, the challenges it poses, and how to potentially tease out great wisdom from the crowd and get to one of the holy grails of modern radiation therapy: building standard datasets against which AI autosegmentation can be measured and potentially validated....

Top 10 Lessons Learned after 10 Years of Plan Studies

Posted on

I couple weeks ago, I gave a webinar for the “Best of QADS” series put on by Sun Nuclear Corporation. The webinar was called “The Top 10 Lessons Learned after 10 Years of International Plan Studies.” The presentation (including the live Q&A session) was recorded. You can watch it by visiting the following link.

Click here to register and watch that presentation. (It’s free of course)....

Invariably Variable: The Art of Contouring

Posted on

PART 1: THE INERTIA OF WRONG ASSUMPTIONS

An object at rest tends to stay at rest…
Newton’s First Law

I’ll never forget the reactions of those two physicians, those many years ago. Or, at least I won’t let myself. Not yet at least.

Allow me a moment to retell.

It was about seven years ago and I had just given a talk at a regional meeting in the Midwest. This particular audience was made up of medical dosimetrists and radiation therapists, with a smattering of medical physicists and radiation oncologists. My topic: “Variation in Anatomical Contouring.”

One of my first slides was a clumsy cartoon I had sketched together in PowerPoint. It showed a horse (labelled “treatment planning”) pulling a train of carts, each labelled with a specific technology dependent on the preceding one. And under the horse, representing the road on which the horse and all carts depended, was written one big, bold word: CONTOURING. I found that old cartoon and I’ve reproduced it in Figure 1, below.

Figure 1. My slide (circa ~2010) used to say, essentially, “We can talk about the cart and the horse all we want, but let’s not forget the condition of the road…”

My simple argument was that if you don’t get your anatomy volumes defined correctly – both for targets and critical organs – then everything else downstream suffers. Or, following the horse-and-cart metaphor, inaccurate contours make for a really bumpy ride. All the benefits of the elegant technology of radiation therapy – inverse planning and dose optimization, dose calculation, DVH and other plan metrics, image-guidance, and precision delivery – don’t even matter if your patient anatomy blueprint is wrong in the first place. The anatomy contours are the original “design input” to the personalized medicine that is radiation oncology. Get that wrong, and you’re in trouble.

For the talk, I showed some preliminary data on inter-observer anatomical contours over a range of critical organs. These were controlled experiments where all clinicians were given the same CT images, and the variation I was seeing in some organs was shocking. While there was not much variation for some organs like the brain or lung which are easily seen as clearly defined pixel regions, there was very large variation for many other organs like the parotid, sub-mandibular glands, brainstem, larynx, and even the spinal cord (!)....

Choose Your Own Adventure

Posted on

Here is talk I did for the annual AAMD meeting (Atlanta, GA) on June 12, 2016. The response I got and the line of people inspired to tell me “their stories” was a big lift.

Sometimes you just have to let it all hang out. Enjoy.

 ...

2016 AAMD Contouring Workshop

Posted on

Dr. Aaron Kusano and I will be leading this year’s AAMD Contouring Workshop. This will be our third workshop of this kind in a row, and we believe this year’s agenda is the best. We have incorporated the most frequent request (namely, for more “hands on” contouring time) while keeping the instruction and testing methods that have been successful the past two years.

Below is a short conversation with both of us that provides a nice summary of what we’re doing this year and why. We hope you sign up and join the workshop! It’s a volunteer effort on our part, and our only motivation is because of our passion and yours.

...

It’s Here: ProKnow

Posted on

I am proud to be collaborating with a multi-institutional team to build a system called “ProKnow.” ProKnow stands for “Profound Knowledge,” a term that many of you will recognize from Deming’s famous “System of Profound Knowledge” to improve quality and pride in workmanship across any company, team, or project.

ProKnow will allow the worldwide community to study, and ultimately improve, the standard of care in radiation oncology. We have powerful analytical modules to help you: ensure accurate anatomy contouring, quantify and study plan quality metrics, identify best practices, and ultimately correlate your methods and modalities with patient outcomes.

Here is the link to our cloud-based system: www.proknowsystems.com. Take a look!...