Shawn W. O'Driscoll, M.D., Ph.D., orthopedic surgeon at Mayo Clinic in Rochester, Minnesota, discusses a study of an outcome scoring system measured for patient-physician agreement. He and colleagues published the study's results in an article in the January 2018 Mayo Clinic Proceedings. Dr. O'Driscoll explains that there can be differences in how patients and their physicians would score the value of their outcomes. He says the system used in the study, the Summary Outcome Determination (SOD), is built on four principles that are simple and value-based, promote patient-centered treatment and are generalizable to any medical specialty. The investigators reviewed post-treatment medical records of 320 patients who had undergone shoulder or elbow procedures and included 164 patients in the study for whom both physicians and patients had completed the SOD. The investigators found a high level of correlation in the SOD categories and numerical values assigned to the patient's outcome by the physician and the patient in the study.
Dr. O'Driscoll advocates use of the SOD, explaining that this easy-to-use-and-understand scoring system drives the physician to assess the patient's priorities pre-treatment and then post-treatment, rating how much better — or not — their condition is compared to pre-treatment.
The full article is available at: https://www.mayoclinicproceedings.org/article/S0025-6196(17)30741-3/fulltext
I'm Sean O'Driscoll professor of orthopedic surgery here at the Mayo clinic and specialize in elbow and shoulder surgery. I'm discussing a paper that we recently published in the Mayo clinic proceedings entitled patient physician agreement. Using summary outcome determination scores. There are four points that I would like to make about this paper that contributes to our practice of medicine, not just here at Mayo Clinic, but anywhere where this would be instituted. The first is that this is a value based uh aspect of looking at the evaluation of clinical outcomes and surgical patients. And of course it does not need to be limited to just surgical patients. The second is that it's patient centered. The third is that it's simple and clear and can be easily implemented to normal clinical workflow. And the fourth is that it's independent of the condition or the severity of the condition. So that adjusts automatically to whatever condition you're evaluating or whatever level within that range the patient functions. So, first I said that its value based and we are in an era right now in which the delivery of medical care necessarily must be focused on value. And so whereas we normally tend to look at outcomes in surgical patients as success or failure or degree of success. We have to start looking at them as as to the value that they deliver to the patient. So for example, I have an operation, I do on the elbow that I also do on the shoulder in arthritic patients in the elbow, I would call this operation and eight, meaning that in general if a patient is this operation, they would tend to value it as eight out of 10 on a scale. And if I do the same operation on the shoulder, I would tend to refer to it as a four, meaning that patients in general will value it At about a four. Now, value brings into consideration the aspect of patient physician agreement and patient reported outcomes have become very, very important because we've realized that the outcome that a patient assigns to a procedure or an intervention is often different from the outcome assigned by the doctor. And in fact, it's generally accepted that doctors and patients don't agree on the outcomes of surgical procedures and perhaps medical interventions as well or other types. And that raises the question of just why we don't agree. Is it that we really don't understand what the patient wants or values or is it that we really value something different altogether than what the patient values? I think that the training of a doctor is such that we should be able to understand and I do believe that we do understand what is important to a patient. So what this means is that what we're evaluating and outcomes is probably different than what the patient is about. So, as a surgeon say, doing elbow surgery, I would value things like the range of motion of the elbow, following the surgery and whether the elbow is stable or it dislocates still, whether it's painful patient might value something else very differently. Let me take for example, something I just operated on. He's one of our law enforcement officers and the federal system. He had a serious injury to his elbow. It was badly broken, very badly broken, was close to unfixable and indeed it was not able to be fixed anywhere near adequately with the first attempt five months ago and he was referred to me with a painful elbow that only moves through a small portion of the arc of motion. It dislocates at times or partially dislocates. And he has a severe problem with the nerve as well that goes past the elbow. When you look at his X ray. Clearly he has an arthritic elbow that needs to be fixed or replaced. When you ask him what matters most and get a list of priorities. He would tell you that the number one priority that would affect his quality of life is the lack of flexion, meaning he can't get his hand anywhere close to his body or his face. And that would be his number one priority is to recover reflection. His number two priority would be to have an album that doesn't slip on a joint and cause them to suddenly lose the ability to use the hand the hand because he has essentially stopped using his hand because of that unpredictability. Now, I would have thought that the number one priority was pain relief. But turns out that was actually his # three priority. And maybe even the fourth because the nerve problems in his hand was such that he was losing function in his hand. And he told me if he could get the other three corrected, he could live with the pain. Interestingly, what I thought was his number one problem. It was not at all his # one problem. In fact, by prioritizing this way using this system of evaluating patients, we necessarily have to get into their minds and understand what matters to them most. In other words, what they value most. He could tell me in advance what he would score his elbow. He told me he would score his elbow as greatly improved if we accomplished that. Now I had thought I was going to have to replace his elbow because of the pain of the arthritis, turns out we didn't have to do that at all. In fact that would have been a disservice to him because it would have put a lifelong limit on the use of his elbow. So it's patient centered Its value based and 3rd, it's simple and clear. The concept of evaluating outcomes is to simplify to the point where it is very clearly understood by both the patient and the doctor and that agreement between the two should be expected rather than uncommon. So the agreement was quite profound. The agreement was perfect in about three quarters of the patients, meaning that the doctor and the patient or the physician assistant assistant or the resident or fellow and the patient agreed on the outcome. In about three quarters of the cases. Now we also agreed if we just ease off the restrictions slightly there, we agreed within one point meaning we were in the same category or if we're not in the same category, we were still within one point on a 10 point numerical scale. Uh 98% of the time. So a 98% of perfect or near perfect agreement is a remarkable correlation between a doctor evaluating the outcome and the patient evaluating his or her own outcome. And so what happens? You have a scale that stretches out. You have a scale that stretches out the outcomes. So when we think of outcomes with surgery or intervention medically the first question we ask the patient is compared to before your surgery. Are you better, worse or not much different and not much different would include. You're not sure if you're better or worse. Now let's assume that they're better. The next is to pick a category of one of four levels. And we ask the patient, are you improved, greatly improved? Almost normal or normal. And we tell them that normal means as though they've never had surgery. They've never had condition or disease or an injury. The difficulty in picking a category is not great. It's very clear to most people which category you're in but it does require that you understand how the patient was before surgery. So it necessarily drives the doctor to inquire about the preoperative status when you evaluate them as well. It requires that you go back and look at the priority and it also has an internal consistency check. So we asked the patient the questions in terms of which four categories they would fall into and then we asked some numerical value. So I say, well if 10 is perfectly normal and zero is just like you were before your surgery, what number would you give to your elbow now or to your knee or your hip? And so they pick a number And a 10 point scale is a familiar concept to patients from the pain scales and other scales. So if the patient says I would rate myself as improved and eight out of 10, Then you wonder, Okay, so why are they only improved and not greatly improved if it's eight out of 10? So it permits an opportunity to verify what you have just heard from the patient and therefore be very clear as to outcomes. Finally, it's independent of the severity of the condition or the condition itself. Scoring systems are very often dependent on the actual um condition or the procedure that has been done and therefore they have limited applicability across systems. What we would like to be able to do is to value an elbow replacement in a crippled person crippled with rheumatoid arthritis and compare the value to society of that operation to the value of an operation done on an elite athlete or the value of a medication prescription in somebody with another condition. Ultimately, that will help us to make wise decisions in the delivery of healthcare. In summary the summary outcome determination that we have reported is indeed a summary outcome determination. In other words, it's like saying when all the dust is settled and all is said and done. What was the impact? What was the value? That's what matters to the patient. That's what matters in health care. And that's what matters as we move forward. Trying to derive values for the health care measures that we employ. So we have been implementing and are now looking back and trying to determine how they fit in as we move forward in the future. We hope you found this presentation from the content of Mayo Clinic proceedings Valuable. Our journal's mission is to promote the best interests of patients by advancing the knowledge and professionalism of the physician community. If you are interested in more information about us. Our homepage is www dot Mayo clinic proceedings dot org. There you will find access information for our social media content, such as additional videos on our Youtube channel or journal updates on facebook. You can also follow us on twitter. More information about health care at Mayo clinic is available at www dot mayoclinic dot org. This video content is copyrighted by Mayo Foundation for Medical Education and Research.
