|Year : 2018 | Volume
| Issue : 1 | Page : 50-55
Procedural competency training during diagnostic radiology residency: Time to go beyond “See one, do one, teach one”!
Driss Raissi1, Qiong Han2, Michael Winkler1, Edward J Escott3
1 Department of Medicine, University of Kentucky; Department of Diagnostic Radiology, Lexington, KY 40506, USA
2 Department of Medicine, University of Kentucky, Lexington, KY 40506, USA
3 Department of Diagnostic Radiology, University of Kentucky; Department of Otolaryngology-Head and Neck Surgery, Lexington, KY 40506, USA
|Date of Submission||13-Jul-2017|
|Date of Acceptance||31-Oct-2017|
|Date of Web Publication||23-Apr-2018|
Dr. Driss Raissi
Department of Diagnostic Radiology, College of Medicine, University of Kentucky, HX-319D, 800 Rose Street, Lexington 40506, KY
Source of Support: None, Conflict of Interest: None
Objectives: Achieving procedural competency during diagnostic radiology residency can impact the radiologist's future independent practice after graduation, especially in a private practice setting. However, standardized procedure competency training within most radiology residency programs is lacking, and overall procedural skills are still mainly acquired by the traditional “see one, do one, teach one” methodology. We report the development of a simple standardized procedural training protocol that can easily be adopted by residency programs currently lacking any form of structured procedural training.
Materials and Methods: An ad hoc resident procedural competency committee was created in our radiology residency program. A procedural certification protocol was developed by the committee which was composed of attending radiologists from the involved divisions and two chief residents. A road map to achieve procedural competency certification status was finalized. The protocol was then implemented through online commercial software.
Results: Our procedural certification protocol took effect in September 2014. We reviewed all resident records from September 2014 to December 2016. Eighteen residents of various levels of training participated in our training protocol. About 72% became certified in paracentesis, 11% in thoracentesis, 83% in feeding tube placement, 55% in lumbar puncture/myelogram, and 77% in tunneled catheter removal.
Conclusions: Our single-center experience demonstrates that a simple to adopt structured approach to procedural competency training is feasible and effective. Our “certified” radiology residents were deemed capable of performing those procedures under indirect supervision.
The following core competencies are addressed in this article: Patient care, Medical knowledge, and Systems-based practice.
Keywords: Procedural competency, resident education, standardized training
|How to cite this article:|
Raissi D, Han Q, Winkler M, Escott EJ. Procedural competency training during diagnostic radiology residency: Time to go beyond “See one, do one, teach one”!. Int J Acad Med 2018;4:50-5
|How to cite this URL:|
Raissi D, Han Q, Winkler M, Escott EJ. Procedural competency training during diagnostic radiology residency: Time to go beyond “See one, do one, teach one”!. Int J Acad Med [serial online] 2018 [cited 2022 Oct 5];4:50-5. Available from: https://www.ijam-web.org/text.asp?2018/4/1/50/230854
| Introduction|| |
Clinical competency has been defined as “having sufficient knowledge and skills such that a procedure can be performed to obtain the intended outcome without harm to the patient”. Specific standards to achieve minimum procedural competency during diagnostic radiology residency training have not been addressed by the Residency Review Committee (RRC) although several regulatory agencies require continuous oversight and performance assessment of physician practice.
“Light interventional radiology” procedures, also known as “light IR,” is an umbrella term used for relatively less complex invasive procedures, including but not limited to paracentesis, thoracentesis, lumbar puncture, and feeding tube placement. While a significant number of noninterventional radiologists perform “light IR” procedures, especially in private practice settings, it is largely unknown whether they underwent a thorough competency assessment for those procedures during residency training.
Procedural competency in diagnostic radiology residency is of paramount importance to prepare graduates for independent practice. However, standardized procedure competency training within US residency programs is lacking, and overall procedural skills are still mainly acquired by the traditional “see one, do one, teach one” methodology, which is largely passé., While the benefit of on-site direct resident supervision has been validated in many medical specialties, there are still no standardized algorithms to ensure resident transition from direct supervision toward indirect supervision., Several models to measure and achieve procedural competency have been developed; however, standardized implementation is lacking, in part due to complex methodology and much heterogeneity across different medical specialties.,,
With this in mind, we developed a simple standardized procedural competency protocol for our diagnostic radiology residency program to ensure high-quality resident education as well as safe and effective patient care. A standardized approach would likely improve residents' overall hands-on experience and graded responsibility. Our protocol is simple enough that it can be easily adopted in a short period at most institutions lacking any procedural competency protocol.
This article reviews data and conclusions from our radiology department procedural certification committee's initial review of this protocol. The term “certified” or “certification” carried a narrow scope definition with validity within our institution only, and it only implied that the “certified residents” were allowed to perform procedures under indirect supervision without direct physical presence of a supervising attending during its performance. The concepts of direct and indirect supervision used in our report are as defined by the Accreditation Council for Graduate Medical Education (ACGME).
| Materials and Methods|| |
An ad hoc Procedural Competency Committee was formed to create a procedural competency protocol inspired by the available literature. Core stakeholders were identified as all radiology subspecialty divisions performing “light IR” procedures. Division chiefs of abdominal radiology, neuroradiology, vascular and interventional radiology, a faculty representative from our resident education committee, and both of our chief residents formed the Committee. The Committee outlined the policy with specific guidelines and milestones to ensure procedural competency in predetermined procedures deemed of significant value to the resident's future independent practice. A draft protocol outlined minimal hands-on requirements; resident procedural responsibilities such as preprocedure patient evaluation, review of the case with the supervising attending, and postprocedure care; faculty expectations regarding on-site direct supervision; and a predefined roadmap to procedural certification that would enable the certified resident to perform that specific procedure under indirect supervision [Figure 1].
|Figure 1: Procedural competency timeline with procedure-specific thresholds|
Click here to view
The protocol outlined the required reading material, procedural videos, preprocedural checklist, minimum threshold of successfully completed procedures under direct supervision, without major complication as defined by the Society of Interventional Radiology, and requirement for passing the end-of-rotation written examinations, including procedure-related questions. The required minimal thresholds were extrapolated from the available literature on procedural competency training., Fulfilling all these criteria should render the diagnostic radiology resident competent to perform these procedures under indirect supervision. Otherwise, the resident will continue to require direct procedural supervision [Figure 2].
|Figure 2: Diagram of the proposed procedural competency training protocol|
Click here to view
The residency coordinator used our online graduate medical education web-based software MedHub ® (Ascend Learning ©, Dexter, MI, USA) as a tracking method for these procedures, establishing a minimum certification threshold for each procedure outlined in our protocol [Figure 3]. Each time the resident performed a certifiable procedure, the resident would log this procedure in MedHub ® and the supervising faculty would receive an electronic alert to review the logged case. The faculty member would either sign off as accepted or rejected, depending on the resident's performance during direct observation. A free text box is associated with each procedural electronic submission to note any special circumstances in the intra- or postprocedural period, to provide feedback and document any adverse events.
|Figure 3: MedHub® interface demonstrating certifiable procedures with their minimal threshold|
Click here to view
Our study was deemed a nonfunded quality improvement project by our institutional review board (IRB), and per university policy, our IRB does not regularly review quality improvement projects; hence, formal review was waived.
| Results|| |
Our procedural competency training protocol took effect in September 2014. We reviewed our residents' procedural logs on MedHub ® to evaluate progress and identify opportunities for improvement. Only procedures performed under direct supervision for certification purposes were analyzed. The review period was from September 2014 to December 2016. Eighteen diagnostic radiology residents participated in our training protocol, seven of whom were PGY4, five PGY3, and six PGY2, at the beginning of the review period [Figure 4].
|Figure 4: Certification rate (%) by different groups of residents. FT: Feeding tube; LP: Lumbar puncture; TC: Tunneled catheter|
Click here to view
By the end of December 2016, 72% of that initial pool of residents were certified in paracentesis, 11% in thoracentesis, 83% in feeding tube placement, 55% in lumbar puncture/myelogram, and 77% in tunneled catheter removal. The PGY4 residents from the 2014/2015 academic year have all graduated with six of them pursuing further fellowship training and only one joining private practice; their certification rates are as follows: 85% in paracentesis, 57% in thoracentesis, 100% in feeding tube placement, 100% in lumbar puncture/myelogram, and 100% in tunneled catheter removal. The current PGY5 residents were the PGY3 class at the beginning of our protocol. They currently have the following certification rates halfway through their last year of residency training: 80% in paracentesis, 20% in thoracentesis, 100% in feeding tube placement, 100% in lumbar puncture/myelogram, and 80% in tunneled catheter removal. The current PGY4 residents were the PGY2 at the beginning of our review period, and they are still in the midst of their procedural rotations. Their results are included in the overall certification rates [Figure 4].
No minor or major complications were reported during our review period.
| Discussion|| |
While the ACGME requires diagnostic radiology residency programs to track residents' experiences in several diagnostic imaging modalities, the only invasive procedures tracked are image-guided biopsies and drainages. A combined minimum of 25 cases is recommended by the time of graduation. There are 185 diagnostic radiology programs in the US with about 1132 residents graduating every year based on the 2014/2015 ACGME data. While more than 90% of graduating residents are pursuing fellowship training in at least one subspecialty, the rest will enter independent practice directly, and their acquired skillset during residency will impact the safety of their practice. Furthermore, many of the graduating fellows from noninterventional subspecialties may be performing “light IR” procedures, especially in private practice where most radiologists share the general radiology workload while performing procedures that may be beyond the scope of their subspecialty training. To the best of our knowledge, there is no reported data on how many diagnostic radiology programs actually have a discrete procedural certification protocol.
The current health care shift from fee-for-service to value-based reimbursement is a government mandate to ensure that the best-qualified physicians are performing these procedures; hence, a standardized procedure competency program during diagnostic radiology residency can be of great value to radiology as a specialty. As an example, the American Board of Internal Medicine has a procedure competency requirement in certain procedures viewed as essential to the practice of that specialty. However, there is no mention of a standardized method of “how to get there”.
The ACGME in 2015 introduced a new set of training guidelines known as the diagnostic radiology milestone project. While it offers the framework for a more comprehensive approach in the assessment of radiology residents, there is no specific evidence-based approach of “how to get there,” leaving significant leeway for the residency programs to self-govern.
Without an RRC procedural competency requirement, each residency program may or may not develop an evidence-based, standardized, and validated procedural competency curriculum to ensure a more or less homogeneous level of procedural competency on graduation. With this concern in mind, we drafted a simple protocol for our residency program to transition radiology residents from performing procedures under direct supervision to performing them under indirect supervision, awarding them a “certified status” [Figure 5].
During the training period, we placed great emphasis on the need to discuss the case with the supervising attending before proceeding with an indirectly supervised procedure to ensure sound judgment, patient safety, and availability of the faculty member for on-site direct supervision.
Radiology residents performing procedures under indirect supervision was not a common practice at our institution before our protocol; however, currently, a significant share of “light IR” procedures are performed under indirect supervision by the certified trainees marking a positive shift in resident hands-on experience and graded responsibility. Whether our approach will impact resident postgraduation employability or affect the safety of their practice remains to be seen. This far, four of our former graduates who did not pursue a VIR fellowship have applied and obtained “light IR” hospital privileges using their MedHub® case log from the “certification” process.
The “certification” process was only intended to identify residents who can perform light IR procedures under indirect supervision, with the goal of enhancing the resident's procedural training experience in a standardized fashion and allow them to experience a more real-life experience than they would with traditional direct supervision. However, one has to be aware that procedural billing of indirectly supervised residents has potential billing challenges given the fact that Medicare and Medicaid require direct supervision of most therapeutic services to provide reimbursement. In our own experience, these “light IR” procedures offered a tremendous learning opportunity for our trainees while allowing the faculty to directly supervise complex invasive procedures.
Besides the single-center nature of our project, our protocol has several limitations. During the 2015/2016 year, the interventional radiology division hired two physician assistants to provide “light IR” procedures under indirect supervision. Their initial training process may have affected the caseload availability of our rotating residents. This, and the fact that some procedures are provided by other services at our institution, introduces confounding bias. Our findings, particularly the suboptimal thoracentesis certification rates, should be interpreted with this in mind.
For the purpose of our review, only procedures performed under direct supervision were taken into account to evaluate the results of our certification process. This may explain our null complications rate. A future review of procedures performed under indirect supervision by “certified residents” may shed more light on the overall safety of our protocol.
Simulation-based training was not part of our protocol and could have had a positive impact as reported by other institutions and specialties., Simulation may have hastened our certification process allowing for a less restrictive live-procedure threshold. In addition, the use of standardized assessment tools to evaluate resident procedural competency such as those based on the Angoff method could have validated our results as reproducible and applicable elsewhere., However, one must note that direct observation remains the cornerstone of procedural training and was heavily emphasized during our training protocol.
The mere performance and documentation of a number of procedures does not ensure adequate training or competency. Our protocol streamlined a series of requirements to achieve procedural competency to perform procedures under indirect supervision during residency with the primary goals of ensuring patient safety and postgraduation procedural competency. While several competency assessment protocols are reported in the literature, widespread adoption is lacking. In addition, some of these proposed competency assessment tools do not specifically address procedural training.
Our protocol is a straightforward and simple way to get started. However, it is evident that a multipronged approach using simulation-based training, multisource feedback, validated assessment tools, and lifelong self-assessment methodology would offer better career-long results.
| Conclusions|| |
Despite several agencies requiring continuous oversight and performance assessment of physician practice during and after postgraduate training, lack of evidence-based procedural competency standards affects most residency programs.
Several large academic radiology programs may have already established evidence-based procedural competency training; however, this remains the exception rather than a common practice across all programs. In this article, we report a simple way to start a procedural competency protocol that can be easily adopted by many diagnostic radiology residency programs in need of structured procedural training and competency assessment.
More research would be needed to evaluate the widespread feasibility of our procedural competency protocol and its impact on patient safety. A special task force at the national level may be better suited to take on this challenge.
The authors wish to thank Dr. M. Elizabeth Oates for her editorial support and the valuable feedback.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Miller MD. Office procedures. Education, training, and proficiency of procedural skills. Prim Care 1997;24:231-40.
Philadelphia Center for Risk Management, Health Resources and Services Administration, U.S. Department of Health and Human Services. Policy Information Notice 01-16: Credentialing and Privileging of Health Center Practitioners; 2001. Available from: http://www.bphc.hrsa.gov/
. [Last accessed on 2010 Nov 16; Last revised on 2006 Jul 27].
Vozenilek J, Huff JS, Reznek M, Gordon JA. See one, do one, teach one: Advanced technology in medical education. Acad Emerg Med 2004;11:1149-54.
Rohrich RJ. “See one, do one, teach one”: An old adage with a new twist. Plast Reconstr Surg 2006;118:257-8.
Defilippis AP, Tellez I, Winawer N, Di Francesco L, Manning KD, Kripalani S, et al.
On-site night float by attending physicians: A Model to improve resident education and patient care. J Grad Med Educ 2010;2:57-61.
Trowbridge RL, Almeder L, Jacquet M, Fairfield KM. The effect of overnight in-house attending coverage on perceptions of care and education on a general medical service. J Grad Med Educ 2010;2:53-6.
Kashner TM, Byrne JM, Henley SS, Golden RM, Aron DC, Cannon GW, et al.
Measuring progressive independence with the resident supervision index: Theoretical approach. J Grad Med Educ 2010;2:8-16.
Siragusa DA, Cardella JF, Hieb RA, Kaufman JA, Kim HS, Nikolic B, et al.
Requirements for training in interventional radiology. J Vasc Interv Radiol 2013;24:1609-12.
Pastis NJ, Nietert PJ, Silvestri GA, American College of Chest Physicians Interventional Chest/Diagnostic Procedures Network Steering Committee. Variation in training for interventional pulmonary procedures among US pulmonary/critical care fellowships: A survey of fellowship directors. Chest 2005;127:1614-21.
Mendiratta-Lala M, Eisenberg RL, Steele JR, Boiselle PM, Kruskal JB. Quality initiatives: Measuring and managing the procedural competency of radiologists. Radiographics 2011;31:1477-88.
Sacks D, McClenny TE, Cardella JF, Lewis CA. Society of interventional radiology clinical practice guidelines. J Vasc Interv Radiol 2003;14:S199-202.
Tariq M, Bhulani N, Jafferani A, Naeem Q, Ahsan S, Motiwala A, et al.
Optimum number of procedures required to achieve procedural skills competency in internal medicine residents. BMC Med Educ 2015;15:179.
Cation LJ, Durning SJ. Procedure skill competence and certification in internal medicine residency training. Teach Learn Med 2003;15:175-9.
Vydareny KH, Amis ES Jr., Becker GJ, Borgstede JP, Bulas DI, Collins J, et al.
Diagnostic radiology milestones. J Grad Med Educ 2013;5:74-8.
Barsuk JH, Cohen ER, Vozenilek JA, O'Connor LM, McGaghie WC, Wayne DB, et al.
Simulation-based education with mastery learning improves paracentesis skills. J Grad Med Educ 2012;4:23-7.
Smith CC, Huang GC, Newman LR, Clardy PF, Feller-Kopman D, Cho M, et al.
Simulation training and its effect on long-term resident performance in central venous catheterization. Simul Healthc 2010;5:146-51.
Angoff WH. Scales, norms, and equivalent scores. In: Thorndike RL, editor. Educational Measurement. 2nd
ed. Washington, DC: American Council on Education; 1971. p. 508-600.
Huang GC, Newman LR, Schwartzstein RM, Clardy PF, Feller-Kopman D, Irish JT, et al.
Procedural competence in internal medicine residents: Validity of a central venous catheter insertion assessment instrument. Acad Med 2009;84:1127-34.
Kilminster SM, Jolly BC. Effective supervision in clinical practice settings: A literature review. Med Educ 2000;34:827-40.
Schmitt JE, Scanlon MH, Servaes S, Levin D, Cook TS. Milestones on a shoestring: A Cost-effective, semi-automated implementation of the new ACGME requirements for radiology. Acad Radiol 2015;22:1287-93.
Isupov I, McInnes MD, Hamstra SJ, Doherty G, Gupta A, Peddle S, et al.
Development of RAD-score: A Tool to assess the procedural competence of diagnostic radiology residents. AJR Am J Roentgenol 2017;208:820-6.
[Figure 1], [Figure 2], [Figure 3], [Figure 4], [Figure 5]