One of the benefits of myTIPreport is that at evaluation time, a teacher doesn't need to think about the traditional level system. Instead, they are only asked to indicate whether or not they have observed a learner being able to perform a given milestone. With that data, myTIPreport is then able to calculate the milestone level they have achieved for ACGME reporting purposes. This article will walk through how that calculation is made.
First, let's talk through some of the basics of the views.
In the summary view of the left-hand side, you'll see each of the subcompetencies along with some statistics. Each entry has a progress bar that tells you the current level of the learner for the subcompetency along with how many evaluations they've received for it.
The insights page has more granular data on each of the milestones and also gives insight into how the levels are calculated. Let's walk through a few examples:
First, some basics of the view:
Next you'll see a list of the milestones for this subcompetency and some stats about each of them. Here's how to interpret those numbers:
In the above picture, you can see that the learner demonstrated the level 1 milestones in the one evaluation she received. You can also see that she had the opportunity to demonstrate all milestones (and wasn't able to).
How is this data then taken to give a single, overall level number? There are three rules:
In this example, even though the learner has achieved all of the milestones in level 3, she has not achieved all of the milestones in level 2. This will result in a level score of 1.5 since she has not achieved all of the milestones in level 2, even though she has achieved those in level 3.
By default, only the last 10 opportunities to demonstrate a capability are considered. This helps prevent a large number of failures on advanced milestones early in someone's career plaguing them later. When this limit is hit, it will be displayed like this:
If you want to change the number of evaluations considered when calculating a level, you can do so by following the directions here.
We understand that level calculations can be a bit confusing at first. If you're still unclear about anything explained here, feel free to ask clarifying questions at anytime by emailing [email protected]