Something I forgot to mention in my post on how I passed the CompTIA A+ Core 1 is important enough, to me, to create a separate post. This is the concept popularized by Former Secretary of Defense Donald Rumsfeld.
...There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don't know we don't know...
Former United States Secretary of Defense Donald Rumsfeld, February 12, 2002
I think applying this concept to knowledge is an extension of Make it Stick's principle of avoiding "Illusions of Knowing." I applied this concept to practice exams by doing the following:
Any question that I did not immediately know the answer, or was not 100% sure I knew, I marked with a star.
How does this apply to Rumsfeld's model? Each question on an exam falls into one of 4 categories. Which category it falls into depends on whether I was certain I would get the question right, and how that compares to whether I actually got it right. What I mean by certainty is when reading the question and the options, I knew immediately what the answer was (in my opinion) without hesitation.
Known Knowns
This is an answer I was certain I got right and was actually correct.
This means I have confirmed the subject of this question to be a known known. I know it, and I know I know it. This is the category that I try to maximize on practice exams. I aim for as many answers as possible to be in this category. This is only possible if I am:
- Able to identify what I do not know
- Able to put the effort in to fill those gaps
Known Unknowns
This is an answer I wasn't sure if I was correct and was wrong.
Out of the two unknowns, this is the one I want to see the most of. Everyone has limits of their knowledge. We cannot know everything, but we should know the limits of our knowledge. Adding this extra dimension to practice tests changes the metacognitive effect of wrong answers for me. I can be happy that I was aware of the limits of my knowledge, and then actively work towards filling in the gap.
Additionally, this casts "Illusions of Knowing" to the wayside. By the time I finish a 90 minute practice exam and start grading my answers, if I did not mark questions, I may no longer remember how confident I was in my answer for question 10, so I can convince myself that it was a mistake and I know the answer. By marking the question before hand, I remove this ambiguity.
Notice that Known Unknowns can also be answers I wasn't sure about, and were correct. These require a similar review of the material, but this is to gain confidence or clarification. Not much re-learning needs to be done. Obviously, the number of Known Unknowns went down as I studied and tested and retested myself, but I also noticed that along with this progress, the Known Unknowns left would become ones that ended up being correct.
This, I think, builds confidence in test taking. When I wrote the real exam, even when I was unsure of an answer, I had the experience that the majority of my Known Unknowns were correct in my final practice tests, so I had confidence in my knowledge.
Unknown Unknowns
An answer I was certain was correct, but was incorrect.
This is a true example of "Illusions of Knowing". I was wrong, yet for some reason I was convinced I was correct. These are solved in the same way as Known Unknowns, but normally they require more research. I've found Unknown Unknowns come up because there was something I misunderstood, or I was being a bad test taker and not reading the question in full.
In my own experience, Unknown Unknowns are uncommon, I would find them once or twice in a practice exam. They are however the most serious and the ones I re-studied the most.
Unknown Knowns
Critics of Rumsfeld's argue that there is a fourth category, Unknown Knowns, which are things we know, but choose to ignore.
I'm not sure if this fits into the testing paradigm, but this could be an example of unlearning or expertise bias. This could be a question I got wrong because I wanted to believe (because of past experience, or personal values) something else was the answer.
Conclusion
When I was taking my practice exams I always marked the questions I was uncertain of the answer, or had to use a process of elimination to come up with the best fit. This provided a certainty to the review process that otherwise might be plagued with wishful thinking or bad memory. I would recommend this technique to anyone studying for any test, whether about IT, politics, or plant science.