Before 1994, the NCLEX® was your typical paper-and-pencil exam and was only administered twice a year due to the logistics of administration (i.e. ordering testing materials, hiring proctors, shipping test booklets to be graded, etc.). However, today both the NCLEX-RN® and NCLEX-PN® employ Computerized Adaptive Testing (CAT).
This makes taking the NCLEX a much different testing experience than what many students are used to. On a conventional paper-and-pencil exam, questions are predetermined and standardized. Computer adaptive testing asks test-takers different questions based on their previous responses.
How Does CAT Work?
Computer Adaptive Testing (CAT) scoring is a dynamic process that adapts to a test taker’s performance in real time. On the NCLEX, the difficulty of each question is determined by the candidate’s previous responses. The scoring process relies on an algorithm that selects questions and estimates the candidate’s ability level based on their performance.
To put it simply, if you answer a question correctly, the computer re-estimates your ability and presents a new question that is a little more difficult — a question that you should have a 50% chance of answering correctly. Conversely, if you answer a question incorrectly, the computer will present a new question that is slightly less difficult. This pattern continues until you run out of time or until the computer identifies your competency level as above or below the passing standard. For both the NCLEX-RN and the NCLEX-PN the maximum amount of time is five hours, the minimum number of items is 85, and the maximum number of items is 150.
Here’s a step-by-step breakdown of how CAT scoring works:
- Start with a medium-difficulty question — The test begins by presenting the candidate with a question of medium difficulty.
- Assess the response — The algorithm assesses whether the answer was correct or incorrect.
- Update the ability estimate — Based on the response, the algorithm updates its estimate of the candidate’s ability level.
- Select the next question — The algorithm then selects a new question that matches the updated ability estimate. If the candidate answered the question correctly, the ability estimate is increased; if they answered incorrectly, the estimate is decreased.
- Repeat the process — Steps 2 through 4 are repeated for each subsequent question until a stopping criterion is met. This typically occurs when the algorithm has gathered enough information to determine the candidate’s ability level with a high level of confidence.
- Determine the pass/fail outcome — Once the stopping criterion has been met, the candidate’s ability level is compared to the predetermined passing standard. If the candidate’s ability estimate is above the passing standard, they pass the exam; if it’s below the standard, they fail.
- Confidence interval — The algorithm also calculates a confidence interval around the candidate’s ability estimate. This interval represents the range within which the true ability level is likely to fall, with a certain level of confidence (e.g., 95%). The confidence interval is used to ensure that the test outcome is as accurate as possible.
CAT Scoring: Pros and Cons
The NCSBN has adopted CAT for some good reasons. However, when it comes to the self-assessment exams that you take before the NCLEX, CAT really doesn’t make sense. Here are the pros and cons of CAT:
Pros of CAT
The CAT format is used for the NCLEX because it offers a number of important benefits:
- Individualized — Tailors questions to each candidate’s ability by presenting questions based on previous responses.
- Secure— Reduces the likelihood that two test-takers will encounter the same sets of questions which makes it more difficult to share information.
- Balanced — Ensures that questions will appropriately represent all content areas with algorithmic selection.
- Quick— Provides real-time scoring which allows examinees to receive their results more quickly.
In short, the CAT format ensures a more accurate assessment of a candidate’s nursing knowledge, skills, and abilities, while also providing a more efficient and secure testing process.
Cons of CAT
The above benefits are clear. CAT is a legitimate scoring method for exam day, but what about before exam day? The truth is that only the NCSBN has the resources to create thousands of questions to deliver a CAT exam. A question bank of this size includes questions that range from very easy to very difficult across all subjects and systems. It is an enormous task to build and maintain a question bank large enough to deliver a true CAT exam. Here are some cons to consider when thinking about adaptive tests:
- Efficiency — Practicing with easy questions doesn’t prepare you for difficult questions, but practicing with difficult questions prepares you for easy questions. Self-assessment tests that focus on difficult questions prepare you for the NCLEX more efficiently.
- Bandwidth — It is nearly impossible to keep a current and accurate bank of questions that parallel the quality of the NCLEX, and trying to deliver a CAT exam with a smaller bank of questions minimizes the likelihood that the exam will be balanced and unique for the individual test taker.
- Transparency — The NCLEX scoring algorithm is proprietary, so there is no way to replicate results on a practice exam that would be valid in comparison to the NCLEX. Practicing with something other than the actual NCLEX questions that are customized for each user can promote a false sense of security, and for those that “pass” with the minimal number of questions, less opportunity to view a larger selection of test questions.
What Are the Pass/Fail Rules
Whether you pass or fail the NCLEX is determined by one of the following three rules:
- 95% Confidence Interval – The exam ends when the computer is 95% certain that the candidate’s ability is clearly above (pass) or below (fail) the passing standard.
- Maximum-Length Exam – If a candidate receives the maximum number of questions and the candidate’s ability is close to the passing standard, the computer will determine whether the candidate receives a pass or fail score.
- Run-Out-of-Time – If time runs out and a candidate has not answered the minimum number of questions, they will fail. If the candidate has answered the minimum number of questions, the computer will determine a pass or fail based on existing responses.
NGN Scoring Models
Previously, each NCLEX item would be scored as either wholly correct or incorrect. However, The Next Generation NCLEX (NGN) has introduced a scoring methodology known as polytomous (partial credit) scoring. This means that examinees can receive partial credit for correct responses within items that require multiple responses. Here is a breakdown of how NGN scoring works:
Each of the 15 question types falls under different scoring method categories. For example, multiple choice questions are scored according to the 0/1 rule, while Drop-Down Cloze Dyad/Triad items are scored with “Rationales Scoring.” Learn more about NCLEX question types and try out some sample questions.
Now that you understand how CAT works, it’s time to start preparing for the NCLEX. Check out our in-depth guide about the NCLEX, and learn how to register for the NCLEX if you haven’t already.
CAT Terminology
Candidate ability
Represents the test-takers readiness and competence for nursing practice. The ability is typically represented on a continuous scale, with higher scores indicating greater knowledge or skill in the subject area.
Ability estimate
A numerical representation of a test taker's level of knowledge, skills, or competence in the subject matter being assessed. It is a calculated value based on the candidate's performance during the test and serves as an indicator of their proficiency in the content areas covered by the exam.
Passing standard
After the test reaches its stopping criterion, the candidate's final ability estimate is compared to the passing standard. If the ability estimate meets or exceeds the passing standard, the candidate passes the exam; if it is below the standard, they fail.
Logit
A unit of measurement to represent how well someone is doing on the test compared to the difficulty of the questions they're answering. This makes it easier for the scoring system to understand and compare the performance of different test-takers.
Represents the test-takers readiness and competence for nursing practice. The ability is typically represented on a continuous scale, with higher scores indicating greater knowledge or skill in the subject area.
A numerical representation of a test taker's level of knowledge, skills, or competence in the subject matter being assessed. It is a calculated value based on the candidate's performance during the test and serves as an indicator of their proficiency in the content areas covered by the exam.
After the test reaches its stopping criterion, the candidate's final ability estimate is compared to the passing standard. If the ability estimate meets or exceeds the passing standard, the candidate passes the exam; if it is below the standard, they fail.
A unit of measurement to represent how well someone is doing on the test compared to the difficulty of the questions they're answering. This makes it easier for the scoring system to understand and compare the performance of different test-takers.