CPA Computer-Based Examination FAQs

CANDIDATE PERFORMANCE REPORTS

How does the candidate performance report relate to my score for this section?

Both your score and your performance information deal with your examination performance. However, they present different views of your performance and are separately derived.

Your score – the official result – was obtained by means of IRT (Item Response Theory)* scoring and the application of various quality control measures and verification processes. It was calculated as a whole, taking into account all your responses.

Your Candidate Performance Report information was derived separately. Performance results by content area (the first section of your Candidate Performance report) are calculated across multiple-choice questions and simulations only. They do not include written communication responses which are represented in overall performance (the second part of the report).

*For information about IRT, please refer to http://io.psych.uiuc.edu/irt/main.asp or to http://en.wikipedia.org/wiki/Item_response_theory   

My score for this section is 72 but my performance ratings by content area are 48, 48, 77, 63, and 65. When I add these numbers together and divide by 5, I don’t get 72. Why?

I multiplied my performance results by the percentage of the test devoted to each content area and then added them up but this did not equal my total score. Why?

Neither adding and dividing performance ratings by content area, nor multiplying performance ratings by the percentage of the test devoted to each area and then adding them will generate your score. Your score and your content area assessments are calculated on different scales. It is NOT possible to derive your score from diagnostic results.

My score is 74 and my performance in all content areas is “comparable.” Why didn’t I pass?
Your score is so close to passing that it is not surprising that your profile is very similar to the profiles of candidates in the “just passed” group. The fact that you did not pass indicates, however, that there was a weakness in your performance. Keep in mind that even if your performance was comparable to the performance of the “just passed” group, the “just passed” candidates may have done better in some categories to compensate for weaknesses in others.
My report shows comparable results in most of the content and overall performance areas, but I still failed. How is this possible?

It is likely that although your performance was weaker in some content areas, you compensated for that weakness by performing well in other content areas, and your overall performance was comparable as a result. Remember, your performance is measured against the performance of candidates who passed with scores of between 75 and 80, so it makes sense that your performance could be comparable in more than one area if your failing score is close to the reference group.

Why is my performance compared to the performance of candidates who earned scores of between 75 and 80? Why isn’t it compared to the performance of all who passed?

The assumption underlying the choice of this comparison group was that you would want to know how much better you needed to perform in order to pass. The candidates who earned scores of between 75 and 80 are in the “just passed” category. A comparison of their performance and yours is the best way to give you some idea of how your performance was like and unlike the performance of the “just passed” group.

On what basis are Performance results determined to be “weaker,” “comparable” or “stronger”?

Performance is compared to a range of one-half of one standard deviation of the average score of candidates who earned scores of between 75 and 80. Performance within the range is “comparable,” below the range “weaker,” and above the range “stronger.”

Example:
  • a candidate earns a performance result of 73 in one content area
  • the mean performance result in that content area is 70 for candidates who earned scores of between 75 and 80
  • in this instance, ½ of a standard deviation is 2
  • the candidate whose performance result is 73 is in the “stronger” category (Candidates whose performance results are between 68 and 72 are in the “comparable” category; candidates whose performance results are below 68 are in the “weaker” category)
What does standard deviation mean?

Standard deviation is a standardized unit used to report how close your score is to the average score. This unit of measure takes into account how spread out the data is. A low standard deviation means that the scores are tightly clustered; a high standard deviation means that they are widely scattered.

My report shows weaker performance in one content area. Was my performance weaker in multiple-choice questions or simulations?

It's impossible to tell. Candidate Performance results are based on both multiple-choice and simulation responses and it is not possible to separate the two.

Do all examinations have the same distribution of questions by content area when different candidates get different questions to answer?
All examinations are programmed to include the required percentage of questions in each content area. It doesn’t matter that candidates take different examinations because all examinations are constructed to have equal content area distribution of questions.
Should I study only the content areas that are indicated as being weaker?

It is always best to study everything. If you only study the areas where you are weak, you might do better on those areas, but worse on others when you re-test.

How does my performance in each content area relate to my overall performance?

The content area Performance results give an indication of performance by content category. The overall performance diagnostic results reflect performance by examination segment: multiple-choice questions, simulations, and written communication responses. Both are calculated across multiple-choice questions and simulations.

My friend and I have identical performance results in some report sections. However, our “performance compared to passing candidates” falls into different categories. How is this possible?

Performance results are displayed on the Candidate Performance report after rounding to the nearest integer. However, the cut scores (the demarcation points separating one category from another) used to compare your performance to the performance of those in the “just passed” category are NOT rounded. Therefore, it is possible that two results may appear to be the same because of rounding but the underlying results vary by decimal points and fall above or below the cut scores for weaker, comparable, or stronger categories.

What information is included on the diagnostic report for BEC (Business Environment and Concepts) as that section does not contain simulations?

The diagnostic report for BEC displays the candidate’s performance by content area only, based on responses to multiple-choice questions.

What does this statement mean? “….there is some imprecision inherent in all performance assessments. Your examination performance could differ from one day to the next depending on a variety of factors”

In the testing world, the term standard error of measurement describes the imprecision inherent in all performance assessments. An individual’s examination performance could well differ from one day to the next, based (or not having) reviewed the topics emphasized on the test. The individual performance assessment should be viewed as a general indication of performance on a given day, recognizing that results would vary if the same examination were given to the same individual several times.



© 2006 American Institute of Certified Public Accountants.
All Rights Reserved. Contact Us

      Last revised