How the questions are written and verified

That last sentence is the one most question banks would leave out, so it comes first. You are entitled to know how the thing you are studying from was made, and to decide for yourself how far to trust it. The rest of this page is the process in detail, including what it cannot catch.

1. The blueprint comes from the exam owner

Each exam has a blueprint file built from the owner’s own published material: the content outline or test plan, and the candidate handbook. It records the domains and their weights, the tasks under each domain, the number of questions, the time limit, the fee and the retake rules, each with the page it was taken from and the date it was checked.

Task statements in our blueprints are our own short paraphrases. The owner’s codes and domain names are kept as published, because they are identifiers, but the outline text itself is the owner’s copyright and is not copied. Where a fact could not be confirmed on the owner’s site, the blueprint says so and the site does not state it.

2. Each question is written to one task

A question is generated by an AI model for one task in the blueprint, with the standard references for that exam in front of it. It must produce the question, the answer options, the key, a written reason for every option (why the right one is right and why each wrong one is wrong), a one-line takeaway, and a citation to a reference.

Nothing is taken from an exam. We do not have exam questions, we do not accept questions that candidates remember, and the models are not asked to reproduce any.

3. Mechanical checks and a duplicate check

Before any model looks at a draft again, it has to pass a set of fixed rules: exactly four distinct options with one key, no “all of the above” or “none of the above”, a key that is not conspicuously the longest option, a negative such as NOT or EXCEPT written in capitals, a reason of real length for every option, and at least one reference. It is then compared with every other question in that exam’s bank. Near-duplicates are rejected, and of two similar drafts the older one is kept, so the count on an exam page is a count of different questions rather than of rewordings.

4. A second model solves it blind, twice

A different model from the one that wrote the question is given only the question and its options, with no key and no explanations, and asked to answer it. It does this twice, with the options in a different order the second time. Both answers must match the key, and the solver must not say that a second option is defensible or that none is correct. A question that fails is rejected. This catches the most damaging kind of error, a wrong key, and a common second kind: a question with two defensible answers.

5. A flaw audit

The item is then read whole, with its key, explanations, task and references, against a checklist of item-writing flaws: an option that gives itself away, a rationale that contradicts the key, a reference that does not support the answer, or a question that tests something outside the task it is tagged to. Only an item that passes every check can be published, and the result of each check is kept with the item.

6. After publication: reports and statistics

Every question has a report button. A report goes to a person, and a question with two open reports is suspended until it has been looked at. Separately, each question’s own answer data is rebuilt regularly and flags three patterns for a person to review: a wrong option chosen more often than the key, a question that stronger performers miss more often than weaker ones, and a question almost nobody gets right. A flag is a request to look, not a verdict.

When a question is corrected or withdrawn, the change is recorded in the public corrections log.

What this process cannot do

• It is not expert review. Two models agreeing is evidence that a question has one best answer. It is not the judgement of a practitioner, and a subtle error that both models share can pass.
• It cannot see the exam. The questions follow the public outline. How closely they resemble the exam in style and difficulty is something we cannot measure directly and do not make claims about.
• It cannot predict your result. We do not publish a pass rate for people who use this site, and the readiness estimate in the product is a band, not a probability.

This is why every question carries its reference. If an explanation matters to you, check it against the source, and if it is wrong, tell us.

Keeping up with outline changes

Exam owners revise their outlines. When one changes, the new outline is loaded as a new blueprint, questions are re-tagged or retired against it, and the change is noted in the corrections log. Each exam page states which edition of the outline it follows and the date its facts were last checked.

Common questions

See also the editorial policy and the trademark notices.