CCA Prep
CCAR-FUA

Lesson 4.1Lesson 19 of 30

Explicit review criteria

Design prompts with explicit criteria to improve precision and reduce false positives

Explicit review criteriaVague prompt"Check that comments are accurate""Be conservative, onlyhigh-confidence"One noisy category in every PRExplicit criteriaReport bugs, security · skip styleSeverity levels with code examplesNoisy category off until its promptis fixedPrecision comes from defining what counts, not from asking for caution

Key points

  1. 1Explicit criteria beat vague instructions. "Flag a comment only when its claimed behaviour contradicts what the code actually does" works; "check that comments are accurate" does not.
  2. 2"Be conservative" or "only report high-confidence findings" barely moves precision. The model still has no definition of a reportable issue, so it just reports less, real bugs included.
  3. 3Say what to report (bugs, security vulnerabilities, data loss) and what to skip (minor style, naming, patterns consistent with the surrounding code).
  4. 4One noisy category erodes developers' trust in the accurate ones. Temporarily disable it while you rewrite and re-measure its prompt; keep the precise categories running.
  5. 5Define each severity level with an explicit condition and a concrete code example, so the same issue gets the same severity on every run.
What the exam guide checks here

Objective wording from the exam guide (English).

  • 4.1.K1Explicit criteria beat vague instructions
  • 4.1.K2"Be conservative" / "only high-confidence" do not improve precision like categorical criteria
  • 4.1.K3High false-positive categories erode trust in accurate categories
  • 4.1.S1Define what to report (bugs, security) vs skip (style, local patterns) instead of confidence filtering
  • 4.1.S2Temporarily disable high false-positive categories while improving their prompts
  • 4.1.S3Define severity levels with concrete code examples

Lock it in

Answer scenario questions on this topic. Every option comes with an explanation.

Practise 4.1 · 16 questions