Fameproof / Methodology
New audit
Open methodology · v1.0

How Fameproof measures AI visibility

A defined test—not a universal claim about a person, a model, or public opinion.

Definition

What we measure

AI Visibility Score measures how often and how prominently a named public person is mentioned across a declared prompt set, model configuration, and measurement date. Each audit is a reproducible observation of those tests. It does not measure how often a person appears “in AI” generally.

The unit of evidence is a model response to one declared prompt. A response is not treated as a factual source about the measured person.

Prompt design

English prompt pack v1 contains 24 prompts across four categories: general recognition, domain prominence, geographic or cultural association, and bounded recommendation lists. Prompts never name the target or comparators. They may use non-identifying declared metadata such as professional category and region.

  • Every prompt has a stable key, ordinal, rationale, known bias note, and expected response shape.
  • The shared system instruction asks for direct, concise answers without revealing who is being measured.
  • Rendered prompt text and the exact pack version are stored immutably with the audit.

Mention extraction and identity

Text is Unicode-normalized and case-folded before matching the canonical name and safe aliases at word boundaries. Short aliases marked ambiguous never count alone. The matched sentence, alias, list position, method, and confidence are stored. A person counts at most once per sample.

Identity context helps distinguish namesakes. The product does not infer sensitive traits, reputation, trustworthiness, health, beliefs, or character.

Memory visibility formula

Scores are calculated deterministically from stored extraction results; a model never chooses or judges the final score.

Memory Visibility Score (0–100) = round(100 × (
  0.45 × mention_rate +
  0.30 × prominence +
  0.15 × topic_coverage +
  0.10 × consistency
))

Mention rate is mentioned samples divided by successful samples. Prominence gives full credit to unranked mentions and linearly discounts list positions 2–10. Topic coverage is the fraction of tested categories with a mention. Consistency is one minus the standard deviation of per-prompt mention rates across repetitions.

Grounded visibility

Grounded Visibility Score (0–100) = round(100 × (
  0.40 × mention_rate + 0.25 × prominence +
  0.15 × topic_coverage + 0.10 × consistency +
  0.10 × citation_rate
))

Grounded results are kept separate from memory results. Models are equally weighted in the all-model aggregate, so no provider disappears behind request volume.

Eligibility and confidence

Failed and skipped samples are excluded from component denominators and reported separately. At least 70% of planned samples must succeed for an aggregate; otherwise the result is marked insufficient data. When at least 10 eligible samples exist, Fameproof computes a 95% bootstrap confidence interval using 1,000 resamples.

Numeric trend claims are shown only between audits with the same prompt-pack version, mode, model set, repetitions, temperature, and other request settings.

Leaderboard eligibility and recent-result reuse

The AI Fame Leaderboard ranks the latest sufficient completed measurement for each public figure using the standard English v1 pack, memory mode, all three standard models, two repetitions, and temperature zero. The board shows the measurement date and confidence interval. People and organizations are never mixed; organization rankings require their own equivalent prompt pack and audit data.

Before queueing work, Fameproof checks for a completed audit with the exact same target, comparator set, prompt pack, mode, model set, repetitions, and temperature. If it finished within the configured 30-day freshness window, that result is returned and no new AI model requests are issued. Any material configuration difference creates a new measurement.

Known limitations

  • Model outputs vary over time, even when a model identifier is unchanged.
  • Training data and list prompts may reproduce geographic, linguistic, popularity, and historical biases.
  • Provider model IDs describe API configurations, not consumer chat applications.
  • A mention is an observation, not proof of importance, quality, sentiment, or factual accuracy.
  • Prompt coverage is curated but cannot represent every possible question or culture.
This audit measures visibility within the selected prompts, model versions, settings, and date range. It does not measure universal AI knowledge, public reputation, or the behavior of consumer chat applications.

Retention and deletion

Raw model outputs are retained for 90 days by default and then deleted or redacted. Aggregate scores, immutable audit configuration, and non-sensitive extraction metadata remain until the workspace deletes the audit. Deleting a person purges their aliases, audits, raw responses, exports, and share links.

Public share pages contain aggregates only—never raw model responses.