Separate facts, judgments and evidence
Start with concrete information, then inspect scores when useful. Every assessment belongs to a task, date, fixture and tested scope.
How to read a report
- Check the test date, account or tier, fixture and available native output.
- Distinguish observations, vendor statements, evaluator judgments and unverified information.
- Compare plan terms and actual output against your task.
- Inspect evidence and score rationale when needed. Scores across unrelated tasks are not averaged into a product rating.
Product entities & task topic connections
RealTest connects independent product profiles with task-specific benchmark topics: product-wide facts are maintained once, while individual capabilities are tested on shared fixtures.
Each product contract records operator provenance, site-wide billing model, privacy terms, verified pricing snapshots, and the Path A trust baseline.
Task topics test candidate tools on the same benchmark input, combining 30% honesty & reputation with 70% task capability into an integer score.
Strong performance in one task does not guarantee quality in another. Product profiles list each tested capability with its date and score without averaging.
Developers can submit products, sync plan changes or embed SVG badges. All published scores require passing native export and SHA-256 evidence gates.
Seven assessment dimensions
| Dimension | Information to inspect |
|---|---|
| Pricing & rules | Plans, payment periods, allowances, renewal and refunds, with sources and dates. |
| Interaction & conversion | Observed registration, payment and download flows, including obstacles. |
| Background & reputation | Verifiable operator and external information. Missing facts stay unknown. |
| Engineering & privacy | Distinguish observed uploads, local processing and vendor retention statements. |
| Task quality | Task-specific fixtures, native outputs and failure cases; images and documents use different observations. |
| Delivery | Separate tested exports from advertised maxima. Record formats, resolution, batch and API support. |
| Speed & stability | State the environment and timing boundaries. One network wait is not a general speed guarantee. |
Automated checks, human judgment & untested scope
| Category | What is covered | Limits & how to verify |
|---|---|---|
| Deterministic script checks | Reads exported file header dimensions and formats; computes alpha-channel edge error (MAD/SAD), compression PSNR/SSIM or DOCX text and editable table cells; verifies SHA-256 evidence digests and 30/70 arithmetic totals. | Establishes properties of the attached artifact only; cannot reconstruct processing wait or paid entitlements from an uploaded file alone. |
| Human editorial judgment | Evaluates when pricing or watermark limits are disclosed and how much login/upsell friction occurs; synthesizes seven-aspect bilingual summaries separate from fixture quality; records fingerprint-bound human decisions on cap candidates. | Rationales and component scores are open under each report’s scoring basis for inspection against screenshots and native exports. |
| Explicitly untested scope | Does not generalize a single portrait or text PDF to every material or scanned OCR; does not treat one network wait as an uptime guarantee; keeps login-gated or unstated bills marked unverified with reasons. | Evaluation effort prioritizes practical tool choice rather than exhaustive scenario coverage; source checks and performance tests are separated on the updates page. |
Evidence boundaries
Native output supports inspection of a delivered file. A screenshot establishes only the visible interface or preview; absent native output is stated explicitly.
Privacy policies are vendor statements. Observing local or cloud processing is not a security audit. A dated record does not automatically describe the current service.
Network-observed server wait can include queues, inference and transfer overhead. End-to-end timing also includes network and interaction effects; neither is pure model inference time.
When results are comparable
Align the task, fixture and version, tested tier, scope and rubric. When historical records lack these conditions, compare the recorded facts without assigning a unified rank or winner.
Most historical pricing consists of prose and does not cover every plan. Pages mark incomplete coverage. Starting prices do not imply full plans, and one-time trials are not recurring free allowances.
Scoring formula, historical rubrics & publication
The first four dimensions form honesty & reputation; the final three form capability. Overall score weights them 30% and 70%. An applicable, validly reviewed cap may affect the final score.
Component maxima follow each report’s rubric. Historical scores are not recalculated for this redesign. Cap candidates without valid review, insufficient evidence and inconsistent records are withheld.
The report contract validates publication eligibility. Pages, public data and badges share the released set. Submitting a lead does not automatically change findings or approve a report.
Test fixtures & file inspector →Editorial independence & commercial disclosure
RealTest evaluates tools anonymously from an ordinary-user perspective: we do not notify vendors beforehand, never accept vendor-supplied VIP test accounts, and never sell rankings, badges, or score modifications.
A tool’s business model (free, freemium, subscription, or credits) and login requirements are treated as neutral filter attributes rather than score penalties. Deductions apply only to undisclosed restrictions, unfulfilled promises, or deceptive conversion friction.
Vendor corrections, re-testing & appeals
We publish reproducible test observations and dated official snapshots. If your product has updated its pricing tiers, expanded free allowances, resolved a backend error, or if you believe a recorded fact is inaccurate, you can request a verification review:
- Online queue (recommended, issues a trackable receipt UUID): /submit/
- Editorial & developer email: hello@realtest.org
- Handling rules: Verified pricing or term changes are recorded as dated fact supplements; capability changes require passing a fresh fixed-fixture test and SHA-256 evidence gate, with public logs on Updates.