Methodology v1.2.0
How the grades are calculated
This page is a first-class product artefact. Everything the grades depend on lives here — the indicator registry, the normalisation function, the weights, the grade bands, the trend calculation, the peer-comparison method, and the sensitivity analysis we run against every quarterly revision.
In three minutes
How we grade, in three minutes
- 1. Collect. We pull raw numbers from audited or statutory sources — AGSA, National Treasury, DWS, SAPS, Stats SA, city entity annual reports. Independent sources (GCRO, CSIR) supplement and are flagged as Independent.
- 2. Normalise. Every indicator is scored 0–100 by distance to a reference — a published standard where one exists, a regulatory threshold otherwise, or the metro-peer median as a fallback.
- 3. Aggregate. Indicators are weighted within a pillar (advisory-panel approved) into a 0–100 pillar score.
- 4. Letter grade. A ≥85, B 70–84, C 55–69, D 40–54, E 25–39, F <25. Bands are frozen per methodology version.
- 5. Trend & peer. Trend is a separate linear fit over the trailing 3 years; peer rank is a separate percentile against the other 7 metros. Neither is blended into the grade.
Neutrality charter
- We grade services, not parties or people.
- Government-data-first sourcing.
- If we can't source it, we don't publish it.
- Right of reply, 10 working days before publication.
- Corrections logged in a public changelog.
- Constructive framing: what would move this grade.
- Independent advisory panel reviews methodology annually.
Data-quality tiers
TAAuditedAudited / statutory
AGSA outcomes, audited annual-report KPIs, Treasury S71 data, SAPS official releases, Stats SA.
TBOfficialOfficial self-reported
SDBIP quarterlies, entity operational KPIs not yet audited.
TCOperationalOperational / derived
Metrics we derive from scraped official notices. Derivation methodology published.
TDIndependentIndependent / third-party
GCRO, CSIR, ISS, Moody's, Ratings Afrika. Flagged 'Independent' throughout.
Tier A/B indicators dominate grade weight (≥60% per pillar). Tier C metrics move the monthly pulse displays but have capped grade weight (≤15%).
Sensitivity analysis
Before every quarterly and annual revision we perturb the weights, swap the normalisation function, and drop each indicator in turn. If a grade flips under modest perturbation we publish it with a "borderline" marker.
Latest revision: 92% grade stability under ±20% weight perturbation; three pillar grades flagged borderline.
Data table
| Period | Value (% stable) |
|---|---|
| ±0% | 100 |
| ±10% weights | 96 |
| ±20% weights | 92 |
| alt normalisation | 89 |
| leave-1-out | 91 |
Indicator registry
The full technical spec, rendered directly from indicators.json.
| Pillar | Indicator | Unit | Tier | Weight | Reference |
|---|---|---|---|---|---|
| Water | Non-revenue water Percentage of water lost to leaks, theft, or unbilled use before reaching customers. | % | TAAudited | 30% | DWS target ≤ 35% |
| Water | Pipe bursts per 100 km of main Reported bursts on distribution mains, normalised to 100 km network length. | bursts | TAAudited | 20% | Metro median 42 |
| Water | Blue Drop score (drinking water quality) DWS composite compliance score for drinking water safety and system management. | score | TAAudited | 25% | SANS 241 excellence ≥ 95 |
| Water | Households reporting water outages > 2 days (last year) GHS metro cut: share of households reporting a water interruption longer than 2 days. | % | TAAudited | 15% | Metro median 18% |
| Water | Supply disruption notices (this month) Count of Joburg Water daily supply disruption notices, aggregated monthly (Tier C — derived from scraped official notices). | notices | TCOperational | 10% | 12-mo median 340 |
| Electricity | Total energy losses Electricity purchased vs sold, expressed as % losses (technical + commercial). | % | TAAudited | 35% | NERSA benchmark ≤ 10% |
| Electricity | Unplanned outage notices (this month) City Power published unplanned outage notices per month (Tier C — derived). | notices | TCOperational | 15% | 12-mo median 890 |
| Electricity | Load-shedding hours affecting Joburg (rolling 12 mo) CSIR + Eskom hours of load shedding weighted by City Power schedule exposure. | hours | TDIndependent | 30% | 2023 peak 3200h |
| Electricity | NERSA compliance findings Number of open NERSA licence-compliance findings against the distributor. | findings | TAAudited | 20% | Zero |
| Roads | Road network in poor/very-poor condition (VCI) Percentage of the paved network rated poor or very poor by the Visual Condition Index. | % | TAAudited | 35% | COTO target < 20% |
| Roads | Kilometres resurfaced vs target Actual km resurfaced against the JRA integrated report annual target. | % of target | TAAudited | 25% | 100% of published target |
| Roads | Pothole repairs completed (this month — throughput, not backlog) JRA-reported potholes repaired. Throughput, not backlog: repairs completed ≠ potholes remaining. | repairs | TCOperational | 10% | 12-mo median 4 300 |
| Roads | Traffic signal availability Percentage of intersections with working traffic signals (JRA self-reported). | % | TBOfficial | 15% | JRA target ≥ 90% |
| Roads | Household satisfaction with roads (GCRO) GCRO Quality of Life survey — % of respondents rating roads as good. | % | TDIndependent | 15% | Metro median 32% |
| Billing | Revenue collection rate Cash collected as % of billed revenue (National Treasury Municipal Money). | % | TAAudited | 35% | Treasury norm ≥ 95% |
| Billing | Billing-query resolution KPI Percentage of billing queries resolved within CoJ target (self-reported). | % | TBOfficial | 20% | CoJ target ≥ 90% |
| Billing | Ombudsman billing complaints resolved % of billing complaints closed by the Joburg Ombudsman within a cycle. | % | TAAudited | 25% | Ombudsman charter 80% |
| Billing | Repairs & maintenance as % of PPE Leading indicator of future service quality. Treasury norm ≥ 8%. | % | TAAudited | 20% | Treasury norm ≥ 8% |
| Safety | Serious contact crime per 100 000 Aggregate of ~30 SAPS precincts within CoJ, per 100k population. | per 100k | TAAudited | 35% | Metro median 780 |
| Safety | Residential robbery per 100 000 SAPS residential robbery aggregated across CoJ precincts. | per 100k | TAAudited | 20% | Metro median 205 |
| Safety | Feel safe walking alone at night (GCRO) % of GCRO respondents who report feeling safe walking alone after dark. | % | TDIndependent | 25% | Metro median 22% |
| Safety | Vehicle-related crime per 100 000 SAPS hijacking + theft of/from vehicle, aggregated across CoJ precincts. | per 100k | TAAudited | 20% | Metro median 640 |
| Governance | AGSA audit opinion (CoJ) Auditor-General's audit opinion for the City of Johannesburg. Scored: unqualified=100, unqual w/ findings=80, qualified=55, adverse=25, disclaimed=0. | score | TAAudited | 30% | Unqualified, no findings |
| Governance | Unauthorised/irregular/fruitless expenditure (R bn) Total UIFW expenditure per AGSA. Lower is better. | R bn | TAAudited | 20% | Zero |
| Governance | Cash coverage (months) Cash and equivalents divided by monthly operating expenditure. | months | TAAudited | 20% | Treasury norm ≥ 3 months |
| Governance | Credit rating (Moody's scale, mapped) Moody's rating mapped to 0–100. Ba3=45, Ba2=55, Ba1=65, Baa3=72, positive outlook adds 3. | score | TDIndependent | 10% | Metro median Ba2 (55) |
| Governance | % SDBIP targets achieved Share of Service Delivery & Budget Implementation Plan targets achieved (self-reported, cross-checked against AGSA performance findings). | % | TBOfficial | 20% | CoJ target 90% |
Independent advisory panel
The panel reviews the methodology annually. Minutes are published as PDF.
Prof. N. Molefe
Public governance, Wits School of Governance
Prof. K. van Wyk
Development economics, UJ SARChI Chair
T. Dlamini, CA(SA)
Ex-AGSA senior manager, MFMA audits
R. Naidoo
Civil-society data specialist, Open Cities Lab
Eng. M. Ferreira
Water & sanitation engineering, SAICE fellow
L. Mabaso
Journalism / civic tech, Media Hack