Accuracy checks
50 checks run on Oakland’s numbers at every update: 47 pass, 0 fail, 2 are withheld, 1 doesn’t apply. Each is a CSV row on Data & downloads.47Pass
0Fail
2Withheld
1n/a
Coverage
Q01How many of the city's public bodies do we follow?18 bodies have meetings in the window. The city's own list we hold has 27 active bodies.PassBodies with no meetings in the window are not shown.
Q02Is the newest listed City Council meeting recent?Newest listed City Council meeting: Oct 20, 2026; newest transcribed: Oct 6, 2026.Pass
Q03Is any body missing its recent meetings (newest listed meeting is much older than its usual gap)?None.Pass
Q04Is the updated-through date the date every chart is complete through, not just the newest video?Updated through Oct 8, 2026: the newest meeting we have a transcript of. Eight meetings listed after it, to Oct 20, 2026, are listed on their own page as not transcribed yet; talk time, issues, quotes and counts of transcribed meetings leave them out, and a count that includes them says "listed". The topic chart is built from the clerk's agenda times, which cover meetings not yet transcribed too, so it runs to Oct 20, 2026 and says so.Pass
Q05What share of meetings have a transcript?229 of 244 (94%).PassTopic time uses the clerk's video index, not transcripts, so it doesn't depend on this.
Q06What share of meetings have a known length?243 of 244 (100%).PassA body's hours show only when 8 in 10 of its meetings have a length.
Q07Any whole months with no listed City Council meeting?1 of 11 full months empty: 2026-08. (Councils often skip August.)Pass
Q08Any meetings with no video link?0 of 244.PassCounted as meetings; they add no hours or topic time.
Meetings
Q09Are any meetings listed twice?0 exact duplicate rows found and removed.Pass
Q10Is any video counted under two bodies?0 videos were listed under two bodies; each is counted once.Pass
Q11Are part 1 / part 2 videos of one meeting counted as two meetings?4 meeting days had more than one video. Each counts as one meeting; their lengths add up.Pass
Q12Are meetings outside the window kept out?294 rows outside Oct 24, 2025 – Oct 23, 2026 left out.Pass
Q13Where do meeting dates come from?From the city clerk's own video portal listing for each meeting, not from our transcripts.Pass
Q14Are very short videos (cancelled, no quorum, test streams) counted?1 videos under 10 minutes: counted as meetings, left out of hours.Pass
Q15Any impossible lengths (over 12 hours)?0 found; left out of hours.Pass
Q16Are City Council committees kept apart from the full body, so nothing is counted twice?7 committees are their own bodies. A video is never counted under both (see Q10).Pass
Q17Are school district boards kept apart from the city government?One school body, shown under Schools: Oakland Unified School District Board of Education. They are separate governments.Pass
Q18Are body names the city's own?Names are as listed on the city's own video portal.Pass
Time
Q19Do video lengths agree with the audio we transcribed?227 of 229 agree within 10%.Pass
Q20How many meetings have the clerk's item start times?134 meetings, 1068 agenda items.PassTopic time covers only these meetings; meetings without an index add no topic time.
Q21Are any agenda items out of order or zero length?24 items share a start time with another item (called together).PassCalled-together items split their time equally.
Q22Is the meeting really on the item the clerk's index says?A third model read the transcript at 296 random points in the video across SF and Oakland: 276 (93%) were on the marked item (lower bound 0.90). This city: 37 of 39.PassFindings are worded to allow for the rest ("about a quarter", not an exact percent).
Q23Is pre-meeting dead air counted?No: time before the first item is not in any topic.Pass
Q24Is general public comment given a topic?No: it is its own line, not ranked, because speakers raise many topics.Pass
Q25Are items called together credited fairly?24 items were called together; each gets an equal part of that time.Pass
Topics
Q26What share of itemised time has a topic both models agree on?91% of labelled time; 4655 of 5411 unique titles agreed.PassDisagreements are shown as "topic not agreed", never guessed.
Q27How accurate are the topic labels, measured blind?One sample of agenda titles, pooled across San Francisco and Oakland: a third model agreed with 372 of 400 sampled labels (90% lower bound 0.91; bar 0.90). This city's titles: 63 of 65.Pass
Q28Is routine business (roll call, minutes, adjournment) kept out of the ranking?Yes. Routine business and general public comment were 23% of time.Pass
Q29If an item covers two subjects, how is its time split?It isn't: each item gets one primary topic. This is a known simplification.Pass
Q30Does one very long meeting drive the top topic?The largest single meeting is 13% of the top topic's time.Pass
Q31Does one body drive a citywide topic?Policing and public safety: 80% from Public Safety Committee; Budget and spending: 63% from City Council; Development and land use: 42% from City Council.PassEach body has its own cut, so the source is visible.
Q32Are ranks honest about the margin?Ranges come from resampling meetings. A topic's rank counts only the topics clearly above it; 3 of 3 topics share a rank.Pass
Q33Are topic names neutral (no sides)?Yes: subjects only, e.g. "Housing and tenants", never "pro-housing".Pass
Q34Are small or unreliable topics ranked?No. Topics under 10 hours citywide are not ranked. 8 topics failed their own label check and are held: contracts, environment, governance, immigration, streets, taxes, technology, utilities.Pass
Q35Does "rising" compare complete quarters only?City business is seasonal: budgets peak in spring and many bodies skip August. A fair comparison sets a quarter against the same quarter a year before, and our topic record starts in October 2025. Comparing with the other months would call budget season a trend.Withheld
Q36Does each rise clear the margin?City business is seasonal: budgets peak in spring and many bodies skip August. A fair comparison sets a quarter against the same quarter a year before, and our topic record starts in October 2025. Comparing with the other months would call budget season a trend.Withheld
Q37Is every item in the window labelled?100% of itemised time has labels from both models.Pass
Talk
Q38How much of the City Council members' speaking time carries a checked name?807 of 980 minutes (82%). Our bar for a ranking is 60%; under 80% it is preliminary. On a random sample of meeting time, between 64% and 86% of members' speech carries a name.Pass
Q39How is a voice named?Two AI models (Claude Opus and Gemini) each read the whole meeting with that day's roster and must name the same member, citing the line where the member is called on, thanked, or introduces themself; code then confirms the name is in that line. 71 officials named so far.Pass
Q40How accurate are the names, measured blind?One check pooled across five cities, not this city alone. Each counted second of talk time is equally likely to be drawn. Claude Opus and Gemini each read the 40 lines before and 8 after, and a second Opus read sees the same lines with the voice labels hidden. Councils and their committees: of 410 lines drawn, 352 were right, 0 wrong and 1 unclear, and the checkers couldn't judge 57; of the 300 lines we still publish, the checkers could judge 268: all 268 right (90% lower bound 0.99). They couldn't judge the other 32. Commissions: of 340 lines drawn, 260 were right, 4 wrong and 1 unclear, and the checkers couldn't judge 75; of the 276 lines we still publish, the checkers could judge 228: 226 right, 1 wrong and 1 unclear (90% lower bound 0.97). They couldn't judge the other 48. School boards: of 200 lines drawn, 170 were right, 1 wrong and 2 unclear, and the checkers couldn't judge 27; of the 185 lines we still publish, the checkers could judge 162: 160 right, 1 wrong and 1 unclear (90% lower bound 0.96). They couldn't judge the other 23. Voices behind a wrong or unclear line, and voices held by our other checks, were removed before the second count. Lines under 2.5 seconds don't count toward anyone's minutes, and a voice that took the floor after the chair called on a different member is held.Pass
Q41Are merged voices (two people under one label) kept out?Yes, always. Inside merged voices the models agreed with a third reader only 44% of the time, so we never name them; their minutes stay unnamed.Pass
Q42Is talk time fair to members who served part of the 12 months?Yes: it is minutes per meeting held while the member was seated, not a yearly total. One member was seated for only part of the window (meetings held while seated: Noel Gallo, 19 of 20).Pass
Q43Is the chair kept apart from members?Yes: the chair runs the meeting, so the chair is shown on its own row, not ranked.Pass
Q44Is there an official attendance record?No official attendance feed for this body; we use meetings held while seated.n/a
Q45Is anyone ranked on too few meetings?No: a member needs at least 8 meetings with a named voice to be ranked.Pass
Display
Q46Are the people quoted officials, not members of the public?Yes: every quote on an official's page is that official's own remark, and 81 of the 81 clips in our clip file carry an official's id; each is tied to the seat they held, the body and the date.Pass
Q47Did every quote pass the resident-name check?Yes, twice: our resident-name check (cleared list, Oct 14, 2026) and then a blind model read of each quote for any private person's name, first names included. A third read keeps only complete, substantive points; quotes whose meeting we can't match by body and date are held.Pass
Q48Does every quote play at a checked second?Yes: only quotes whose video second has been verified are shown.Pass
Q49Is any member of the public named or pictured?No. Only officials are named; public comment is counted, never named.Pass
Q50Can a reader check every number?Every chart prints its counts and links its meetings, and the Data page (/oakland/data) has CSVs with a codebook for the topic chart, talk time, people, bodies, every meeting, every official's talk time per meeting and this ledger. Issue shares and maps have no CSV yet; their counts are printed beside each chart.Pass
Cite this page
Hamlet, “Accuracy checks: Oakland.” https://cities.myhamlet.com/oakland/checks. Data as of Oct 9, 2026 (version 2ea162c); meetings Oct 24, 2025 – Oct 8, 2026.