San Francisco · read through Oct 8, 2026
Accuracy ledger
The 50 questions we ask of this page’s numbers before it ships, the answers, and what the page does because of them. The ledger and every chart’s numbers are CSV files on the Data page, with a codebook.46Pass
1Fail
2Held
1n/a
Coverage
Q01How many of the city's public bodies do we follow?51 bodies have meetings in the window. The city's own list we hold has 109 active bodies.PassBodies with no meetings in the window are not shown.
Q02Is the newest Board of Supervisors meeting on file recent?Newest Board of Supervisors meeting on file: Oct 20, 2026; newest one read: Oct 6, 2026.Pass
Q03Is any body missing its recent meetings (newest on file is much older than its usual gap)?Four bodies look behind: Budget and Appropriations Committee, San Francisco County Transportation Authority Board, Urban Forestry Council, SFUSD Board Committees.FailThese bodies' charts cover less of the window; their newest date is shown in the commissions table.
Q04Is the read-through date the date every chart is complete through, not just the newest video?Read through Oct 8, 2026: the newest meeting we have a transcript of. 32 meetings on file after it, to Oct 23, 2026, are listed on their own page as not read yet; talk time, issues, quotes and counts of meetings read leave them out, and a count that includes them says "on file". The topic chart is built from the clerk's agenda times, which cover meetings we haven't read too, so it runs to Oct 23, 2026 and says so.Pass
Q05What share of meetings have a transcript?763 of 812 (94%).PassTopic time uses the clerk's video index, not transcripts, so it doesn't depend on this.
Q06What share of meetings have a known length?812 of 812 (100%).PassA body's hours show only when 8 in 10 of its meetings have a length.
Q07Any whole months with no Board of Supervisors meeting on file?1 of 11 full months empty: 2026-08. (Councils often skip August.)Pass
Q08Any meetings with no video link?0 of 812.PassCounted as meetings; they add no hours or topic time.
Meetings
Q09Are any meetings listed twice?0 exact duplicate rows found and removed.Pass
Q10Is any video counted under two bodies?0 videos were listed under two bodies; each is counted once.Pass
Q11Are part 1 / part 2 videos of one meeting counted as two meetings?10 meeting days had more than one video. Each counts as one meeting; their lengths add up.Pass
Q12Are meetings outside the window kept out?936 rows outside Oct 24, 2025 – Oct 23, 2026 left out.Pass
Q13Where do meeting dates come from?From the city clerk's own video portal listing for each meeting, not from our transcripts.Pass
Q14Are very short videos (cancelled, no quorum, test streams) counted?8 videos under 10 minutes: counted as meetings, left out of hours.Pass
Q15Any impossible lengths (over 12 hours)?0 found; left out of hours.Pass
Q16Are Board of Supervisors committees kept apart from the full body, so nothing is counted twice?10 committees are their own bodies. A video is never counted under both (see Q10).Pass
Q17Are school district boards kept apart from the city government?One school body, shown under Schools: Board of Education (SFUSD). They are separate governments.Pass
Q18Are body names the city's own?Names are as listed on the city's own video portal.Pass
Time
Q19Do video lengths agree with the audio we transcribed?751 of 763 agree within 10%.Pass
Q20How many meetings have the clerk's item start times?692 meetings, 8811 agenda items.PassTopic time covers only these meetings; meetings without an index add no topic time.
Q21Are any agenda items out of order or zero length?172 items share a start time with another item (called together).PassCalled-together items split their time equally.
Q22Is the meeting really on the item the clerk's index says?A third model read the transcript at 296 random points in the video across SF and Oakland: 276 (93%) were on the marked item (lower bound 0.90). This city: 239 of 257.PassFindings are worded to allow for the rest ("about a quarter", not an exact percent).
Q23Is pre-meeting dead air counted?No: time before the first item is not in any topic.Pass
Q24Is general public comment given a topic?No: it is its own line, not ranked, because speakers raise many topics.Pass
Q25The Board of Supervisors' commendations: is all that time really commendations?A third model read the transcript every 5 minutes through every commendation segment: 370 of 398 checks (93%) were commendations, ceremonies or memorials; the rest were mostly introductions or public comment the clerk hadn't marked yet.PassWe say "about a quarter" of the board's indexed time, which holds after taking the unmarked minutes out.
Topics
Q26What share of itemised time has a topic both models agree on?87% of labelled time; 4655 of 5411 unique titles agreed.PassDisagreements are shown as "topic not agreed", never guessed.
Q27How accurate are the topic labels, measured blind?One sample of agenda titles, pooled across San Francisco and Oakland: a third model agreed with 372 of 400 sampled labels (90% lower bound 0.91; bar 0.90). This city's titles: 309 of 335.Pass
Q28Is routine business (roll call, minutes, adjournment) kept out of the ranking?Yes. Routine business and general public comment were 24% of time.Pass
Q29If an item covers two subjects, how is its time split?It isn't: each item gets one primary topic. This is a known simplification.Pass
Q30Does one very long meeting drive the top topic?The largest single meeting is 3% of the top topic's time.Pass
Q31Does one body drive a citywide topic?Development and land use: 36% from Board of Appeals; Budget and spending: 15% from Budget and Appropriations Committee; Policing and public safety: 46% from Police Commission.PassEach body has its own cut, so the source is visible.
Q32Are ranks honest about the margin?Ranges come from resampling meetings. A topic's rank counts only the topics clearly above it; 17 of 17 topics share a rank.Pass
Q33Are topic names neutral (no sides)?Yes: subjects only, e.g. "Housing and tenants", never "pro-housing".Pass
Q34Are small or unreliable topics ranked?No. Topics under 10 hours citywide are not ranked. 8 topics failed their own label check and are held: contracts, environment, governance, immigration, streets, taxes, technology, utilities.Pass
Q35Does "rising" compare complete quarters only?City business is seasonal: budgets peak in spring and many bodies skip August. A fair comparison sets a quarter against the same quarter a year before, and our topic record starts in October 2025. Comparing with the other months would call budget season a trend.Held
Q36Does each rise clear the margin?City business is seasonal: budgets peak in spring and many bodies skip August. A fair comparison sets a quarter against the same quarter a year before, and our topic record starts in October 2025. Comparing with the other months would call budget season a trend.Held
Q37Is every item in the window labelled?100% of itemised time has labels from both models.Pass
Talk
Q38How much of the Board of Supervisors members' speaking time carries a checked name?2,754 of 2,891 minutes (95%). Our bar for a ranking is 60%; under 80% it is an early read. On a random sample of meeting time, between 80% and 93% of members' speech carries a name.Pass
Q39How is a voice named?Two AI models (Claude Opus and Gemini) each read the whole meeting with that day's roster and must name the same member, citing the line where the member is called on, thanked, or introduces themself; code then confirms the name is in that line. 265 officials named so far.Pass
Q40How accurate are the names, measured blind?One check pooled across five cities, not this city alone. Each counted second of named speech is equally likely to be drawn. Claude Opus and Gemini each read the 40 lines before and 8 after, and a second Opus read sees the same lines with the voice labels hidden. Councils and their committees: of 410 lines drawn, 352 were right, 0 wrong and 1 unclear, and the checkers couldn't judge 57; of the 300 lines we still publish, the checkers could judge 268: all 268 right (90% lower bound 0.99). They couldn't judge the other 32. Commissions: of 340 lines drawn, 260 were right, 4 wrong and 1 unclear, and the checkers couldn't judge 75; of the 276 lines we still publish, the checkers could judge 228: 226 right, 1 wrong and 1 unclear (90% lower bound 0.97). They couldn't judge the other 48. School boards: of 200 lines drawn, 170 were right, 1 wrong and 2 unclear, and the checkers couldn't judge 27; of the 185 lines we still publish, the checkers could judge 162: 160 right, 1 wrong and 1 unclear (90% lower bound 0.96). They couldn't judge the other 23. Voices behind a wrong or unclear line, and voices held by our other checks, were removed before the second count. Lines under 2.5 seconds don't count toward anyone's minutes, and a voice that took the floor after the chair called on a different member is held.Pass
Q41Are merged voices (two people under one label) kept out?Yes, always. Inside merged voices the models agreed with a third reader only 44% of the time, so we never name them; their minutes stay unnamed.Pass
Q42Is talk time fair to members who served part of the 12 months?Yes: it is minutes per meeting held while the member was seated, not a yearly total. One member was seated for only part of the window (meetings held while seated: Alan Wong, 34 of 37).Pass
Q43Is the chair kept apart from members?Yes: the chair runs the meeting, so the chair is shown on its own row, not ranked.Pass
Q44Is there an official attendance record?No official attendance feed for this body; we use meetings held while seated.n/a
Q45Is anyone ranked on too few meetings?No: a member needs at least 8 meetings with a named voice to be ranked.Pass
Display
Q46Are the people quoted officials, not members of the public?Yes: every quote on an official's page is that official's own remark, and 291 of the 291 clips in our clip file carry an official's id; each is tied to the seat they held, the body and the date.Pass
Q47Did every quote pass the resident-name check?Yes, twice: our resident-name check (cleared list, Oct 14, 2026) and then a blind model read of each quote for any private person's name, first names included. A third read keeps only complete, substantive points; quotes whose meeting we can't match by body and date are held.Pass
Q48Does every quote play at a checked second?Yes: only quotes whose video second has been verified are shown.Pass
Q49Is any member of the public named or pictured?No. Only officials are named; public comment is counted, never named.Pass
Q50Can a reader check every number?Every chart prints its counts and links its meetings, and the Data page (/sf/data) has CSVs with a codebook for the topic chart, talk time, people, bodies, every meeting, every official's named speech per meeting and this ledger. Issue shares and maps have no CSV yet; their counts are printed beside each chart.Pass
Cite this page
Hamlet, “Accuracy ledger: San Francisco.” https://cities.myhamlet.com/sf/checks. Data as of Oct 9, 2026 (version 9637340); meetings Oct 24, 2025 – Oct 8, 2026.