Skip to main content

Weekly Sync - 11 August 2026

Attendees

Summary

  • Nirjas PR #77 merged (7 August): pyproject / poetry, ruff, pyright.
  • Opened Nirjas PR #82 (5 August): hermetic test suite — committed per-language hazard fixtures + 78 vendored real-world files, golden extractor/source assertions, provenance manifest. Goldens exposed a quoted-literal false positive in Swift and HTML (/* … */ / <!-- … --> inside strings counted as comments and corrupted *Source() output).
  • Rebased PR #78 onto #82. Merge order is now #82 → #78. Kaushl asked to rebase #78 after #76/#77; we asked to land #82 first because master tests were failing and the rebase cost is smaller that way.
  • Published rycerzes/atarashi-dataset (9 August): 25,119 / 1,729 / 1,643 rows, 3,015 licenses, leak-free splits (0 cross-split overlap).
  • Minerva PR #6 updates (9–10 August): split hygiene moved into the exporter, eval suites packaged (src/evaluation/), clean Atarashi build promoted to output/atarashi.

Notes

  • Packaged Atarashi eval (non-circular, regime-split): the-stack-smol grew from 445 → 2,985 labelled queries; 72% are tag-only, where UNKNOWN is the right answer. SPDX-tag eval: R@1 0.605, macro 0.645. Lexical cascade: 59% coverage at 0.924 precision (tau=0.30).
  • Important caveat on the Hub dataset: ~86% of fragments are verbatim substrings of the reference texts they would be scored against. Recall@k on that test split measures copy-detection, not identification. Do not report it as an Atarashi result.
  • #82 review (11 August): Kaushl asked about Tree-Sitter quoted-parent handling, a fallback when an unaudited grammar construct appears, dropping per-file .ABOUT in favour of one provenance manifest, and optional timing. Follow-up commits address the warning/fallback and provenance; timing left for #78.

Action Items

  • Finish #82 review comments and get it merged, then #78.
  • Treat the 3,015-class classifier track as a dead end for production Atarashi; next eval set needs labels that no matcher produced.