The scoring got smarter and the results still weren't good enough, until I found the actual bug: my own vocabulary, not the code.

Part of my job running operations for the Beiwe Research Platform at Harvard's Onnela Lab is finding money to keep it funded. Beiwe is open source, used by 100+ research collaborators worldwide, and like most academic infrastructure it survives grant cycle to grant cycle. Finding the right grant means checking Grants.gov, the NIH Guide, PCORI, half a dozen foundation sites, on a rolling basis, by hand, and hoping I didn't miss something with a deadline three weeks out. It's exactly the kind of tedious, repetitive search problem I can't leave alone. So I built a tool to do it for me.

What I didn't expect was that the hardest part wouldn't be the scraping. It would be figuring out how to teach a relevance filter to think the way a funder thinks instead of the way I think, and realizing partway through that the two are further apart than they look.

Starting With a Fake

The first version I built was a slick interactive widget: search box, filter buttons for NIH, PCORI, Gates, NSF, DoD, a relevance percentage next to every grant. It looked complete enough that I almost didn't ask the obvious question.

Then I did: is this real data, or mock data?

It was mock data. Eight fabricated grant listings, invented to demonstrate the UI. Nothing wrong with prototyping that way, but it was a good reminder to check before I built a workflow around a demo. So I scrapped the frontend-first approach and rebuilt it as a Python script that hits actual public APIs: Grants.gov's search endpoint, the NIH Guide RSS feed, NSF's solicitation filter, PCORI's current funding cycles, and Massachusetts Life Sciences Center's program list. All free, no subscriptions.

One decision mattered more than it seemed like it should. NIH Reporter and the NSF Awards API were the obvious first choice for "grant data," and I excluded both of them on purpose. They report funded projects, not open solicitations. A tool meant to tell me what I can still apply to is useless if half its results are grants that already closed a year ago. It's a small distinction, awards versus opportunities, but a tool that ignores it just produces a longer list, not a more useful one.

The early debugging was ordinary engineering: an NIH RSS feed returning 406 errors until I set an explicit XML header instead of inheriting a shared JSON one, and a regression I introduced myself when I rewrote the search logic to require every keyword to match instead of any of them, which silently cut my result count by a third. Both fixable in a few minutes once I found them. Neither was the real problem.

Teaching Claude to Think Like a Program Officer

Once the pipeline was pulling real opportunities, I needed a way to sort roughly a hundred listings by whether they were actually worth reading. I built a scoring step where Claude rates every grant 0 to 100 against a written profile of the Beiwe Service Center: what the platform does technically (passive sensor collection, an AWS backend, the Forest analytics library), who funds work like it (NIMH, NIDA, NIA, NINDS), and what award sizes are realistic.

That got better the moment I stopped writing the profile from my own head and started writing it from evidence. I pulled JP Onnela's research profile, the lab's actual funding history, and built the scoring criteria around the mechanisms and institutes that have funded his work before. It's the difference between guessing what a funder wants and reading what they've already paid for.

I ended up writing the whole scoring rubric out as its own document, mission fit, funder alignment, mechanism fit, keyword tiers, award range realism, each with worked examples, so I could hand it to JP directly rather than asking him to trust a black box percentage next to a grant title.

"This R24 infrastructure mechanism directly aligns with BSC's mission to provide shared digital phenotyping research resources… though the NIGMS/ODSS focus is a softer fit compared to NIMH or NIDA which more directly fund BSC's behavioral and psychiatric health domains." — Claude, scoring PAR-23–236

Readable reasoning like that, not just a number, is what made it something I could actually defend in a meeting.

The Scores Went Up. The Problem Didn't Go Away.

After sharpening the rubric, the average relevance score across a run climbed noticeably. I still wasn't finding enough grants that were genuinely worth applying to. My instinct was to blame the scoring again, tune the keywords further, adjust the funder weights.

That instinct was wrong, and figuring out why it was wrong was the most useful part of the whole project.

The scraper was searching using Beiwe's own vocabulary: digital phenotyping, passive sensing, mobile behavioral monitoring. That's the language we use to describe the platform. It is almost never the language a grant solicitation uses to describe itself. A funding call titled "Behavioral Biomarkers in Adolescent Depression" or "Remote Monitoring for Multiple Sclerosis" is often a perfect fit for exactly what Beiwe does, and it will never once use the word phenotyping. Funders write in the language of the disease or the clinical question they're trying to solve. The tool that ends up solving it is an implementation detail from their point of view, not a search term.

I'd built a very good filter for a vocabulary that the people posting grants don't actually use.

The fix wasn't a smarter scoring model. It was going back and rewriting the search terms themselves around disease and method language instead of platform language: schizophrenia and bipolar disorder remote monitoring, substance use and ecological momentary assessment, aging and cognitive decline via smartphone, sleep and actigraphy. Same platform, completely different set of search queries, because the platform was never going to be what the grant was looking for by name.

What Actually Building the Rubric Taught Me

Writing that scoring document turned out to be its own kind of prompt engineering discipline, distinct from the code around it. A relevance score is only useful if two different people would defend the same number for the same grant. That meant writing down, explicitly, what "good fit" means in terms specific enough that a language model, or a colleague, could apply them consistently: which NIH institutes count as strategic versus peripheral, which mechanisms match an infrastructure platform versus a hypothesis-driven study, which keywords are a real signal versus surface-level noise. Every time a score looked wrong, the fix wasn't "adjust the number," it was "find the sentence in the rubric that's ambiguous and make it precise." That's a different skill than writing a scraper, and it's the one that actually generalizes to other places I need Claude to make a judgment call instead of a lookup.

The most recent run pulled sixty opportunities across eleven sources and fifteen distinct funders, with the strongest hit an 88 percent match on PAR-25–136, an NIH mechanism literally titled "Laboratories to Optimize Digital Health." It exists, it's open, and I would never have found it by scrolling Grants.gov manually at 11pm.

The rubric document is still sitting in JP's inbox. What's next is finishing the disease-vocabulary rewrite, adding SAMHSA and the disease-specific foundations (NARSAD, the Michael J. Fox Foundation, the MS Society) that never show up in a federal API, and wiring the results into a Notion database so a new run only ever shows me what's actually new.

Every project like this ends up teaching me the same discipline: write down a judgment call specifically enough that someone else, human or model, makes the same call I would.