Scoring companies is the part of this system that looks like arithmetic and behaves like weather forecasting. Every signal we can collect cheaply is unreliable in a specific, documented way, and the useful response is not to hunt for cleaner inputs but to weight the dirty ones according to how each of them fails. What follows is the weighting scheme we settled on, the reasoning behind each rule, and a scorer we shipped that returned the same number for every company and looked, for about a week, like it was working.
01Every cheap signal lies in its own way
A read is a candidate a source returned. Before it can be qualified it has to clear gates, and gates need evidence. The evidence available to us without a contract or a card on file comes in roughly four kinds, and each kind fails in a way we can name.
- Job posts go stale and stay online. Applicant tracking systems rarely prune, so a role filled in January is often still advertised in June, and a repost is indistinguishable from a first post. Hiring three data engineers is a strong statement about the company that wrote the advert. It is a weak statement about the company today.
- Changelogs lie by omission. A team that ships constantly and writes nothing looks dormant. A team with a marketing-owned changelog looks relentless on four entries a quarter. The absence of a changelog tells you almost nothing about the presence of engineering.
- Funding data lands late. The announcement is not the wire date, the wire is not the close, and the close was usually a quarter before anyone published. By the time a round appears in a public database, the spending decisions it paid for have often already been made.
- Stack detection reports one page view. A tag firing on the marketing site says nothing about the product. A tag left behind by a vendor that churned two years ago fires exactly as convincingly as one belonging to a customer who renewed last week.
None of that is a reason to throw a signal away. It is a reason to be precise about what the signal measures, which is usually not the thing the field name implies. A column called job_posted_at measures when a row was last touched in somebody else's applicant tracking system. Weight it as that and it becomes useful again.
The second-order problem is correlation. Three of those four are downstream of the same act of publishing, so a company with an active marketing team emits more of all of them and a company that ships quietly emits fewer. Adding them up does not give you three views of a business. It gives you one view of a communications budget, counted three times, and it systematically turns down the quiet engineering-led companies that tend to be the best accounts.
02How we weight signals when scoring companies
Four rules, in the order they matter.
- Prefer signals with a timestamp you can trust. A commit date, a release tag, a domain registration record and a directory entry that records when it was added all carry a time nobody had to infer. Anything where the date is a guess gets a fraction of the weight of anything where the date is a fact, and we would rather have one dated signal than four undated ones.
- Decay everything by age, with a half-life per signal class rather than one global constant. A funding round earns weight slowly and holds it for about a year. A job post is worth most in its first three weeks and close to nothing after four months. Running both through the same curve is how a stale job board outranks a fresh one.
- Never let a single signal qualify a company. Nothing reaches qualified on one input, however loud that input is. A company clears the gate when signals from at least two independent families agree, where independent means the two would not both be wrong for the same reason. Two directory listings are one family. A directory listing plus a competitor SDK imported in a public repository are two.
- Treat the absence of a signal as unknown rather than as a negative. We did not detect a scraping SDK is a different claim from this company does not scrape, and most of what we fail to see is a consequence of where we looked. A missing signal comes out of the numerator and the denominator both.
That fourth rule is why a score here is two numbers instead of one. Every company carries a score and a confidence, because a missing signal has to reduce certainty without pretending to be a finding. A company with a thin public surface ends up with a low-confidence score rather than a low score, and the two are handled differently: low confidence sends the company back for more research, low score turns it down with a written reason in the ledger. Collapsing them into a single number quietly rejects every business that does not blog.
03The scorer that gave every company the same answer
For about a week, our scorer returned effectively the same value for every read in the batch. It did not look broken. The queue had an order, the order was stable between runs, and the companies at the top were plausible, because they were the ones the directory source happened to return first and that source skews toward the largest providers. Reviewers worked down the list, approved drafts, and nothing in the experience suggested the score was doing no work at all.
The cause was ordinary. A normalisation step clamped each factor to a ceiling before dividing by the batch maximum, and the ceiling was low enough that nearly every input arrived already sitting on it. Divide a column of near-identical numbers by its own maximum and you get a column of numbers near one. The arithmetic was correct throughout. The output was a constant with rounding noise on the end.
We found it by plotting the distribution, which we had never once done. One spike, nothing on either side of it. A histogram that takes four minutes to produce would have caught this on the first day, and the reason nobody produced it is that the ranking read as reasonable when you looked at it as a list. Lists hide shape. A sorted list of identical values is still a sorted list, and it will keep its order forever.
The wider lesson is about what a plausible artefact actually proves. An ordered queue is evidence that something produced an order. It is not evidence that the thing which produced it was the scorer. Wherever a downstream consumer imposes structure of its own, and a queue imposes order by definition, you can lose the entire upstream computation without the output changing shape. Insertion order, alphabetical order and a broken model all present as a ranked list.
04What we refuse to score on
We do not buy intent data we cannot explain. This costs us reach, and we accept the cost.
The offer is familiar. A vendor sells a feed of accounts said to be surging on topics adjacent to your category, sourced from a publisher panel it will describe only in outline. The signal may well be real. The problem is that it is unfalsifiable at our end: we cannot tell you which page anyone read, on what date, or whether the reader works within three floors of a buying decision.
That breaks two things we depend on. The refusal ledger records a written reason for every company we turn down, and a reason has to be checkable by the person reading it. A third-party feed said no is not a reason, it is a citation of a black box. And the review screen puts each draft beside the evidence it was written from, so a factor that cannot be shown there cannot be argued with by the reviewer whose entire job is to argue with it.
A score you cannot show the reasoning for cannot be argued with, and a score nobody can argue with will be wrong for months before anyone notices. That is the flat scorer again, wearing a better suit. The difference is that we caught the flat scorer with a histogram, and there is no histogram that catches a purchased number whose provenance is a trade secret.
So the weighting stays boring: timestamps we trust, decay by age, two independent families before anything is qualified, and absence recorded as absence. The score orders the queue and does nothing else. It does not open a door, it does not write a draft, and it certainly does not deliver a knock. A human approves every message before it goes anywhere, and the score is one of the things they are allowed to disagree with.