# Method

Licensed CC BY 4.0.

## The firms

Every investor permalink in the FundingWatcher startup file was ranked by the
number of portfolio companies with a usable domain. The top 40 whose Crunchbase
`types` contain none of accelerator, incubator, government_office,
entrepreneurship_program or university_program form the set. Nothing else was
excluded. Corporate venture arms (Intel Capital, Google Ventures, Salesforce
Ventures), investment banks (Goldman Sachs, Silicon Valley Bank), an angel group
(Alumni Ventures), a syndicate (Gaingels) and one individual (Edward Lando) all
clear that rule and are in. Every firm in the set has at least 150 portfolio
companies with a vector; the smallest is 191.

Four permalinks above the cut have no row in the investor file, so their `types`
could not be tested and they were dropped: sequoia-capital-china,
google-for-entrepreneurs, easme and tencent.

## The measure

Each company is one 1,024-dimension embedding of a standardised description of
what it sells, who buys it, how it is delivered, what market it sits in and what
it charges for, written from the company's current homepage. Similarity is
cosine between unit-normalised vectors. A pair is two portfolio companies of the
same firm scoring at or above the threshold. pairs.csv stores scores to six
decimal places and the threshold is applied at full precision. 0.80 is roughly "same product
category"; 0.85 is "a reader would call these competitors".

## The control

A firm with 500 portfolio companies has 124,750 possible pairs and will hold
some close ones by arithmetic alone. So every firm is compared against itself at
random: 50 draws of the same size as its vectored portfolio, taken from the
87,210 funded startups in the file that have a vector, counted at the same
thresholds and cleaned by the same rules. `expected` is the mean of those 50
draws. Draws use numpy default_rng with seed 17.

## Cleaning

Applied to portfolio pairs and to the random draws alike, before any count.

- Same entity under two names: names identical after lowercasing and stripping
  trailing domain words and legal or filler tokens.
- Same entity under two domains: identical first domain label, or one name and
  one domain label each a prefix of the other.
- One company acquired the other, from the acquirer field in Crunchbase.
- Both companies acquired by the same acquirer.

42 pairs were dropped at 0.80, 31 of them at 0.85. Every one is listed in
excluded_pairs.csv with its reason. Company records are keyed by cleaned domain
before pairing, so two rows sharing a domain are one company and cannot form a
pair.

## Funnel

| Step | Rows |
| --- | --- |
| Rows in the startup file | 297,511 |
| Rows with a usable domain | 290,757 |
| Unique company records | 288,215 |
| Rows listing at least one investor | 107,821 |
| Unique funded domains | 107,436 |
| Funded domains with a vector (the random frame) | 87,210 |
| Portfolio rows across the 40 firms | 17,834 |
| Of those, with a vector | 13,823 |
| Pairs at 0.85 before cleaning | 973 |
| Pairs at 0.85 after cleaning | 942 |
| Pairs at 0.80 before cleaning | 3,729 |
| Pairs at 0.80 after cleaning | 3,687 |

## Limits

The vectors describe each company as its website describes it today, not as it
was when the firm invested. Two companies that converged after the fact score
the same as two that were competitors on day one; `flag_year_gap` in pairs.csv
marks the pairs founded more than 8 years apart, where that reading is most
likely. The investor list carries no round, date or lead role, so a small angel
cheque and a lead Series B count alike, and which of the two deals came first
cannot be read from this data. Vector coverage runs from 61% to 94% by firm; the
control is drawn at the same size as the vectored portfolio, so coverage cannot
inflate a ratio.