Finding missing links by shared neighbours lost to simple popularity on three corpora

Tools that promise to find undiscovered connections often rank unlinked pairs of concepts by the neighbours they share: if A goes with B and B goes with C, perhaps A goes with C. We tested that ranking against the plainest baseline there is, which pairs the most popular concepts. Each corpus was cut by date, and a ranker scored a hit when a pair it proposed from the early documents appeared together at least twice in the next stretch of documents. Ranking unlinked pairs by shared neighbours found fewer later links than ranking them by popularity on all three corpora: precision in the top 200 of 0.127 against 0.268 on our research notes, 0.018 against 0.088 on Biology Stack Exchange and 0.007 against 0.043 on Skeptics Stack Exchange.c1 On degree-preserving rewired graphs the shared-neighbour score reached a 95th percentile of 0.164, above its real score of 0.127, so at the top of the list it added nothing we could detect beyond popularity.c2 Among pairs of similar popularity the shared-neighbour score still separated later links from the rest: area under the curve 0.569 on the research notes, 0.757 on Biology and 0.696 on Skeptics.c3 Three of four preregistered verdicts failed.

Swanson [1] described knowledge that is public and still undiscovered: two bodies of literature that each hold half of an argument and never cite each other. His method, now called literature-based discoveryDefinition: literature-based discovery information science, after Swanson The search for connections that no single document states but that follow from linking what separate documents say. Excludes finding links that some document already states, which is ordinary retrieval. full definition, looks for a term C that is linked to a term B in one literature while B is linked to A in another, so that A and C become a candidate pair. Later work recast the problem as link predictionDefinition: link prediction network science Ranking pairs of nodes that are not yet joined in a network by how likely they are to become joined later. Excludes recovering links that exist but were hidden at random from the same snapshot. full definition, the ranking of node pairs in a network by how likely they are to become joined. Liben-Nowell and Kleinberg [2] set out the problem on coauthorship networks, built rankers from measures of how close two nodes sit in the network, and found that subtle proximity measures can outperform direct ones. One such measure is the Adamic-Adar index [3]. Kastrin, Rindflesch and Hristovski [4] applied proximity measures, among them common neighbours, the Adamic-Adar index and preferential attachment, to a network of co-occurring MeSH terms with its history of link formation, and reported that supervised methods outperform the unsupervised ones.

Krenn and Zeilinger [5] built a semantic network of quantum physics concepts from 750,000 papers, linked two concepts when one article studies both, and trained a neural network to predict which concept pairs researchers will study in the next five years. Krenn and colleagues [6] later built a benchmark, Science4Cast, from about 143,000 papers on artificial intelligence and compared ten prediction methods on it; in the authors' public summary, the most effective methods used a curated set of network features.

This study asks a narrower question that sits under all of that work. Before a gap ranking is trusted to point at undiscovered connections, does it beat a ranking that knows only how popular each concept is? The shared-neighbour score in our test discounts popular concepts on purpose, in the form used for time-sliced tests by Krenn and Zeilinger. The baseline is preferential attachmentDefinition: preferential attachment network science A ranking of unlinked pairs by the product of the two concepts' degrees, so the most popular pairs come first. Excludes any use of which neighbours the two concepts share. full definition, the product of the two concepts' degrees.

The design was written and hash-locked on 31 October 2026 as a preregistrationDefinition: preregistration research method (metascience) A dated written plan for a study, fixed and hash-locked before any result is seen: the question, the data, the statistic, the controls, the forecasts and the kill conditions. Excludes a plan revised after looking at the outcome, and a protocol written only for the record after the run. full definition, before any score of the study was computed. Exploration before the lock read schemas, field names and date ranges only. The locked file names the corpora, the slicing, the vocabulary rules, the definition of a link and of a target, the ranker and its baseline, both controlsDefinition: control statistics and experimental design A version of the data with the structure under test removed (for example shuffled or pooled), scored the same way as the real data, so the real score has something to beat. Excludes a comparison with a different dataset or with a published number. full definition, the measures, four verdicts, the forecasts and one kill condition. The preregistration's SHA-256 is given in the reproduction section.

Each verdictDefinition: verdict research method (our usage) The pass or fail of one preregistered rule set against the run's numbers, read off by code. Excludes an interpretation written after reading the numbers. full definition is a rule read off by code:

The kill conditionDefinition: kill condition research method (our usage, after Popper's falsification) A rule written in the preregistration that, when met, decides against the claim with no room for judgement after the run. Excludes reasons found after the run for setting a result aside. full definition reads: if the ranking fails to beat popularity and also fails with popularity fixed, absent-link scores say nothing beyond how common a word is, and its top gaps get reported only as prompts for reading.

Three corpora were used. The first is our own research notes: entries of our findings ledgerDefinition: findings ledger our records The append-only, dated list of every result we have recorded, numbered F1, F2 and on. Corrections are added as new dated lines and old entries are never rewritten. Excludes a curated list of successes. full definition, the titles of saved readings, a catalogue of laws and a table of checks, each dated. They are internal records and are not public, so no one outside can reproduce that arm. Their dates span days of reading, not years, so a slice of them is a stretch of attention, not a stretch of a field's history.

The other two are the question posts of Biology Stack Exchange (28,798 questions) and Skeptics Stack Exchange (11,202 questions), 2011 to 2024, taken from the sites' public data dump. Each question's tags serve as its concepts. Stack Exchange content is licensed CC BY-SA [7]; this paper reports statistics only.

Each corpus was ordered by date and cut into five time slicesDefinition: time slice forecast evaluation One of five blocks of equal document count cut from a corpus ordered by date. A fold trains on the early slices and tests on the next one. Excludes random splits, which let later documents inform the training graph. full definition of equal document count. The fifth slice was held back for a later study and was not read here. A vocabulary was built from the training slices only: at most 1,500 concepts, each appearing in at least four documents and in no more than 10 percent of them, with each document keeping at most 40 concepts by tf-idf weight. On the research notes the concepts are single words and word pairs; on the Stack Exchange sites they are tags.

A linkDefinition: link our usage for this study Two concepts appearing in the same document. A concept here is a question tag on the Stack Exchange sites and a word or word pair on the research notes. Excludes any judgement that the two concepts are related in fact. full definition is two concepts in one document. A targetDefinition: target our usage for this study A pair of concepts never linked in the training slices that appears together in at least two documents of the next slice. Excludes pairs that appear together only once in the test slice, and any check that the later pairing is true. full definition is a pair with no link in the training slices that appears together in at least two documents of the test slice.

Three folds trained on slice 1, slices 1 to 2 and slices 1 to 3, and each tested on the slice after its training block. Every number in the results is the mean over the three folds unless a fold is named.

Let $A$ be the training graph, $d_i$ the degreeDefinition: degree network science The number of other concepts a concept is linked to in the training graph, our measure of how popular it is. Excludes how many documents mention the concept. full definition of concept $i$, and $E$ the number of edges. Each edge between $i$ and $k$ gets a weight that falls as both ends become popular, floored at zero:

The A-B-C rankingDefinition: A-B-C ranking literature-based discovery, after Swanson Scoring an unlinked pair A and C by the intermediate concepts B that each is linked to. Here a cosine over shared neighbours with each edge weighted down when both of its ends are popular. Excludes methods trained on known outcomes, and the expert reading that Swanson applied to each candidate. full definition scores an unlinked pair $i$ and $j$ by the cosine of their weighted neighbour rows, so that it is high when the two concepts share specific neighbours:

The popularity baseline is

The Adamic-Adar indexDefinition: Adamic-Adar index network science A count of the neighbours two nodes share in which each shared neighbour is weighted by one over the logarithm of its own degree. Excludes the popularity discount on edges that the A-B-C score uses. full definition was reported beside them as a secondary measure. It was not in the preregistration, and its result is read as a lead only:

where the sum runs over the neighbours $k$ that $i$ and $j$ share. Ties in any score are broken by a fixed small noise term, the same in every run.

The main measure is precision in the top 200Definition: precision in the top 200 information retrieval The share of a ranker's 200 highest-scored unlinked pairs that turn out to be targets. Excludes how the ranker orders pairs below the top 200. full definition:

where $h$ is the number of targets among a ranker's 200 highest-scored unlinked pairs. It is set beside the base rateDefinition: base rate statistics The share of all unlinked candidate pairs that are targets, the precision a random ranking would reach on average. Excludes the precision of any particular ranker. full definition, the share of all unlinked candidate pairs that are targets.

The interval for the difference in precision comes from a document bootstrapDefinition: document bootstrap statistics (resampling) Redrawing the test slice's documents with replacement, recomputing which pairs are targets, and rescoring the fixed top-200 lists each time, so the interval shows how much the result depends on which documents happened to arrive. Excludes uncertainty from the training graph, which is held fixed. full definition: 200 times, the test slice's documents are reweighted by a multinomial draw, the targets are recomputed, and both fixed top-200 lists are rescored. The difference is averaged over the three folds within each draw, and the 2.5th and 97.5th percentiles of the 200 averages form the 95 percent intervalDefinition: 95% interval statistics The range that holds the middle 95 percent of the bootstrap or control draws for a number. We report it beside the number. Excludes a statement that the true value lies inside with 95 percent probability. full definition.

The second measure holds popularity fixed. Candidate pairs are split into tenths by the product of their degrees, and the area under the ROC curveDefinition: area under the ROC curve statistics and machine learning The chance that a randomly chosen target scores above a randomly chosen non-target. 0.5 is chance and 1 is perfect. Excludes how the top of the list performs, which precision in the top 200 measures. full definition is computed inside each tenth from 40,000 sampled pairs of a target and a non-target. The within-decile AUCDefinition: within-decile AUC our usage for this study The area under the ROC curve computed separately inside each tenth of the candidate pairs ranked by degree product, then averaged with weights equal to each tenth's number of targets. Excludes any advantage that comes from ranking popular pairs above unpopular ones. full definition is their weighted mean:

where $A_b$ is the area in tenth $b$ and $n_b$ its number of targets.

Two controls were run on the research notes, 20 draws each, scored against the true targets of each fold. Degree-preserving rewiringDefinition: degree-preserving rewiring network science, after Maslov and Sneppen Repeatedly swapping the ends of two randomly chosen edges, so every concept keeps its degree while who is linked to whom is scrambled. Excludes changes to any concept's popularity. full definition attempts two random double-edge swaps per edge on the training graph, skipping any swap that would duplicate an edge. Every concept keeps its popularity and loses its real neighbourhood. If the shared-neighbour score finds anything beyond popularity at the top of the list, it must beat this control. Node-label permutationDefinition: node-label permutation statistics (permutation test) Shuffling which concept sits at which node of the training graph, so the graph's shape stays and its link to the real concepts is broken. Excludes any change to the shape of the graph. full definition keeps the graph's shape and breaks its tie to the real concepts. It is the weaker control: a score that loses to it would be worse than noise. The 95th percentile of each control's null distributionDefinition: null distribution statistics The spread of scores a control gives over many random draws. Its 95th percentile is the bar a real score must clear. Excludes a theoretical distribution assumed rather than drawn from the data. full definition is the bar.

There was no planted check. No corpus with a known set of future links was built, so the study cannot say how often the instrument misses a real effect.

Table 1 gives the main measure on each corpus and Figure 1 draws it. Ranking unlinked pairs by shared neighbours found fewer later links than ranking them by popularity on all three corpora: precision in the top 200 of 0.127 against 0.268 on our research notes, 0.018 against 0.088 on Biology Stack Exchange and 0.007 against 0.043 on Skeptics Stack Exchange.c1 All three bootstrap intervals for the difference lie below zero.

Figure 2 sets the research-notes score against its controls. On degree-preserving rewired graphs the shared-neighbour score reached a 95th percentile of 0.164, above its real score of 0.127, so at the top of the list it added nothing we could detect beyond popularity.c2 The rewired draws had a mean of 0.146, and 19 of 20 met or beat the real score (p value from drawsDefinition: p value (from draws) statistics The share of control draws scoring at or above the real score, with one added to the count and to the number of draws, so it is never zero. Excludes a p value from a textbook formula that assumes a distribution. full definition 0.95). The score did beat the node-permuted graphs, whose 95th percentile was 0.048 (p 0.048, the smallest value 20 draws allow).

Among pairs of similar popularity the shared-neighbour score still separated later links from the rest: area under the curve 0.569 on the research notes, 0.757 on Biology and 0.696 on Skeptics.c3 Over all candidate pairs, without holding popularity fixed, its area was 0.739 on the research notes against 0.782 for popularity, 0.810 against 0.815 on Biology, and 0.762 against 0.836 on Skeptics.

Adamic-Adar, not preregistered, matched or beat popularity: 0.273 against 0.268 on the research notes and 0.122 against 0.088 on Biology.c4 No interval was computed for that comparison.

The folds explain much of the research-notes result. Table 2 shows that the first fold, with 285 concepts and a base rate of 0.094, holds almost all of the hits. The later folds have 1,500 concepts and base rates near 0.001, and the shared-neighbour score found no target in its top 200 in either of them.

Table 3 sets the forecasts written before the run against the outcomes.

Kastrin and colleagues [4] carry preferential attachment among their unsupervised measures, and our test adds a narrow fact to that line of work: a popularity-discounted shared-neighbour score falls below it at the top of the list on these three corpora. The rewired control sharpens that. A graph that keeps every concept's popularity and scrambles its neighbours gives the shared-neighbour score a higher top-200 precision than the real graph does. On the research notes, then, the neighbourhoods the score reads add nothing at the top that popularity alone does not carry.

The within-decile result points the other way, and both can hold at once. Inside a band of similar popularity, shared neighbours do separate future pairs from the rest. The discount in eq. 1 pushes popular pairs down the list, and popular pairs are where most of the later links are. A gap tool built on shared neighbours may therefore be useful as a re-ranker within a popularity band, which this study did not test.

The Adamic-Adar result is the lead worth a new test. It weights shared neighbours by their rarity without discounting the pair itself, and on two corpora it met or passed the popularity baseline. That was not preregistered and carries no interval here.

Krenn and Zeilinger [5] and Krenn and colleagues [6] predict concept pairs with richer features and learned models. Our result does not bear on those methods. It bears on any tool that shows shared-neighbour gaps to a reader as candidate discoveries without first showing how a popularity list would do.