The graph and its sources¶
The graph builder is the first stage of a run. It queries the biomedical
sources for one disease, merges duplicate entities, records where every
edge came from, and returns a DiseaseGraph.
What matters here is not the number of sources but which attributes
each one is allowed to set. The causal engine reads only edge_type
and evidence_class, so an edge's source decides whether it ever becomes
an arrow — see Which edges earn an arrow.
The sources¶
| Source | Supplies | Access | Causal-admissible edges? |
|---|---|---|---|
| Monarch Initiative | Gene–disease associations from OMIM, ClinGen, Orphanet, HPO | REST v3 | Only where the evidence class is genetic |
| Open Targets | Target–disease evidence, per-datatype scores, tractability | GraphQL v4 | Only genetic_association / somatic_mutation |
| KEGG | Pathway membership | REST | No — participates_in is not directed |
| Reactome | Hierarchical pathway membership | REST | No |
| STRING | Protein–protein interactions, 0–1000 confidence | REST, batched | No — PPIs are undirected |
| OmniPath | Signed, directed regulatory interactions | REST | Yes — the only source of gene→gene arrows |
| UniProt | Protein function, PDB cross-references, location | REST | Enrichment only |
| RCSB PDB | 3D structures for docking | Search v2 + Data API | Enrichment only |
| ChEMBL | Drug–target relationships, incl. pathogen targets | Local SQLite (chembl_36.db, ~28 GB) |
No |
OmniPath is the load-bearing one for causal inference, and it is worth being clear why. STRING has far more interactions, but they are undirected: an arrow drawn from a STRING row records which protein appeared in the first column, not which acts on which. OmniPath aggregates SIGNOR, TRRUST, PhosphoSite and around a hundred other resources and resolves direction and sign by consensus across them.
Aggregation does not cost provenance: each OmniPath interaction carries the primary databases that assert it and the PubMed IDs behind them, both recorded on the edge. So multi-source corroboration counts distinct primary evidence rather than counting OmniPath twice for an interaction STRING also reported.
ChEMBL is the only source that can surface a non-human target —
Plasmodium DHFR-TS, HIV-1 protease — which become PATHOGEN_GENE
nodes rather than merging into the human gene population.
Building one¶
Requires network
This block queries every live source, so it is not executed as a doctest.
from neorx import build_disease_graph
graph = build_disease_graph(
"HIV",
max_genes=20, # cap on genes carried forward
string_min_score=400, # STRING combined-score threshold, 0-1000
use_cache=True,
allow_mocks=False,
)
print(graph.n_genes, graph.n_proteins, graph.n_pathways)
print(graph.sources_queried)
allow_mocks=False is the default and it matters: with mocks disabled, a
failed API call produces an empty result rather than curated stand-in
data. Mock fallbacks exist for offline development and tests, and turning
them on in anger would mean a reported number resting on hand-written
data.
Node merging¶
Monarch calls it CCR5; Open Targets calls it ENSG00000160791. Nodes
are normalised to gene symbols and merged, keeping the highest confidence
score and the union of metadata, with every contributing source recorded
in the node's source field — which is what the multi-source consensus
term in causal confidence counts.
The data model¶
A DiseaseGraph is Pydantic, so it serialises losslessly and validates
on construction. Edges carry more than a weight:
>>> from neorx import GraphEdge, EdgeType
>>> edge = GraphEdge(
... source_id="CCL5", target_id="CCR5",
... edge_type=EdgeType.REGULATES,
... weight=0.7,
... source_db="OmniPath",
... evidence_class="regulatory",
... sign=1,
... primary_sources=["SIGNOR", "TRRUST"],
... pmids=["12345678"],
... )
>>> edge.edge_type.value
'regulates'
>>> edge.primary_sources
['SIGNOR', 'TRRUST']
sign records activation versus inhibition, primary_sources the
databases behind an aggregated edge, evidence_class the attribute
admissibility turns on.
The twelve edge types and eight node types:
>>> from neorx import EdgeType, NodeType
>>> [e.value for e in EdgeType]
['activates', 'inhibits', 'phosphorylates', 'binds', 'regulates',
'participates_in', 'associated_with', 'causes', 'treats',
'interacts_with', 'upregulates', 'downregulates']
>>> [n.value for n in NodeType]
['gene', 'protein', 'pathway', 'phenotype', 'disease', 'drug',
'metabolite', 'pathogen_gene']
To NetworkX and back¶
The causal engine works on a nx.DiGraph. The conversion is explicit:
>>> from neorx import (DiseaseGraph, GraphNode, NodeType,
... disease_graph_to_networkx)
>>> graph = DiseaseGraph(
... disease_name="HIV",
... nodes=[
... GraphNode(node_id="CCR5", name="CCR5", node_type=NodeType.GENE),
... GraphNode(node_id="MONDO:0005109", name="HIV",
... node_type=NodeType.DISEASE),
... ],
... edges=[edge.model_copy(update={"source_id": "CCR5",
... "target_id": "MONDO:0005109"})],
... )
>>> G = disease_graph_to_networkx(graph)
>>> sorted(G.nodes)
['CCR5', 'MONDO:0005109']
>>> G.edges["CCR5", "MONDO:0005109"]["evidence_class"]
'regulatory'
Edge attributes survive the conversion, which is the whole point —
is_causal_admissible reads them off the NetworkX graph.
Persistence and export¶
>>> import tempfile
>>> from neorx import save_graph, load_graph
>>> with tempfile.TemporaryDirectory() as tmp:
... path = save_graph(graph, output_dir=tmp, fmt="json")
... reloaded = load_graph(path)
... for fmt in ("graphml", "gexf", "cytoscape"):
... _ = save_graph(graph, output_dir=tmp, fmt=fmt)
>>> reloaded.disease_name
'HIV'
>>> len(reloaded.nodes)
2
Four formats: json (lossless, round-trips through load_graph),
graphml, gexf for Gephi, and cytoscape for Cytoscape. The default
directory is ~/.neorx/graphs/, and filenames carry a UTC timestamp.
With DATABASE_URL set, graphs and jobs are also written to PostgreSQL —
save_graph_to_db, load_graph_from_db.
Caching¶
Every HTTP client goes through a cache layer, backed by JSON files under
~/.neorx/cache/ by default or by Redis when
NEORX_CACHE_BACKEND=redis. API responses are held for 24 hours and
assembled graphs for 7 days. Pass use_cache=False, or --no-cache on
the CLI, to bypass it.
All HTTP traffic in neorx.core.sources uses requests, which is what
lets the experiment runner freeze a run's traffic into
a cassette for replay. A source added with a different HTTP client
would silently escape that recording.
Dated builds¶
A build can be pinned to a past release, so that a run reads what that date would have read rather than what the sources say today:
from neorx import build_disease_graph
from neorx.snapshots.reader import SNAPSHOTS_DIR, SnapshotStore
from neorx.snapshots.releases import release_for
from neorx.snapshots.resolver import SourceResolver
store = SnapshotStore(SNAPSHOTS_DIR, on_read=None)
# An empty `live` mapping on purpose: a dated build must not be able to
# reach a live client for a pinned source even by accident.
resolver = SourceResolver(store=store, release_for=release_for, live={})
graph = build_disease_graph(
"HIV infection",
as_of="2021-11",
resolver=resolver,
disease_id="EFO_0000764",
)
as_of is a key in the pinned release table — 2018-06, 2021-11,
2025-06 — not a free-form date, and an unknown one raises rather than
being rounded to something nearby:
>>> from neorx.snapshots.releases import TIME_POINTS, release_for
>>> sorted(TIME_POINTS)
['2018-06', '2021-11', '2025-06']
>>> release_for("opentargets", "2021-11")
'21.11'
>>> release_for("omnipath", "2021-11")
'20211113-20220114'
as_of requires both resolver and disease_id, and refuses
allow_mocks outright — curated mock data is a measurement of no
release, and a dated build would carry it as a node score for that
date.
A dated score and a live score are different quantities
On a live build, a gene node's score is Open Targets' overall
association score — a harmonic sum over datasources that the platform
publishes. The derived extract does not carry that number and it is
not recoverable from the per-datatype rows; reconstructing it would
mean inventing a quantity and presenting it as a measurement.
So a dated node carries the maximum genetic-evidence score in the
pinned release instead. Both are real Open Targets numbers. They are
not the same number, and they must not be placed in the same column,
ranked against each other, or pooled. DiseaseGraph.as_of is how a
consumer tells which it is holding. The full per-datatype breakdown
is on both, in metadata["datatype_scores"].
Unpinned sources stay live on a dated build, because their edges are not
causal-admissible and identification filters them out. Their nodes are
restricted to the pinned release's own gene population: they may enrich a
node the release put in the graph, never introduce one. What they offered
and did not get to contribute is recorded in
DiseaseGraph.frame_exclusions, with a reason per entry, because a
silent filter is how frame leakage returns after a refactor.
See Versioned snapshots for how the extracts are built.