Efficient long context
A sparse mixture-of-experts backbone routes each nucleotide to only 8 of 576 experts, extending context from 8K to 1M tokens at a fraction of dense-model cost.
Genome foundation model China Agricultural University · BioMap Research
DeepD is a sparse mixture-of-experts genome foundation model that learns from genes and the full intergenic landscape across all domains of life. It combines single-nucleotide resolution with a million-token context and efficient sparse computation, supporting broad sequence-to-function prediction with particular strength in regulatory and non-coding tasks. Beyond experimentally validated promoter design, its Context–Anchor–Body (CAB) strategy enables controllable generation of entire yeast chromosomes and megabase-scale plant genomic segments. These sequences preserve coherent genic regions while introducing variations consistent with natural population patterns.
Genomes are long, heterogeneous and evolutionarily diverse, and most of their sequence is non-coding. DeepD was built to model them as they occur in nature. It pairs efficient sparse computation with pretraining on unselected whole genomes and a new Context–Anchor–Body strategy for controllable long-sequence generation.
DeepD was trained at two scales: DeepD-2B (1.8 billion parameters, 1 trillion tokens) and DeepD-60B (63 billion parameters, 5.69 trillion tokens). The same model supports zero-shot and supervised sequence-to-function prediction, exposes interpretable genomic features through its expert routing, and generates experimentally validated promoters, complete yeast chromosomes and megabase-scale Arabidopsis segments.
A sparse mixture-of-experts backbone routes each nucleotide to only 8 of 576 experts, extending context from 8K to 1M tokens at a fraction of dense-model cost.
Species labels embedded from an established phylogenetic tree condition a species-informed router, so DeepD learns lineage-specific composition, codon usage and coding rules.
Pretraining keeps every genic region and the full intergenic space, including regulatory and repetitive DNA, rather than enriching for genes or sampling short segments.
The Context–Anchor–Body strategy generates each segment between upstream and downstream constraints and relays forward, decoupling output length from the context window.
DeepD integrates DNA sequence, species label embeddings and annotation features in a sparse mixture-of-experts transformer, trained with next-token prediction on byte-level nucleotides.
| DeepD-2B | DeepD-60B | |
|---|---|---|
| Total parameters | 1.8B | 63B |
| Training tokens | 1T | 5.69T |
| Context length | 8K | 8K → 1M |
| Layers / heads | 24 / 16 | 24 / 32 |
| Hidden size | 768 | 2,048 |
| Experts (activated) | 128 (8) | 576 (8) |
Each block mixes 24 sliding-window attention heads for local motifs with 8 full-attention heads for long-range dependencies, and a species-informed router that selects experts per token. Both modules below are interactive.
Hover any head to see the shape of its attention mask.
The DeepD-60B corpus comprises about 5.90 Tb of nucleotide sequence. The eukaryotic partition covers 3,600 animal, 928 plant, 767 fungal and 139 protist genomes. Coding sequence, introns, repeats, other intergenic DNA and viral genomes are all represented, so the model learns genome-wide organization rather than genes alone.
Bar length on a logarithmic scale.
Pretraining runs at an 8K context. Midtraining then extends the window step by step to 1M tokens. Validation perplexity keeps improving as the context grows, and sparse routing keeps the cost of ultra-long sequences manageable.
DeepD's experts organize themselves around biology. Probing them reveals species-specific coding rules, and routing weights alone are enough to decode genes, repeats and distal regulatory elements.
Early layers contain distinct AT-activated and GC-activated expert clusters. Species tags then steer generation toward each lineage's GC content and codon usage. Zero-shot likelihoods also respect synonymous codon preferences and penalize disruptive mutations.
Redrawn after Figure 2B to convey the activation switch as GC rises. For expert indices and exact values, see the original panel.
Dark diamonds are natural GC content, teal circles the sequence generated under that species label, for 11 species in Figure 2C; the dashed diagonal is y = x. Toggling the label off drops every point onto the ~39% unconditional baseline.
We trained lightweight classifiers on DeepD's 576-dimensional gating vectors from layers 2–24. Routing alone resolves CRISPR arrays and operons in E. coli, gene structure and repeat classes in human (mean AUPRC 0.92), and distal enhancers in the repeat-rich maize genome.
DeepD scores variants zero-shot by the change in sequence likelihood, with no task-specific labels. Its embeddings also support supervised prediction of regulatory function. It is strongest where genomes are hardest to read: non-coding and regulatory DNA.
Across five zero-shot human benchmarks DeepD reaches AUROCs of 0.814–0.965. It is competitive with the Evo 2 series and Nucleotide Transformer (NT) v2, and outperforms AlphaGenome on protein fitness, ClinVar, BRCA1 and BRCA2, with particular strength on synonymous, splice-related and non-coding variants.
Classifiers trained on DeepD representations lead all general genome language models on human TF binding, promoter, core-promoter and splice-site prediction.
Figure 4F. AUROC of classifiers trained on frozen model representations. Labels above each group show DeepD’s margin over the best Evo model; n/a marks tasks without an Evo result.
AlphaGenome, trained directly on human functional genomic tracks, scores higher on TF binding (0.695–0.918) and splice sites (0.815). DeepD scores higher on promoters (0.860 vs 0.692) and core promoters (0.620 vs 0.601). The two models are complementary (Figure S3E).
DeepD moves generation from open-ended continuation toward specification. It writes regulatory elements with a requested strength and builds chromosome-scale sequences that keep gene structure and order while varying within natural ranges.
Fine-tuned on STARR-seq activity labels, DeepD generates novel 170-bp synthetic core promoters conditioned on High or Low activity. The designs are dissimilar to natural maize sequences, yet dual-luciferase assays in maize protoplasts confirm the intended direction.
Early errors in autoregressive generation compound with length. CAB instead writes each 8-kb Body between an upstream Anchor and a downstream Context, then relays forward: the freshly generated segment anchors the next round. Total output length is no longer tied to the training window.
Schematic after Figure 7A. Each round generates one 8-kb Body chunk.
Fine-tuned on 762 S. cerevisiae genomes with the S288C reference held out, DeepD generated 29 full-length chromosome III sequences on the S288C backbone. They keep reference-like gene content and order while carrying variation that mirrors the natural yeast pangenome.
Applied to a 1-Mb interval of Arabidopsis thaliana chromosome 1 (0.13–1.13 Mb), CAB generated three independent sequences. They preserve coherent genic regions and introduce new variation mostly in intergenic DNA, with divergence from TAIR12 in the middle of the natural range.
A feature-by-feature comparison with representative published genome models.
| Feature | DeepD | Evo 2 | NT-2.5B | AlphaGenome |
|---|---|---|---|---|
| Architecture | Sparse mixture-of-experts | Dense StripedHyena 2 | Dense transformer | U-Net–transformer |
| Size and efficiency | 63B sparse, ~1B activated per token; computationally efficient | 40B, no sparse activation; computationally demanding | 2.5B, no sparse activation | ~450M parameters (distilled student model); computationally efficient |
| Pretraining data | Unselected whole genomes: all genic and the full intergenic regulatory and repetitive space | Not whole-genome; selected genic and nearby regions (promoters, ncRNA) | Not whole-genome; sampled 6-kb segments of reference genomes | Not whole-genome; human and mouse genomic intervals |
| Phylogeny information | Yes Explicit species labels embedded from an established phylogenetic tree | No | No | No |
| Target species | All domains of life, including human and mouse | All domains of life, including human and mouse | Limited species, including human and mouse | Human and mouse |
| Prediction performance | Competitive in zero-shot protein-fitness and disease-variant prediction in human and other species; outperforms AlphaGenome zero-shot and surpasses the Evo series in supervised classification of intergenic regulatory sequences | Competitive in zero-shot protein-fitness and disease-variant prediction | Weak | Weak zero-shot; strong in supervised classification of intergenic regulatory sequences |
| Genomic feature interpretation | Exon–intron boundaries, DNA transposons, LTR repeats and distal CREs | Exon–intron boundaries; TEs and distal CREs unknown | Promoters, enhancers and splice sites; TEs and distal CREs unknown | Promoters, TF binding, splicing and distal regulatory interactions; TEs unknown |
| Long-sequence generation strategy | Yes Context–Anchor–Body strategy for controllable genome-scale generation | Guided only by a starting sequence | No | No |
| Long-sequence generation capability | High-quality yeast chromosomes that are hard to distinguish from natural isolates, and megabase Arabidopsis segments | Limited yeast chromosome generation: gene number, gene order and tRNA number unmatched; gene structure unknown | No | No |
Evo 2: Brixi et al., Nature 2026. NT: Dalla-Torre et al., Nature Methods 2025. AlphaGenome: Avsec et al., Nature 2026.
Analysis code, generated sequences and a hosted inference API are available now. Source code, model weights and the full inference pipeline will be released in the same repository no later than publication.
The DeepD API returns nucleotide-resolution embeddings, per-position logits and prompt-conditioned DNA generation. Keys are free for non-commercial research.
@article{xin2026deepd,
title = {DeepD enables efficient genome modeling and controllable
long-context sequence generation across domains of life},
author = {Xin, Beibei and Wang, Zijian and Yu, An and Chen, Ziyang and
Guo, Yucheng and Zhang, Mingpei and Hu, Bozhen and Ye, Chonghang and
Chen, Jian and Liu, Qiujie and Zhao, Haiming and Song, Weibin and
Dong, Nanqing and Yang, Fan and Yang, Qirong and Zhao, Hainan and
Lai, Jinsheng},
journal = {bioRxiv},
year = {2026},
doi = {10.64898/2026.10.13.758170},
url = {https://doi.org/10.64898/2026.10.13.758170}
}
bioRxiv preprint: https://doi.org/10.64898/2026.10.13.758170
China Agricultural University
State Key Laboratory of Maize Bio-breeding
BioMap Research
Shanghai Artificial Intelligence Laboratory
Yazhouwan National Laboratory