Genome foundation model China Agricultural University  ·  BioMap Research

Reading and writing genomes
across all domains of life

DeepD is a sparse mixture-of-experts genome foundation model that learns from genes and the full intergenic landscape across all domains of life. It combines single-nucleotide resolution with a million-token context and efficient sparse computation, supporting broad sequence-to-function prediction with particular strength in regulatory and non-coding tasks. Beyond experimentally validated promoter design, its Context–Anchor–Body (CAB) strategy enables controllable generation of entire yeast chromosomes and megabase-scale plant genomic segments. These sequences preserve coherent genic regions while introducing variations consistent with natural population patterns.

63B
parameters, sparse MoE
with ~1B active per token
1M
token context at
single-nucleotide resolution
5.69 Tb
unselected whole-genome
pretraining sequence
3.07M
genomes spanning eukaryotes,
archaea, bacteria and viruses
Introducing DeepD

One model to predict, interpret and design genomes

Genomes are long, heterogeneous and evolutionarily diverse, and most of their sequence is non-coding. DeepD was built to model them as they occur in nature. It pairs efficient sparse computation with pretraining on unselected whole genomes and a new Context–Anchor–Body strategy for controllable long-sequence generation.

DeepD was trained at two scales: DeepD-2B (1.8 billion parameters, 1 trillion tokens) and DeepD-60B (63 billion parameters, 5.69 trillion tokens). The same model supports zero-shot and supervised sequence-to-function prediction, exposes interpretable genomic features through its expert routing, and generates experimentally validated promoters, complete yeast chromosomes and megabase-scale Arabidopsis segments.

Efficient long context

A sparse mixture-of-experts backbone routes each nucleotide to only 8 of 576 experts, extending context from 8K to 1M tokens at a fraction of dense-model cost.

Phylogeny-informed

Species labels embedded from an established phylogenetic tree condition a species-informed router, so DeepD learns lineage-specific composition, codon usage and coding rules.

Unselected whole genomes

Pretraining keeps every genic region and the full intergenic space, including regulatory and repetitive DNA, rather than enriching for genes or sampling short segments.

Controllable generation

The Context–Anchor–Body strategy generates each segment between upstream and downstream constraints and relays forward, decoupling output length from the context window.

01 · Model

A sparse, phylogeny-aware architecture

DeepD integrates DNA sequence, species label embeddings and annotation features in a sparse mixture-of-experts transformer, trained with next-token prediction on byte-level nucleotides.

DeepD mixture-of-experts architecture: species embedding and DNA sequence feed multi-scale gated attention and a species-informed router selecting shared and routed experts.
Figure 1A. Species embedding and DNA sequence pass through multi-scale gated attention and a species-informed router.
  • Species-informed router. The gating network sees both the token and a species representation, so experts can capture sequence-level and lineage-specific signals.
  • Multi-scale gated attention. In each block 24 of 32 heads use sliding-window attention and 8 use full attention, capturing local motifs and long-range dependencies. A sigmoid output gate stabilizes long-context training.
  • 576 experts, 8 active. One shared expert plus routed experts. Each token activates about 1 billion of the 63 billion parameters.
  • Reweighted objective. Soft-masked repeats stay in their native context but contribute less to the loss, so abundant repeats do not dominate optimization.
DeepD-2BDeepD-60B
Total parameters1.8B63B
Training tokens1T5.69T
Context length8K8K → 1M
Layers / heads24 / 1624 / 32
Hidden size7682,048
Experts (activated)128 (8)576 (8)

Multi-scale attention and species-informed routing

Each block mixes 24 sliding-window attention heads for local motifs with 8 full-attention heads for long-range dependencies, and a species-informed router that selects experts per token. Both modules below are interactive.

Attention head roles

mask is schematic
SWA · 256-token window ×24 FA · full attention ×8

Hover any head to see the shape of its attention mask.

Species-informed routing

Pretrained on whole genomes from every domain of life

The DeepD-60B corpus comprises about 5.90 Tb of nucleotide sequence. The eukaryotic partition covers 3,600 animal, 928 plant, 767 fungal and 139 protist genomes. Coding sequence, introns, repeats, other intergenic DNA and viral genomes are all represented, so the model learns genome-wide organization rather than genes alone.

Pretraining corpus by domain
  • Eukaryotes
  • Bacteria
  • Viruses
  • Archaea

Bar length on a logarithmic scale.

Token composition of pretraining and midtraining stages and number of nucleotides for DeepD-2B and DeepD-60B.
Figure 1C, E. Token composition by stage, and total training nucleotides.
Circular phylogenetic tree of 5,434 eukaryotic genomes in the pretraining corpus, highlighting Animalia, Plantae, Fungi and Protista.
Figure 1B. Phylogenetic tree of the 5,434 eukaryotic genomes in the corpus.

Long context, efficiently

Pretraining runs at an 8K context. Midtraining then extends the window step by step to 1M tokens. Validation perplexity keeps improving as the context grows, and sparse routing keeps the cost of ultra-long sequences manageable.

Validation perplexity of DeepD-60B midtraining decreases as context length increases from 8K to 1M.
Figure 1F. Validation perplexity falls as the context extends from 8K to 1M.
Total iteration time on 64 GPUs for MoE-60B versus Dense-60B across training sequence lengths.
Figure 1G. Iteration time of the sparse MoE-60B versus a dense 60B model on 64 GPUs. The gap widens with sequence length.
02 · Interpretability

What DeepD learns about genomes

DeepD's experts organize themselves around biology. Probing them reveals species-specific coding rules, and routing weights alone are enough to decode genes, repeats and distal regulatory elements.

Phylogeny-informed modeling captures species-specific rules

Early layers contain distinct AT-activated and GC-activated expert clusters. Species tags then steer generation toward each lineage's GC content and codon usage. Zero-shot likelihoods also respect synonymous codon preferences and penalize disruptive mutations.

Heatmap of Layer 2 expert activation across samples sorted by GC content showing AT-activated and GC-activated clusters.
Figure 2B. As input GC content rises from 25% to 75%, activation hands over from AT-activated to GC-activated experts, with no supervision.
GC content of natural sequences versus sequences generated with species labels across diverse taxa.
Figure 2C. With species tags, generated sequences match natural GC content across a wide spectrum. Without a tag they default to about 40% GC.
Radar chart comparing relative synonymous codon usage of natural and generated sequences for H. sapiens and S. cerevisiae.
Figure 2D. Coding sequences generated from a 50-bp prompt reproduce the GC-rich codon preferences of human and the AT-biased preferences of yeast.
Mean change in log-likelihood for G-to-A transitions at the third position of CAG codons across prokaryotes and eukaryotes.
Figure 2E. In low-CAG species DeepD favors the synonymous G→A change. In high-CAG species it penalizes it, all at single-nucleotide resolution and without fine-tuning.
Violin plots of likelihood change after 10-bp deletions across functional genomic regions in prokaryotes and eukaryotes.
Figure 2F. 10-bp deletions cost the most likelihood in coding sequence and structured RNAs, and little in introns and intergenic DNA, mirroring relaxed evolutionary pressure.

Drag GC content and watch the experts hand over

redrawn schematic
GC 25%samples sorted by GC contentGC 75%
Input sequence GC content 45.0%
—mean Z-score, AT-activated experts
—mean Z-score, GC-activated experts

Redrawn after Figure 2B to convey the activation switch as GC rises. For expert indices and exact values, see the original panel.

Generated GC content with and without a species label

R² = 0.706generated vs natural GC
ρ = 0.816Spearman correlation
≈ 39%baseline without a label

Dark diamonds are natural GC content, teal circles the sequence generated under that species label, for 11 species in Figure 2C; the dashed diagonal is y = x. Toggling the label off drops every point onto the ~39% unconditional baseline.

Expert routing interprets coding, regulatory and repetitive DNA

We trained lightweight classifiers on DeepD's 576-dimensional gating vectors from layers 2–24. Routing alone resolves CRISPR arrays and operons in E. coli, gene structure and repeat classes in human (mean AUPRC 0.92), and distal enhancers in the repeat-rich maize genome.

Probing framework: DeepD-60B 8K or 1M, 576 expert gating weights from 23 layers used to train binary classifiers for CDS, intron, TE and other annotations.
Figure 5A. For each annotation class, a classifier is trained on gating vectors and the best-performing layer is kept.
Predicted probability tracks in E. coli: alternating CRISPR repeat and spacer signals, and lac operon gene boundaries.
Figure 5B. Repeat and spacer probabilities alternate in step with the array, and gene signals align with lacA, lacY, lacZ and lacI.
Predicted probability tracks for CDS, LTR, LINE and SINE across the human HBB gene cluster on chromosome 11.
Figure 5C. CDS signals at HBB and HBD, with class-specific LTR, LINE and SINE tracks. Genome-wide AUPRC reaches 0.96 for CDS and 0.95 for SINE.
Bar chart of maize annotation AUPRC for 8K and 1M models with best layer per category.
Figure 5D. In maize, where more than 85% of the genome is repetitive, the 1M-context model decodes gene structure, CREs and TE families better than the 8K model.
Routing probability tracks across the maize tb1 locus, including the upstream Hopscotch retrotransposon enhancer.
Figure 5E. At the domestication gene tb1, DeepD recovers the gene and a prominent signal at the functionally validated Hopscotch enhancer about 60 kb upstream.
03 · Prediction

Sequence-to-function prediction across species

DeepD scores variants zero-shot by the change in sequence likelihood, with no task-specific labels. Its embeddings also support supervised prediction of regulatory function. It is strongest where genomes are hardest to read: non-coding and regulatory DNA.

DNA, RNA and protein, from phage to mammals

0.551Mean |ρ|, bacterial protein fitness7 datasets · ahead of Evo series and ProGen2
0.478Mean |ρ|, eukaryotic protein fitness5 datasets · on par with the strongest models
0.297Mean |ρ|, ncRNA fitness7 datasets · best overall, including RiNALMo
0.603AUROC, phage gene essentialityComparable to the best Evo 2 models
Bar chart of zero-shot ncRNA fitness prediction; DeepD-60B mean 0.297 is highest.
Figure 3A. Zero-shot ncRNA fitness across rRNA, tRNA and ribozyme datasets.
Supervised classifier AUROC on yeast H3/H4 histone marks and five mouse transcription factor binding tasks; DeepD-60B ranks first on each.
Figure 3B. On embeddings, DeepD reaches AUROC 0.764 (H3) and 0.811 (H4) in yeast and ranks first on all five mouse TF-binding tasks.

Human variant effects and regulatory DNA

Across five zero-shot human benchmarks DeepD reaches AUROCs of 0.814–0.965. It is competitive with the Evo 2 series and Nucleotide Transformer (NT) v2, and outperforms AlphaGenome on protein fitness, ClinVar, BRCA1 and BRCA2, with particular strength on synonymous, splice-related and non-coding variants.

    DeepD-60B Evo 2 (40B → 1B) Evo 1 NT-2.5B NT-500M AlphaGenome Protein LMs

    Supervised human regulatory prediction

    Classifiers trained on DeepD representations lead all general genome language models on human TF binding, promoter, core-promoter and splice-site prediction.

      Figure 4F. AUROC of classifiers trained on frozen model representations. Labels above each group show DeepD’s margin over the best Evo model; n/a marks tasks without an Evo result.

      0.860Promoter detection
      0.620Core-promoter detection
      0.714Splice-site prediction
      0.497–0.688Five TF-binding tasks

      AlphaGenome, trained directly on human functional genomic tracks, scores higher on TF binding (0.695–0.918) and splice sites (0.815). DeepD scores higher on promoters (0.860 vs 0.692) and core promoters (0.620 vs 0.601). The two models are complementary (Figure S3E).

      04 · Design

      Controllable design, from promoters to chromosomes

      DeepD moves generation from open-ended continuation toward specification. It writes regulatory elements with a requested strength and builds chromosome-scale sequences that keep gene structure and order while varying within natural ranges.

      Maize core promoters with specified strength, validated in vivo

      Fine-tuned on STARR-seq activity labels, DeepD generates novel 170-bp synthetic core promoters conditioned on High or Low activity. The designs are dissimilar to natural maize sequences, yet dual-luciferase assays in maize protoplasts confirm the intended direction.

      Pipeline for conditional generation of maize core promoters: STARR-seq labels, fine-tuning with strength tokens, conditional generation, filtering and protoplast reporter assays.
      Figure 6A. Fine-tune on strength tokens, generate conditionally, filter, then test in maize protoplasts.
      76%of High designs above the High-activity threshold (n = 83)
      83%of Low designs below the Low-activity threshold (n = 71)
      Violin plots of predicted and measured promoter strength for High and Low generated candidates with activity thresholds.
      Figure 6B–C. Predicted (left) and measured (right) strength of High and Low designs.
      AT content along the promoter relative to TSS and enriched TATA box and Y patch motifs in High designs.
      Figure 6D. High designs carry a TATA box and Y patch at positions typical of natural plant core promoters.
      Context–Anchor–Body

      Generating beyond a single context window

      Early errors in autoregressive generation compound with length. CAB instead writes each 8-kb Body between an upstream Anchor and a downstream Context, then relays forward: the freshly generated segment anchors the next round. Total output length is no longer tied to the training window.

      Prompt CA Response Bgen

      Schematic after Figure 7A. Each round generates one 8-kb Body chunk.

      Complete yeast chromosomes

      Fine-tuned on 762 S. cerevisiae genomes with the S288C reference held out, DeepD generated 29 full-length chromosome III sequences on the S288C backbone. They keep reference-like gene content and order while carrying variation that mirrors the natural yeast pangenome.

      309.9–316.6kbLength of 29 generated ChrIII sequencesS288C reference: 316.62 kb
      138–140Coding genesvs 139 in S288C · tRNAs 10 vs 10
      138 / 140Genes collinear with S288CRepresentative AIYeast25
      100%of AIYeast25 SNPs found in natural isolates3,490 SNPs · no stop-gain
      Whole-chromosome alignment between S288C ChrIII and generated AIYeast25 showing gene-level collinearity.
      Figure 7C. Extensive gene-level collinearity across the whole chromosome, telomere to telomere.
      Bar chart of gene, Pfam and tRNA counts for generated yeast chromosomes versus S288C, and protein length histograms.
      Figure 7B. Reference-like numbers of coding genes, Pfam-supported genes and tRNAs, and matching protein-length distributions.
      Variant landscape of generated and natural ChrIII sequences relative to S288C across collinear blocks.
      Figure 7D. Generated chromosomes carry 8–10 SVs and 3,288–4,025 SNPs each, with population-like variant density. They fall across natural clades rather than forming one artificial cluster.
      Enlarged views of a variable TE region and a coding region in generated and natural yeast chromosomes.
      Figure 7E–F. A TE present in S288C is absent from all generated chromosomes, as in 54.99% of natural isolates. A ~3.5-kb gene gains synonymous and missense changes but no stop-gains.

      Megabase-scale plant genome segments

      Applied to a 1-Mb interval of Arabidopsis thaliana chromosome 1 (0.13–1.13 Mb), CAB generated three independent sequences. They preserve coherent genic regions and introduce new variation mostly in intergenic DNA, with divergence from TAIR12 in the middle of the natural range.

      99.2%Core genes recovered243 of 245 in AIAtRegion1_01
      296 / 299Reference protein-coding genes recovered
      1,753Exonsvs 1,760 in TAIR12 · Pfam proteins 266 vs 269
      88%of generated SNPs are non-codingAll 12 SVs in intergenic or intronic DNA
      Gene organization comparison between TAIR12 and generated AIAtRegion1_01 over 1 Mb with one-to-one gene matches.
      Figure 8B. One-to-one gene matches along the full megabase. Of the three genes not fully recovered, two have disrupted reading frames and one lies at the very end of the segment.
      Feature counts for AIAtRegion1_01 versus TAIR12: genes 299/299, Pfam 266/269, exons 1,753/1,760, tRNA 8/8. Protein length distributions in TAIR12 and AIAtRegion1_01.
      Figure 8C–D. Genomic composition and protein-length distributions closely match the reference.
      Density of sequence divergence from TAIR12 across 255 natural accessions with three generated sequences marked.
      Figure 8E. All three generated sequences fall in the middle of the natural divergence distribution.
      Sequence identity of CDS, gene body, promoter and intergenic regions for natural and generated sequences.
      Figure 8F. Variation follows functional constraint, as in natural genomes. CDSs are the most conserved and intergenic DNA the most divergent.
      Representative structural variants in AIAtRegion1_01 including an ATHILA7-derived insertion and deletions at SIMPLEGUY1 and ATLINE1A.
      Figure 8G. An ATLINE1A deletion overlaps deletions in 236 natural accessions. A new 1.8-kb insertion derives from an endogenous ATHILA7 retrotransposon found elsewhere in the genome, not from arbitrary sequence.
      05 · Comparison

      How DeepD compares

      A feature-by-feature comparison with representative published genome models.

      Feature DeepD Evo 2 NT-2.5B AlphaGenome
      Architecture Sparse mixture-of-experts Dense StripedHyena 2 Dense transformer U-Net–transformer
      Size and efficiency 63B sparse, ~1B activated per token; computationally efficient 40B, no sparse activation; computationally demanding 2.5B, no sparse activation ~450M parameters (distilled student model); computationally efficient
      Pretraining data Unselected whole genomes: all genic and the full intergenic regulatory and repetitive space Not whole-genome; selected genic and nearby regions (promoters, ncRNA) Not whole-genome; sampled 6-kb segments of reference genomes Not whole-genome; human and mouse genomic intervals
      Phylogeny information Yes Explicit species labels embedded from an established phylogenetic tree No No No
      Target species All domains of life, including human and mouse All domains of life, including human and mouse Limited species, including human and mouse Human and mouse
      Prediction performance Competitive in zero-shot protein-fitness and disease-variant prediction in human and other species; outperforms AlphaGenome zero-shot and surpasses the Evo series in supervised classification of intergenic regulatory sequences Competitive in zero-shot protein-fitness and disease-variant prediction Weak Weak zero-shot; strong in supervised classification of intergenic regulatory sequences
      Genomic feature interpretation Exon–intron boundaries, DNA transposons, LTR repeats and distal CREs Exon–intron boundaries; TEs and distal CREs unknown Promoters, enhancers and splice sites; TEs and distal CREs unknown Promoters, TF binding, splicing and distal regulatory interactions; TEs unknown
      Long-sequence generation strategy Yes Context–Anchor–Body strategy for controllable genome-scale generation Guided only by a starting sequence No No
      Long-sequence generation capability High-quality yeast chromosomes that are hard to distinguish from natural isolates, and megabase Arabidopsis segments Limited yeast chromosome generation: gene number, gene order and tRNA number unmatched; gene structure unknown No No

      Evo 2: Brixi et al., Nature 2026. NT: Dalla-Torre et al., Nature Methods 2025. AlphaGenome: Avsec et al., Nature 2026.

      06 · Resources

      Get DeepD

      Analysis code, generated sequences and a hosted inference API are available now. Source code, model weights and the full inference pipeline will be released in the same repository no later than publication.

      Inference API

      Request an API key

      The DeepD API returns nucleotide-resolution embeddings, per-position logits and prompt-conditioned DNA generation. Keys are free for non-commercial research.

      1. Apply with your institutional email. University addresses are activated automatically; other addresses are reviewed by our team.
      2. Check your inbox. Your API key is sent to the email you provide, in English.
      3. Start calling DeepD. Follow the usage guide and notebooks on GitHub.

      Citation

      @article{xin2026deepd,
        title   = {DeepD enables efficient genome modeling and controllable
                   long-context sequence generation across domains of life},
        author  = {Xin, Beibei and Wang, Zijian and Yu, An and Chen, Ziyang and
                   Guo, Yucheng and Zhang, Mingpei and Hu, Bozhen and Ye, Chonghang and
                   Chen, Jian and Liu, Qiujie and Zhao, Haiming and Song, Weibin and
                   Dong, Nanqing and Yang, Fan and Yang, Qirong and Zhao, Hainan and
                   Lai, Jinsheng},
        journal = {bioRxiv},
        year    = {2026},
        doi     = {10.64898/2026.10.13.758170},
        url     = {https://doi.org/10.64898/2026.10.13.758170}
      }

      bioRxiv preprint: https://doi.org/10.64898/2026.10.13.758170

      07 · Team

      Authors and affiliations

      Beibei Xin1,2,3,5,†, Zijian Wang1,†, An Yu1,†, Ziyang Chen1,†, Yucheng Guo6,†, Mingpei Zhang1,3, Bozhen Hu6, Chonghang Ye6, Jian Chen1,2,3,5, Qiujie Liu1,2,3,5, Haiming Zhao1,2,3,5, Weibin Song1,2,3,5, Nanqing Dong7,8, Fan Yang9, Qirong Yang6, Hainan Zhao1,2,3,5,*, Jinsheng Lai1,2,3,4,5,*

      † These authors contributed equally  ·  * Correspondence: jlai@cau.edu.cn (J.L., lead contact) and hnzhao@cau.edu.cn (H.Z.)

      1. State Key Laboratory of Maize Bio-breeding, National Maize Improvement Center, Department of Plant Genetics and Breeding, China Agricultural University, Beijing, China
      2. Center for Crop Functional Genomics and Molecular Breeding, China Agricultural University, Beijing, China
      3. School of Interdisciplinary Life Sciences, China Agricultural University, Beijing, China
      4. Sanya Institute of China Agricultural University, Sanya, China
      5. Ministry of Agriculture and Rural Affairs Key Laboratory of Gene Editing Technologies (Beijing), Beijing, China
      6. BioMap Research, Beijing, China
      7. Shanghai Artificial Intelligence Laboratory, Shanghai, China
      8. Shanghai Innovation Institute, Shanghai, China
      9. Yazhouwan National Laboratory, Sanya, China