Human host-filter bundle for the rype browser app ================================================= One URL: paste the URL of this directory (with a trailing slash) into the Index field of the rype web app and it configures both filtering stages. Contents -------- index.json bundle manifest: relative paths of the two indexes below human-hprcv1-w20.toml rype config the .ryxdi was built from human.ryxdi/ rype index of human references (see below) manifest.toml, buckets.parquet, inverted/shard.0.parquet k = 64, w = 20, 1 bucket, 1 shard, 414,737,338 minimizers chm13_sr_parts.mmi minimap2 multi-part index of T2T-CHM13v2.0 (masked Y, rCRS mitochondrion), short-read preset, 5 parts, 9.07 GB chm13_sr_parts.mmi.parts.json byte offset, size and sequence count of each part of the .mmi (minimap2 writes parts back to back with no table of contents; the app needs one to open parts independently) How the app uses them --------------------- The app runs entirely in the browser. Reads never leave the machine; the indexes are read from this directory by HTTP range requests and nothing is downloaded whole or stored. 1. rype pass: every batch of reads is classified against human.ryxdi by streaming the inverted shard once per batch. Reads (or pairs) that hit the index are dropped. 2. minimap2 pass (short reads only): the survivors are mapped with the minimap2 `sr` preset against each part of chm13_sr_parts.mmi in turn, parts mounted by the byte ranges in .parts.json. A pair is dropped if either mate maps in any part. How the indexes were built -------------------------- rype index (human.ryxdi): built with `rype index from-config` from human-hprcv1-w20.toml (included here; k = 64, w = 20, default salt, sequences orientation-normalised). One bucket over 97 reference FASTA files: - GCA_009914755.4 T2T-CHM13v2.0 - GCF_000001405.40 GRCh38.p14 - GENCODE v47 transcripts - 94 haplotype assemblies (maternal + paternal) of 47 HPRC year-1 samples (*.f1_assembly_v2_genbank.fa) rype index from-config -c human-hprcv1-w20.toml # output renamed to human.ryxdi The file list in the config gives bare filenames; place the FASTAs beside it to rebuild. minimap2 index (chm13_sr_parts.mmi): minimap2 2.30, from chm13v2.0_maskedY_rCRS.fa.PanSN.fa.gz (T2T-CHM13v2.0 with the Y masked and the rCRS mitochondrion, PanSN sequence names). Multi-part (-I) so that each part fits a browser's memory: minimap2 -x sr -t 10 -I 600M -d chm13_sr_parts.mmi chm13v2.0_maskedY_rCRS.fa.PanSN.fa.gz python3 tools/mmi_parts.py chm13_sr_parts.mmi # writes chm13_sr_parts.mmi.parts.json # (tools/ in github.com/michaeliter/rype-web)