Genomicsqlite

* **Compressed database files can be queried over the web directly** ([docs](https://mlin.github.io/GenomicSQLite/guide_db/#reading-databases-over-the-web))
* [Open routines](https://mlin.github.io/GenomicSQLite/guide_db/#opening-a-compressed-database) accept http/https URLs (read-only) as well as local filenames
* Adaptive batching & prefetching to optimize the HTTP request pattern (will be tuned over time)
* Builds in [sqlite_web_vfs](https://github.com/mlin/sqlite_web_vfs) extension (also usable standalone)
* **Improves the `genomicsqlite` command-line tool installed with the Python package** ([docs](https://mlin.github.io/GenomicSQLite/guide_db/#genomicsqlite-interactive-shell))
* Opens local filename or remote URL
* Removes extraneous messages on entering interactive shell
* New `--compact` subcommand to re-compress and defragment a database file, potentially shrinking it further and making it more suitable for web access
* The extension is now compatible with host programs *statically* linking SQLite3.

<img src="https://user-images.githubusercontent.com/356550/107273470-33164900-69f3-11eb-9f29-628d2ff9974d.gif" width="800"/>

<hr/>

**Start here** 👉 [](https://mlin.github.io/GenomicSQLite/) 👈

[](https://pypi.org/project/genomicsqlite/) [](https://anaconda.org/mlin/genomicsqlite) [](https://crates.io/crates/genomicsqlite)

Most developers should install a package through pip/conda/maven/cargo, which bundle the same prebuilt library files available below for use with C/C++ and yet-to-be-packaged languages.

* **libgenomicsqlite.so** for Linux x86-64 (Ivy Bridge), built in CentOS 7
* **libgenomicsqlite.dylib** for macOS Catalina x86-64

An up-to-date version of SQLite itself (&geq;3.31.0) is also required, as detailed in the [docs](https://mlin.github.io/GenomicSQLite/). If unable to upgrade the host SQLite, a prebuilt Linux x86-64 `libsqlite3.so.0` is available below, for deployment with suitable `LD_LIBRARY_PATH`.

**USE AT YOUR OWN RISK:** This project is not associated with the SQLite developers.

<hr/>

_{SHA-256 digests:}


610d5de2f12906b098528c391a1af9a9bdb09561f1b6e795a2c739d533fcf5c6 libgenomicsqlite.so
80e22ff126561d379210363c842f95101a1cdd9a94d687a342cabadf0e0add24 libgenomicsqlite.dylib
9d6cd9c6724d276db4d3b1a344d72cd96bfa2bda99e4702033647b6bf481f8b6 genomicsqlite-jdbc-v0.8.0.jar
2dae78da03f2dd1bdeac5089cf4ec976912efab23186e5b6c0c8ce91048d16b9 libsqlite3.so.0

Linux x86-64 build improvements:

* Extends prebuilt shared library (.so) compatibility back to CentOS 7 and Ivy Bridge CPUs
* Each Release will now include a prebuilt `libsqlite3.so.0`, as a backup option for those unable to upgrade their host's SQLite

**USE AT YOUR OWN RISK:** The extension makes fundamental changes to the SQLite storage layer. While designed to preserve ACID transaction safety, it's young and unlikely to have zero bugs. This project is not associated with the SQLite developers.

<hr/>

_{SHA-256 digests:}


6d03fc963752b82ae7cfbffb56aef21d9ccbeaf3c5728d3f3d499c58d53d1917 libgenomicsqlite.so
732e4f39842206d49e6cc01c8d620957aa810ba63fd12ce3c010fc5bbf02aa92 libgenomicsqlite.dylib
507b2542fb8dd8b2c9359102dbf0b304061effae481b4b914f9d60a16200b29e genomicsqlite-jdbc-v0.7.0.jar
3873c8aa004a88e27b9029d5d9442b9ae52875cacf9aa339b773154844d27bf2 libsqlite3.so.0

* Add [in-SQL `dna_revcomp(?)`](https://mlin.github.io/GenomicSQLite/guide/#dna-reverse-complement) to reverse-complement a DNA sequence TEXT value
* The SQLite developers' [UINT collating sequence](https://www.sqlite.org/uintcseq.html) is now bundled and enabled automatically
* useful for sorting by chromosome names (e.g. 'chr2' < 'chr10') and other identifiers
* simply `ORDER BY chrom COLLATE UINT`
* Use speedier [Zstandard v1.4.7](https://github.com/facebook/zstd/releases/tag/v1.4.7) internally
* Portability tweaks to prebuilt shared library

* Add [SQL functions for parsing genomic range strings](https://mlin.github.io/GenomicSQLite/guide/#parse-genomic-range-string) like `chr1:2,345-6,789`
* Fix Rust `genomicsqlite_attach_sql()` binding

* Rust bindings, published on [crates.io](https://crates.io/crates/genomicsqlite) and covered in the [docs](https://mlin.github.io/GenomicSQLite/)
* Adds SQL helper functions for storing DNA/RNA sequences as compact BLOBs using two bits per nucleotide.
* `nucleotides_twobit()`
* `twobit_dna()`
* `twobit_rna()`
* `twobit_length()`
* see [Programming Guide for details](https://mlin.github.io/GenomicSQLite/guide/#two-bit-encoding-for-nucleotide-sequences)