Glaredb

Highlights

- Python bindings:
- You can now query intermediate results, simply referencing a variable:
python
intermediate = con.sql("select * from tbl where a > 2;")
con.sql("select * from intermediate where b > 4;")

- `show()` was added to print results without other libraries


- SQL commands:
- `COPY TO`:
- Add support for remote URLs, including GCS and S3. For more information, see <https://github.com/GlareDB/glaredb/pull/1280>
- Tables can now be copied with `COPY TO` (instead of only queries). For more information see <https://github.com/GlareDB/glaredb/pull/1280>

- SQL functions:
- All functions received support for passing in identifiers and named arguments. For more information, see <https://github.com/GlareDB/glaredb/pull/1314>
- `csv_scan`:
- Add support for GCS and S3 with credentials
- `ndjson_scan`:
- Add support for GCS and S3 with credentials
- `parquet_scan`:
- Add support for GCS and S3 with credentials
- `generate_series`:
- Added support for floats

- Recipes:
- We've added a new [examples](https://github.com/GlareDB/glaredb/tree/main/examples/) directory. Our first example demonstrates how to connect to glaredb with R and query a Snowflake database. Check it out here: <https://github.com/GlareDB/glaredb/blob/main/examples/R/out-of-process.R>
- We added an example of using our Python bindings with OpenAI! See it here: <https://github.com/GlareDB/glaredb/blob/main/py-glaredb/examples/openai_ex.py>

What's Changed

* chore: add semicolon to sql snippet by greyscaled in https://github.com/GlareDB/glaredb/pull/1299
* chore: Add example R script for connecting to glaredb by scsmithr in https://github.com/GlareDB/glaredb/pull/1296
* feat: composable plans by universalmind303 in https://github.com/GlareDB/glaredb/pull/1300
* feat: Support URLs for `COPY .. TO ..`. by vrongmeal in https://github.com/GlareDB/glaredb/pull/1280
* feat: Add support for float in generate_series by adhish20 in https://github.com/GlareDB/glaredb/pull/1298
* chore: move builtins.rs to sqlbuiltins by adhish20 in https://github.com/GlareDB/glaredb/pull/1302
* chore: Add Dependabot configuration for Cargo by 0xf333 in https://github.com/GlareDB/glaredb/pull/1301
* fix csv_scan for object_stores by universalmind303 in https://github.com/GlareDB/glaredb/pull/1303
* use pr-target instead of pr by universalmind303 in https://github.com/GlareDB/glaredb/pull/1305
* chore: Add example for running openai on data from postgres by scsmithr in https://github.com/GlareDB/glaredb/pull/1309
* feat: Support object store in `*_scan` functions. by vrongmeal in https://github.com/GlareDB/glaredb/pull/1306
* fix: stop parsing datasource and tunnel limits from cloud response by f0ssel in https://github.com/GlareDB/glaredb/pull/1313
* chore: move "client" stuff to metastoreproto by adhish20 in https://github.com/GlareDB/glaredb/pull/1311
* fix: Fix not being able to read huggingface parquets by scsmithr in https://github.com/GlareDB/glaredb/pull/1319
* feat: Support named arguments and idents in table functions. by vrongmeal in https://github.com/GlareDB/glaredb/pull/1314
* fix: Explicitly use `Ident` in table function parameters. by vrongmeal in https://github.com/GlareDB/glaredb/pull/1322
* fix: Add xtask alias back (for zip) by vrongmeal in https://github.com/GlareDB/glaredb/pull/1325
* chore: Update deltalake to `da1012`. by vrongmeal in https://github.com/GlareDB/glaredb/pull/1326
* feat: adding show() method to python execution results by adhish20 in https://github.com/GlareDB/glaredb/pull/1315
* chore: Use `SchemaReference` from datafusion instead of our own. by vrongmeal in https://github.com/GlareDB/glaredb/pull/1327
* chore: simplify feat template by greyscaled in https://github.com/GlareDB/glaredb/pull/1328
* build: Bump version to 0.2.1 by greyscaled in https://github.com/GlareDB/glaredb/pull/1332

New Contributors
* 0xf333 made their first contribution in https://github.com/GlareDB/glaredb/pull/1301

**Full Changelog**: https://github.com/GlareDB/glaredb/compare/v0.2.0...v0.2.1

Highlights

- python bindings:
- GlareDB can now be used in Python with Pandas, Polars and external data sources. [Install the module from PyPI](https://pypi.org/project/glaredb/) with

console
pip install glaredb

The instance of `glaredb` is currently limited to in-memory. We're working on connecting to remote instances of `glaredb` for a future release. For more information, we have examples at <https://github.com/GlareDB/glaredb/tree/main/py-glaredb/examples>.
- `COPY TO`:
- With `COPY TO`, a local copy of a query can be persisted. We're adding support for remote persistence in the future. For example, you can save a local copy in `test.csv` of the query `select 1` with: `copy (select 1) to local (location 'test.csv');`
- Currently, `json`, `parquet` and `csv` formats are supported. The default format is CSV, but you can specify a different format as follows:

sql
copy (select 1) to local format parquet (location 'test.parquet');


- `ndjson_scan`:
- You can now read [newline delimited JSON](https://github.com/ndjson/ndjson-spec) files, ex: `select * from ndjson_scan('./your/file.json');`
- misc:
- Added timestamp support for native tables

What's Changed

* chore: Prevent python linking for macos by scsmithr in https://github.com/GlareDB/glaredb/pull/1198
* remove python dep from docker by universalmind303 in https://github.com/GlareDB/glaredb/pull/1199
* feat: Allow reading from the python environment by scsmithr in https://github.com/GlareDB/glaredb/pull/1195
* feat: Add pandas support by scsmithr in https://github.com/GlareDB/glaredb/pull/1201
* feat: Use multi-threaded runtime in python by scsmithr in https://github.com/GlareDB/glaredb/pull/1203
* chore: Break up slt tests into batches by scsmithr in https://github.com/GlareDB/glaredb/pull/1205
* chore: Add slt tests for parquet_scan and csv_scan by scsmithr in https://github.com/GlareDB/glaredb/pull/1206
* chore: add link to protoc installation by greyscaled in https://github.com/GlareDB/glaredb/pull/1209
* feat: Make data dir optional in python bindings by scsmithr in https://github.com/GlareDB/glaredb/pull/1211
* chore: Use raw github link in csv_scan test by scsmithr in https://github.com/GlareDB/glaredb/pull/1212
* feat: add support for scanning ndjson by universalmind303 in https://github.com/GlareDB/glaredb/pull/1213
* ci: setup python workflow by universalmind303 in https://github.com/GlareDB/glaredb/pull/1210
* remove py-glaredb from default members by universalmind303 in https://github.com/GlareDB/glaredb/pull/1224
* build(deps): Bump deps by scsmithr in https://github.com/GlareDB/glaredb/pull/1241
* check permissions on tmp dir by universalmind303 in https://github.com/GlareDB/glaredb/pull/1239
* chore: Bump rust version in docker image by scsmithr in https://github.com/GlareDB/glaredb/pull/1250
* chore: Remove starting metastore from pgproto test script by scsmithr in https://github.com/GlareDB/glaredb/pull/1252
* fix: Downgrade datadriven crate by scsmithr in https://github.com/GlareDB/glaredb/pull/1253
* fix: Fix explain queries by scsmithr in https://github.com/GlareDB/glaredb/pull/1251
* feat: `COPY TO` support by vrongmeal in https://github.com/GlareDB/glaredb/pull/1159
* refactor: xtask --> justfile by universalmind303 in https://github.com/GlareDB/glaredb/pull/1242
* ci: Keep py-glaredb and glaredb versions in sync by universalmind303 in https://github.com/GlareDB/glaredb/pull/1255
* fix: Allow create tables with timestamps by vrongmeal in https://github.com/GlareDB/glaredb/pull/1257
* chore: Add pytest by scsmithr in https://github.com/GlareDB/glaredb/pull/1258
* feat: support compute_engine authentication parameter by f0ssel in https://github.com/GlareDB/glaredb/pull/1256
* chore: Add more info for pypi by scsmithr in https://github.com/GlareDB/glaredb/pull/1263
* build(deps): Bump deps by scsmithr in https://github.com/GlareDB/glaredb/pull/1277
* refactor: use native execs instead of custom execs by universalmind303 in https://github.com/GlareDB/glaredb/pull/1262
* chore: Add test cases for parsing compute engine from conn string by scsmithr in https://github.com/GlareDB/glaredb/pull/1285
* fix: don't fetch filetype eagerly by adhish20 in https://github.com/GlareDB/glaredb/pull/1286
* feat: Add s3 to object store registry create by scsmithr in https://github.com/GlareDB/glaredb/pull/1291
* feat(py): add lazy execution to python by universalmind303 in https://github.com/GlareDB/glaredb/pull/1292
* chore: Add tests asserting logical plan reuse in python by scsmithr in https://github.com/GlareDB/glaredb/pull/1293
* build: Bump version to 0.2.0 by scsmithr in https://github.com/GlareDB/glaredb/pull/1295

New Contributors
* adhish20 made their first contribution in https://github.com/GlareDB/glaredb/pull/1286

**Full Changelog**: https://github.com/GlareDB/glaredb/compare/v0.0.16...v0.2.0

**Full Changelog**: https://github.com/GlareDB/glaredb/compare/v0.0.15...v0.0.16

Highlights

- table create/insert:
- added native table support (ie: `CREATE TABLE my_table ( my_col text );`
- table functions:
- `parquet_scan` (ie: `SELECT * FROM parquet_scan('path/to/file.parquet')`
- `generate_series` (ie: `SELECT * FROM generate_series(1, 4)`
- misc:
- added credentials as a database object (see: https://github.com/GlareDB/glaredb/pull/1140)
- an `--ignore-auth` flag was added when running `glaredb` locally for development convenience
- fixes:
- `list_schemas` no longer returns hidden schemas for BigQuery

What's Changed

* build(deps): Bump deps by scsmithr in https://github.com/GlareDB/glaredb/pull/1139
* chore: Allow ignoring auth messages by scsmithr in https://github.com/GlareDB/glaredb/pull/1145
* feat: Add native (internal) table support using delta-rs by scsmithr in https://github.com/GlareDB/glaredb/pull/1129
* feat: Add support for credentials objects by vrongmeal in https://github.com/GlareDB/glaredb/pull/1140
* feat: Add `generate_series` function by scsmithr in https://github.com/GlareDB/glaredb/pull/1154
* feat: add "parquet_scan" function by universalmind303 in https://github.com/GlareDB/glaredb/pull/1160
* fix: Don't return hidden data sets from bigquery by scsmithr in https://github.com/GlareDB/glaredb/pull/1162
* add support for reading from "file://" for `parquet_scan` by universalmind303 in https://github.com/GlareDB/glaredb/pull/1164
* feat: Stream writes to tables by scsmithr in https://github.com/GlareDB/glaredb/pull/1165
* chore: Remove native table inner by scsmithr in https://github.com/GlareDB/glaredb/pull/1169
* chore: bump version to 0.0.15 by f0ssel in https://github.com/GlareDB/glaredb/pull/1170


**Full Changelog**: https://github.com/GlareDB/glaredb/compare/v0.0.14...v0.0.15

What's Changed
* feat: Support temporary tables by vrongmeal in https://github.com/GlareDB/glaredb/pull/1089
* build(deps): Bump deps by scsmithr in https://github.com/GlareDB/glaredb/pull/1114
* feat: Allow specifying user and password as commandline by scsmithr in https://github.com/GlareDB/glaredb/pull/1118
* build(deps): Update datafusion and object_store by scsmithr in https://github.com/GlareDB/glaredb/pull/1121
* feat: Add `list_schemas` and `list_tables` table functions by vrongmeal in https://github.com/GlareDB/glaredb/pull/1122
* chore: add documentation for demo postgres data by f0ssel in https://github.com/GlareDB/glaredb/pull/1123
* chore: add docs for demo user setup by f0ssel in https://github.com/GlareDB/glaredb/pull/1124
* chore: Add demo data source to readme by scsmithr in https://github.com/GlareDB/glaredb/pull/1125
* feat: Delta lake data source (initial implementation) by scsmithr in https://github.com/GlareDB/glaredb/pull/1119
* chore: Bump version to 0.0.14 by scsmithr in https://github.com/GlareDB/glaredb/pull/1130


**Full Changelog**: https://github.com/GlareDB/glaredb/compare/v0.0.13...v0.0.14

New Contributors
* universalmind303 made their first contribution in https://github.com/GlareDB/glaredb/pull/1091

**Full Changelog**: https://github.com/GlareDB/glaredb/compare/v0.0.12...v0.0.13