Repository navigation
Conversation
ClickHouse 25.9 or later is now supported for analytics workloads. The
adapter talks to the HTTP interface through curl instead of PDO: one
keep-alive connection per adapter, compressed responses, settings sent
as URL parameters.
- Results come back as TabSeparatedWithNamesAndTypes, which parsed as
fast as the JSON formats in PHP, is safe for any byte and makes an
error in the middle of a result abort the transfer instead of looking
like data. The statement parses one row per fetch() and casts values
from their ClickHouse type; integers beyond PHP_INT_MAX, Int128/256
and Decimal stay strings, Array/Map/Tuple become PHP arrays.
- Values are bound client-side as literals, skipping ? inside quotes,
quoted identifiers and comments. Numeric strings stay quoted: an
unquoted integer beyond Int64 is read as a lossy Float64.
- Errors carry the ClickHouse code, including those reported after the
result started streaming (both the 25.9 and the tagged 26.x formats).
- Column metadata comes from system.columns. The sorting key is not
taken as the primary key, as it is not unique: the key is a column
named id or the model's $primary_key. MATERIALIZED and ALIAS columns
are left out.
- A UInt128 (or Int128, UInt256, Int256, UUID, String) key gets a UUIDv7
generated in PHP on insert. ActiveRecord\Uuid generates them and
converts between the decimal the model holds and UUID text. UInt128
sorts numerically, so ORDER BY id follows creation order.
- DateTime values are written as quoted Unix timestamps and read back
in PHP's time zone. session_timezone is not used: before 26.2 INSERT
and lightweight UPDATE did not agree on how it applies.
- Inserts are synchronous by default: the fastest per save() and errors
are reported. Connections can opt into async_insert=1, which keeps
wait_for_async_insert=1 so failed inserts are never lost silently.
- update_all() and delete_all() return null, as ClickHouse does not
report affected rows. Transactions throw. DELETE without conditions
gets the WHERE ClickHouse requires.
- Any other query string parameter of the connection URL becomes a
ClickHouse setting. parse_connection_url() now parses the query string
properly: explode('/&/') never split more than one parameter.
Model::insert_all($rows, ['batch_size' => N]) inserts many records with
multi-row INSERT statements on every adapter. It builds no models, so
validations and callbacks do not run. Aliases and inflected names are
resolved, dates and JSON converted, timestamps filled in and missing
keys generated as save() would. Rows setting the same columns share
statements, and batches shrink to fit the bound parameters the database
accepts. ClickHouse writes the rows as literals and sends batches of
1000 rows or more synchronously even on an async connection.
The core also copes with tables without a primary key: Model::insert()
and Table no longer read a missing pk[0].
New tests: ClickhouseAdapterTest and ClickhouseModelTest with their own
schema and models, InsertAllTest for every adapter, UuidTest. CI runs
ClickHouse 26.8 in every job and 25.9 in one row; compose.yaml has both.
Verified on MySQL 5.7 and 8.4, PostgreSQL 18, SQLite, ClickHouse 25.9
and 26.8.
PHP 8.5 deprecates ReflectionMethod::setAccessible(), which has done nothing since 8.1. The test suite needs PHP 8.2 (PHPUnit 11.5), so the call is never needed where the tests can run.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a ClickHouse adapter for analytics workloads (ClickHouse 25.9 or later) and a bulk insert API,
Model::insert_all(), that works on every adapter.ClickHouse adapter
The adapter talks to the HTTP interface through curl, not PDO. It keeps one keep-alive connection per adapter, asks for compressed responses and sends settings as URL parameters. The minimum is 25.9 because that is the first release with lightweight
UPDATEon by default.TabSeparatedWithNamesAndTypes.fetch()and casts values from their ClickHouse type. Integers beyondPHP_INT_MAX,Int128/256andDecimalstay strings;Array/Map/Tuplebecome PHP arrays.?inside quotes, quoted identifiers or comments is left alone. Numeric strings stay quoted, because an unquoted integer beyond Int64 is read as a lossy Float64.system.columns. The sorting key is not taken as the primary key, because it is not unique: the key is a column namedidor the model's$primary_key.MATERIALIZED/ALIAScolumns are left out.UInt128key (alsoInt128,UInt256,Int256,UUIDorString) gets a UUIDv7 generated in PHP on insert. The newActiveRecord\Uuidclass generates them and converts between the decimal the model holds and UUID text.UInt128sorts numerically, soORDER BY idfollows creation order; theUUIDtype would not.session_timezoneis not used, because before 26.2INSERTand lightweightUPDATEdid not agree on how it applies.save()(about 2 ms) and errors are reported.async_insert=1. It keepswait_for_async_insert=1, as ClickHouse recommends, so failed inserts are never lost silently.update_all()anddelete_all()returnnull, because ClickHouse does not report affected rows.DELETEwithout conditions gets theWHEREClickHouse requires. Transactions throw.Model::insert_all()Model::insert_all($rows, ['batch_size' => N])inserts many records with multi-rowINSERTstatements on MySQL, PostgreSQL, SQLite and ClickHouse.save()would generate them.Core fixes
parse_connection_url()parses the query string properly.explode('/&/')never split more than one parameter.Model::insert()andTablecope with tables without a primary key, instead of reading a missingpk[0].Tests
ClickhouseAdapterTestandClickhouseModelTest, with their own schema (test/sql/clickhouse.sql) and models (test/models/Clickhouse). The shared fixtures rely on auto-increment keys and transactions, which ClickHouse does not have.InsertAllTestruns against the default adapter.UuidTestneeds no database.compose.yamlhas both versions.Verified locally: 994 tests green on MySQL 5.7 and 8.4, PostgreSQL 18 and SQLite, with each of mysql, pgsql and sqlite as the default adapter, against ClickHouse 25.9 and 26.8. phpcs is clean.