Home

JQuick-SQL Wiki

Welcome to the JQuick-SQL documentation.

JQuick-SQL is a lightweight, embeddable federated SQL query engine for Java. It addresses a common challenge: how to use a unified SQL interface to perform federated analysis across data scattered across heterogeneous data sources, such as different databases, in-memory collections, or even REST APIs.

The engine follows a classic Coordinator-Worker architecture and includes a built-in query optimizer that supports predicate pushdown, projection pushdown, and join reordering. This allows for powerful data federation capabilities even in lightweight deployments.

✨ Key Features

Feature Area	Capabilities
SQL Support	Standard SQL: `SELECT`, `WHERE`, `GROUP BY/HAVING`, `ORDER BY`, `LIMIT/OFFSET`; `INNER/LEFT/RIGHT/FULL/CROSS JOIN`; `UNION/UNION ALL`, Subqueries, Recursive CTE
Aggregations	`COUNT`, `SUM`, `AVG`, `MIN`, `MAX` with two-phase aggregation (Partial → Shuffle → Final)
Query Optimization	Predicate Pushdown, Projection Pushdown, Join Reordering
Execution Modes	Embedded (in-memory) and Distributed (Coordinator-Worker) with gRPC-based data exchange
Data Types	Integer, Float, String, Boolean, Date, and support for arithmetic/logical expressions and functions (e.g., `toUpper`)

🏗️ Architecture

JQuick-SQL follows a classic database architecture, transforming a SQL statement into an executable plan through a clear pipeline. SQL Input → Parser → AST → Logical Plan → Optimizer → Physical Plan → Fragmenter → Coordinator → Workers (gRPC) → Result (DataSet)

Parser: Converts SQL text into an Abstract Syntax Tree (AST).
Logical Plan & Optimizer: Transforms the AST into a logical plan and applies optimization rules (e.g., Predicate Pushdown, Projection Pushdown, Join Reordering).
Physical Plan: Generates an executable physical execution plan.
Fragmenter: Splits the physical plan into distributable task fragments.
Coordinator-Worker: Manages task distribution and data exchange using a Master-Worker architecture over gRPC.

🚀 Quick Start

1. Add Maven Dependency

<dependency>
    <groupId>io.github.paohaijiao</groupId>
    <artifactId>jquick-sql</artifactId>
    <version>${latest.version}</version>
</dependency>

2. Create Engine and Register Tables

You can register tables programmatically, similar to using an in-memory database.

import io.github.paohaijiao.jquick.sql.*;

// 1. Create the embedded SQL engine
JQuickSQL sql = JQuickSQL.embedded();

// 2. Prepare table schema and data
List<JQuickColumnMeta> columns = Arrays.asList(
    new JQuickColumnMeta("id", Integer.class, "users"),
    new JQuickColumnMeta("name", String.class, "users"),
    new JQuickColumnMeta("age", Integer.class, "users")
);

List<JQuickRow> rows = Arrays.asList(
    createRow("id", 1, "name", "Alice", "age", 25),
    createRow("id", 2, "name", "Bob", "age", 30)
);

// 3. Register the table
sql.registerTable("users", columns, rows);

3. Execute SQL Queries

// Execute a SQL query
JQuickDataSet result = sql.execute("SELECT * FROM users WHERE age > 25");
result.printTable();

// Shutdown the engine
sql.shutdown();

📊 SQL Syntax Examples

Basic Queries and Expressions

-- Use functions and expressions
SELECT id, toUpper(name) as upperName, (age+1)*3 as age FROM users;

-- Use CASE WHEN logic
SELECT id, name, age, 
  CASE WHEN age >= 30 THEN 'Middle-aged' 
       WHEN age >= 20 THEN 'Young' 
       ELSE 'Teen' END AS age_group 
FROM users;

Filtering (WHERE Clause)

Supports various conditions, including BETWEEN, IN, LIKE, REGEXP, and EXISTS subqueries.

-- Logical operations
SELECT * FROM users WHERE age > 25 AND status = 'active';

-- Subquery with EXISTS
SELECT * FROM users u 
WHERE EXISTS (SELECT 1 FROM orders o WHERE o.user_id = u.id);

Aggregation and Grouping

SELECT status, COUNT(*) as count, AVG(age) as avg_age 
FROM users 
GROUP BY status 
HAVING COUNT(age) > 1;

JOIN Operations

-- INNER JOIN
SELECT u.id, u.name, o.amount 
FROM users u INNER JOIN orders o ON u.id = o.user_id;

-- LEFT JOIN
SELECT u.id, u.name, o.amount 
FROM users u LEFT JOIN orders o ON u.id = o.user_id;

🔧 API Reference

Core Classes

Class	Description
`JQuickSQL`	The main entry point for creating engine instances, registering tables, and executing SQL.
`JQuickSQL.Builder`	A builder class for constructing `JQuickSQL` instances with custom configurations.
`JQuickDataSet`	The result set returned by `execute()`, supporting iteration and printing.

JQuickSQL Core Methods

// Create an embedded engine
JQuickSQL sql = JQuickSQL.embedded();

// Use Builder for advanced configuration
JQuickSQL sql = JQuickSQL.builder()
    .embedded(2)              // Set number of workers
    .parallelism(2)           // Set parallelism
    .table("products", columns, rows)  // Register tables
    .build();

// Register a table
sql.registerTable("users", columns, rows);

// Execute a query
JQuickDataSet result = sql.execute("SELECT * FROM users");

// Shutdown the engine
sql.shutdown();

🎯 Use Cases

Embedded Data Analysis: Perform complex SQL queries on in-memory datasets within a Java application.
Federated Queries Across Sources: Unify querying across different data sources, with potential for Calcite-like model configuration in the future.
Lightweight BI Reporting: Provide SQL-based analysis for small to medium-sized datasets.

📄 License

This project is licensed under the Apache License 2.0.

Navigation Menu

Repository navigation