Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
206 changes: 206 additions & 0 deletions labs/GSP1288/HASHNODE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,206 @@
# Configuring Vector Search in Spanner - GSP1288

## Overview

This Google Skills lab uses an existing Spanner instance and a Google-hosted embedding model to demonstrate semantic product search. The exercise creates a remote model, loads ten product records, generates vectors, and ranks available products by cosine distance. This guide is independently written from the publicly accessible task requirements.

### What you'll do

- Configure a remote embeddings model in a lab-provisioned Spanner database.
- Create a product schema and insert ten product records.
- Authorize the Spanner service agent to call the model and generate embeddings.
- Run a similarity query and verify the four graded checkpoints.

## Setup and requirements

### Before you click the Start Lab button

Open the [official GSP1288 lab](https://www.skills.google/focuses/119411). Use a private browsing window and sign in with the temporary lab account shown in the lab panel. Do not use your personal Google Cloud account, and do not publish the temporary username, password, project ID, or any access token. The lab is timed and resources are temporary.

The lab provides a Spanner instance named `cymbal-retail` and a GoogleSQL database named `cymbal-bicycles`. Wait for these resources to become ready. The model name and region must match your lab's current configuration.

### Activate Cloud Shell

Open Cloud Shell in the Google Cloud console, authorize it using the lab account, and confirm the active project:

```bash
gcloud config get-value project
gcloud spanner instances describe cymbal-retail --format='value(config)'
gcloud spanner databases describe cymbal-bicycles --instance=cymbal-retail
```

Set variables from your active project and lab configuration. The sample model identifier below can be overridden if the lab displays another value:

```bash
export PROJECT_ID="$(gcloud config get-value project)"
export INSTANCE=cymbal-retail
export DATABASE=cymbal-bicycles
export REGION="$(gcloud spanner instances describe "$INSTANCE" --format='value(config)' | sed -nE 's/.*regional-([a-z]+-[a-z0-9]+[0-9]).*/\1/p')"
export MODEL_ID="${MODEL_ID:-gemini-embedding-001}"
printf 'Project: %s | Region: %s | Model: %s\n' "$PROJECT_ID" "$REGION" "$MODEL_ID"
```

If the instance is multi-regional or the displayed region differs, set `REGION` to the lab-provided Agent Platform endpoint location. Do not guess the region.

## Task 1. Create an embeddings model in Spanner and configure it to an Agent Platform model endpoint

In the Google Cloud console, open Spanner, select the `cymbal-retail` instance and `cymbal-bicycles` database, then open Spanner Studio. Create a new SQL tab. The remote model passes text to the Google model and returns a vector and statistics.

From Cloud Shell, the equivalent DDL is:

```bash
gcloud spanner databases ddl update "$DATABASE" --instance="$INSTANCE" --project="$PROJECT_ID" --ddl="
CREATE MODEL EmbeddingsModel
INPUT(content STRING(MAX),)
OUTPUT(embeddings STRUCT<statistics STRUCT<truncated BOOL, token_count FLOAT32>, values ARRAY<FLOAT32>>,)
REMOTE OPTIONS(endpoint='//aiplatform.googleapis.com/projects/$PROJECT_ID/locations/$REGION/publishers/google/models/$MODEL_ID')"
```

Only create the model if it does not already exist; rerunning the same `CREATE MODEL` statement produces an already-exists error.

### Test completed task

Use the first Google Skills **Check my progress** checkpoint to verify creation of `EmbeddingsModel`. Check its DDL using:

```bash
gcloud spanner databases ddl describe "$DATABASE" --instance="$INSTANCE" --project="$PROJECT_ID"
```

## Task 2. Create a new table and load products data

Create the products table in Spanner Studio or with the CLI. The `ARRAY<FLOAT32>` column will hold the model output; the timestamp column allows commit timestamps.

```sql
CREATE TABLE products (
categoryId INT64 NOT NULL,
productId INT64 NOT NULL,
productName STRING(MAX) NOT NULL,
productDescription STRING(MAX) NOT NULL,
productDescriptionEmbedding ARRAY<FLOAT32>,
createTime TIMESTAMP NOT NULL OPTIONS (allow_commit_timestamp = true),
inventoryCount INT64 NOT NULL,
priceInCents INT64
) PRIMARY KEY(categoryId, productId);
```

To run this from Cloud Shell, save the DDL as `products.sql` and use `gcloud spanner databases ddl update "$DATABASE" --instance="$INSTANCE" --ddl-file=products.sql`.

Load ten bicycle-related products. The independently authored sample below uses the same ten product identities and inventory fields as the lab, with shorter original descriptions rather than reproducing copyrighted source prose. For exact reference data, consult the official lab. Execute in Spanner Studio:

```sql
INSERT INTO products (categoryId,productId,productName,productDescription,createTime,inventoryCount,priceInCents)
VALUES
(1,1,'Cymbal Helios Helmet','Adjustable youth helmet with vents and a comfortable protective fit.',PENDING_COMMIT_TIMESTAMP(),100,10999),
(1,2,'Cymbal Sprout','Small balance bicycle designed for toddlers aged two to four.',PENDING_COMMIT_TIMESTAMP(),10,13999),
(1,3,'Cymbal Spark Jr.','First pedal bicycle for young riders learning confident handling.',PENDING_COMMIT_TIMESTAMP(),34,13900),
(1,4,'Cymbal Summit','Mountain bicycle with suspension for rugged off road trails.',PENDING_COMMIT_TIMESTAMP(),0,79999),
(1,5,'Cymbal Breeze','Electric commuter bicycle with assisted pedaling and battery power.',PENDING_COMMIT_TIMESTAMP(),72,129999),
(1,6,'Cymbal Trailblazer Backpack','Weather resistant backpack for cycling tools and supplies.',PENDING_COMMIT_TIMESTAMP(),24,7999),
(1,7,'Cymbal Phoenix Lights','Front and rear bicycle lights for visibility after sunset.',PENDING_COMMIT_TIMESTAMP(),87,3999),
(1,8,'Cymbal Windstar Pump','Compact portable bicycle tire pump for quick inflation.',PENDING_COMMIT_TIMESTAMP(),36,24999),
(1,9,'Cymbal Odyssey Multi-Tool','Pocket tool kit with screwdrivers and bicycle repair keys.',PENDING_COMMIT_TIMESTAMP(),52,999),
(1,10,'Cymbal Nomad Water Bottle','Reusable leak resistant bottle for hydration on rides.',PENDING_COMMIT_TIMESTAMP(),42,1299);
```

Confirm the table contains ten rows:

```sql
SELECT COUNT(*) AS product_count FROM products WHERE categoryId=1;
```

### Test completed task

Use the second Google Skills checkpoint after the table and ten records exist. Do not insert duplicates on a rerun; the automated script uses per-product existence checks.

## Task 3. Generate and store text embeddings for the products data

Find the Google Cloud project number and grant the lab-specified **Cloud Spanner API Service Agent** role to the Spanner-managed service agent. This permission is specifically required for the model endpoint integration.

```bash
PROJECT_NUMBER="$(gcloud projects describe "$PROJECT_ID" --format='value(projectNumber)')"
SPANNER_AGENT="service-${PROJECT_NUMBER}@gcp-sa-spanner.iam.gserviceaccount.com"
gcloud projects add-iam-policy-binding "$PROJECT_ID" \
--member="serviceAccount:$SPANNER_AGENT" \
--role='roles/spanner.serviceAgent' --condition=None
```

After IAM propagation, run the following SQL in Spanner Studio:

```sql
UPDATE products p1
SET productDescriptionEmbedding = (
SELECT embeddings.values FROM ML.PREDICT(
MODEL EmbeddingsModel,
(SELECT productDescription AS content FROM products p2 WHERE p2.productId=p1.productId)
)
)
WHERE categoryId=1 AND productDescriptionEmbedding IS NULL;
```

Check for ten populated vectors:

```sql
SELECT COUNT(*) AS embedded_products
FROM products
WHERE categoryId=1 AND productDescriptionEmbedding IS NOT NULL;
```

If model invocation returns a permission error, wait briefly for IAM propagation and retry. Verify that the model ID and region are correct before changing permissions.

### Test completed task

Run the third Google Skills checkpoint after the product embeddings have been stored.

## Task 4. Perform vector search using text embeddings in Spanner

Generate an embedding for a search phrase and rank products by cosine distance. Filter out unavailable inventory and return the five closest matches:

```sql
SELECT productName, productDescription, inventoryCount,
COSINE_DISTANCE(productDescriptionEmbedding, (
SELECT embeddings.values
FROM ML.PREDICT(MODEL EmbeddingsModel,
(SELECT 'starter bicycle for a three year old child' AS content))
)) AS distance
FROM products
WHERE inventoryCount > 0 AND productDescriptionEmbedding IS NOT NULL
ORDER BY distance
LIMIT 5;
```

Smaller distances indicate closer semantic similarity. The balance bicycle should be relevant to a toddler search, although the exact ranking may vary with the embedding model and sample descriptions.

### Test completed task

Run the fourth Google Skills checkpoint. The automated script performs a final count of total rows and non-null vectors, but cannot invoke the grader itself.

---

## Solution of Lab

### Quick

Run these two commands **only after the Draft PR has been merged into `main`**. Until then the main-branch URL may return 404.

```bash
curl -LO https://raw.githubusercontent.com/ePlus-DEV/storage/refs/heads/main/labs/GSP1288/lab.sh
source lab.sh
```

The script auto-detects the active project and Spanner region, honors `NO_COLOR` and `FORCE_COLOR=1`, checks for existing schema/data, and retries transient failures. Override `MODEL_ID` or `REGION` only when your lab configuration requires it.

### Manual

Chưa tìm thấy video YouTube hướng dẫn thủ công đã xác minh cho lab này.

The full manual task sequence and SQL commands are provided above; a video is optional and does not replace the lab instructions.

## Sources and verification

- [Official Google Skills GSP1288](https://www.skills.google/focuses/119411) — official task and checkpoint order.
- [Google Codelab: Getting started with Spanner Vector Search](https://codelabs.developers.google.com/spanner-getting-started-vector-search) — related technical background, not a substitute for the lab.
- [Spanner DML via gcloud](https://docs.cloud.google.com/spanner/docs/dml-tasks) and [Spanner DDL CLI](https://docs.cloud.google.com/sdk/gcloud/reference/spanner/databases/ddl/update).

Image assets from the Google Skills page have not been embedded because reuse and public hotlink permissions have not been verified. Consult the linked official page for original screenshots.

**Verification status:** static validated only; runtime/grader unverified. No Google Cloud lab credentials were used, and no grader result is claimed.
115 changes: 115 additions & 0 deletions labs/GSP1288/lab.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
#!/usr/bin/env bash
# Copyright (c) 2026 ePlus.DEV | Unlicense
# GSP1288 - Configuring Vector Search in Spanner
# Static validation only; requires an authorized, active Google Skills lab.
set -Eeuo pipefail

if [[ -z "${NO_COLOR:-}" ]] && { [[ -t 1 ]] || [[ "${FORCE_COLOR:-0}" == "1" ]]; }; then
GREEN=$'\033[0;32m'; CYAN=$'\033[0;36m'; YELLOW=$'\033[0;33m'; RED=$'\033[0;31m'; RESET=$'\033[0m'
else
GREEN=''; CYAN=''; YELLOW=''; RED=''; RESET=''
fi
info() { printf '%s▶ %s%s\n' "$CYAN" "$*" "$RESET"; }
ok() { printf '%s✔ %s%s\n' "$GREEN" "$*" "$RESET"; }
warn() { printf '%s! %s%s\n' "$YELLOW" "$*" "$RESET" >&2; }
die() { printf '%s✖ %s%s\n' "$RED" "$*" "$RESET" >&2; exit 1; }
trap 'die "Stopped near line $LINENO. Inspect the preceding command."' ERR
printf '%s\n' "${CYAN}ePlus.DEV – Starting Execution${RESET}"
printf '%s\n' "Copyright (c) 2026 ePlus.DEV | Unlicense"
command -v gcloud >/dev/null || die 'Google Cloud CLI is required.'
command -v python3 >/dev/null || die 'Python 3 is required.'

PROJECT_ID="${PROJECT_ID:-${GOOGLE_CLOUD_PROJECT:-$(gcloud config get-value project 2>/dev/null || true)}}"
[[ -n "$PROJECT_ID" && "$PROJECT_ID" != "(unset)" ]] || die 'No active project. Set PROJECT_ID.'
INSTANCE="${INSTANCE:-cymbal-retail}"
DATABASE="${DATABASE:-cymbal-bicycles}"
gcloud spanner instances describe "$INSTANCE" --project="$PROJECT_ID" --format='value(name)' >/dev/null ||
die "Lab-provisioned Spanner instance $INSTANCE not found in $PROJECT_ID."
gcloud spanner databases describe "$DATABASE" --instance="$INSTANCE" --project="$PROJECT_ID" --format='value(name)' >/dev/null ||
die "Lab-provisioned database $DATABASE not found."
CONFIG="$(gcloud spanner instances describe "$INSTANCE" --project="$PROJECT_ID" --format='value(config)' 2>/dev/null || true)"
SPANNER_REGION="$(printf '%s' "$CONFIG" | sed -nE 's/.*regional-([a-z]+-[a-z0-9]+[0-9]).*/\1/p')"
META_REGION="$(gcloud compute project-info describe --project="$PROJECT_ID" --format=json 2>/dev/null | python3 -c 'import json,sys; d=json.load(sys.stdin); print(next((i.get("value","") for i in d.get("commonInstanceMetadata",{}).get("items",[]) if i.get("key")=="google-compute-default-region"),""))' 2>/dev/null || true)"
REGION="${REGION:-${META_REGION:-${SPANNER_REGION:-$(gcloud config get-value compute/region 2>/dev/null || true)}}}"
[[ -n "$REGION" && "$REGION" != "(unset)" ]] || die 'Unable to determine region; set REGION from the lab details.'
META_ZONE="$(gcloud compute project-info describe --project="$PROJECT_ID" --format=json 2>/dev/null | python3 -c 'import json,sys; d=json.load(sys.stdin); print(next((i.get("value","") for i in d.get("commonInstanceMetadata",{}).get("items",[]) if i.get("key")=="google-compute-default-zone"),""))' 2>/dev/null || true)"
ZONE="${ZONE:-${META_ZONE:-$(gcloud config get-value compute/zone 2>/dev/null || true)}}"
if [[ "$ZONE" == "(unset)" ]]; then ZONE=''; fi
# Spanner is regional/multi-regional and does not consume ZONE; it is logged for diagnostics.
MODEL_ID="${MODEL_ID:-gemini-embedding-001}"
[[ "$MODEL_ID" =~ ^[a-zA-Z0-9._-]+$ ]] || die 'Invalid MODEL_ID.'
[[ "$REGION" =~ ^[a-z]+-[a-z0-9]+[0-9]$ ]] || die 'Invalid REGION.'
info "Project: $PROJECT_ID | Spanner: $INSTANCE/$DATABASE | Region: $REGION | Zone: ${ZONE:-n/a} | Model: $MODEL_ID"

run_sql() {
gcloud spanner databases execute-sql "$DATABASE" --instance="$INSTANCE" --project="$PROJECT_ID" --sql="$1"
}
run_ddl() {
gcloud spanner databases ddl update "$DATABASE" --instance="$INSTANCE" --project="$PROJECT_ID" --ddl="$1" --quiet
}
retry() {
local attempt
for attempt in 1 2 3 4; do
if "$@"; then return 0; fi
if (( attempt == 4 )); then return 1; fi
warn "Attempt $attempt failed; retrying in 12 seconds..."
sleep 12
done
}
ddl_contains() {
gcloud spanner databases ddl describe "$DATABASE" --instance="$INSTANCE" --project="$PROJECT_ID" --format=json |
grep -Eiq "$1"
}

info 'Task 1/4 — Create the remote embeddings model.'
if ddl_contains 'CREATE MODEL EmbeddingsModel'; then
ok 'EmbeddingsModel already exists.'
else
retry run_ddl "CREATE MODEL EmbeddingsModel INPUT(content STRING(MAX),) OUTPUT(embeddings STRUCT<statistics STRUCT<truncated BOOL, token_count FLOAT32>, values ARRAY<FLOAT32>>,) REMOTE OPTIONS(endpoint = '//aiplatform.googleapis.com/projects/$PROJECT_ID/locations/$REGION/publishers/google/models/$MODEL_ID')" ||
die 'Failed to create EmbeddingsModel.'
fi

info 'Task 2/4 — Create the products table and load 10 original sample records.'
if ddl_contains 'CREATE TABLE products'; then
ok 'Products table already exists.'
else
retry run_ddl 'CREATE TABLE products (categoryId INT64 NOT NULL, productId INT64 NOT NULL, productName STRING(MAX) NOT NULL, productDescription STRING(MAX) NOT NULL, productDescriptionEmbedding ARRAY<FLOAT32>, createTime TIMESTAMP NOT NULL OPTIONS (allow_commit_timestamp = true), inventoryCount INT64 NOT NULL, priceInCents INT64) PRIMARY KEY(categoryId, productId)' ||
die 'Failed to create products table.'
fi
# Independently written descriptions, not copied from the lab or codelab.
rows=(
'1|Cymbal Helios Helmet|Adjustable youth helmet with vents and a comfortable protective fit.|100|10999'
'2|Cymbal Sprout|Small balance bicycle designed for toddlers aged two to four.|10|13999'
'3|Cymbal Spark Jr.|First pedal bicycle for young riders learning confident handling.|34|13900'
'4|Cymbal Summit|Mountain bicycle with suspension for rugged off road trails.|0|79999'
'5|Cymbal Breeze|Electric commuter bicycle with assisted pedaling and battery power.|72|129999'
'6|Cymbal Trailblazer Backpack|Weather resistant backpack for cycling tools and supplies.|24|7999'
'7|Cymbal Phoenix Lights|Front and rear bicycle lights for visibility after sunset.|87|3999'
'8|Cymbal Windstar Pump|Compact portable bicycle tire pump for quick inflation.|36|24999'
'9|Cymbal Odyssey Multi-Tool|Pocket tool kit with screwdrivers and bicycle repair keys.|52|999'
'10|Cymbal Nomad Water Bottle|Reusable leak resistant bottle for hydration on rides.|42|1299'
)
for record in "${rows[@]}"; do
IFS='|' read -r product_id name description inventory price <<< "$record"
sql="INSERT INTO products (categoryId,productId,productName,productDescription,createTime,inventoryCount,priceInCents) SELECT 1,$product_id,'$name','$description',PENDING_COMMIT_TIMESTAMP(),$inventory,$price WHERE NOT EXISTS (SELECT 1 FROM products WHERE categoryId=1 AND productId=$product_id)"
retry run_sql "$sql" >/dev/null || die "Failed to load product $product_id."
done
ok '10 product keys inserted or already present.'

info 'Task 3/4 — Grant Spanner service-agent role and generate embeddings.'
PROJECT_NUMBER="$(gcloud projects describe "$PROJECT_ID" --format='value(projectNumber)')"
[[ "$PROJECT_NUMBER" =~ ^[0-9]+$ ]] || die 'Could not resolve project number.'
SERVICE_AGENT="service-$PROJECT_NUMBER@gcp-sa-spanner.iam.gserviceaccount.com"
retry gcloud projects add-iam-policy-binding "$PROJECT_ID" \
--member="serviceAccount:$SERVICE_AGENT" --role='roles/spanner.serviceAgent' --condition=None --quiet >/dev/null ||
die 'Cannot grant the documented Spanner service-agent role.'
retry run_sql 'UPDATE products p1 SET productDescriptionEmbedding = (SELECT embeddings.values FROM ML.PREDICT(MODEL EmbeddingsModel, (SELECT productDescription AS content FROM products p2 WHERE p2.productId=p1.productId))) WHERE categoryId=1 AND productDescriptionEmbedding IS NULL' ||
die 'Embedding update failed; verify model ID, IAM propagation and region.'

info 'Task 4/4 — Run semantic similarity search.'
retry run_sql "SELECT productName,inventoryCount,COSINE_DISTANCE(productDescriptionEmbedding,(SELECT embeddings.values FROM ML.PREDICT(MODEL EmbeddingsModel,(SELECT 'starter bicycle for a three year old child' AS content)))) AS distance FROM products WHERE inventoryCount>0 AND productDescriptionEmbedding IS NOT NULL ORDER BY distance LIMIT 5" ||
die 'Similarity query failed.'
info 'Read-back: check the number of products and non-null embeddings.'
run_sql 'SELECT COUNT(*) AS products,COUNTIF(productDescriptionEmbedding IS NOT NULL) AS embedded FROM products WHERE categoryId=1'
ok 'Finished SQL workflow. Check each Google Skills grader checkpoint manually.'
warn 'static validated only; runtime/grader unverified.'
Loading