Plan: Xano buildout
Status: Phases 1 to 3 are done; Phase 4 steps 1 to 4 are done (the generator, the four new tables, five columns on the consent log, two on the user table); steps 5 and 6 wait on real exports and the signer endpoint:
cargo run --bin xano-buildout -- --workspace <id> --data-home <home>, tested by tests/xano.rs. The remaining Phase 4 steps
change a live workspace and are applied one confirmed step at a time.
Everything is generated from the same map (site/ddic.json) as the Rust structs, the GraphQL schema and these pages, so the tables,
the CSV headers and the docs can’t drift apart.
What Phase 3 changed
The dry run against the target workspace found most of the consent stack already in place:
| Planned in Phase 1 | Already in the workspace | Now |
|---|---|---|
| A new consent ledger | An append-only consent log, hash-chained and signed, written by the CRM Sync worker | Reused. Add the missing columns; rows go through the worker’s signer so the chain stays valid. |
| A consent entitlement function | A release gate every Google and Shopify payload must pass: latest decision wins, a tie revokes, no decision means denied | Reused. One decision point. A second would disagree with it. |
| A customer table | The storefront user table the gate keys on, with erasure and opt-out tags | Reused. Add kunnr and shopify_customer_id. |
A --hold KR flag | A region policy table with a data home per country | Reused. A person whose data home isn’t the workspace’s is held. |
merchant_product, identity_link, dsr_request, dsr_receipt | Nothing equivalent | Created (step 2). |
One rule follows from how the gate works: it reads consent category marketing as all three ad signals (ad storage, ad user data,
ad personalization). So e-mail and SMS opt-ins are imported as categories newsletter (the worker’s own word for e-mail marketing) and sms, never marketing, and an
e-mail opt-in can never open Customer Match. Shopify’s sale opt-out becomes the user’s opt-out tag, which the gate treats as a
hard stop, not a consent row.
ECC export (DD03L + records) ─┐
Shopify customers (Admin API) ─┼─▶ xano-buildout ─▶ xano/tables/*.xs the 4 new tables
Region policy (snapshot) ─┘ ├▶ plan.json steps 2-6, with the column additions
├▶ csv/storefront_users.csv upserts, matched on e-mail
├▶ csv/consent_records.csv input for the consent signer
├▶ csv/identity_link.csv, merchant_product.csv
├▶ expected_release_decisions.csv what the gate should answer
└▶ report.md what was skipped or held, and why
Phase 4
| Step | What | Changes the workspace |
|---|---|---|
| 1 | Reshape the generator to the existing tables (this page) | No: built |
| 2 | Create merchant_product, identity_link, dsr_request, dsr_receipt (each with a primary id and a unique idempotency_key; the last three point at the user table). Optional columns are nullable, so a missing price is null, never 0. | Yes: done |
| 3 | Add to the consent log: idempotency_key (unique), seq, gpc, expires_at, evidence_sha256. All optional and nullable: existing rows have none of them. Region, regime and banner version are not added: the log already keeps them in its systems_notified JSON. The chain hash and signature cover a fixed set (subject, category, action, consent ID, time, previous hash), so new columns don’t disturb them, and aren’t tamper-evident either. | Yes: done |
| 4 | Add to the user table: kunnr, shopify_customer_id (nullable, sensitive) | Yes: done |
| 5 | Import: users as upserts on e-mail (a blank field is left as is, tags are merged); identity links and products as inserts; consent rows one call at a time through the signer, oldest first. Needs a new admin-only worker endpoint that anchors and signs a backfill row for a known user with its historical time: no existing endpoint does. | Yes |
| 6 | Verify against expected_release_decisions.csv. A live gate call writes a ledger row, so verification uses verify- idempotency keys, or is checked offline. | Only ledger rows |
Phase 5: the session-to-order join (planned)
SAP never sees a shopper’s visit: its sessions belong to logged-in SAP users. What reaches SAP is the order, so the visit has to ride on the order. The join carries the consent session and the ad click from the landing page to the SAP sales order, so a confirmed sale can be reported to Google Ads against the click that led to it, and only if the release gate allows it.
landing ─▶ consent banner ─▶ add to cart ──────────▶ checkout ─▶ Shopify order ─▶ middleware ─▶ SAP sales order
gclid session_id cart attributes: customAttributes: VBAK-BSTNK (or a Z field)
gbraid (consent row) _crm_session, _crm_click _crm_session, _crm_click = Shopify order ID
wbraid (only if ad_storage granted)
│ │
└─▶ visit_click (Xano): session_id → click IDs, landed_at ◀──── SAP confirms the sale ─────────────┘
└─▶ conversion endpoint: order → session → click
→ release gate (google_ads_conversion) → export
| Step | Where | What | Rule |
|---|---|---|---|
| 5.1 | CRM Sync worker, landing request | Read gclid / gbraid / wbraid from the landing URL; write one visit_click row: session_id, the click IDs, landed_at, region | Stored only if ad_storage is granted for the session at that moment. Otherwise nothing is kept: Consent Mode’s own signals cover the denied case. |
| 5.2 | Storefront, add to cart | Set cart attributes _crm_session (the consent session_id) and _crm_click (the visit_click row’s key, not the raw click ID) | Underscore-prefixed attributes are hidden from the shopper at checkout. Never a raw e-mail or click ID in the cart. |
| 5.3 | Shopify | Cart attributes become the order’s customAttributes | No code: Shopify copies them. |
| 5.4 | Middleware (order to SAP) | Keep writing the Shopify order ID to VBAK-BSTNK (or the Z field used today); carry nothing else | SAP holds the order reference only, never the session or the click. |
| 5.5 | CRM Sync worker, on SAP confirmation (sale, return, cancellation) | Look up the Shopify order’s _crm_click → visit_click → click IDs; call the existing conversion endpoint with the order ID, conversion_time from SAP, value and currency | The endpoint already runs the release gate (google_ads_conversion) and the region policy, and stores the order reference only as an HMAC. |
| 5.6 | Xano | Expire visit_click rows after Google’s click-ID lookback (90 days) | A click older than the window can’t be credited; keeping it serves nothing. |
What this needs that doesn’t exist yet: the visit_click table (generated with the others when the step is approved), the
worker’s landing and confirmation hooks, and the storefront’s cart attribute write. Nothing in it changes the consent log or the
release gate, and SAP needs no change beyond the order reference it already carries.
Acceptance:
- A session without
ad_storageleaves novisit_clickrow, and its order carries no_crm_click. - An order whose shopper withdrew ad user data after the click is refused by the gate at confirmation time, not at the click.
- A return or cancellation in SAP reaches Google as an adjustment against the same order reference.
- No raw e-mail, click ID or session ID appears in SAP, in the order’s visible attributes, or in the export.
The CSVs
| File | Rows from | Rules |
|---|---|---|
storefront_users.csv | ECC customer (KNA1 + default e-mail in ADR6) and Shopify customers | Matched on the normalised e-mail. One-time customer numbers are never a key. Deleted customers are left out. tags_add carries rejects_sale_of_data for Shopify’s dataSaleOptOut. |
consent_records.csv | Shopify emailMarketingConsent / smsMarketingConsent (with consentUpdatedAt); the ECC consent table | session_id (the consent log’s existing column) records the visit a decision came from; backfill rows have none, and today only the signed-in path writes it (the anonymous banner path writes it empty, which Phase 5 needs changed). Jurisdiction and regime go into systems_notified. Categories newsletter, sms; an ECC ads row becomes marketing only as a withdrawal: imports never grant ads. Rows without a time are skipped (the gate ignores them anyway). The ECC ZZ_MKT_OPTIN flag is never imported. |
identity_link.csv | People with both a KUNNR and a Shopify ID | user_email is resolved to the user’s ID on import. |
merchant_product.csv | The converter in this book over the ECC export | No personal data. |
Every row carries an idempotency key, so a replayed file writes nothing the second time. The CSVs, plan.json and report.md
are written to xano/ but not committed: from real exports they hold personal data.
On the fixtures the run writes 2 users, 1 identity link, 4 consent rows and 1 product, holds the KR customer (data home kr, not
the workspace’s us) with all of their rows, and reports the ECC flag, the ECC ads grant (its later withdrawal is imported), the
SMS state with no decision, and the sale opt-out turned into a tag.
Expected decisions
expected_release_decisions.csv is computed by an offline copy of the gate’s rules (src/release_gate.rs, tested on its own):
hard stops first (erasure; the opt-out tag refuses the ad signals), then per signal the latest timed decision wins, a tie
revokes, test sources are ignored, and no decision means denied. For the sample customer with an e-mail opt-in, Customer Match is
still refused: the opt-in is category newsletter, which covers no ad signal. The live gate is the authority; the copy follows it.
Open decisions
- Who runs the signer import (step 5), and from where.
- Retention per purpose and regime (sets
expires_at); the gate doesn’t check expiry yet. - Whether the gate should hold ad user data and ad personalization separately:
marketinggrants or revokes them together. - Whether B2B contact persons are in scope for the first import.
- Phase 5: which middleware carries the Shopify order ID into SAP today, and in which field (
BSTNKor a Z field), so the confirmation hook can find the order.