Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Plan: Xano buildout

Status: Phases 1 to 3 are done; Phase 4 steps 1 to 4 are done (the generator, the four new tables, five columns on the consent log, two on the user table); steps 5 and 6 wait on real exports and the signer endpoint: cargo run --bin xano-buildout -- --workspace <id> --data-home <home>, tested by tests/xano.rs. The remaining Phase 4 steps change a live workspace and are applied one confirmed step at a time.

Everything is generated from the same map (site/ddic.json) as the Rust structs, the GraphQL schema and these pages, so the tables, the CSV headers and the docs can’t drift apart.

What Phase 3 changed

The dry run against the target workspace found most of the consent stack already in place:

Planned in Phase 1Already in the workspaceNow
A new consent ledgerAn append-only consent log, hash-chained and signed, written by the CRM Sync workerReused. Add the missing columns; rows go through the worker’s signer so the chain stays valid.
A consent entitlement functionA release gate every Google and Shopify payload must pass: latest decision wins, a tie revokes, no decision means deniedReused. One decision point. A second would disagree with it.
A customer tableThe storefront user table the gate keys on, with erasure and opt-out tagsReused. Add kunnr and shopify_customer_id.
A --hold KR flagA region policy table with a data home per countryReused. A person whose data home isn’t the workspace’s is held.
merchant_product, identity_link, dsr_request, dsr_receiptNothing equivalentCreated (step 2).

One rule follows from how the gate works: it reads consent category marketing as all three ad signals (ad storage, ad user data, ad personalization). So e-mail and SMS opt-ins are imported as categories newsletter (the worker’s own word for e-mail marketing) and sms, never marketing, and an e-mail opt-in can never open Customer Match. Shopify’s sale opt-out becomes the user’s opt-out tag, which the gate treats as a hard stop, not a consent row.

ECC export (DD03L + records) ─┐
Shopify customers (Admin API) ─┼─▶ xano-buildout ─▶ xano/tables/*.xs                 the 4 new tables
Region policy (snapshot)      ─┘                  ├▶ plan.json                        steps 2-6, with the column additions
                                                  ├▶ csv/storefront_users.csv         upserts, matched on e-mail
                                                  ├▶ csv/consent_records.csv          input for the consent signer
                                                  ├▶ csv/identity_link.csv, merchant_product.csv
                                                  ├▶ expected_release_decisions.csv   what the gate should answer
                                                  └▶ report.md                        what was skipped or held, and why

Phase 4

StepWhatChanges the workspace
1Reshape the generator to the existing tables (this page)No: built
2Create merchant_product, identity_link, dsr_request, dsr_receipt (each with a primary id and a unique idempotency_key; the last three point at the user table). Optional columns are nullable, so a missing price is null, never 0.Yes: done
3Add to the consent log: idempotency_key (unique), seq, gpc, expires_at, evidence_sha256. All optional and nullable: existing rows have none of them. Region, regime and banner version are not added: the log already keeps them in its systems_notified JSON. The chain hash and signature cover a fixed set (subject, category, action, consent ID, time, previous hash), so new columns don’t disturb them, and aren’t tamper-evident either.Yes: done
4Add to the user table: kunnr, shopify_customer_id (nullable, sensitive)Yes: done
5Import: users as upserts on e-mail (a blank field is left as is, tags are merged); identity links and products as inserts; consent rows one call at a time through the signer, oldest first. Needs a new admin-only worker endpoint that anchors and signs a backfill row for a known user with its historical time: no existing endpoint does.Yes
6Verify against expected_release_decisions.csv. A live gate call writes a ledger row, so verification uses verify- idempotency keys, or is checked offline.Only ledger rows

Phase 5: the session-to-order join (planned)

SAP never sees a shopper’s visit: its sessions belong to logged-in SAP users. What reaches SAP is the order, so the visit has to ride on the order. The join carries the consent session and the ad click from the landing page to the SAP sales order, so a confirmed sale can be reported to Google Ads against the click that led to it, and only if the release gate allows it.

landing ─▶ consent banner ─▶ add to cart ──────────▶ checkout ─▶ Shopify order ─▶ middleware ─▶ SAP sales order
gclid       session_id        cart attributes:                    customAttributes:              VBAK-BSTNK (or a Z field)
gbraid      (consent row)     _crm_session, _crm_click            _crm_session, _crm_click       = Shopify order ID
wbraid                         (only if ad_storage granted)
   │                                                                                                  │
   └─▶ visit_click (Xano): session_id → click IDs, landed_at ◀──── SAP confirms the sale ─────────────┘
                                                                     └─▶ conversion endpoint: order → session → click
                                                                         → release gate (google_ads_conversion) → export
StepWhereWhatRule
5.1CRM Sync worker, landing requestRead gclid / gbraid / wbraid from the landing URL; write one visit_click row: session_id, the click IDs, landed_at, regionStored only if ad_storage is granted for the session at that moment. Otherwise nothing is kept: Consent Mode’s own signals cover the denied case.
5.2Storefront, add to cartSet cart attributes _crm_session (the consent session_id) and _crm_click (the visit_click row’s key, not the raw click ID)Underscore-prefixed attributes are hidden from the shopper at checkout. Never a raw e-mail or click ID in the cart.
5.3ShopifyCart attributes become the order’s customAttributesNo code: Shopify copies them.
5.4Middleware (order to SAP)Keep writing the Shopify order ID to VBAK-BSTNK (or the Z field used today); carry nothing elseSAP holds the order reference only, never the session or the click.
5.5CRM Sync worker, on SAP confirmation (sale, return, cancellation)Look up the Shopify order’s _crm_click → visit_click → click IDs; call the existing conversion endpoint with the order ID, conversion_time from SAP, value and currencyThe endpoint already runs the release gate (google_ads_conversion) and the region policy, and stores the order reference only as an HMAC.
5.6XanoExpire visit_click rows after Google’s click-ID lookback (90 days)A click older than the window can’t be credited; keeping it serves nothing.

What this needs that doesn’t exist yet: the visit_click table (generated with the others when the step is approved), the worker’s landing and confirmation hooks, and the storefront’s cart attribute write. Nothing in it changes the consent log or the release gate, and SAP needs no change beyond the order reference it already carries.

Acceptance:

  • A session without ad_storage leaves no visit_click row, and its order carries no _crm_click.
  • An order whose shopper withdrew ad user data after the click is refused by the gate at confirmation time, not at the click.
  • A return or cancellation in SAP reaches Google as an adjustment against the same order reference.
  • No raw e-mail, click ID or session ID appears in SAP, in the order’s visible attributes, or in the export.

The CSVs

FileRows fromRules
storefront_users.csvECC customer (KNA1 + default e-mail in ADR6) and Shopify customersMatched on the normalised e-mail. One-time customer numbers are never a key. Deleted customers are left out. tags_add carries rejects_sale_of_data for Shopify’s dataSaleOptOut.
consent_records.csvShopify emailMarketingConsent / smsMarketingConsent (with consentUpdatedAt); the ECC consent tablesession_id (the consent log’s existing column) records the visit a decision came from; backfill rows have none, and today only the signed-in path writes it (the anonymous banner path writes it empty, which Phase 5 needs changed). Jurisdiction and regime go into systems_notified. Categories newsletter, sms; an ECC ads row becomes marketing only as a withdrawal: imports never grant ads. Rows without a time are skipped (the gate ignores them anyway). The ECC ZZ_MKT_OPTIN flag is never imported.
identity_link.csvPeople with both a KUNNR and a Shopify IDuser_email is resolved to the user’s ID on import.
merchant_product.csvThe converter in this book over the ECC exportNo personal data.

Every row carries an idempotency key, so a replayed file writes nothing the second time. The CSVs, plan.json and report.md are written to xano/ but not committed: from real exports they hold personal data.

On the fixtures the run writes 2 users, 1 identity link, 4 consent rows and 1 product, holds the KR customer (data home kr, not the workspace’s us) with all of their rows, and reports the ECC flag, the ECC ads grant (its later withdrawal is imported), the SMS state with no decision, and the sale opt-out turned into a tag.

Expected decisions

expected_release_decisions.csv is computed by an offline copy of the gate’s rules (src/release_gate.rs, tested on its own): hard stops first (erasure; the opt-out tag refuses the ad signals), then per signal the latest timed decision wins, a tie revokes, test sources are ignored, and no decision means denied. For the sample customer with an e-mail opt-in, Customer Match is still refused: the opt-in is category newsletter, which covers no ad signal. The live gate is the authority; the copy follows it.

Open decisions

  • Who runs the signer import (step 5), and from where.
  • Retention per purpose and regime (sets expires_at); the gate doesn’t check expiry yet.
  • Whether the gate should hold ad user data and ad personalization separately: marketing grants or revokes them together.
  • Whether B2B contact persons are in scope for the first import.
  • Phase 5: which middleware carries the Shopify order ID into SAP today, and in which field (BSTNK or a Z field), so the confirmation hook can find the order.