Skip to main content

QRY Nexus: Your Data, as a Product

Status: Production Ready

Ever wish you could just hand someone a clean, curated dataset without them needing to understand your 47-table schema, figure out the right JOINs, or ask you "which column is the customer ID again?" for the hundredth time?

QRY Nexus turns your raw data assets into data products — curated, documented, quality-checked datasets that consumers can discover, access, and even query via REST API. Think of it as your organization's internal data marketplace, minus the six-month procurement process.

What's Inside​

ComponentWhat it doesThink of it as...
Data ProductsCurated datasets with quality contractsYour data's product catalog
API GatewayREST API for external appsA vending machine for data
DiscoverUnified search across all data assetsGoogle, but for your company's data

Data Products​

The Problem​

You have 3,000 tables in your data warehouse. Someone from marketing asks for "customer data." Which table? customers? customer_dim? stg_customer_v2_final? TEMP_DO_NOT_DELETE_customer?

Data Products solve this by letting you define exactly what consumers get — the right query, the right filters, the right columns — wrapped in a clean, versioned, documented package.

How It Works​

A Data Product is a named container of one or more datasets. Each dataset is a SQL query pinned to a specific datasource. You publish it, consumers discover and query it. Nobody needs to know your schema looks like a Jackson Pollock painting.

Data Product: "Customer 360"
├── Dataset: "active_customers"     → SELECT ... FROM customers WHERE active = true
├── Dataset: "customer_orders"      → SELECT ... FROM orders JOIN customers ...
└── Dataset: "monthly_metrics"      → SELECT ... FROM analytics.customer_kpis

Lifecycle​

DRAFT → PUBLISHED → DEPRECATED
  ↓         ↓            ↓
(build)  (consume)   (sunset)
  • Draft: Work in progress. Add datasets, tweak queries, nobody sees it yet.
  • Published: Live and discoverable. Consumers can access it. Version bumps on changes.
  • Deprecated: Sunset notice. Consumers get a warning, existing access still works temporarily.

Quality Scoring​

Every Data Product gets an automatic quality score (0–100) based on:

  • Documentation coverage — Do your datasets have descriptions? Do columns have context?
  • Query health — Do the underlying queries actually work?
  • Freshness — Is the data up to date according to your SLA?
  • Consumer satisfaction — Are people actually using this?

Think of it as a health check for your data offerings. Nobody wants to consume a product with a score of 23.

Consumer Access​

Access is closed by default — nobody queries your data product unless you grant them access. Grants support:

  • Row filters: "API client X only sees data WHERE region = 'EU'"
  • Column masking: "Hide SSN column for this consumer"
  • Rate limit overrides: Different consumers, different throttles
  • Expiration dates: Temporary access that self-destructs

Creating Products via Conversation​

Because this is QRY, you don't need to click through 15 forms:

Create a data product called "customer-360" in the marketing domain.
Add a dataset "active_customers" that queries all active customers
from the analytics datasource with their lifetime value and segment.

QRY will create the product, write the SQL, and set it to draft. Review it, tweak if needed, publish when ready.

API Gateway​

The Problem​

Your mobile app needs customer data. Your partner's BI tool needs order metrics. Your IoT dashboard needs real-time readings. Everyone's asking for a "quick API" and your backend team is booked for the next three sprints.

The Solution​

The API Gateway exposes published Data Products as REST endpoints. External applications authenticate with API keys and query data over HTTP. No custom backend needed.

GET /api/v1/gateway/products/customer-360/query?dataset_name=active_customers&limit=100
Authorization: Bearer qry_abc123...

That's it. JSON response with your curated data.

API Key Management​

  • Keys use the qry_ prefix for easy identification in logs
  • SHA-256 hashed at rest (we never store the raw key)
  • Shown once at creation — copy it or lose it forever
  • Per-client rate limiting with Redis sliding window
  • IP allowlisting for extra paranoia
  • Expiration dates for temporary integrations

Rate Limiting​

Every API client has a configurable rate limit (requests per minute). The Gateway uses a Redis sliding window — no bursting, no gaming, fair for everyone. Exceeding the limit returns 429 Too Many Requests with a helpful Retry-After header.

Response Formats​

FormatUse case
JSON (default)Web apps, mobile, most integrations
CSVExcel users, legacy systems, data imports
ParquetAnalytics tools, data pipelines, bulk transfers

Usage Analytics​

Every API request is logged with:

  • Client ID, product queried, dataset name
  • Response time, rows returned, status code
  • Client IP address

Admins get a usage dashboard showing top consumers, request volumes, and error rates. You'll finally know who's actually using the data (and who said they needed "urgent API access" three months ago and never made a single call).

Discover​

The Problem​

"I know we have customer data somewhere." Sound familiar? With data scattered across BigQuery, SAP HANA, Snowflake, and your colleague's "temporary" PostgreSQL instance, finding the right dataset is like finding a needle in a data warehouse.

The Discover tab searches everything in one place:

  • Data Products — Your curated, published offerings
  • DataHub assets — Datasets, dashboards, and charts from your DataHub catalog

Type a query, get results from both sources, filtered by source tabs. DataHub not configured? No problem — it degrades gracefully and just shows Data Products.

Rich DataHub Results​

DataHub results aren't just names in a list. Each result shows:

  • Breadcrumbs: bigquery › pue-madrid › metromadrid — immediately see where the data lives
  • Type badge: Dataset, Dashboard, or Chart with color-coded icons
  • Qualified name: The full catalog.schema.table path for copy-paste convenience
  • Click-to-navigate: Click a dataset result and jump straight to the Catalog Explorer with the table pre-selected

No more "I found it in DataHub but where is it in QRY?" back-and-forth.

Featured Landing Page​

When you haven't searched yet, Discover shows:

  • Stats: Total products, published count, datasets, consumers — your data catalog at a glance
  • Domain pills: Browse by business domain (Marketing, Finance, Operations...)
  • Recently published: The latest Data Products, ready to explore

Getting Started​

For Data Producers (Creating Products)​

  1. Navigate to Nexus in the left sidebar
  2. Click + New Product in the Explorer panel
  3. Fill in name, domain, description
  4. Add datasets — pick a datasource, write (or paste) a SQL query
  5. Preview results to verify
  6. When ready, click Publish

Your product is now live and discoverable.

For Data Consumers (Accessing Products)​

  1. Go to the Discover tab in Nexus
  2. Search for what you need
  3. Click a Data Product to see its datasets and documentation
  4. Request access from the product owner
  5. Query via the QRY interface or the API Gateway

For Admins (Managing Visibility)​

Nexus visibility is controlled in Admin Panel → AI & Models → Nexus Data Catalog:

  • Enable/Disable: Toggle the entire Nexus feature on or off
  • Admin Only: Restrict Nexus to admin users while you set things up — perfect for a gradual rollout

This follows the same pattern as Lakeflow and ML Hub — flip a switch, no deployment needed.

Setting Up the API Gateway​

  1. Open a Data Product → Consumers tab
  2. Click Create API Client
  3. Set a name, rate limit, optional IP allowlist
  4. Copy the API key immediately (it's shown once, then gone forever)
  5. Grant the client access to the product
  6. Share the endpoint URL and key with the external consumer

Use Cases​

📊 Self-Service Analytics​

Sarah from Marketing needs customer metrics for her quarterly report. Instead of filing a Jira ticket and waiting three days, she searches "customer metrics" in Discover, finds the customer-360 product, and queries the monthly_metrics dataset directly.

🔌 Partner Integration​

Your logistics partner needs daily order data for delivery optimization. Create an API client with a 100 req/min limit, grant access to the orders product with a row filter for status = 'shipped', and hand them the endpoint. Done in 10 minutes, not 10 meetings.

🏗️ Microservices Data Access​

Your mobile app team needs user profiles. Instead of building (and maintaining) yet another internal API, point them to the Gateway endpoint for the user-profiles product. Same data, zero backend work.

🔍 Data Discovery for New Hires​

A new analyst joins and needs to understand what data is available. They open Discover, browse by domain, and find documented Data Products with quality scores. No "go ask Dave, he knows where everything is."

Best Practices​

Data Products​

✅ Do:

  • Write clear descriptions — your future self will thank you
  • Use meaningful dataset names (active_customers not ds_001)
  • Set freshness SLAs for time-sensitive data
  • Deprecate before deleting — give consumers time to migrate
  • Start with a few high-value products, not 200 mediocre ones

❌ Don't:

  • Publish a product with SELECT * queries (curate your columns)
  • Skip quality checks before publishing
  • Create one massive product with 50 datasets (split by domain)
  • Forget to update descriptions when queries change

API Gateway​

✅ Do:

  • Set sensible rate limits (start conservative, increase on request)
  • Use IP allowlists for production integrations
  • Set expiration dates for temporary/pilot integrations
  • Monitor usage dashboards for anomalies
  • Use descriptive client names (mobile-app-prod not test-client-3)

❌ Don't:

  • Share API keys via Slack/email (use a secrets manager)
  • Set rate limits to 10,000 rpm "just in case" (you'll regret it)
  • Create API clients without knowing who/what will use them
  • Ignore the usage analytics (they tell you who actually cares about your data)

Quick Reference​

ActionHow
Create a productNexus → Explorer → + New Product
Add a datasetOpen product → Datasets tab → + Add Dataset
PublishOpen draft product → click Publish
Create API clientOpen product → Consumers → Create API Client
Grant API accessOpen product → Consumers → select client
Search everythingNexus → Discover tab → type your query
Configure visibilityAdmin Panel → AI & Models → Nexus Data Catalog

What's Coming Next​

  • Lineage visualization — See upstream/downstream dependencies in a graph view
  • AI Data Classification — Automatic PII/sensitive data detection using LLM analysis
  • Permissioned discovery — BROWSE-style permissions for controlling who can discover what

See Also​

Last updated: March 03, 2026

QRYA product of IXEN.