tabular-optimization-ingestion

Tabular Optimization Ingestion

Use this skill when the user provides raw or semi-structured data and asks a question that may require optimization.

The purpose of this skill is to bridge the gap between messy uploaded data and solver-ready model construction.

This skill does not solve the optimization problem itself. It inspects the data, infers likely modeling roles, and identifies what still needs clarification.

It does not authorize heuristic, greedy, or backtracking schedules as answers. In the NemoClaw sandbox, read cuopt-sandbox: the first solver that produces assignments or a schedule must be cuOpt after probe → env → smoke gates pass. Ingestion output is a modeling interpretation (entities, objective fields, constraints) — never a completed plan.

This skill refines the optimization interpretation using the uploaded data; it does not replace the earlier intent decision unless the data clearly contradicts it.

Purpose

Users do not upload:

• objective vectors
• sparse matrices
• explicit decision variable definitions

They upload things like:

• products.csv
• capacity.xlsx
• orders.json
• travel_times.csv
• dealers.csv
• depots.csv

This skill turns raw tables into a candidate optimization interpretation.

What this skill should produce

After inspection, produce a compact working interpretation containing:

• likely problem type: LP / MILP / QP / routing / unclear
• likely entities: products, resources, orders, locations, vehicles, customers, depots, time periods, etc.
• likely objective fields: profit, cost, margin, time, risk, distance, penalty
• likely constraint fields: capacity, demand, budget, inventory, hours, availability, limits, min/max bounds
• likely decision grain: per product, per route, per order, per plant, per vehicle-stop, etc.
• minimum unresolved ambiguities that require a user clarification

Ingestion workflow

1. Identify table shape and grain

For each uploaded source, determine:

• what one row appears to represent
• whether the table is entity-level, transaction-level, matrix-like, or parameter-like

Examples:

• one row per product
• one row per plant
• one row per customer order
• one row per location pair
• one row per vehicle
• one row per time period

Always state the candidate row meaning before modeling from it.

2. Identify likely optimization roles by column meaning

Inspect column names and values for roles such as:

Objective-like fields:

• profit
• revenue
• margin
• cost
• travel_time
• distance
• risk
• penalty

Constraint-like fields:

• capacity
• demand
• available_hours
• budget
• max_load
• min_order
• inventory
• time_window_start
• time_window_end

Identifier / relationship fields:

• product_id
• plant_id
• order_id
• customer_id
• vehicle_id
• depot_id
• location_id
• origin / destination
• pickup / delivery

3. Infer candidate decision structure

Infer what the model is probably deciding.

Examples:

• units to produce by product
• units to ship from origin to destination
• whether to open a facility
• which vehicle serves which stops
• route sequence through locations
• allocation of budget across campaigns

Do not overcommit when the data supports multiple plausible decisions. Note the candidates and ask one focused question if needed.

4. Match the data pattern to a problem family

Use common table patterns:

LP / MILP patterns

Typical signs:

• product/resource tables
• profit and resource consumption by product
• capacity tables
• demand tables
• assignment tables
• open/close or yes/no columns implying integrality

QP patterns

Typical signs:

• covariance matrix
• risk matrix
• variance terms
• squared error context
• portfolio weights or regression-style structure

Routing patterns

Typical signs:

• locations / coordinates
• travel time or distance matrix
• depot + customer tables
• vehicle tables
• demands and capacities
• time windows
• pickup and delivery pairs

5. Identify only the critical ambiguities

Before handing off, identify the smallest set of unanswered questions that block valid model construction.

Examples:

• Are demands mandatory or forecast-only?
• Must decisions be integers?
• Are these time windows hard constraints?
• Can unmet demand be allowed with penalty?
• Is profit net profit or revenue only?

Do not ask broad generic questions if the data already strongly suggests the answer.

Preferred output format

Summarize findings compactly:

• Candidate problem family: ...
• Row meanings: ...
• Likely decision: ...
• Likely objective: ...
• Likely constraints: ...
• Unresolved blockers: ...

This summary should be short enough that a downstream formulation skill can use it directly.

Column-role heuristics

Use these heuristics carefully. They guide interpretation but do not prove it.

Cost / objective heuristics

• cost, unit_cost, shipping_cost, expense → likely minimization coefficient
• profit, margin, contribution → likely maximization coefficient
• distance, travel_time, duration → likely routing objective term or service constraint
• risk, variance, covariance → likely QP signal

Capacity / demand heuristics

• capacity, available, limit, max_* → upper bound / resource constraint
• demand, required, need → demand fulfillment or service requirement
• min_*, minimum_* → lower bound or service rule

Integrality heuristics

Treat MILP as likely when the data or request suggests:

• whole units
• counts of vehicles, workers, facilities, shifts, items
• yes/no choices
• on/off decisions
• assignment indicators

Routing heuristics

Treat routing as likely when the core question depends on path construction, not merely allocation.

Signs include:

• stop sequence matters
• travel between locations matters
• vehicles start/end at depots
• travel matrix or coordinates are present
• pickup and delivery relationships appear

Cross-row coupling heuristics

Duplicate values in a foreign-key column across rows of a parent table usually signal a shared agent or resource — one entity serving multiple parents. Shared resources need a mutual-exclusion constraint that no other column states.

• coach_id, driver_id, instructor_id, nurse_id, operator_id → shared agent; can serve only one parent at a time
• machine_id, bay_id, tool_id → shared resource; can host only one job at a time
• any FK column where distinct_values < row_count → check whether simultaneous assignment is allowed

A *_unavailability (or *_availability) table documents known absences; duplicated FK values document implicit conflicts. Treat both as constraint sources. Concrete check: for every FK-looking column in a parent table, compare distinct value count to row count, and surface the column when distinct < rows.

Pre-solve capacity arithmetic

Before handing off to formulation, compare aggregate requirements against upper bounds implied by the tables:

• required events or appearances per entity (e.g. round-robin games per team)
• maximum placements allowed by per-week, per-day, or per-slot caps
• total court/slot/machine capacity across the horizon

If required appearances exceed what the time structure can host, flag the conflict before building the full MILP. Example: 8 required games per team with at most 1 game per team per week over 5 weeks allows only 5 games — infeasible without relaxing the weekly cap.

Examples

Example 1: product mix tables

Files include:

• products.csv with columns like product, profit, labor_hours, steel_units
• capacity.csv with columns like resource, available

Likely interpretation:

• one row in products.csv = one product
• one row in capacity.csv = one resource limit
• decision = how much of each product to produce
• objective = maximize profit
• constraints = labor and steel capacities

Example 2: routing tables

Files include:

• customers.csv with customer_id, demand, time_window_start, time_window_end
• vehicles.csv with vehicle_id, capacity
• travel_times.csv with origin/destination or matrix-style travel times

Likely interpretation:

• customer rows are stops with demand and optional time windows
• vehicle rows define fleet capacity
• travel table defines movement cost/time
• likely problem family = routing

Example 3: time-slot / resource assignment (scheduling MILP)

Files include patterns such as:

• games.csv or jobs.csv — items to place (events, tasks, orders)
• time_slots.csv or shifts.csv — when placement can occur
• courts.csv, machines.csv, or rooms.csv — resources
• teams.csv or workers.csv — entities tied to shared agents (coaches, operators)
• *_unavailability.csv — blocked (resource, slot) or (agent, slot) pairs

Likely interpretation:

• decision = assign each item to a (slot, resource) or similar binary/integer placement
• hard constraints = no double-booking, unavailability, capacity, one game per team per slot
• likely problem family = MILP (even if user only says "build a schedule" or "valid plan")
• NemoClaw: read optimization-from-data-orchestrator + cuopt-sandbox before any custom scheduler code

Example 4: historical transaction table

File includes:

• sales_history.csv with order_id, date, region, revenue, units_sold

Likely interpretation:

• this may be analytics, not optimization, unless the user asks for a future decision under constraints
• do not invent decision variables from history alone without a decision-focused question

Guardrails

• Do not assume every table with cost and capacity is automatically a valid optimization model.
• Do not mistake transaction history for decision variables without evidence.
• Do not treat forecast data as hard constraints unless the user implies that.
• Do not infer QP unless there is a real quadratic signal.
• Do not infer routing unless movement between locations is central to the decision.
• Prefer one precise clarification over a long list of speculative questions.

Handoff guidance

• If the data suggests LP / MILP:

  • hand off to numerical-optimization-formulation
  • then to cuopt-numerical-optimization-api-python (or cuopt-numerical-optimization-api-cli for MPS inputs)

• If the data suggests QP:

  • hand off to numerical-optimization-formulation
  • then to cuopt-numerical-optimization-api-python

• If the data suggests routing:

  • hand off to routing-formulation
  • then to cuopt-routing-api-python

• If mode selection is still needed because replayability, audit, export, or reuse may matter:

  • use optimization-mode-router before deep model construction

• If optimization intent itself is still uncertain:

  • use optimization-intent-router

Success criterion

This skill succeeds when the downstream model-building step can proceed with either:

• no clarification, or
• one or a few narrowly targeted clarifying questions

It fails when it produces a vague restatement of the table without narrowing the modeling interpretation.