Day 13 — Perception-driven support affordances / contact discovery

Day 12 owned the ContactSchedule: CLAIM → MEASURE → PROMOTE → HOLD → DEMOTE across FOOT / HAND / OBJECT; schedules are intent that wait for measurement; measured ContactState owns the hard set; one WBC/QP; yield to Day 08 gait and Day 09 recovery. Day 11 owned promoting measured hand contact into the Day 07 hard support set. Day 10 owned soft EE / grasp on FREE limbs. Day 09 owned mid-event abort and RecoveryCommand. Day 08 owned gait modes vs measured contact_set. Day 07 owned the feasible program. Day 06 owned the estimate. Day 05 owned sag honesty.

Today owns support affordance proposals from vision, tactile, and related detectors as schedule candidates only. Affordance detectors propose WHERE / WHAT to CLAIM next. Affordance confidence gates CLAIM (entry into the schedule) — not friction cones. Measured contact still owns the hard set. No detector-as-constraint-truth. Perception never attaches cones, never replaces ContactState, never writes JointCommand, never fights gait or recovery.

Affordance proposals are candidates, not constraint truth

A detector that attaches hand or object cones because the vision score crossed $0.9$ is just CLAIM-as-truth wearing a neural net. Proposals feed schedule slots; measurement still admits them; demotion is still mandatory when confidence, wrench agreement, or ceilings fail — including when the detector still “believes” the rail is there.

Core thesis: perception-driven affordances consume the same EstimatorState + Day 07 friction / unilaterality + torque_scale stack. An AffordanceProposal (or thin ContactAffordance feed) is a soft candidate into ContactSchedule CLAIM slots. Confidence / geometry / reachability gates whether a CLAIM is allowed — not whether cones attach. MEASURE is still required before PROMOTE; detector score $\neq$ MEASURED. Soft EE remains for FREE limbs. There is still one QP face — not a vision-driven second plant that finishes a wall brace because the detector looked sure.

Ownership rule: one writer for affordance candidates, one writer for schedule intent, one writer for measured contact_set. When they disagree, measured contact wins for constraints. Affordance confidence never attaches cones. Intent aborts, softens, retimes, or waits — it does not promote OBJECT because the detector said “rail at $(x,y,z)$.” Day 08 gait and Day 09 recovery still outrank a stuck schedule hold, even when perception is loud.

How today fits the Days 05–12 stack

Do not bolt a “perception optimizer” that writes JointCommand or attaches cones beside the QP when a detector fires. That is Day 10’s second-plant anti-pattern wearing a depth camera.

Affordance honesty / confidence gates

Day 12’s per-slot lifecycle still holds. Day 13 adds a pre-CLAIM gate: proposals must earn the right to open a CLAIM slot.

Sequencing rules (all required):

1. Affordance confidence gates CLAIM entry, not cone attach — a high score without MEASURE still cannot promote.
2. One CLAIM admission at a time (or a documented simultaneous pair) — no “CLAIM every rail the detector listed this tick.”
3. MEASURE still required before PROMOTE; detector score $\neq$ MEASURED.
4. Wrong normal / flipped geometry in the proposal must fail the gate or fail MEASURE / wrench agreement — never silently attach inverted cones.
5. Mid-sequence POWER_GATED / stale / recovery / perception dropout → freeze or abort the schedule; do not advance phases on stale pixels.

Demotion is success when the detector hallucinated a rail, the tactile spike was grease, or the pack sagged. A robot that always finishes the vision-driven handoff into a collapsing bus is ignoring Day 05 with a prettier depth demo.

Detector vs measure vs schedule vs QP ownership

Perception is where teams accidentally create a fourth writer on support constraints. Freeze the split:

Hard vs soft: support constraints on measured FOOT / HAND / OBJECT stay hard. Soft EE authority lives only on FREE limbs. Affordance proposals are softer still — they feed CLAIM, they do not feed cones. If the QP goes infeasible under the expanded set, demote / abort / soften — do not silently drop an object cone so the detector “makes it.”

Mid-perception / mid-CLAIM abort honesty

Vision and tactile are where teams disable safety “just until the detector settles.” Do not.

Abort, demote, and soften are success paths when the detector hallucinated, the wall was grease, or the pack sagged mid-handoff.

Shared API: affordance feed into Day 12 schedule shapes

Keep Day 02 / 04 wire honesty. Cyclic commands still carry BusHeader. Extend Day 12 ContactSchedule / ManipulationCommand / WbcTaskSet and Day 06–08 ContactState — do not invent a parallel “perception bus” or second plant:

BusHeader:
  node_id, seq
  t_sample, clock_domain, age_budget_ms
  faults

# Upstream (read-only here)
EstimatorState, PowerState  ⊇ BusHeader

ContactState ⊇ BusHeader:
  contact_set[]        # feet, hands, AND objects when measured
  SupportContact:
    link_id / frame
    kind               # FOOT | HAND | OBJECT
    mu, normal, wrench_est?
    confidence, age_ms
    flags              # settling / flicker / gated / grasp_lost
    parent_grasp_id?

# Soft perception feed (host / mid-rate) — NEVER attaches cones
AffordanceProposal ⊇ BusHeader:       # or ContactAffordance[]
  proposals[]:
    target_frame       # candidate FOOT / HAND / OBJECT frame
    kind_hint          # FOOT | HAND | OBJECT
    pose_hint          # WHERE in world / base
    normal_hint?       # soft — not hard cone truth
    mu_hint?
    confidence         # gates CLAIM entry, NOT cone attach
    geometry_ok        # planar / edge / graspable checks
    reachability_ok
    age_ms             # expire stale pixels / tactile spikes
    source             # VISION | TACTILE | FUSED | ...
  relax_on_fault       # stale / dropout → expire candidates

# Thin multi-contact face (host / mid-rate) — Day 12 + affordance admit
ContactSchedule ⊇ BusHeader:          # or MultiContactCommand
  intent               # IDLE | MULTI_CONTACT | OBJECT_SUPPORT | ABORT
  slots[]:
    target_frame
    phase              # FREE | CLAIM | PROMOTE | HOLD | DEMOTE
    earliest_t / latest_t?   # soft timing hints — NOT constraint truth
    priority_hint
    affordance_ref?    # which AffordanceProposal admitted this CLAIM
  yield_to             # GAIT | RECOVERY | POWER
  admit_policy         # min confidence / geom / reach to open CLAIM
  relax_on_fault       # stale / POWER_GATED / flicker / grasp_lost /
                       # perception_dropout → abort/demote

# Extends Day 10–12 manipulation face (optional merge)
ManipulationCommand ⊇ BusHeader:
  intent               # … | CLAIM_SUPPORT | LOCO_MANIP
                       # | OBJECT_SUPPORT | MULTI_CONTACT | ABORT
  ee_refs[] / hand_refs[]
  hand_role[] / support_claim[]
  object_claim[]
  schedule_ref?
  affordance_ref?      # optional link to active proposal set
  relax_on_fault

# Extended Day 07–12 task face — unchanged ownership
WbcTaskSet ⊇ BusHeader:
  mode                 # STANCE_DS | STANCE_SS | SWING | IMPACT | RECOVER_IP
                       # | MANIP | HAND_SUPPORT | LOCO_MANIP
                       # | OBJECT_SUPPORT | MULTI_CONTACT | ...
  com_ref / capture_ref?
  foot_refs[]
  hand_support_refs[]?
  object_support_refs[]?
  ee_refs[]               # soft EE — FREE limbs only
  hand_refs[]
  posture_bias?
  priority_order[]
  relax_on_fault

# Downstream muscle (Day 02) — ceilings still reflect torque_scale
JointCommand ⊇ BusHeader:
  q_des? / qd_des? / tau_ff?
  iq_ceiling? / tau_ceiling?
  flags                # STO / soften / hold

Higher layers publish affordance candidates and schedule intent. Measured ContactState owns support membership. Gait and recovery keep their writers. Rate loops enforce BMS ceilings. Nobody re-implements FOC, AHRS, or friction cones inside the affordance detector.

Physical ↔ digital pairing for perception-driven CLAIMs

If the twin always promotes when the detector fires with sticky contacts and a full pack while hardware sees delayed contact_set, false positives, and sag, you will tune vision thresholds forever and never ship a CLAIM that survives a tired bus or a greased rail.

Bring-up sequence (before free vision-driven multi-contact)

Days 07–12 green (stance, step, recovery, FREE-hand reach, measured hand SUPPORT / loco-manip, scripted multi-contact / OBJECT handoffs with mid-transition faults proven). Then:

1. Days 07–12 green — schedule abort paths still behave; no open recovery or gait fights; OBJECT / HAND promotes still require MEASURE.
2. Offline / logged proposals — replay detectors into the CLAIM admit gate only; confirm high score without MEASURE never attaches cones; confirm low score never opens CLAIM.
3. Single live gated CLAIM — one known surface or fixed object; vision or tactile proposes → confidence gate → CLAIM → wait for MEASURED → PROMOTE → HOLD → DEMOTE; feet remain primary; confirm QP feasible.
4. Perception honesty injects — false positives, missed detections, delayed perception, wrong normals in twin and on hardware; confirm drop / wait / demote, not heroic promote-on-score.
5. Non-fight + mid-CLAIM faults — force affordance-CLAIM-during-swing, recovery-during-HOLD, stale / gated / flicker / perception dropout; confirm yield / abort / demote.
6. Then add free vision-driven or learned residuals that choose affordance thresholds / schedule slots above the QP — never replace cones, FOC, torque_scale, gait, recovery, or measured ContactState. Richer scene graphs and teleop+affordance blend wait for Day 14+.

Skip to “end-to-end vision multi-contact in sim” and hardware’s first real rail grasp becomes an unplanned tip / brownout experiment with better depth video.

Failure gallery

What to freeze this week

Extend frames and your control config with:

• AffordanceProposal / ContactAffordance fields: pose / normal hints, confidence, geometry_ok, reachability_ok, age_ms, source — soft only
• Who owns affordance candidates vs ContactSchedule intent vs measured contact_set vs gait vs recovery (disagree → wait/yield/demote; measured contact wins for cones; recovery and gait outrank schedule holds)
• Admit policy: confidence / geometry / reachability gate CLAIM, never cone attach
• ContactSchedule affordance_ref + admit_policy; phases FREE → CLAIM → PROMOTE → HOLD → DEMOTE unchanged from Day 12
• Explicit rule: detector score $\neq$ MEASURED; perception never writes JointCommand or friction cones
• Mid-CLAIM / mid-perception abort / demote for stale, POWER_GATED, low torque_scale, flicker, grasp integrity loss, and perception dropout
• Twin hooks: false positives, missed detections, delayed perception, wrong normals, sag under handoff peaks, schedule-vs-gait collisions — same as hardware
• Bring-up gate: no free vision-driven multi-contact / learned affordance residuals until steps 1–5 above are green

The lesson is not “let the detector drive the robot until the depth map looks busy.” It is make perception-driven support the same contact-rich program in silicon and in the twin: soft affordance candidates into schedule CLAIM slots, confidence gating entry not cones, one estimate, honest power ceilings, measured FOOT/HAND/OBJECT in the hard support set, schedules that wait for measurement and yield to gait/recovery, soft tasks only on FREE limbs, and abort / demote paths that stay honest when the detector or the pack lies mid-CLAIM.

Next

Day 14 — Learned residuals / richer scene graphs on the affordance→schedule stack: residual ranking of CLAIM candidates and light scene structure above the same measured hard set — still no detector-as-constraint-truth; twin must keep false+/miss/delay honesty before free learned multi-contact.