NVIDIA / NVIDIA/simready-foundation

Replace BasisCurves grasp-vector convention (GSP.001) with a proper grasp affordance USD schema

Open
#4 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
88
Forks
18
PR merge metrics
No merged PRs in 30d

Description

Summary

The current way to declare grasp affordances — requirement GSP.001
(graspable-vector-line)
in the physics_graspable capability — relies on a
UsdGeom.BasisCurves prim named with the prefix grasp_identifier_*. This works as a
first pass, but it is a naming-convention workaround rather than a real contract. This
issue proposes replacing it with a proper typed/applied USD schema for grasp affordances.

Problems with the current BasisCurves convention

  1. The contract is a prim-name prefix. "Grasp-ness" is encoded in the name
    grasp_identifier_*, not in the type system or schema registry. Rename the prim and
    it silently stops being a grasp. There is no machine-discoverable way to ask "is this
    object graspable?" without string-matching prim names across the hierarchy — which
    does not survive references, payloads, or flattening cleanly.

  2. A line under-specifies the grasp. Two points give a position and an axis. They do
    not carry: gripper type, jaw opening width, required grip force, payload rating,
    approach-direction sign, pre-grasp standoff, contact normals, or a quality score —
    i.e. none of the metadata a planner or data-generation pipeline needs to decide
    which robot can use this grasp.

  3. It does not generalize across gripper types. A line is a parallel-jaw mental
    model. A suction/vacuum grasp is a point + approach normal (a line misrepresents
    it). A multi-finger grasp is a palm pose + per-finger contacts (a line cannot
    express it). The convention quietly assumes one gripper class.

  4. Geometry and semantics are conflated. The grasp's meaning is welded to a piece of
    renderable curve geometry. Visualization should be derived from the grasp data, not
    be the source of truth.

  5. No typed query path. Consumers cannot do prim.HasAPI<Graspable>() or
    prim.IsA<GraspPoint>(). Discovery is a brittle traversal + string match.

Proposed direction: a proper grasp schema

Two schemas, each using the right USD mechanism:

  • GraspableAPI — a single-apply applied API schema on the object root /
    rigid-body prim. It is the discovery entry point: prim.HasAPI<GraspableAPI>()
    answers "does this object expose grasps?" in one call, no traversal or string-match.

  • GraspPoint — a typed schema (IsA Xformable) for each grasp candidate. The
    prim transform is the 6-DOF grasp pose. Gripper class and continuous requirements
    are typed attributes, so the same structure serves every gripper type. Each grasp is
    its own posed prim (USD allows only one transform per prim), which is also the clean
    answer to "how are multiple grasp points represented?"

# Object-level marker — the HasAPI discovery entry point
class "GraspableAPI" (
    inherits = </APISchemaBase>
    customData = { token apiSchemaType = "singleApply" }
) {
    rel grasp:points   # optional → the GraspPoint prims for this object
}

# One posed grasp candidate; transform = grasp pose
class GraspPoint "GraspPoint" ( inherits = </Xformable> )
{
    uniform token grasp:gripperKind = "parallelJaw" (
        allowedTokens = ["parallelJaw", "vacuum", "multiFinger", "softGripper", "magnetic"]
    )
    uniform token grasp:approachAxis = "Z" (allowedTokens = ["X","Y","Z","-X","-Y","-Z"])
    float grasp:approachStandoff = 0.10   # meters, pre-grasp offset
    float grasp:openingWidth     = 0.0    # meters, parallel-jaw
    float grasp:requiredForce    = 0.0    # Newtons
    float grasp:maxPayload       = 0.0    # kg
    float grasp:suctionRadius    = 0.0    # meters, vacuum
    point3f[] grasp:contactPoints = []    # finger/suction contacts, local frame
    float grasp:quality = 1.0             # [0,1]
    uniform token grasp:source = "manual" (
        allowedTokens = ["manual", "analytic", "learned", "physicsValidated"]
    )
    rel grasp:gripperProfile              # optional → full gripper capability descriptor
}

Authored example (replaces a grasp_identifier_01 curve on a cup):

def Xform "CoffeeCup" ( prepend apiSchemas = ["PhysicsRigidBodyAPI", "GraspableAPI"] )
{
    rel grasp:points = [ </CoffeeCup/Grasps/handle_side> ]
    def Mesh "CupGeometry" ( prepend apiSchemas = ["PhysicsCollisionAPI"] ) {}
    def Scope "Grasps" {
        def GraspPoint "handle_side" {
            double3 xformOp:translate = (0.0, 0.10, 0.0)
            quatf   xformOp:orient    = (0.707, 0, 0.707, 0)
            uniform token[] xformOpOrder = ["xformOp:translate", "xformOp:orient"]
            uniform token grasp:gripperKind  = "parallelJaw"
            float  grasp:openingWidth        = 0.085
            float  grasp:requiredForce       = 15.0
            uniform token grasp:approachAxis = "-Y"
            float  grasp:quality             = 0.92
            uniform token grasp:source       = "physicsValidated"
        }
    }
}

Suction grasp — same structure, no line, no width: gripperKind = "vacuum",
suctionRadius, approach normal = the prim's approach axis.

Consumption

if prim.HasAPI(GraspableAPI):                 # one typed call
    for gp in iter_grasp_points(prim):        # prims that IsA GraspPoint
        pose = UsdGeom.Xformable(gp).ComputeLocalToWorldTransform(Usd.TimeCode.Default())
        # use directly for planning, OR as ground truth to compare against where a
        # robot actually gripped / a policy predicted a grip

This supports both direct use (a simulator reads grasps and acts) and ground-truth use
(compare authored grasps to executed/predicted grips for accuracy and failure analysis).

Notes / open questions

  • Codeful vs codeless schema: recommend codeful. String discovery via
    GetAppliedSchemas() works for both, but the ergonomic typed HasAPI<T>() / IsA<T>()
    query and generated C++/Python accessors that Isaac Sim consumers expect require
    codeful. Codeless's only advantage (change schema without recompiling USD) does not
    apply to a foundational standard.
  • Migration: convert each grasp_identifier_* curve to a GraspPoint (translate =
    curve midpoint, approachAxis from curve direction, openingWidth = endpoint distance);
    apply GraspableAPI to the root; regenerate the line as a derived purpose="guide"
    visualization; update the validator. Accept both during a deprecation window.
  • Standardization: propose to the AOUSD Robotics WG after a SimReady-local version.
    No grasp affordance schema exists in OpenUSD core or the WG roadmap yet (current focus
    is URDF/MJCF/SDFormat → USD mapping and UsdPhysics deformables), so there is room to
    lead.
  • Typed GraspPoint vs applied GraspPointAPI for the per-grasp prim is an open
    call; attribute set is identical either way.

Happy to open a PR with a draft schema.usda + migration script + updated validator if
there's appetite for this direction.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the proposed schema.usda, then define the migration script and updated validator described in the issue. Resolve the typed GraspPoint versus GraspPointAPI choice and the migration/deprecation behavior; done means the replacement schema, conversion path, and validator support the legacy convention during the stated window.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
robotics
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.