NVIDIA / NVIDIA/simready-foundation
Replace BasisCurves grasp-vector convention (GSP.001) with a proper grasp affordance USD schema
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 88
- Forks
- 18
- PR merge metrics
- No merged PRs in 30d
Description
Summary
The current way to declare grasp affordances — requirement GSP.001
(graspable-vector-line) in the physics_graspable capability — relies on a
UsdGeom.BasisCurves prim named with the prefix grasp_identifier_*. This works as a
first pass, but it is a naming-convention workaround rather than a real contract. This
issue proposes replacing it with a proper typed/applied USD schema for grasp affordances.
Problems with the current BasisCurves convention
-
The contract is a prim-name prefix. "Grasp-ness" is encoded in the name
grasp_identifier_*, not in the type system or schema registry. Rename the prim and
it silently stops being a grasp. There is no machine-discoverable way to ask "is this
object graspable?" without string-matching prim names across the hierarchy — which
does not survive references, payloads, or flattening cleanly. -
A line under-specifies the grasp. Two points give a position and an axis. They do
not carry: gripper type, jaw opening width, required grip force, payload rating,
approach-direction sign, pre-grasp standoff, contact normals, or a quality score —
i.e. none of the metadata a planner or data-generation pipeline needs to decide
which robot can use this grasp. -
It does not generalize across gripper types. A line is a parallel-jaw mental
model. A suction/vacuum grasp is a point + approach normal (a line misrepresents
it). A multi-finger grasp is a palm pose + per-finger contacts (a line cannot
express it). The convention quietly assumes one gripper class. -
Geometry and semantics are conflated. The grasp's meaning is welded to a piece of
renderable curve geometry. Visualization should be derived from the grasp data, not
be the source of truth. -
No typed query path. Consumers cannot do
prim.HasAPI<Graspable>()or
prim.IsA<GraspPoint>(). Discovery is a brittle traversal + string match.
Proposed direction: a proper grasp schema
Two schemas, each using the right USD mechanism:
-
GraspableAPI— a single-apply applied API schema on the object root /
rigid-body prim. It is the discovery entry point:prim.HasAPI<GraspableAPI>()
answers "does this object expose grasps?" in one call, no traversal or string-match. -
GraspPoint— a typed schema (IsA Xformable) for each grasp candidate. The
prim transform is the 6-DOF grasp pose. Gripper class and continuous requirements
are typed attributes, so the same structure serves every gripper type. Each grasp is
its own posed prim (USD allows only one transform per prim), which is also the clean
answer to "how are multiple grasp points represented?"
# Object-level marker — the HasAPI discovery entry point
class "GraspableAPI" (
inherits = </APISchemaBase>
customData = { token apiSchemaType = "singleApply" }
) {
rel grasp:points # optional → the GraspPoint prims for this object
}
# One posed grasp candidate; transform = grasp pose
class GraspPoint "GraspPoint" ( inherits = </Xformable> )
{
uniform token grasp:gripperKind = "parallelJaw" (
allowedTokens = ["parallelJaw", "vacuum", "multiFinger", "softGripper", "magnetic"]
)
uniform token grasp:approachAxis = "Z" (allowedTokens = ["X","Y","Z","-X","-Y","-Z"])
float grasp:approachStandoff = 0.10 # meters, pre-grasp offset
float grasp:openingWidth = 0.0 # meters, parallel-jaw
float grasp:requiredForce = 0.0 # Newtons
float grasp:maxPayload = 0.0 # kg
float grasp:suctionRadius = 0.0 # meters, vacuum
point3f[] grasp:contactPoints = [] # finger/suction contacts, local frame
float grasp:quality = 1.0 # [0,1]
uniform token grasp:source = "manual" (
allowedTokens = ["manual", "analytic", "learned", "physicsValidated"]
)
rel grasp:gripperProfile # optional → full gripper capability descriptor
}
Authored example (replaces a grasp_identifier_01 curve on a cup):
def Xform "CoffeeCup" ( prepend apiSchemas = ["PhysicsRigidBodyAPI", "GraspableAPI"] )
{
rel grasp:points = [ </CoffeeCup/Grasps/handle_side> ]
def Mesh "CupGeometry" ( prepend apiSchemas = ["PhysicsCollisionAPI"] ) {}
def Scope "Grasps" {
def GraspPoint "handle_side" {
double3 xformOp:translate = (0.0, 0.10, 0.0)
quatf xformOp:orient = (0.707, 0, 0.707, 0)
uniform token[] xformOpOrder = ["xformOp:translate", "xformOp:orient"]
uniform token grasp:gripperKind = "parallelJaw"
float grasp:openingWidth = 0.085
float grasp:requiredForce = 15.0
uniform token grasp:approachAxis = "-Y"
float grasp:quality = 0.92
uniform token grasp:source = "physicsValidated"
}
}
}
Suction grasp — same structure, no line, no width: gripperKind = "vacuum",
suctionRadius, approach normal = the prim's approach axis.
Consumption
if prim.HasAPI(GraspableAPI): # one typed call
for gp in iter_grasp_points(prim): # prims that IsA GraspPoint
pose = UsdGeom.Xformable(gp).ComputeLocalToWorldTransform(Usd.TimeCode.Default())
# use directly for planning, OR as ground truth to compare against where a
# robot actually gripped / a policy predicted a grip
This supports both direct use (a simulator reads grasps and acts) and ground-truth use
(compare authored grasps to executed/predicted grips for accuracy and failure analysis).
Notes / open questions
- Codeful vs codeless schema: recommend codeful. String discovery via
GetAppliedSchemas()works for both, but the ergonomic typedHasAPI<T>()/IsA<T>()
query and generated C++/Python accessors that Isaac Sim consumers expect require
codeful. Codeless's only advantage (change schema without recompiling USD) does not
apply to a foundational standard. - Migration: convert each
grasp_identifier_*curve to aGraspPoint(translate =
curve midpoint, approachAxis from curve direction, openingWidth = endpoint distance);
applyGraspableAPIto the root; regenerate the line as a derivedpurpose="guide"
visualization; update the validator. Accept both during a deprecation window. - Standardization: propose to the AOUSD Robotics WG after a SimReady-local version.
No grasp affordance schema exists in OpenUSD core or the WG roadmap yet (current focus
is URDF/MJCF/SDFormat → USD mapping and UsdPhysics deformables), so there is room to
lead. - Typed
GraspPointvs appliedGraspPointAPIfor the per-grasp prim is an open
call; attribute set is identical either way.
Happy to open a PR with a draft schema.usda + migration script + updated validator if
there's appetite for this direction.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the proposed schema.usda, then define the migration script and updated validator described in the issue. Resolve the typed GraspPoint versus GraspPointAPI choice and the migration/deprecation behavior; done means the replacement schema, conversion path, and validator support the legacy convention during the stated window.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- robotics
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100