openai / openai/codex

Bundled skill helpers need preflight validation before installation and image API side effects

Open
#44,187 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug imagen skills
Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

Component / version

Bundled skill assets in codex-rs/skills/src/assets/samples, tag rust-v0.153.4. Seven inspected installed files matched the tag byte-for-byte.

Actual and reproduction

Real helper functions were exercised with temporary synthetic files and fake network clients:

  • Skill validation accepts empty required metadata instead of rejecting it.
  • Batch installation can begin side effects before later invalid/conflicting targets are identified; partial success needs accurate reporting.
  • Image generation can reach the client API call before discovering an existing/conflicting output path. This proves a needless call in the test, not an actual charge.

Related bundled instructions also need to remain consistent with the consuming host's source-ordering and image-input contracts, accurately describe overwrite refusal, and use manifest examples accepted by the validator. These text changes are proposed compatibility improvements for the inspected host contract, not a claim that every host has identical instructions.

Expected

Validate deterministic input/output conflicts before side effects, retain accurate partial-success errors for unavoidable later I/O failures, and keep bundled instructions/examples consistent with actual tools and the host contract.

Evidence and proposed patch

The candidate applies cleanly to the exact public tag. Isolated actual-function checks use synthetic files and fake clients; no real image call, installation, credential or private document was used. This report does not claim a released fix. The locally modified cache was later observed at original bytes again; the writer was not identified, and this is not evidence blaming a particular updater.

Candidate source diff
--- a/codex-rs/skills/src/assets/samples/openai-docs/SKILL.md
+++ b/codex-rs/skills/src/assets/samples/openai-docs/SKILL.md
@@ -9,9 +9,7 @@
 
 Provide current, cited OpenAI product, API, model, and Codex guidance. Read zero or one primary reference.
 
-**First substantive action:** Search the user's exact requested official OpenAI documentation topic and any explicitly named model using a concise, topic-specific query of 2-6 essential terms. When an already-available direct official documentation search and page-retrieval capability is present, use it first: search, then fetch or open the matching official page before general web search. Otherwise, immediately use official-domain web search, then actually open or fetch the relevant official page. Complete this source order before reading a reference, inspecting local or repository files, running a Codex manual or model resolver, drafting a plan, or answering from memory. Use the actual fetched page, not a search snippet or an unopened link. If one official search or page does not establish the answer, search another appropriate official domain and actually open or fetch the result. Preserve the exact requested model; never substitute a newer model.
-
-**Only exception:** An explicitly requested, genuinely broad, cross-topic Codex setup, orientation, or system-map synthesis may use the manual first when shell execution and an allowed temporary cache are available. A specific Codex feature, setting, command, error, model, or requested citation remains docs-first. Mixed Chat/Work/Codex comparisons are official documentation questions, not manual-first Codex requests.
+Follow the current host's source order. For installed state, configuration, and troubleshooting, inspect the relevant local facts first; use official documentation when local evidence is insufficient or the question needs current public product/API information. An explicit request to search or cite sources still requires retrieval. Use a concise topic-specific query, then actually open the relevant official page; do not treat a search snippet as the full source. Preserve the exact requested model.
 
 For generic software tasks, answer the software task directly. OpenAI implementation, debugging, SDK, API, prompting, agent, and eval requests are not generic.
 
@@ -31,7 +29,7 @@
 
 ## Source and execution boundaries
 
-- Search, open, fetch, and cite only `developers.openai.com`, `platform.openai.com`, and `learn.chatgpt.com`. Cite the page that supports the claim. State uncertainty when official sources do not establish pricing, availability, account access, limits, or behavior.
+- For web research, use official OpenAI websites under the current host's domain policy, including the official support site when relevant. Cite the page that supports the claim. State uncertainty when official sources do not establish pricing, availability, account access, limits, or behavior.
 - Preserve an explicitly requested model for selection, migration, and prompting. Resolve an unspecified latest or current migration target only after searching and fetching current official guidance.
 - Use `references/latest-model.md` only as a disclosed fallback after current official model guidance does not answer the question. Read `references/upgrading-to-gpt-5p6-sol.md` only for an actual, requested GPT-5.6-family migration; read `references/prompting-guide.md` only for requested prompting work.
 - Before building, running, editing, debugging, or testing an API-backed app or tool, use `openai-platform-api-key` first when available. Documentation, conceptual examples, model selection, and read-only guidance do not require an API key.
--- a/codex-rs/skills/src/assets/samples/imagegen/SKILL.md
+++ b/codex-rs/skills/src/assets/samples/imagegen/SKILL.md
@@ -74,10 +74,11 @@
 - If the user provides no images, treat the request as **generate**.
 
 Built-in edit semantics:
-- Built-in edit mode is for images already visible in the conversation context, such as attached images or images generated earlier in the thread.
-- If the user wants to edit a local image file with the built-in tool, first load it with built-in `view_image` tool so the image is visible in the conversation context, then proceed with the built-in edit flow.
-- Do not promise arbitrary filesystem-path editing through the built-in tool.
-- If a local file still needs direct file-path control, masks, or other explicit CLI-only parameters, use the explicit CLI fallback only when the user asks for it.
+- Inspect a local target with `view_image` before editing it.
+- When every target has a local path and the current tool supports it, use `referenced_image_paths`.
+- When a target has no local path, use the smallest `num_last_images_to_include` that includes all targets, within the current tool limit. Never provide both mechanisms.
+- If neither mechanism can include all targets, ask for the missing images to be attached again.
+- Use explicit CLI fallback for CLI-only controls only when the user has requested that path; local file input alone is not a reason to switch.
 - For edits, preserve invariants aggressively and save non-destructively by default.
 
 Execution strategy:
--- a/codex-rs/skills/src/assets/samples/skill-installer/SKILL.md
+++ b/codex-rs/skills/src/assets/samples/skill-installer/SKILL.md
@@ -54,5 +54,5 @@
 - Curated listing is fetched from `https://github.com/openai/skills/tree/main/skills/.curated` via the GitHub API. If it is unavailable, explain the error and exit.
 - Private GitHub repos can be accessed via existing git credentials or optional `GITHUB_TOKEN`/`GH_TOKEN` for download.
 - Git fallback tries HTTPS first, then SSH.
-- The skills at https://github.com/openai/skills/tree/main/skills/.system are preinstalled, so no need to help users install those. If they ask, just explain this. If they insist, you can download and overwrite.
+- The skills at https://github.com/openai/skills/tree/main/skills/.system are preinstalled, so no need to help users install those. If they ask, just explain this. System package updates belong to the host update path. This installer rejects existing destinations; do not delete an existing directory to simulate an overwrite. A separately requested replacement must preserve its explicit scope and rollback.
 - Installed annotations come from `$CODEX_HOME/skills`.
--- a/codex-rs/skills/src/assets/samples/plugin-creator/references/plugin-json-spec.md
+++ b/codex-rs/skills/src/assets/samples/plugin-creator/references/plugin-json-spec.md
@@ -15,7 +15,6 @@
   "license": "MIT",
   "keywords": ["keyword1", "keyword2"],
   "skills": "./skills/",
-  "hooks": "./hooks.json",
   "mcpServers": "./.mcp.json",
   "apps": "./.app.json",
   "interface": {
@@ -62,7 +61,6 @@
 - `license` (`string`): License identifier (for example `MIT`, `Apache-2.0`).
 - `keywords` (`array` of `string`): Search/discovery tags.
 - `skills` (`string`): Relative path to skill directories/files.
-- `hooks` (`string`): Hook config path.
 - `mcpServers` (`string` or `object`): MCP config path, or an object whose keys are MCP server names and whose values are MCP server config objects.
 - `apps` (`string`): App manifest path for plugin integrations.
 - `interface` (`object`): Interface/UX metadata block for plugin presentation.
@@ -114,7 +112,7 @@
 ### Path conventions and defaults
 
 - Path values should be relative and begin with `./`.
-- `skills`, `hooks`, and string-valued `mcpServers` are supplemented on top of default component discovery; they do not replace defaults.
+- `skills` and string-valued `mcpServers` are supplemented on top of default component discovery; they do not replace defaults.
 - Custom path values must follow the plugin root convention and naming/namespacing rules.
 - This repo’s scaffold writes `.codex-plugin/plugin.json`; treat that as the manifest location this skill generates.
 
--- a/codex-rs/skills/src/assets/samples/skill-creator/scripts/quick_validate.py
+++ b/codex-rs/skills/src/assets/samples/skill-creator/scripts/quick_validate.py
@@ -57,23 +57,24 @@
     if not isinstance(name, str):
         return False, f"Name must be a string, got {type(name).__name__}"
     name = name.strip()
-    if name:
-        if not re.match(r"^[a-z0-9-]+$", name):
-            return (
-                False,
-                f"Name '{name}' should be hyphen-case (lowercase letters, digits, and hyphens only)",
-            )
-        if name.startswith("-") or name.endswith("-") or "--" in name:
-            return (
-                False,
-                f"Name '{name}' cannot start/end with hyphen or contain consecutive hyphens",
-            )
-        if len(name) > MAX_SKILL_NAME_LENGTH:
-            return (
-                False,
-                f"Name is too long ({len(name)} characters). "
-                f"Maximum is {MAX_SKILL_NAME_LENGTH} characters.",
-            )
+    if not name:
+        return False, "Name must not be empty"
+    if not re.match(r"^[a-z0-9-]+$", name):
+        return (
+            False,
+            f"Name '{name}' should be hyphen-case (lowercase letters, digits, and hyphens only)",
+        )
+    if name.startswith("-") or name.endswith("-") or "--" in name:
+        return (
+            False,
+            f"Name '{name}' cannot start/end with hyphen or contain consecutive hyphens",
+        )
+    if len(name) > MAX_SKILL_NAME_LENGTH:
+        return (
+            False,
+            f"Name is too long ({len(name)} characters). "
+            f"Maximum is {MAX_SKILL_NAME_LENGTH} characters.",
+        )
 
     description = frontmatter.get("description", "")
     if not isinstance(description, str):
@@ -81,14 +82,15 @@
     description = description.strip()
     if description.startswith("[TODO:"):
         return False, "Description contains an unfinished TODO placeholder"
-    if description:
-        if "<" in description or ">" in description:
-            return False, "Description cannot contain angle brackets (< or >)"
-        if len(description) > 1024:
-            return (
-                False,
-                f"Description is too long ({len(description)} characters). Maximum is 1024 characters.",
-            )
+    if not description:
+        return False, "Description must not be empty"
+    if "<" in description or ">" in description:
+        return False, "Description cannot contain angle brackets (< or >)"
+    if len(description) > 1024:
+        return (
+            False,
+            f"Description is too long ({len(description)} characters). Maximum is 1024 characters.",
+        )
 
     body = content[match.end() :]
     fence_marker = None
--- a/codex-rs/skills/src/assets/samples/skill-installer/scripts/install-skill-from-github.py
+++ b/codex-rs/skills/src/assets/samples/skill-installer/scripts/install-skill-from-github.py
@@ -13,6 +13,7 @@
 import urllib.error
 import urllib.parse
 import zipfile
+import unicodedata
 
 from github_utils import github_request
 
@@ -326,7 +327,8 @@
         tmp_dir = tempfile.mkdtemp(prefix="skill-install-", dir=_tmp_root())
         try:
             repo_root = _prepare_repo(source, args.method, tmp_dir)
-            installed = []
+            planned = []
+            destinations = set()
             for path in source.paths:
                 skill_name = args.name if len(source.paths) == 1 else None
                 skill_name = skill_name or os.path.basename(path.rstrip("/"))
@@ -334,19 +336,22 @@
                 if not skill_name:
                     raise InstallError("Unable to derive skill name.")
                 dest_dir = os.path.join(dest_root, skill_name)
-                if os.path.exists(dest_dir):
-                    raise InstallError(f"Destination already exists: {dest_dir}")
+                # Reject portable name aliases before any batch write.
+                destination_key = unicodedata.normalize("NFC", skill_name).casefold()
+                if os.path.lexists(dest_dir) or destination_key in destinations:
+                    raise InstallError(f"Destination already exists or is duplicated: {dest_dir}")
+                destinations.add(destination_key)
                 skill_src = os.path.join(repo_root, path)
                 _validate_skill(skill_src, repo_root)
+                planned.append((skill_name, skill_src, dest_dir))
+            for skill_name, skill_src, dest_dir in planned:
                 _copy_skill(skill_src, dest_dir)
-                installed.append((skill_name, dest_dir))
+                print(f"Installed {skill_name} to {dest_dir}", flush=True)
         finally:
             if os.path.isdir(tmp_dir):
                 shutil.rmtree(tmp_dir, ignore_errors=True)
-        for skill_name, dest_dir in installed:
-            print(f"Installed {skill_name} to {dest_dir}")
         return 0
-    except InstallError as exc:
+    except (InstallError, OSError) as exc:
         print(f"Error: {exc}", file=sys.stderr)
         return 1
 
--- a/codex-rs/skills/src/assets/samples/imagegen/scripts/image_gen.py
+++ b/codex-rs/skills/src/assets/samples/imagegen/scripts/image_gen.py
@@ -18,6 +18,7 @@
 import re
 import sys
 import time
+import unicodedata
 from typing import Any, Dict, Iterable, List, Optional, Tuple
 
 from io import BytesIO
@@ -329,6 +330,27 @@
     if suffix and not suffix.startswith("-") and not suffix.startswith("_"):
         suffix = "-" + suffix
     return path.with_name(f"{path.stem}{suffix}{path.suffix}")
+
+
+def _validate_output_paths(outputs, *, force, downscale_max_dim, downscale_suffix, seen=None):
+    seen = set() if seen is None else seen
+    targets = list(outputs)
+    if downscale_max_dim is not None:
+        targets.extend(_derive_downscale_path(p, downscale_suffix) for p in outputs)
+    for path in targets:
+        resolved = unicodedata.normalize("NFC", str(path.resolve())).casefold()
+        if resolved in seen:
+            _die(f"Duplicate output target: {path}")
+        seen.add(resolved)
+        if path.is_dir() or path.is_symlink():
+            _die(f"Output must be a regular file target: {path}")
+        if path.exists() and not force:
+            _die(f"Output already exists: {path} (use --force to overwrite)")
+        for parent in path.parents:
+            if parent.exists() or parent.is_symlink():
+                if not parent.is_dir():
+                    _die(f"Output parent is not a directory: {parent}")
+                break
 
 
 def _downscale_image_bytes(
@@ -597,41 +619,46 @@
         "moderation": args.moderation,
     }
 
+    prepared = []
+    seen = set()
+    for i, job in enumerate(jobs, start=1):
+        prompt = str(job["prompt"]).strip()
+        fields = _merge_non_null(base_fields, job.get("fields", {}))
+        # Allow flat job keys as well (use_case, scene, etc.)
+        fields = _merge_non_null(
+            fields, {k: job.get(k) for k in base_fields.keys()}
+        )
+        augmented = _augment_prompt_fields(args.augment, prompt, fields)
+
+        job_payload = dict(base_payload)
+        job_payload["prompt"] = augmented
+        job_payload = _merge_non_null(
+            job_payload, {k: job.get(k) for k in base_payload.keys()}
+        )
+        job_payload = {k: v for k, v in job_payload.items() if v is not None}
+
+        _validate_generate_payload(job_payload)
+        effective_output_format = _normalize_output_format(
+            job_payload.get("output_format")
+        )
+        _validate_transparency(
+            job_payload.get("background"), effective_output_format
+        )
+        job_payload["output_format"] = effective_output_format
+
+        n = int(job_payload.get("n", 1))
+        outputs = _job_output_paths(
+            out_dir=out_dir,
+            output_format=effective_output_format,
+            idx=i,
+            prompt=prompt,
+            n=n,
+            explicit_out=job.get("out"),
+        )
+        _validate_output_paths(outputs, force=args.force, downscale_max_dim=args.downscale_max_dim, downscale_suffix=args.downscale_suffix, seen=seen)
+        prepared.append((i, job_payload, outputs, effective_output_format))
     if args.dry_run:
-        for i, job in enumerate(jobs, start=1):
-            prompt = str(job["prompt"]).strip()
-            fields = _merge_non_null(base_fields, job.get("fields", {}))
-            # Allow flat job keys as well (use_case, scene, etc.)
-            fields = _merge_non_null(
-                fields, {k: job.get(k) for k in base_fields.keys()}
-            )
-            augmented = _augment_prompt_fields(args.augment, prompt, fields)
-
-            job_payload = dict(base_payload)
-            job_payload["prompt"] = augmented
-            job_payload = _merge_non_null(
-                job_payload, {k: job.get(k) for k in base_payload.keys()}
-            )
-            job_payload = {k: v for k, v in job_payload.items() if v is not None}
-
-            _validate_generate_payload(job_payload)
-            effective_output_format = _normalize_output_format(
-                job_payload.get("output_format")
-            )
-            _validate_transparency(
-                job_payload.get("background"), effective_output_format
-            )
-            job_payload["output_format"] = effective_output_format
-
-            n = int(job_payload.get("n", 1))
-            outputs = _job_output_paths(
-                out_dir=out_dir,
-                output_format=effective_output_format,
-                idx=i,
-                prompt=prompt,
-                n=n,
-                explicit_out=job.get("out"),
-            )
+        for i, job_payload, outputs, effective_output_format in prepared:
             downscaled = None
             if args.downscale_max_dim is not None:
                 downscaled = [
@@ -654,33 +681,9 @@
 
     any_failed = False
 
-    async def run_job(i: int, job: Dict[str, Any]) -> Tuple[int, Optional[str]]:
+    async def run_job(i, payload, outputs, effective_output_format):
         nonlocal any_failed
-        prompt = str(job["prompt"]).strip()
         job_label = f"[job {i}/{len(jobs)}]"
-
-        fields = _merge_non_null(base_fields, job.get("fields", {}))
-        fields = _merge_non_null(fields, {k: job.get(k) for k in base_fields.keys()})
-        augmented = _augment_prompt_fields(args.augment, prompt, fields)
-
-        payload = dict(base_payload)
-        payload["prompt"] = augmented
-        payload = _merge_non_null(payload, {k: job.get(k) for k in base_payload.keys()})
-        payload = {k: v for k, v in payload.items() if v is not None}
-
-        n = int(payload.get("n", 1))
-        _validate_generate_payload(payload)
-        effective_output_format = _normalize_output_format(payload.get("output_format"))
-        _validate_transparency(payload.get("background"), effective_output_format)
-        payload["output_format"] = effective_output_format
-        outputs = _job_output_paths(
-            out_dir=out_dir,
-            output_format=effective_output_format,
-            idx=i,
-            prompt=prompt,
-            n=n,
-            explicit_out=job.get("out"),
-        )
         try:
             async with sem:
                 print(f"{job_label} starting", file=sys.stderr)
@@ -711,7 +714,7 @@
             return i, str(exc)
 
     tasks = [
-        asyncio.create_task(run_job(i, job)) for i, job in enumerate(jobs, start=1)
+        asyncio.create_task(run_job(*job)) for job in prepared
     ]
 
     try:
@@ -752,6 +755,7 @@
     _validate_transparency(args.background, output_format)
     payload["output_format"] = output_format
     output_paths = _build_output_paths(args.out, output_format, args.n, args.out_dir)
+    _validate_output_paths(output_paths, force=args.force, downscale_max_dim=args.downscale_max_dim, downscale_suffix=args.downscale_suffix)
     downscaled = None
     if args.downscale_max_dim is not None:
         downscaled = [
@@ -823,6 +827,7 @@
     payload["output_format"] = output_format
     _validate_input_fidelity(args.input_fidelity)
     output_paths = _build_output_paths(args.out, output_format, args.n, args.out_dir)
+    _validate_output_paths(output_paths, force=args.force, downscale_max_dim=args.downscale_max_dim, downscale_suffix=args.downscale_suffix)
     downscaled = None
     if args.downscale_max_dim is not None:
         downscaled = [

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the bundled files under codex-rs/skills/src/assets/samples, especially skill-creator/scripts/quick_validate.py and the installer and imagegen helpers. Reproduce the synthetic-file and fake-client checks described in the issue, then verify validation and conflict checks occur before side effects. Done means accurate partial-success reporting and consistent bundled instructions, manifest examples, and overwrite guidance.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
api, documentation, tooling
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.