DataTalksClub / DataTalksClub/website
Accept safe localized podcast resources without weakening external URL policy
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Product outcome
Restore every current podcast season hub and episode detail after the projection builder correctly localized three same-site show-note resources. Runtime podcast resources accept either:
- a bounded external HTTPS URL; or
- a safe root-relative same-site URL, preserving its safe query and fragment.
The correction must not weaken guest-path security, shared URL helpers, source/projection reproducibility, podcast canonical routes, or the external-resource policy.
PM disposition: proceed as one bounded P0 runtime/render regression. The builder already owns the accepted localization. Engineering only needs to make the podcast runtime enforce the same resource-specific destination union.
Normative authority and exact evidence
- URL, link, and SEO compatibility: established podcast
.htmldetails, safe internal/external destinations, suffix preservation, and crawlable render behavior. - GitHub content and people: immutable podcast source/projection adapter and checked public projection.
- Security/privacy/operations: fail-closed, bounded, non-disclosing public behavior.
- Verification strategy: podcast hub/detail, link, desktop/mobile, no-JS, and release gates.
- #162 owns podcast detail canonical/alias behavior; #223 owns path-only guest-profile links; #253 owns complete source-to-projection reproducibility.
- Exact regression base:
face8e4808d65afbf0374d1ced7a88079950d663. - Introducing semantic change:
912f60fff9aed1dfd306fe9eba9d465674976dcc. Its builder keeps safe external HTTPS resources and localizes accepteddatatalks.club/www.datatalks.clubresources with_localize_internal_url, retaining parsed path, query, and fragment. Runtime_resourcesstill calls the external-only_safe_external_url(..., https_only=True).
Authoritative checked census
The raw phrase “23 podcast detail pages” is superseded by the reproducible checked inventory:
- 205 episodes total;
- 153 episodes with resources;
- 461 resources total;
- 458 external HTTPS resources;
- exactly 3 localized root-relative resources on exactly 3 episode details, affecting season hubs 8, 12, and 20;
- zero resource destinations outside those two classes.
| Season / episode | Owning detail | Exact localized resource |
|---|---|---|
| 20 / 9 | /podcast/data-freelancing-career-strategy-market-demand-and-client-acquisition.html |
/podcast/s16e09-become-data-freelancer.html |
| 12 / 7 | /podcast/how-to-grow-your-ml-engineering-career.html |
/podcast/s11e03-from-data-science-to-dataops.html |
| 8 / 6 | /podcast/hiring-for-data-engineering-jobs-in-europe.html |
/podcast/s07e02-recruiting-data-professionals.html |
Current runtime therefore fails those three detail compositions and the three hubs that eagerly compose their season records. The other 202 details and 21 season hubs are not permitted to change.
Scope
1. Add a podcast-resource destination validator
In content/podcast_content.py, add one private resource-specific validator used only by _resources. Do not change _safe_external_url, safe_public_graph_url, or their listening-platform, watch/player, guest, Person, Wiki, or graph consumers.
The validator accepts a non-empty string after the existing surrounding-whitespace trim, with a maximum of 2,048 characters, in exactly one class:
External HTTPS
- parsed scheme is HTTPS and an authority/hostname is present;
- username and password are absent;
- any explicit port parses as a valid numeric port;
- safe path, query, and fragment are retained byte-for-byte after surrounding trim;
- no other scheme, scheme-relative form, missing authority, credential, backslash, control character, malformed percent escape, or raw/once-percent-decoded path traversal is accepted.
Existing query/fragment-bearing safe external HTTPS resources remain accepted and unchanged. Plain HTTP remains rejected at runtime; the builder's existing source rule continues omitting reviewed historical HTTP-only resources rather than upgrading them.
Same-site root-relative
- begins with exactly one
/; - parsed scheme, authority, username, password, and port are absent;
- path is non-empty and root-relative;
- safe query and fragment are allowed and returned unchanged, including the accepted builder example
/podcast/example.html?from=notes#topic; - no backslash, control character, malformed percent escape, or raw/once-percent-decoded path traversal is accepted.
For both classes, preflight raw and decoded path/query/fragment before returning. Reject every C0 control and DEL, raw or percent-decoded backslash, malformed/incomplete/non-hex % escape, invalid UTF-8 percent decoding, a .. path segment after one decode (including encoded separators that expose it), invalid bracketed host, and invalid/out-of-range port. Do not reflect the candidate URL or arbitrary parser exception in the error or logs.
An invalid checked runtime resource raises one stable bounded ImproperlyConfigured category such as Public podcast resource URL is unsafe. before a response renders. It is not silently dropped at runtime and is never emitted as an href. This preserves the existing fail-closed distinction: the source builder alone owns intentional omission of historical HTTP-only source links; runtime projection drift blocks.
2. Preserve resource and page behavior
- Keep the existing exact resource mapping/list/title validation and source order. Do not deduplicate, sort, relabel, normalize suffixes, upgrade HTTP, resolve redirects, fetch destinations, or infer replacements.
- All 461 checked titles and URLs survive in their exact order; the 458 external URLs remain byte-identical and the three local URLs remain the exact values above.
templates/public/podcast_detail.htmlremains unchanged. Every resource stays in Show Notes with the current escaped title, exact DOMhref,target="_blank",rel="noopener noreferrer", and screen-reader suffix. For a query containing&, serialized HTML escaping may use&, while the DOM destination must remain the original URL.- Season hubs do not display resource links, but all 24 must compose their complete season and return their existing response/canonical/navigation behavior. In particular
/podcast?season=8,?season=12, and?season=20return200again. - All 205 established detail paths return their existing
GET/HEAD, canonical, metadata, cache, section/tab, guest, transcript, media, and navigation behavior. The three affected details render their local resource once; the other 202 details are unchanged. - Unsafe synthetic runtime data fails before unsafe markup. It does not cause a fallback link, partial resource list, external request, redirect, secret/PII disclosure, or source value in the error.
3. Preserve builder and adjacent contracts
The existing builder contract is authoritative and is not changed here:
- resource input is an optional list of exact
{title, url}mappings; - title is non-empty and at most 500 characters;
- URL is non-empty and at most 2,048 characters;
- unsafe schemes/shapes fail source adaptation;
- reviewed plain HTTP resources are omitted without manufactured HTTPS;
- safe HTTPS external resources keep source order;
- safe same-site absolute resources localize to root-relative while preserving query/fragment.
Existing focused builder tests must pass unchanged. #223's podcast-local guest validator remains path-only and must still reject every guest query/fragment; no resource policy may be reused at that boundary. #162's .html canonical/alias contract and #253's source/projection/pin/manifest authority remain untouched.
Allowed files
Runtime ownership:
content/podcast_content.py
Focused test ownership:
content/tests/test_podcast_catalog.py- if persistent browser coverage is missing, only
playwright_tests/test_podcast_episode_parity.pyand/orplaywright_tests/test_podcast_seasons.py
content/tests/test_public_projection_builder.py must run unchanged as the builder regression contract. No template, builder, shared helper, generated projection, spec, manifest, or source file is allowed. If another production path is necessary, stop and return to PM.
Dependencies and ordered combined delivery
There is no product/interface dependency on #162, #223, or #253: #270 neither consumes a new domain contract from them nor repairs their product scope. #253 remains separate source/projection recovery; #270 changes no generated byte, and a later authoritative regeneration must independently reproduce the three localized links.
#223 and #270 do overlap content/podcast_content.py and content/tests/test_podcast_catalog.py. Current #223 smoke cannot terminate green until #270 repairs the already-present localized-resource 500s, so the former land-#223-first rule is superseded by one ordered combined integration candidate after the shared current-main recovery blockers are repaired and accepted:
- Freeze the repaired-base #223 guest-boundary patch and exact fingerprint first. Its engineer owns only #223 focused behavior and does not implement, test, or accept #270.
- The #270 engineer prepares only this issue's resource validator/runtime/render delta with the frozen #223 patch as ordered input. It must not edit or reinterpret the guest wrapper, guest path-only grammar, fallback/mismatch behavior, or #223 tests, and it does not test or accept #223.
- A named integration owner—not either independent tester—constructs one uncommitted candidate from the same repaired base: exact #223 patch first, exact #270 delta second. Mechanical overlap is resolved without scope change; any semantic conflict or fingerprint drift returns to the owning engineer/PM.
- Generate one fresh full plan/envelope for the exact combined source fingerprint. One independent tester validates both patch identities and runs that full envelope once, but posts separate tester-final reports and screenshot evidence/dispositions for #223 and #270. All required #270 route screenshots remain under
.tmp/screenshots/issue-270/; they do not prove #223 acceptance. - PM accepts each issue separately. Only after both PASS, create focused commits from the accepted patch boundaries in exact order:
Closes #223, thenCloses #270. The two-commit tip must be byte-identical to the tested tree. Merge/push once; on-call observes the one resulting exact release SHA.
No engineer is a tester for the other issue. No focused engineer run, old face8e4 failure/partial output, or cross-issue assertion is independent acceptance evidence.
Acceptance criteria
- Runtime resources accept exactly safe external HTTPS or safe same-site root-relative destinations, with safe query/fragment retained byte-for-byte and every stated bound/type/parser/security rule enforced.
- Invalid runtime resources raise the stable non-disclosing domain/configuration error before rendering; no builtin parser exception, raw URL, unsafe
href, partial list, redirect, or network request escapes. - The 205/153/461 census is exact: 458 unchanged external HTTPS resources plus the three exact local resources in unchanged source order, with no third destination class.
- All 24 season hubs and all 205 detail records compose. Hubs 8/12/20 and the three named details return
200; the other 21 hubs/202 details remain unchanged. - Each named detail renders its local resource once in Show Notes with exact title/destination and existing target/rel/accessibility behavior; external resources and no-resource episodes retain their current states.
- #223 direct/resolved guest paths remain path-only and reject query/fragment; #162 canonical/alias/SEO behavior is unchanged.
- The builder's HTTP omission, HTTPS acceptance, same-site localization, suffix preservation, order, shape/length, and unsafe rejection tests pass unchanged.
- No generated projection/source/pin/provenance/manifest/hash, builder/shared helper, template/static, route/SEO/cache, copy/design, model/migration, or dependency change occurs.
- Focused tests and fresh full format/lint/type, Django, Playwright, screenshots, content/source invariants, migration drift, container, and versioned verification gates terminate green on one frozen candidate. No retry, skip, xfail, quarantine, timeout extension, assertion weakening, partial run, or unchanged-red evidence is success.
- Engineer and independent tester record exact base/candidate/diff/plan/run/report/screenshot digests and terminal counts; PM evaluates all named routes and states before acceptance.
Required test scenarios
- External HTTPS: hostname/root/path, valid port, query, fragment, query+fragment, mixed-case scheme, and surrounding trim remain accepted without suffix rewriting.
- Local:
/, a.htmlpath, and/podcast/example.html?from=notes#topicremain accepted exactly. - Reject non-string/empty/whitespace/over-2,048; HTTP and every non-HTTPS scheme;
//host; local authority/scheme/credentials; missing external host; username/password; backslash; raw C0/DEL; malformed%; invalid UTF-8; invalid bracketed host/port. - Reject raw and percent-encoded
..path segments, encoded separators that expose traversal, and raw/decoded controls/backslashes in path, query, or fragment. Safe dots inside an ordinary segment and safe percent-encoded data remain accepted. - Resource list/mapping/title behavior: missing/null/list variants, non-mapping row, missing/blank/non-string title/URL, several valid resources, one late invalid resource, stable order, generic bounded error, and no mutation of input.
- Exact projection census: 205 episodes, 153 with resources, 461 total, 458 external plus three exact local; every record composes and each output resource equals its checked input in order.
- Django route matrix: all 24 hubs and 205 details
GET/HEAD; explicitly assert 8/12/20 and the three named details are green, local resource DOM/link behavior is exact, unaffected and no-resource controls remain unchanged, and unsafe injected projection cannot emit markup. - Regression boundaries: unchanged builder tests; #223 direct and resolved guest
?/#rejection; #162.htmlfinal/alias/canonical tests; current Wiki query/fragment behavior where selected by the graph.
Browser and screenshot scenarios
The issue has current rendered impact, so browser verification and screenshots are applicable.
- At desktop
1440x900and mobile390x844, open each of/podcast?season=8,/podcast?season=12, and/podcast?season=20. Verify200, correct season/episode count and navigation, no error/debug page, accessibility regression, failed request, console error, or horizontal overflow. - At both viewports open each of the three named details. Verify the Show Notes tab/section, exact localized link title and DOM
href, existingtarget/rel/screen-reader text, canonical path, surrounding external links/order, and layout. Do not follow an external resource during acceptance. - Run a representative affected detail with JavaScript disabled at
320x800: Show Notes and its local link remain native, visible, and reachable; no script is required. A separate no-JS screenshot is not required because JavaScript behavior does not change, but the browser assertion is required. - Independent tester captures and reads all 12 desktop/mobile affected-route screenshots under
.tmp/screenshots/issue-270/, recording route, viewport, candidate/diff/render digest, PNG digest, overflow/accessibility/console verdict, and proof the image is the intended page rather than an error.
Verification contract
Use only uv-backed commands/Make targets. The #270 engineer runs this issue's focused Django/resource/builder/route/SEO modules and focused podcast Playwright files without acting as #223 tester. The independent tester then generates the versioned plan from the exact ordered combined fingerprint and executes one fresh full profile: locked format/lint/type/quality, full Django, full Playwright, both issues' separately attributed screenshot envelope/dispositions, content/source/projection invariants, migration drift, container, evidence validation, and report validation. Unknown graph/history selects full; no broad evidence is reused merely because #223 touched the same files.
- Migrations: no model/migration change; new migration execution
not_applicable, migration drift required. - Render/screenshots:
rerun, as above; nevernot_applicable. - Generated-content/#253 attestation: no generated path may change. The final diff must prove this component
not_applicable; any generated/source/pin/manifest/hash change stops the lane and returns to PM/#253. - Deployment: not part of engineering/testing. After tester PASS and PM acceptance, engineer commits with
Closes #270; orchestrator merges/pushes and on-call alone observes exact-SHA CI/deployment.
Non-goals
- No change to generated JSON/assets, source checkout/pin/attestation, projection builder/localizer, manifest/tree/hash, terminology policy, or #253 recovery.
- No change to podcast detail paths/canonicals/aliases, season query grammar/order/navigation, sitemap/SEO/cache, or #162 work.
- No change to podcast guest or Person public-path policy; queries/fragments remain forbidden there under #223.
- No change to listening/platform/watch/player/external-profile URL policy,
_safe_external_url,safe_public_graph_url, Wiki URL policy, or any global/shared helper. - No template/CSS/JavaScript/copy/design/accessibility redesign, link-label cleanup, target/rel policy change, resource deduplication/sort, URL normalization, HTTP upgrade, redirect following, synchronous fetch, or broken-link crawler.
- No model/migration/dependency/tool/configuration change, broad refactor, unrelated test/gate repair, production/provider action, commit/push/merge/deploy during engineering or grooming.
Lifecycle
The #270 engineer owns only its resource-policy/runtime/render delta and focused report, using the repaired-base #223 patch as ordered immutable input. The integration owner constructs the exact #223-then-#270 uncommitted tree. A separate tester recomputes and runs the shared full envelope, captures/inspects #270 screenshots, and posts a distinct #270 tester-final report; PM accepts #270 separately. After both issues pass, create Closes #223 then Closes #270, verify the tip matches the tested tree, and merge/push once. On-call alone observes the resulting exact-SHA CI/deployment. Neither scope engineer tests or accepts the other issue.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in content/podcast_content.py at _resources and the existing _safe_external_url path, then read content/tests/test_podcast_catalog.py and the unchanged builder tests. Add focused coverage for the two permitted destination classes, rejection and fail-closed behavior, census preservation, and affected hub/detail rendering; done means all 205 details, 24 hubs, and the specified regression tests retain their contracts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, security, testing-qa
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100