Black hole when trying to access a koppelevrzoek

See https://servicedesk.surf.nl/jira/browse/CXT-81149

User follows accept link from mail, gets stuck in black hole. Possibly relevant: IdP in blackhole message is SA-GW SFO.

Dashboard log is rather unhelpful:

2025-01-20 12:27:45,589  INFO [http-nio-8080-exec-9] d.s.ShibbolethPreAuthenticatedProcessingFilter:142 - Received headers {sec-fetch-mode=[cors], referer=[https://dashboard.surfconext.nl/apps/14375/saml20_sp/about/CXT-81149/accept], samesitesupport=[samesite_supported], sec-fetch-site=[same-origin], x-tls-client=[TLS_AES_256_GCM_SHA384,TLSv1.3,h2], accept-language=[en-US,en;q=0.5], cookie=[lang=en], x-forwarded-proto=[https], x-forwarded-port=[443], dnt=[1], x-forwarded-for=[77.251.xx.yy], priority=[u=4], accept=[application/json], x-real-ip=[195.169..xx.yy], sec-gpc=[1], x-forwarded-server=[09b3c5c0cb47, dashboard.surfconext.nl], x-forwarded-host=[dashboard.surfconext.nl, dashboard.surfconext.nl], host=[dashboard.surfconext.nl], connection=[Keep-Alive], accept-encoding=[gzip, deflate, br, zstd], user-agent=[Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:134.0) Gecko/20100101 Firefox/134.0], sec-fetch-dest=[empty]}
2025-01-20 12:27:45,763  INFO [http-nio-8080-exec-1] d.s.ShibbolethPreAuthenticatedProcessingFilter:142 - Received headers {sec-fetch-mode=[cors], referer=[https://dashboard.surfconext.nl/apps/14375/saml20_sp/about/CXT-81149/accept], samesitesupport=[samesite_supported], sec-fetch-site=[same-origin], x-tls-client=[TLS_AES_256_GCM_SHA384,TLSv1.3,h2], accept-language=[en-US,en;q=0.5], cookie=[lang=en; JSESSIONID=E5CE66909CAA1805A63726D6E435BB8C], x-forwarded-proto=[https], x-forwarded-port=[443], dnt=[1], x-forwarded-for=[77.251.xx.yy], priority=[u=4], accept=[application/json], x-real-ip=[195.169..xx.yy], sec-gpc=[1], x-idp-entity-id=[undefined], x-forwarded-server=[09b3c5c0cb47, dashboard.surfconext.nl], x-forwarded-host=[dashboard.surfconext.nl, dashboard.surfconext.nl], host=[dashboard.surfconext.nl], connection=[Keep-Alive], accept-encoding=[gzip, deflate, br, zstd], user-agent=[Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:134.0) Gecko/20100101 Firefox/134.0], sec-fetch-dest=[empty]}
2025-01-20 12:27:49,878  INFO [http-nio-8080-exec-1] d.s.ShibbolethPreAuthenticatedProcessingFilter:142 - Received headers {sec-fetch-mode=[cors], referer=[https://dashboard.surfconext.nl/apps/14375/saml20_sp/about/CXT-81149/accept], samesitesupport=[samesite_supported], sec-fetch-site=[same-origin], x-tls-client=[TLS_AES_256_GCM_SHA384,TLSv1.3,h2], accept-language=[en-US,en;q=0.5], cookie=[lang=en; JSESSIONID=7719E3AD00A4DD06865C734C55688596; _shibsession_64656661756c7468747470733a2f2f64617368626f6172642e73757266636f6e6578742e6e6c2f73686962626f6c657468=_1c125c500ea1f97bcc4e49bcfdf28e37], x-forwarded-proto=[https], x-forwarded-port=[443], dnt=[1], x-forwarded-for=[77.251.xx.yy], priority=[u=4], accept=[application/json], x-real-ip=[195.169..xx.yy], sec-gpc=[1], x-idp-entity-id=[undefined], x-forwarded-server=[09b3c5c0cb47, dashboard.surfconext.nl], x-forwarded-host=[dashboard.surfconext.nl, dashboard.surfconext.nl], host=[dashboard.surfconext.nl], connection=[Keep-Alive], accept-encoding=[gzip, deflate, br, zstd], user-agent=[Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:134.0) Gecko/20100101 Firefox/134.0], sec-fetch-dest=[empty]}
2025-01-20 12:27:51,888  INFO [http-nio-8080-exec-10] d.s.ShibbolethPreAuthenticatedProcessingFilter:142 - Received headers {sec-fetch-mode=[cors], referer=[https://dashboard.surfconext.nl/apps/14375/saml20_sp/about/CXT-81149/accept], samesitesupport=[samesite_supported], sec-fetch-site=[same-origin], x-tls-client=[TLS_AES_256_GCM_SHA384,TLSv1.3,h2], accept-language=[en-US,en;q=0.5], cookie=[lang=en; JSESSIONID=FD9D61695540F7ABB1838E2CD9746D48; _shibsession_64656661756c7468747470733a2f2f64617368626f6172642e73757266636f6e6578742e6e6c2f73686962626f6c657468=_21ec1d95f0bc9dc0a5ba35424b3ba356], x-forwarded-proto=[https], x-forwarded-port=[443], dnt=[1], x-forwarded-for=[77.251.xx.yy], priority=[u=4], accept=[application/json], x-real-ip=[195.169..xx.yy], sec-gpc=[1], x-idp-entity-id=[undefined], x-forwarded-server=[09b3c5c0cb47, dashboard.surfconext.nl], x-forwarded-host=[dashboard.surfconext.nl, dashboard.surfconext.nl], host=[dashboard.surfconext.nl], connection=[Keep-Alive], accept-encoding=[gzip, deflate, br, zstd], user-agent=[Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:134.0) Gecko/20100101 Firefox/134.0], sec-fetch-dest=[empty]}
2025-01-20 12:27:54,154  INFO [http-nio-8080-exec-2] d.s.ShibbolethPreAuthenticatedProcessingFilter:142 - Received headers {sec-fetch-mode=[cors], referer=[https://dashboard.surfconext.nl/apps/14375/saml20_sp/about/CXT-81149/accept], samesitesupport=[samesite_supported], sec-fetch-site=[same-origin], x-tls-client=[TLS_AES_256_GCM_SHA384,TLSv1.3,h2], accept-language=[en-US,en;q=0.5], cookie=[lang=en; JSESSIONID=5E63FA05988A41173D6ADD94C353ED74; _shibsession_64656661756c7468747470733a2f2f64617368626f6172642e73757266636f6e6578742e6e6c2f73686962626f6c657468=_e916d3a12af39367ff573fc933bd1ddc], x-forwarded-proto=[https], x-forwarded-port=[443], dnt=[1], x-forwarded-for=[77.251.xx.yy], priority=[u=4], accept=[application/json], x-real-ip=[195.169..xx.yy], sec-gpc=[1], x-idp-entity-id=[undefined], x-forwarded-server=[09b3c5c0cb47, dashboard.surfconext.nl], x-forwarded-host=[dashboard.surfconext.nl, dashboard.surfconext.nl], host=[dashboard.surfconext.nl], connection=[Keep-Alive], accept-encoding=[gzip, deflate, br, zstd], user-agent=[Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:134.0) Gecko/20100101 Firefox/134.0], sec-fetch-dest=[empty]}
2025-01-20 12:27:56,393  INFO [http-nio-8080-exec-7] d.s.ShibbolethPreAuthenticatedProcessingFilter:142 - Received headers {sec-fetch-mode=[cors], referer=[https://dashboard.surfconext.nl/apps/14375/saml20_sp/about/CXT-81149/accept], samesitesupport=[samesite_supported], sec-fetch-site=[same-origin], x-tls-client=[TLS_AES_256_GCM_SHA384,TLSv1.3,h2], accept-language=[en-US,en;q=0.5], cookie=[lang=en; JSESSIONID=E6FCC8FE721E66411CB1344B3DE4B838; _shibsession_64656661756c7468747470733a2f2f64617368626f6172642e73757266636f6e6578742e6e6c2f73686962626f6c657468=_28ea7f39359e3591b96d4b5a80fc2d91], x-forwarded-proto=[https], x-forwarded-port=[443], dnt=[1], x-forwarded-for=[77.251.xx.yy], priority=[u=4], accept=[application/json], x-real-ip=[195.169..xx.yy], sec-gpc=[1], x-idp-entity-id=[undefined], x-forwarded-server=[09b3c5c0cb47, dashboard.surfconext.nl], x-forwarded-host=[dashboard.surfconext.nl, dashboard.surfconext.nl], host=[dashboard.surfconext.nl], connection=[Keep-Alive], accept-encoding=[gzip, deflate, br, zstd], user-agent=[Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:134.0) Gecko/20100101 Firefox/134.0], sec-fetch-dest=[empty]}
2025-01-20 12:28:28,435  INFO [http-nio-8080-exec-5] d.s.ShibbolethPreAuthenticatedProcessingFilter:142 - Received headers {sec-fetch-mode=[cors], referer=[https://dashboard.surfconext.nl/apps/14375/saml20_sp/about/CXT-81149/accept], samesitesupport=[samesite_supported], sec-fetch-site=[same-origin], x-tls-client=[TLS_AES_256_GCM_SHA384,TLSv1.3,h2], accept-language=[en-US,en;q=0.5], cookie=[lang=en], x-forwarded-proto=[https], x-forwarded-port=[443], dnt=[1], x-forwarded-for=[77.251.xx.yy], priority=[u=4], accept=[application/json], x-real-ip=[195.169..xx.yy], sec-gpc=[1], x-forwarded-server=[887a73f9929d, dashboard.surfconext.nl], x-forwarded-host=[dashboard.surfconext.nl, dashboard.surfconext.nl], host=[dashboard.surfconext.nl], connection=[Keep-Alive], accept-encoding=[gzip, deflate, br, zstd], user-agent=[Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:134.0) Gecko/20100101 Firefox/134.0], sec-fetch-dest=[empty]}
2025-01-20 12:28:28,803  INFO [http-nio-8080-exec-6] d.s.ShibbolethPreAuthenticatedProcessingFilter:142 - Received headers {sec-fetch-mode=[cors], referer=[https://dashboard.surfconext.nl/apps/14375/saml20_sp/about/CXT-81149/accept], samesitesupport=[samesite_supported], sec-fetch-site=[same-origin], x-tls-client=[TLS_AES_256_GCM_SHA384,TLSv1.3,h2], accept-language=[en-US,en;q=0.5], cookie=[lang=en; JSESSIONID=74E76568A20EA02DA0990E4402EE3D90], x-forwarded-proto=[https], x-forwarded-port=[443], dnt=[1], x-forwarded-for=[77.251.xx.yy], priority=[u=4], accept=[application/json], x-real-ip=[195.169..xx.yy], sec-gpc=[1], x-idp-entity-id=[undefined], x-forwarded-server=[887a73f9929d, dashboard.surfconext.nl], x-forwarded-host=[dashboard.surfconext.nl, dashboard.surfconext.nl], host=[dashboard.surfconext.nl], connection=[Keep-Alive], accept-encoding=[gzip, deflate, br, zstd], user-agent=[Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:134.0) Gecko/20100101 Firefox/134.0], sec-fetch-dest=[empty]}
2025-01-20 12:28:32,970  INFO [http-nio-8080-exec-5] d.s.ShibbolethPreAuthenticatedProcessingFilter:142 - Received headers {sec-fetch-mode=[cors], referer=[https://dashboard.surfconext.nl/apps/14375/saml20_sp/about/CXT-81149/accept], samesitesupport=[samesite_supported], sec-fetch-site=[same-origin], x-tls-client=[TLS_AES_256_GCM_SHA384,TLSv1.3,h2], accept-language=[en-US,en;q=0.5], cookie=[lang=en; JSESSIONID=13468273368DD8F3C3175F54DEA2E29F; _shibsession_64656661756c7468747470733a2f2f64617368626f6172642e73757266636f6e6578742e6e6c2f73686962626f6c657468=_c8003dd4d3f1f029926b157e29b09bf5], x-forwarded-proto=[https], x-forwarded-port=[443], dnt=[1], x-forwarded-for=[77.251.xx.yy], priority=[u=4], accept=[application/json], x-real-ip=[195.169..xx.yy], sec-gpc=[1], x-idp-entity-id=[undefined], x-forwarded-server=[887a73f9929d, dashboard.surfconext.nl], x-forwarded-host=[dashboard.surfconext.nl, dashboard.surfconext.nl], host=[dashboard.surfconext.nl], connection=[Keep-Alive], accept-encoding=[gzip, deflate, br, zstd], user-agent=[Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:134.0) Gecko/20100101 Firefox/134.0], sec-fetch-dest=[empty]}
2025-01-20 12:28:35,064  INFO [http-nio-8080-exec-9] d.s.ShibbolethPreAuthenticatedProcessingFilter:142 - Received headers {sec-fetch-mode=[cors], referer=[https://dashboard.surfconext.nl/apps/14375/saml20_sp/about/CXT-81149/accept], samesitesupport=[samesite_supported], sec-fetch-site=[same-origin], x-tls-client=[TLS_AES_256_GCM_SHA384,TLSv1.3,h2], accept-language=[en-US,en;q=0.5], cookie=[lang=en; JSESSIONID=6ADF54722100A849AAC2ED0D0F8A8C0E; _shibsession_64656661756c7468747470733a2f2f64617368626f6172642e73757266636f6e6578742e6e6c2f73686962626f6c657468=_53ce42fefc405f69f5fa2e7cd5fe3dd8], x-forwarded-proto=[https], x-forwarded-port=[443], dnt=[1], x-forwarded-for=[77.251.xx.yy], priority=[u=4], accept=[application/json], x-real-ip=[195.169..xx.yy], sec-gpc=[1], x-idp-entity-id=[undefined], x-forwarded-server=[887a73f9929d, dashboard.surfconext.nl], x-forwarded-host=[dashboard.surfconext.nl, dashboard.surfconext.nl], host=[dashboard.surfconext.nl], connection=[Keep-Alive], accept-encoding=[gzip, deflate, br, zstd], user-agent=[Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:134.0) Gecko/20100101 Firefox/134.0], sec-fetch-dest=[empty]}
2025-01-20 12:28:38,809  INFO [http-nio-8080-exec-10] d.s.ShibbolethPreAuthenticatedProcessingFilter:142 - Received headers {sec-fetch-mode=[cors], referer=[https://dashboard.surfconext.nl/apps/14375/saml20_sp/about/CXT-81149/accept], samesitesupport=[samesite_supported], sec-fetch-site=[same-origin], x-tls-client=[TLS_AES_256_GCM_SHA384,TLSv1.3,h2], accept-language=[en-US,en;q=0.5], cookie=[lang=en; JSESSIONID=9F06C4179AA5BF6D92EB9F2CF5FC95C7; _shibsession_64656661756c7468747470733a2f2f64617368626f6172642e73757266636f6e6578742e6e6c2f73686962626f6c657468=_d040d29194506e3d783457931b8a52bd], x-forwarded-proto=[https], x-forwarded-port=[443], dnt=[1], x-forwarded-for=[77.251.xx.yy], priority=[u=4], accept=[application/json], x-real-ip=[195.169..xx.yy], sec-gpc=[1], x-idp-entity-id=[undefined], x-forwarded-server=[887a73f9929d, dashboard.surfconext.nl], x-forwarded-host=[dashboard.surfconext.nl, dashboard.surfconext.nl], host=[dashboard.surfconext.nl], connection=[Keep-Alive], accept-encoding=[gzip, deflate, br, zstd], user-agent=[Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:134.0) Gecko/20100101 Firefox/134.0], sec-fetch-dest=[empty]}
2025-01-20 12:28:40,745  INFO [http-nio-8080-exec-3] d.s.ShibbolethPreAuthenticatedProcessingFilter:142 - Received headers {sec-fetch-mode=[cors], referer=[https://dashboard.surfconext.nl/apps/14375/saml20_sp/about/CXT-81149/accept], samesitesupport=[samesite_supported], sec-fetch-site=[same-origin], x-tls-client=[TLS_AES_256_GCM_SHA384,TLSv1.3,h2], accept-language=[en-US,en;q=0.5], cookie=[lang=en; JSESSIONID=CD62BAC7476D434E4694D4592606AC5F; _shibsession_64656661756c7468747470733a2f2f64617368626f6172642e73757266636f6e6578742e6e6c2f73686962626f6c657468=_d6fd5cfb4dc23286f8265b8cbc7ecb9c], x-forwarded-proto=[https], x-forwarded-port=[443], dnt=[1], x-forwarded-for=[77.251.xx.yy], priority=[u=4], accept=[application/json], x-real-ip=[195.169..xx.yy], sec-gpc=[1], x-idp-entity-id=[undefined], x-forwarded-server=[887a73f9929d, dashboard.surfconext.nl], x-forwarded-host=[dashboard.surfconext.nl, dashboard.surfconext.nl], host=[dashboard.surfconext.nl], connection=[Keep-Alive], accept-encoding=[gzip, deflate, br, zstd], user-agent=[Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:134.0) Gecko/20100101 Firefox/134.0], sec-fetch-dest=[empty]}
2025-01-20 12:33:21,865  INFO [http-nio-8080-exec-5] d.s.ShibbolethPreAuthenticatedProcessingFilter:142 - Received headers {sec-fetch-mode=[cors], referer=[https://dashboard.surfconext.nl/], samesitesupport=[samesite_supported], sec-fetch-site=[same-origin], x-tls-client=[TLS_AES_256_GCM_SHA384,TLSv1.3,h2], accept-language=[en-US,en;q=0.9,nl;q=0.8], cookie=[lang=en], x-forwarded-proto=[https], x-forwarded-port=[443], x-forwarded-for=[2001:610:350:216::daf], priority=[u=1, i], accept=[application/json], x-real-ip=[145.101..xx.yy], sec-ch-ua=["Microsoft Edge";v="131", "Chromium";v="131", "Not_A Brand";v="24"], sec-ch-ua-mobile=[?0], x-forwarded-server=[4c74b519fbe4, dashboard.surfconext.nl], x-forwarded-host=[dashboard.surfconext.nl, dashboard.surfconext.nl], sec-ch-ua-platform=["Windows"], host=[dashboard.surfconext.nl], connection=[Keep-Alive], accept-encoding=[gzip, deflate, br, zstd], user-agent=[Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0..xx.yy Safari/537.36 Edg/131.0..xx.yy], sec-fetch-dest=[empty]}
2025-01-20 12:33:26,831  INFO [http-nio-8080-exec-2] d.s.ShibbolethPreAuthenticatedProcessingFilter:142 - Received headers {sec-fetch-mode=[cors], referer=[https://dashboard.surfconext.nl/apps/14375/saml20_sp/about/CXT-81149/accept], samesitesupport=[samesite_supported], sec-fetch-site=[same-origin], x-tls-client=[TLS_AES_256_GCM_SHA384,TLSv1.3,h2], accept-language=[en-US,en;q=0.5], x-forwarded-proto=[https], x-forwarded-port=[443], dnt=[1], x-forwarded-for=[77.251.xx.yy], priority=[u=4], pragma=[no-cache], accept=[application/json], x-real-ip=[145.101.xx.yy], x-forwarded-server=[4c74b519fbe4, dashboard.surfconext.nl], x-forwarded-host=[dashboard.surfconext.nl, dashboard.surfconext.nl], host=[dashboard.surfconext.nl], connection=[Keep-Alive], cache-control=[no-cache], accept-encoding=[gzip, deflate, br, zstd], user-agent=[Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:134.0) Gecko/20100101 Firefox/134.0], sec-fetch-dest=[empty]}
2025-01-20 12:33:26,870  INFO [http-nio-8080-exec-5] d.s.ShibbolethPreAuthenticatedProcessingFilter:142 - Received headers {sec-fetch-mode=[cors], referer=[https://dashboard.surfconext.nl/apps/14375/saml20_sp/about/CXT-81149/accept], samesitesupport=[samesite_supported], sec-fetch-site=[same-origin], x-tls-client=[TLS_AES_256_GCM_SHA384,TLSv1.3,h2], accept-language=[en-US,en;q=0.5], x-forwarded-proto=[https], x-forwarded-port=[443], dnt=[1], x-forwarded-for=[77.251.xx.yy], priority=[u=4], pragma=[no-cache], accept=[application/json], x-real-ip=[145.101.xx.yy], x-idp-entity-id=[undefined], x-forwarded-server=[09b3c5c0cb47, dashboard.surfconext.nl], x-forwarded-host=[dashboard.surfconext.nl, dashboard.surfconext.nl], host=[dashboard.surfconext.nl], connection=[Keep-Alive], cache-control=[no-cache], accept-encoding=[gzip, deflate, br, zstd], user-agent=[Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:134.0) Gecko/20100101 Firefox/134.0], sec-fetch-dest=[empty]}
2025-01-20 12:33:26,942  INFO [http-nio-8080-exec-8] d.s.ShibbolethPreAuthenticatedProcessingFilter:142 - Received headers {sec-fetch-mode=[navigate], referer=[https://dashboard.surfconext.nl/apps/14375/saml20_sp/about/CXT-81149/accept], samesitesupport=[samesite_supported], sec-fetch-site=[same-origin], x-tls-client=[TLS_AES_256_GCM_SHA384,TLSv1.3,h2], accept-language=[en-US,en;q=0.5], x-forwarded-proto=[https], x-forwarded-port=[443], dnt=[1], x-forwarded-for=[77.251.xx.yy], priority=[u=0, i], pragma=[no-cache], accept=[text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8], x-real-ip=[145.101.xx.yy], x-forwarded-server=[887a73f9929d, dashboard.surfconext.nl], x-forwarded-host=[dashboard.surfconext.nl, dashboard.surfconext.nl], host=[dashboard.surfconext.nl], upgrade-insecure-requests=[1], connection=[Keep-Alive], cache-control=[no-cache], accept-encoding=[gzip, deflate, br, zstd], user-agent=[Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:134.0) Gecko/20100101 Firefox/134.0], sec-fetch-dest=[document]}
2025-01-20 12:33:26,946  INFO [http-nio-8080-exec-3] d.s.ShibbolethPreAuthenticatedProcessingFilter:142 - Received headers {sec-fetch-mode=[cors], referer=[https://dashboard.surfconext.nl/apps/14375/saml20_sp/about/CXT-81149/accept], samesitesupport=[samesite_supported], sec-fetch-site=[same-origin], x-tls-client=[TLS_AES_256_GCM_SHA384,TLSv1.3,h2], accept-language=[en-US,en;q=0.5], x-forwarded-proto=[https], x-forwarded-port=[443], dnt=[1], x-forwarded-for=[77.251.xx.yy], priority=[u=4], pragma=[no-cache], accept=[application/json], x-real-ip=[145.101.xx.yy], x-idp-entity-id=[undefined], x-forwarded-server=[d010a1ae873c, dashboard.surfconext.nl], x-forwarded-host=[dashboard.surfconext.nl, dashboard.surfconext.nl], host=[dashboard.surfconext.nl], connection=[Keep-Alive], cache-control=[no-cache], accept-encoding=[gzip, deflate, br, zstd], user-agent=[Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:134.0) Gecko/20100101 Firefox/134.0], sec-fetch-dest=[empty]}

The shib logs are nor preserved, it seems.

Start with the CXT-81149 accept-link flow and the dashboard log entries from ShibbolethPreAuthenticatedProcessingFilter, paying attention to the missing x-idp-entity-id value and the SA-GW SFO IdP. Reproduce the black-hole behavior and trace the request until the cause is identified; done means the user can follow the mail link and complete the accept flow.