Paywall site: businesslive.co.za

businesslive.co.za.txt

requires_login: yes

login_uri: https://www.businesslive.co.za/u/sign-in
login_username_field: email-address
login_password_field: password

not_logged_in_xpath: //a[contains(@href, 'sign-in')]

Running on Dockerised Wallabag

paywall definitions on mounted volume
compose.yml

version: '3'
services:
wallabag:
image: wallabag/wallabag
environment:
- MYSQL_ROOT_PASSWORD=xxxx
- POPULATE_DATABASE=false
- SYMFONY__ENV__DATABASE_DRIVER=pdo_mysql
- SYMFONY__ENV__DATABASE_HOST=xxxx
- SYMFONY__ENV__DATABASE_PORT=3306
- SYMFONY__ENV__DATABASE_NAME=xxx
- SYMFONY__ENV__DATABASE_USER=XXX
- SYMFONY__ENV__DATABASE_PASSWORD=xxx
- SYMFONY__ENV__DATABASE_CHARSET=utf8mb4
- SYMFONY__ENV__MAILER_HOST=mail
- SYMFONY__ENV__MAILER_USER=~
- SYMFONY__ENV__MAILER_PASSWORD=~
- SYMFONY__ENV__FROM_EMAIL=wallabag@foo.bar
- SYMFONY__ENV__DOMAIN_NAME=https://wallabag.foo.bar
- SYMFONY__ENV__TWOFACTOR_SENDER=noreply@foo.bar
- SYMFONY__ENV__SERVER_NAME="Foo Bar Wallabag"
ports:
- "7000:80"
volumes:
- /opt/wallabag/images:/var/www/wallabag/web/assets/images
- /opt/wallabag/graby-site-config:/var/www/wallabag/vendor/j0k3r/graby-site-config
- /opt/wallabag/logs:/var/www/wallabag/var/logs
healthcheck:
test: ["CMD", "wget" ,"--no-verbose", "--tries=1", "--spider", "http://localhost"]
interval: 1m
timeout: 3s
depends_on:
- redis
redis:
image: redis:alpine
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 20s
timeout: 3s

prod.log

[2022-10-25 06:19:09] security.DEBUG: Read existing security token from the session. {"key":"_security_secured_area","token_class":"Symfony\Component\Security\Core\Authentication\Token\RememberMeToken"} []
[2022-10-25 06:19:09] security.DEBUG: User was reloaded from a user provider. {"provider":"Symfony\Bridge\Doctrine\Security\User\EntityUserProvider","username":"XXXXX"} []
[2022-10-25 06:19:10] app.DEBUG: Restricted access config enabled? {"enabled":1} []
[2022-10-25 06:19:10] graby.INFO: Graby is ready to fetch [] []
[2022-10-25 06:19:10] graby.INFO: . looking for site config for businesslive.co.za in primary folder {"host":"businesslive.co.za"} []
[2022-10-25 06:19:10] graby.INFO: ... found site config businesslive.co.za.txt {"host":"businesslive.co.za.txt"} []
[2022-10-25 06:19:10] graby.INFO: Appending site config settings from global.txt [] []
[2022-10-25 06:19:10] graby.INFO: . looking for site config for global in primary folder {"host":"global"} []
[2022-10-25 06:19:10] graby.INFO: ... found site config global.txt {"host":"global.txt"} []
[2022-10-25 06:19:10] graby.INFO: Cached site config with key: businesslive.co.za {"key":"businesslive.co.za"} []
[2022-10-25 06:19:10] graby.INFO: . looking for site config for global in primary folder {"host":"global"} []
[2022-10-25 06:19:10] graby.INFO: ... found site config global.txt {"host":"global.txt"} []
[2022-10-25 06:19:10] graby.INFO: Appending site config settings from global.txt [] []
[2022-10-25 06:19:10] graby.INFO: Cached site config with key: global {"key":"global"} []
[2022-10-25 06:19:10] graby.INFO: Cached site config with key: businesslive.co.za.merged {"key":"businesslive.co.za.merged"} []
[2022-10-25 06:19:10] graby.INFO: Fetching url: https://www.businesslive.co.za/fm/opinion/2022-10-20-duncan-mcleod-is-mark-zuckerberg-lost-in-the-metaverse/ {"url":"https://www.businesslive.co.za/fm/opinion/2022-10-20-duncan-mcleod-is-mark-zuckerberg-lost-in-the-metaverse/"} []
[2022-10-25 06:19:10] graby.INFO: Trying using method "get" on url "https://www.businesslive.co.za/fm/opinion/2022-10-20-duncan-mcleod-is-mark-zuckerberg-lost-in-the-metaverse/" {"method":"get","url":"https://www.businesslive.co.za/fm/opinion/2022-10-20-duncan-mcleod-is-mark-zuckerberg-lost-in-the-metaverse/"} []
[2022-10-25 06:19:10] graby.INFO: Use default user-agent "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.92 Safari/535.2" for url "https://www.businesslive.co.za/fm/opinion/2022-10-20-duncan-mcleod-is-mark-zuckerberg-lost-in-the-metaverse/" {"user-agent":"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.92 Safari/535.2","url":"https://www.businesslive.co.za/fm/opinion/2022-10-20-duncan-mcleod-is-mark-zuckerberg-lost-in-the-metaverse/"} []
[2022-10-25 06:19:10] graby.INFO: Use default referer "http://www.google.co.uk/url?sa=t&source=web&cd=1" for url "https://www.businesslive.co.za/fm/opinion/2022-10-20-duncan-mcleod-is-mark-zuckerberg-lost-in-the-metaverse/" {"referer":"http://www.google.co.uk/url?sa=t&source=web&cd=1","url":"https://www.businesslive.co.za/fm/opinion/2022-10-20-duncan-mcleod-is-mark-zuckerberg-lost-in-the-metaverse/"} []
[2022-10-25 06:19:11] httplug.INFO: Sending request: GET https://www.businesslive.co.za/fm/opinion/2022-10-20-duncan-mcleod-is-mark-zuckerberg-lost-in-the-metaverse/ 1.1 {"request":"[object] (Laminas\Diactoros\Request: {})","uid":"63577fdf035620.43049619"} []
[2022-10-25 06:19:11] app.DEBUG: Crypto: decrypting value: d6 [] []
[2022-10-25 06:19:11] app.DEBUG: Crypto: decrypting value: da [] []
[2022-10-25 06:19:11] graby.INFO: Returning cached and merged site config for businesslive.co.za {"host":"businesslive.co.za"} []
[2022-10-25 06:19:11] graby.DEBUG: Auth: add parameters. {"host":"businesslive.co.za","parameters":{"host":"businesslive.co.za","requiresLogin":true,"loginUri":"https://www.businesslive.co.za/u/sign-in","usernameField":"email-address","passwordField":"password","extraFields":[],"notLoggedInXpath":"//a[contains(@href, 'sign-in')]","username":"masked","password":"masked"}} []
[2022-10-25 06:19:11] httplug.ERROR: Error: Client error response [url] https://www.businesslive.co.za/u/sign-in [status code] 404 [reason phrase] Not Found with response: 404 Not Found 1.1 {"exception":"[object] (Http\Client\Exception\HttpException(code: 404): Client error response [url] https://www.businesslive.co.za/u/sign-in [status code] 404 [reason phrase] Not Found at /var/www/wallabag/vendor/php-http/guzzle5-adapter/src/Client.php:124, GuzzleHttp\Exception\ClientException(code: 404): Client error response [url] https://www.businesslive.co.za/u/sign-in [status code] 404 [reason phrase] Not Found at /var/www/wallabag/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php:89)","milliseconds":671,"uid":"63577fdf035620.43049619"} []
[2022-10-25 06:19:11] graby.WARNING: Request throw exception (with a response): Client error response [url] https://www.businesslive.co.za/u/sign-in [status code] 404 [reason phrase] Not Found {"error_message":"Client error response [url] https://www.businesslive.co.za/u/sign-in [status code] 404 [reason phrase] Not Found"} []
[2022-10-25 06:19:11] graby.INFO: Data fetched: array{"effective_url":"https://www.businesslive.co.za/fm/opinion/2022-10-20-duncan-mcleod-is-mark-zuckerberg-lost-in-the-metaverse/","body":"404 Not Found\n\nThe resource could not be found.\n\n ","headers":{"content-type":"text/plain; charset=UTF-8","x-cloud-trace-context":"e01576313d772683bdd609d81d1dcb85","date":"Tue, 25 Oct 2022 06:19:11 GMT","server":"Google Frontend","content-length":"52"},"status":404} {"data":{"effective_url":"https://www.businesslive.co.za/fm/opinion/2022-10-20-duncan-mcleod-is-mark-zuckerberg-lost-in-the-metaverse/","body":"404 Not Found\n\nThe resource could not be found.\n\n ","headers":{"content-type":"text/plain; charset=UTF-8","x-cloud-trace-context":"e01576313d772683bdd609d81d1dcb85","date":"Tue, 25 Oct 2022 06:19:11 GMT","server":"Google Frontend","content-length":"52"},"status":404}} []
[2022-10-25 06:19:11] graby.INFO: Treating as UTF-8 {"encoding":"utf-8"} []
[2022-10-25 06:19:11] graby.INFO: Filtering HTML to remove XSS [] []
[2022-10-25 06:19:11] graby.INFO: Filtering HTML to remove XSS [] []
[2022-10-25 06:19:11] app.DEBUG: Extracting images from content to provide a default preview picture [] []
[2022-10-25 06:19:11] app.DEBUG: 0 pictures found [] []
[2022-10-25 06:19:11] app.DEBUG: DownloadImagesSubscriber: disabled. [] []
[2022-10-25 06:19:11] security.DEBUG: Stored the security token in the session. {"key":"_security_secured_area"} []

Start by inspecting businesslive.co.za.txt in the mounted graby-site-config volume and comparing its login settings with the site's current sign-in flow. Reproduce the fetch in the Dockerised Wallabag setup using compose.yml, using prod.log to confirm the request path and response. Done means the configured authentication request no longer returns the reported 404.