[FEATURE] A web-scraping tool whose browser session survives between runs
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 58.8k
- Forks
- 8.5k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 109
Description
Feature Area
Integration with external tools
Is your feature request related to a an existing bug? Please link it here.
NA
Describe the solution you'd like
Every browser tool under web-scraping today starts from nothing on each call: a fresh browser, a stock automation fingerprint, and whatever IP the process has. That is fine for a public page and useless behind a login, because the session dies with the browser, so a crew that has to read a signed-in page cannot keep its state between steps.
I would like a tool whose unit of state is a browser profile rather than a session: cookies, storage, a fingerprint and a proxy that all persist, so the second call opens the page already signed in, and one browser is reused across the calls of a single crew run.
Concretely, a AntibrowLoadTool(profile="research-01") that takes a url (and an optional CSS selector), returns the page's visible text truncated to a limit, and exposes close() for cleanup.
Describe alternatives you've considered
SeleniumScrapingToolwith a user-data-dir: keeps cookies, but the fingerprint is a stock automation build and the setup is left to the user.- The hosted-browser tools (
BrowserbaseLoadTool,HyperbrowserLoadTool): sessions are remote and per-call, and they bill by browser time. - Doing it outside the crew and passing text in: works, but then the login step is not something an agent can drive.
Additional Context
I am affiliated with AntiBrow, which is why I can implement it: a PR is ready with the tool, 9 unit tests, the optional extra, and the docs page in en/ar/ko/pt-BR.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the existing browser tools under web-scraping and the proposed AntibrowLoadTool(profile="research-01") interface. Then inspect the nine unit tests, optional extra, and documentation pages in en/ar/ko/pt-BR mentioned in the issue. Done means the profile state and browser reuse behavior work across calls, cleanup is exposed, and the documented interface is covered.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling, web-dev
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100