LLM-controlled headless browser running fully on Cloudflare
This is an experiment to create an AI Agent that can crawl and interact with webpages to achieve desired goal. Fully on Cloudflare (almost).
Services used:
pnpm i
npx wrangler secret put OPENAI_API_KEY # and fill with your OpenAI key
# You can also put it inside .dev.vars (copy .dev.vars.template as a reference)
# Run migrations on local SQLite instance
npx wrangler d1 execute ai-agent-jobs --local --file=migrations/0000_init.sql
# Run migrations on remote SQLite instance
npx wrangler d1 execute ai-agent-jobs --remote --file=migrations/0000_init.sql
# You can use `pnpm run dev` as well but Browser Rendering does not work locally
pnpm run deploy
curl -X POST \
<URL to your deployed worker> \
-d '{"baseUrl": "https://chatwithcloud.ai", "goal": "Extract pricing data" }' \ # Replace with your URL and goal
--no-buffer
baseUrl
from the request's bodywrangler.toml
)KEEP_BROWSER_ALIVE_IN_SECONDS
).env
file and fill with your Cloudflare account ID, D1 Databse ID and Cloudflare D1 token with edit permissions. Use .env.template
as a referencecp .env.template .env
npx drizzle-kit studio
Learn more about Drizzle Studio and its D1 configuration