Search and page content
Search-results APIs, page fetching, scraping and browsing.
-
Pay per call vs API subscriptions: what an agent should buy
Explainer / How agents buy live data
Compare API subscriptions and usage billing for agents with worked search, page-fetching and weather costs, including free allowances and spending caps.
· 5 min read
-
Which AI crawlers should your site allow for agent search?
Explainer / Search and page content
Choose AI crawler rules by purpose: keep agent search open, decide on training, and understand what robots.txt can control across four vendors.
· 5 min read
