Website

Point your agent at your website and it reads the pages and answers from them, with a link to the page each answer came from.

Your AI teammate reads the pages of the website you point it at, and answers from them. Every answer links the page it took the answer from.

Nothing runs on its own. Website is knowledge only: no triggers and no actions.

Your agent answering from crawled pages, with the source page linked.

Quick start

1

Ask your agent to add your website

Open the chat in your dashboard and ask it to add your website as a knowledge source. It asks which URL to crawl. Prefer to click? Integrations > Website in the left sidebar has the same connect form.

The agent asks for the URL it should crawl.

2

Give the URL

Send the URL you want read. The URL decides the scope of the crawl:

• The crawl starts at the exact URL you give.

• A URL for one section, such as one ending in /docs, holds the crawl to that section.

• The root domain crawls the whole of that domain. A subdomain such as help.yourcompany.com is a separate integration.

Your agent starts the crawl and confirms the website is connected. Pages become searchable while the crawl is still running, so you can keep chatting. The left sidebar now lists the Website integration with your domain.

The crawl runs in the background while the chat carries on.

Check it worked. The integration page shows Connected with your domain, and the Website source shows a document count. From a terminal, eesel status lists the integration as connected with its source counts.

Connect-form options: path filters, custom headers, query strings

The connect form under Integrations > Website has extra controls for narrowing the crawl:

Field	What it does
Include paths	Crawl only URLs whose path matches an entry, for example /help, /docs. Comma-separated. The * and ? wildcards work
Exclude paths	Skip URLs whose path matches, for example /blog, /careers. Exclude wins when both are set. Both empty crawls everything
Custom headers	For sites behind a login: add a Cookie, Basic Authorization or Bearer header and the crawler sends it with every request
Query-string URLs	By default, URLs differing only by query string count as one page and tracking parameters are stripped. Turn this on if query strings select real content, like /resources/?category=manuals

In these fields use paths like /docs, not full URLs. The crawler is already held to the domain you gave.

3

Check what it can read

Integrations > Website shows the connection and its sources. The Website source shows a document count once pages are indexed. Its toggle stops your agent answering from those pages. The two Start a chat buttons open a chat about Website, one for a question and one for setup help.

The Website integration page, with the crawled pages counted under Sources.

4

Tell it how to answer

Tell your agent in chat what to always mention, or always leave out. It saves the rule and applies it from then on.

A standing rule given in chat, saved to the agent's instructions.

The rule then sits on the Instructions page, where you can read or change it.

The Instructions page holds every standing rule your agent follows.

5

Put it to work on your site

Add the eesel chat bubble to your website and visitors can ask there. Your agent answers from the crawled pages, links the pages it used, and follows your instructions. See Chat Bubble for how to add the bubble.

The chat bubble on the site, answering from the pages your agent crawled.

From Claude Code, Cursor or a terminal. The same setup works as commands: eesel login, then eesel integrations connect website, which prints the details it needs. Use eesel integrations download start to start the crawl, eesel status to see the integration and its source counts, and eesel chat to ask it a question. Saving a standing rule has no command, so use the dashboard chat for that step. Full command reference: The eesel CLI. Step-by-step: Set up from Claude Code or the terminal. From a Model Context Protocol (MCP) client: Claude Code and MCP clients.

Sources

Sources are what it learns from:

Source	Description	Requires
Web pages	Pages crawled from the URL you give, found through your sitemap and by following links	The URL, plus custom headers if the site is behind a login

A single crawl indexes up to 2,000 pages. Content stays as it was at the last crawl until you re-crawl. There is no automatic re-crawl, so re-save the integration from Integrations > Website to run a fresh one. A fresh crawl also finds newly linked pages.

One integration covers one domain, so www.yourcompany.com and help.yourcompany.com are two integrations. Giving the same domain again re-crawls into the same integration.

Every other integration on the agent is also a source. So one reply can draw on your website, Confluence, Google Drive, your helpdesk and your uploaded files at once. See Integrations.

Troubleshooting

Crawler not finding a page?

The page must be linked from another crawled page or listed in the sitemap. Orphan pages are not found. Then check three things:

• The page is not blocked by your include or exclude filters.

• The page does not need a login. If it does, add custom headers.

• The crawl has not hit the 2,000-page cap.

Too many irrelevant pages crawled?

Add exclude paths for the sections you do not want, such as /blog or /careers. Or use include paths to hold the crawl to /help or /docs. Then re-crawl.

Content out of date after a site change?

There is no automatic re-crawl. Re-save the integration from Integrations > Website to run a fresh crawl with the current content.

Seeing duplicate pages that differ only by their URL query?

By default, query-string variants collapse into one page. If you turned on treating query-string URLs as distinct and do not want that, turn it back off and re-crawl.

FAQ

Can it crawl a site behind a login or a paywall?

Yes. Add custom headers on the connect form: a session Cookie, a Basic Authorization header, or a Bearer token. If the credentials rotate, update the headers and re-save the integration to re-crawl.

Do I need to give it my sitemap?

No. The crawler picks up your sitemap on its own, and also follows links on the pages it reads.

Can I crawl more than one website?

Yes, one integration per domain. A help center on a subdomain like help.yourcompany.com is its own integration, so give that URL directly.

Why did the crawler start partway into my site?

The crawl starts at the exact URL you give, so only pages linked from there are found. To crawl the whole site, give the root domain.

Can I still chat while the crawl is running?

Yes. Pages become searchable as they index, so your agent can answer from the first pages before the crawl finishes.

How is this billed?

Nothing extra for the knowledge source. Billing is pay-as-you-go: a ticket your agent handles is one task, however many replies happen on it. Chats you run yourself from the eesel dashboard are free. See Pricing for rates, the annual commitment, and Enterprise.

Related pages

• Helpdesk

• Chat Bubble

• Integrations overview

• Set up from Claude Code or the terminal