Content Sources > Diagnostics > Real-time Diagnostics

Use the Content Sources > Diagnostics > Real-time Diagnostics page to retrieve and view real-time information for the search appliance, including HTTP headers for a specific URL and traffic on the network layer. This help page contains information about the following topics:

• Viewing HTTP Headers for a URL
• Viewing Crawl and Index Activity
• Running Network Capture

Viewing HTTP Headers for a URL

Use the Manual fetch request section of this page to view HTTP headers for a specified URL and debug headers for SMB URLs. This feature uses the crawler to fetch a URL using all the crawler settings (for example, security and proxy). Fetching a URL does not affect the crawl and indexing of the URL nor does it use crawl pipeline information for the URL, such as If-Modified-Since.

To fetch a URL and view the HTTP headers:

1. Under Manual fetch request, type the URL in the Fetch headers for URL box.
2. Click Fetch.
   The headers for the URL appear in the Headers box.

Viewing Crawl and Index Activity

Use the View crawl and indexing activity section of this page to capture and view information about URLs (including content feeds) that the search appliance is currently crawling and indexing. For each URL that is captured, you can view information, such as HTTP headers.

Limit capture of URLs to documents of interest by specifying a pattern for filtering URLs in the URL pattern to capture box. Only URLs that match the specified pattern appear in the list of captured URLs. For example, to capture only URLs in example.com/, enter this URL pattern in the box. You can specify any valid URL pattern. For information about valid URL patterns, see "Constructing URL Patterns" in "Administering Crawl," which is linked to the Google Search Appliance help center.

To capture and view information about URLs that the search appliance is currently crawling and indexing:

1. Under View crawl and indexing activity, type a URL pattern in the URL pattern to capture box.
2. Click Start Capture.
   The list of captured URLs appears in the Captured URLs box. To view the HTTP headers for a URL, expand the URL by clicking the expand icon.

You can end a capture at any time by clicking the Stop Capture button. Take note that capture automatically stops if 1 GB of data has been collected.

Also, you can export a text capture file that contains the detailed information about URLs.

To export a file:

1. Click the Export link next to the Start/Stop Capture button.
2. Browse to a location on the local computer for the file and click Save.

To request recrawl of a single URL, use the Index > Diagnostics > Index Diagnostics page. To request recrawl for a set of patterns, use the Content Sources > Web Crawl > Freshness Tuning page.

Running Network Capture

Use the Run packet capture section of this page to generate and download a file that contains information about current search appliance network traffic, such as crawling and serving traffic. The file is created by using tcpdump. You can view the downloaded captured file by using a network protocol analyzer tool such as Wireshark.

By default, all traffic is captured, but you can set a filter to limit the capture by hosts. To set a filter, type a comma-separated list of fully qualified hostnames or IP addresses in the box under Run packet capture and select one of the following options:

• Include--Show traffic between the Google Search Appliance and the specified hosts only
• Exclude--Show all traffic, except that which is between the Google Search Appliance and the specified hosts
  Note: If the box is empty and exclude is selected, then all traffic is captured (that is, Include is assumed).

To capture all traffic, leave the value empty.

To run network capture and download the captured file:

1. Under Run packet capture, enter filtering information, as described in the preceding paragraphs (optional).
2. Click Start Capture.
   You can monitor the capture by the checking the values that appear in the file size and packets captured fields next to the Export link. Take note that capture automatically stops if the output file size is greater than 1 GB.
   
3. Click the Export link.
   You can export the file while the capture is running.
4. Browse to a location on the local computer for the file and click Save.