Serving > Head Requestor Deny Rules

The Head Requestor Deny Rules page enables you to identify URLs where content servers deny users access with codes other than HTTP code 401 and define for the search appliance the access-denied responses to expect from the content server. For example, a content server might send an HTTP code 200 instead of a 401 code, or the access-denied response might be in the body of content returned by the content server.

If the content server uses the standard HTTP 401 code, you do not need to configure head requestor deny rules.

If the content server response matches any of the rules configured for its URL, the response is considered an access-denied response. For example, the content might be in a different language.

Use this page to perform the following tasks:

• Setting Head Requestor Deny Rules
• Editing or Deleting Head Requestor Deny Rules

Before Starting this Task

Before you set any Head Requestor Deny Rules, check the status codes returned by content servers in your installation when a user is denied access to a page. If a content server does not return the HTTP 401 status code, determine the URL or URL pattern for the content server.

Setting Head Requestor Deny Rules

To set a head requestor deny rule:

1. Click Serving > Head Requestor Deny Rules.
2. In the URL Pattern field, type the URL or URL pattern identifying a content server that issues a code other than HTTP 401 when it denies access to a user.
3. Click Create New Deny Rule.
4. Select the type of request the search appliance sends to the content server. If you are using content as a deny rule, you must select one of the GET request options.
   • A HEAD request retrieves document headers only and does not detect deny codes in the document content.
   • A GET request of a particular length. Select this type of request when the access-denied content or code is known to be within the number of bytes you enter in the field.
   • A GET request that retrieves the entire response.
5. Type Status Codes the search appliance should interpret as denying access.
6. Click Add Status Code to add additional expected status codes.
7. To add a Header Name and Header Value the search appliance should interpret as denying access, click Add Header, then type in the name and value.
8. To add content that the search appliance should interpret as denying access, click Add Content, then type the expected content returned by
   the webserver to indicate that access was denied, for example "Not Authorized" or "Access Denied." The content is case-sensitive and  doesn't have to be the entire error page or html source. Only the error message text is necessary.
   
9. Click Save.

Editing or Deleting Head Requestor Deny Rules

To edit a head requestor deny rule, click Serving > Head Requestor Deny Rules, then click the Edit link for the rule you want to change. After you make the changes, click Save.

To delete a head requestor deny rule, click Serving > Head Requestor Deny Rules, then click the Delete link for the rule you want to remove and click Okay.

For More Information

For more information about the head requestor, read "How to use the headrequestor process," which is linked to the Google Search Appliance help center.