What is Crawl Rule?
Crawl rules coupled with Content Sources allows:
- Including a path during the crawl
- Excluding a path during the crawl
- Specifing Credentials for a path
- Setting Wildcards like * and Regular Expressions
You can discard Crawl Rules if your Content Source is working first hand.
Steps
Use the following procedure to create a crawl rule.
Step 1: Open Crawl Rules
Open Central Administration then select "Service Applications" > "Search Service Application" > "Crawl Rules".
Step 2: Create Crawl Rule
Choose the New Crawl Rule option and enter the details as shown below.
Please note that you need to include an asterisk (*) to include the URLS.
By default the exclude option is selected, you need to choose the Include option. Check the Follow Links, Complex URLs, SharePoint content as HTTP pages options too.
For external web sites you can use the Anonymous access.
Step 3: Test Crawl Rule
Now we can test our crawl rule. Here we are entering a lengthy URL and make sure it is being caught by our crawl rule.
Once the Test is confirmed, we can proceed with Full Crawl.
References
Manage crawl rules in SharePoint Server 2013
Summary
In this article we have explored how to create a crawl rule in SharePoint 2013.