FDSE :: Help :: Indexing secure SSL / https web pages

Home > Fluid Dynamics Search Engine > Help > 1118

Indexing secure SSL / https web pages

The FDSE web crawler only speaks the normal "http://" protocol, not the secure "https://" protocol. In many cases, this limitation prevents FDSE from being used to index SSL content.

However, indexing SSL content is possible if the FDSE indexer can access the content without using SSL. The common scenarios are:

All SSL content is hosted on a server than accepts normal "http://" connections as well as secure "https://" connections.

For example, https://xav.com and http://xav.com point to identical content. This solution also works if SSL content is hosted in a subfolder of a normal site, like "https://mysite.tld/" == "http://mysite.tld/secure/".
Or, if:
1. all SSL content consists of only static HTML files, without any active ASP, PHP, CGI, etc., content
2. and, all content exists is an folder on the same physical server that hosts FDSE
In this case, you may create a File System realm (see Administration: Creating a "file system" realm). When creating the realm, you must enter a normal "http://" Base URL, even if that URL won't work for visitors.

In either of these cases, you can create a normal FDSE search realm. Once the realm is complete, you must create a URL Rewrite Output Filter so that visitors will be linked to the "https://" version of the URL. To do this:

Confirm that you are running at least FDSE version 2.0.0.0054, the first to support output filters.
Go to Admin Page => Filter Rules => (scroll down) => URL Rewrite Rules => Output Filters.
In one of the open slots labeled "new rewrite rule", create a new rule with the following parameters:

Enabled: [x] (checked)
Verbose: [_] (not checked)
Pattern: http://mysite.tld/
Replace: https://mysite.tld/
Comment: force SSL link

Customize the parts in red to match your site. For the Pattern parameter, use whatever was used as the Base URL when creating your realm. For the Replace parameter, use the corresponding SSL URL.
Test your rule by performing a search. The links returned in the search results should point to the secure "https://" URL.

Warning: the proxy.pl "View with Highlighted Search Terms" utility will not work with these types of results, nor with most solutions that involves URL Rewrite Rules.

    "Indexing secure SSL / https web pages"
    http://www.xav.com/scripts/search/help/1118.html