Main Products: • FDSE • AXS • Guardian   Forum Contact Info About Site Map

 

©2013 FDSC All Rights Reserved

Security: Changing the location of the searchdata folder

By default, the search engine and all of its files reside under a single, publicly-accessible folder named "search":

e:/webroot/search/
	license.html   # optional
	install.html   # optional
	proxy.pl       # public script
	search.pl      # public script
	setperms.bat   # optional
	setperms.sh    # optional
	searchdata/    # private data
	searchmods/    # private libraries

From a security standpoint, the only files that should exist under the publicly-accessible folder are those files which are publicly accessed.

As an example of the risk, people can make direct requests to "http://mysite.tld/search/searchdata/" to access the private files within. Some of those files may contain sensitive information. The best way to prevent these requests is to move the "searchdata" folder to a different location.

To improve security, we begin by make a copy of the "search" folder and placing it somewhere that is not within the publicly-accessible webroot:

e:/private/
	license.html   # optional
	install.html   # optional
	proxy.pl       # public script
	search.pl      # public script
	setperms.bat   # optional
	setperms.sh    # optional
	searchdata/    # private data
	searchmods/    # private libraries

e:/webroot/search/
	license.html   # optional
	install.html   # optional
	proxy.pl       # public script
	search.pl      # public script
	setperms.bat   # optional
	setperms.sh    # optional
	searchdata/    # private data
	searchmods/    # private libraries

Next, we delete all files that we don't need. Those would be the install help files and the setperms scripts:

e:/private/
	proxy.pl       # public script
	search.pl      # public script
	searchdata/    # private data
	searchmods/    # private libraries

e:/webroot/search/
	proxy.pl       # public script
	search.pl      # public script
	searchdata/    # private data
	searchmods/    # private libraries

Next, we delete all publicly-accessible files from the private folder:

e:/private/
	searchdata/    # private data
	searchmods/    # private libraries

and we delete all private data and files from the public folder:

e:/webroot/search/
	proxy.pl       # public script
	search.pl      # public script

The final directory structure looks like this:

e:/private/
	searchdata/    # private data
	searchmods/    # private libraries

e:/webroot/search/
	proxy.pl       # public script
	search.pl      # public script

Note: the proxy.pl utility is disabled by default. You can read more about it at Advanced Search: Highlighting search terms in the actual document. If you don't use proxy.pl, then you should delete that file too. That will enhance your security even more.

By making these changes, we have removed all unneeded files, and partitioned the public from the private. A final change needs to be made in the search.pl file to point to the new location of the data and libraries. To do this, edit search.pl and find the lines that looks like this:

$err = &load_files_ex( '.' );
next Err if ($err);

Replace "." with the full path to your private folder. Remember to always use forward slashes on Windows. Remember to enclose the path string in single quotes:

$err = &load_files_ex( 'e:/private' );
next Err if ($err);

Now save changes and make a normal search request. You should see the normal behavior.

If and only if you are using the proxy.pl utility, open proxy.pl and find the string:

searchdata/search.pending.txt

Replace with the full path to the pending file:

e:/private/searchdata/search.pending.txt

The proxy.pl will then work properly (see Advanced Search: Highlighting search terms in the actual document for more info on proxy.pl).

Things to keep in mind:

• When making a copy of the "search" folder, you may need to re-apply file and folder permissions to ensure that all files and folders under "searchdata" are writable by the CGI script.

• If you move your folders around, but then need to upgrade or re-install the search engine, you may want to temporarily return the folders to their original locations. The advantage of this is that then you can use the auto-installer and it will be able to understand your folder layout and work its magic whereby it updates all code files and keeps your data files intact. Then, after completing the upgrade, you can move your folders back to their hidden area and re-customize the one line in "search.pl". You will also need to re-delete unused files, as the installer will replace them.

• There is an .htaccess file within the "searchdata" folder which completely protects it from web-based viewing when running under the Apache web server. About 80% of customers use Apache. For these customers, there is little risk to begin with.

  Even when an outside user can request into the "searchdata" folder, the most important security information is stored in the file "searchdata/settings.pl" which uses a Perl CGI header followed by a data block. On most servers, the request for that file will result in the Perl CGI header running and returning a null result, while the data remains safe.

  Finally, the most important security setting, the admin password, is encrypted using the relatively secure crypt() call.

  These various measures are all helpful but none are as completely secure as just moving the data folders to a private area.

• There have been no reported instances of problems due to unauthorized access to the private folders in FDSE. Hiding the private files is just a basic common-sense suggestion that applies to all scripts.

• FDSE has also been designed to continue to protect the server even if the admin area is compromised (for example, by somebody cracking or guess the admin password). The FDSE admin area provides arbitrary file-system read access, and limited write access, but it does not allow for arbitrary command execution or customization of source code.

History: this help file applies to FDSE version 2.0.0.0054 and newer.

In FDSE versions 2.0.0.0026 through 53, the method for hiding the "searchdata" folder was to move only the "searchdata" folder, and to modify the load_files( 'searchdata' ); subroutine with its new location. In those versions, the "searchmods" folder was always left in place.

In FDSE versions 2.0.0.0025 and older, the "searchdata" folder could be hidden in the same way, and the $DataFilesDir variable was set with its new location.

    "Security: Changing the location of the searchdata folder"
    http://www.xav.com/scripts/search/help/1079.html