Directory listings in the search results
The Fluid Dynamics Search Engine will index directory listings by default. Directory listings lead to search results that look like this:
Some webmasters do not want these directory listings in their search results. We are generally in favor of directory listings, however, and so they are indexed by default in FDSE. Directory listings are particularly useful for sites that share large numbers of binary files. Major search engines like Google and Altavista include directory listings in their indexes.
Here are a few problems with these types of listings, along with possible work-arounds:
On the Apache web server, there will be nine distinct URL's for each directory listing, corresponding to the different sort options for each folder. These redundant listings can clog up the index.
The nine URL's for this example would be:
http://www.xav.com/latest/ http://www.xav.com/latest/?N=A http://www.xav.com/latest/?N=D http://www.xav.com/latest/?M=A http://www.xav.com/latest/?M=D http://www.xav.com/latest/?S=A http://www.xav.com/latest/?S=D http://www.xav.com/latest/?D=A http://www.xav.com/latest/?D=D
These redundant listings can be skipped using a Filter Rule - see Filter Rules: Avoiding redundant Apache directory listings for more information. Recent releases of FDSE have this filter rule present and enabled by default.
Listings of empty folders will still create a record in the index and may come up in the results, even though no useful information is present.
Listings of empty folders can be skipped using a Filter Rule - see Filter Rules: Bypassing directory listings of empty directories for more information. Recent releases of FDSE have this filter rule present and enabled by default.
Some users are confused by directory listings, since they are not documents in the traditional sense.
See Filter Rules: Bypassing all directory listings to prevent directory listings from appearing in the results. By default, FDSE will include directory listings. You will need to take special steps to disable this.
"Directory listings in the search results" http://www.xav.com/scripts/search/help/1145.html