X-Recipient:	archive-cygwin AT delorie DOT com
X-SWARE-Spam-Status:	No, hits=-4.2 required=5.0 tests=AWL,BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,KHOP_RCVD_TRUST,KHOP_THREADED,RCVD_IN_DNSWL_LOW,RCVD_IN_HOSTKARMA_YE
X-Spam-Check-By:	sourceware.org
MIME-Version:	1.0
In-Reply-To:	<CAEF1h+V_QnJ1R6opS0K3LzpuxB7VAi2Jdja9HSsxjRM+VRq=Nw@mail.gmail.com>
References:	<CAEF1h+V_QnJ1R6opS0K3LzpuxB7VAi2Jdja9HSsxjRM+VRq=Nw AT mail DOT gmail DOT com>
Date:	Fri, 27 Apr 2012 12:27:49 -0600
Message-ID:	<CAFWoy7HTH4uTS4RhRPcGE27h-mGyPOCy-bZhcy2eSYcc+qEEoQ@mail.gmail.com>
Subject:	Re: find.exe vs. cmd.exe dir command vs. filesystem object in vbs script
From:	Keith Christian <keith1christian AT gmail DOT com>
To:	cygwin AT cygwin DOT com
X-IsSubscribed:	yes
Mailing-List:	contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id:	<cygwin.cygwin.com>
List-Unsubscribe:	<mailto:cygwin-unsubscribe-archive-cygwin=delorie DOT com AT cygwin DOT com>
List-Subscribe:	<mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive:	<http://sourceware.org/ml/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender:	cygwin-owner AT cygwin DOT com
Mail-Followup-To:	cygwin AT cygwin DOT com
Delivered-To:	mailing list cygwin AT cygwin DOT com

On Fri, Apr 27, 2012 at 8:28 AM, Cary Lewis <cary DOT lewis AT gmail DOT com> wrote:
> I have a system that makes use of a number of directories which
> contain hundreds of thousands of files.


Cary, I can't comment on any API references, but here is a possible workaround.

The "locate" command works similarly to the "find" command, but
consults a special database (which you can re-generate at any time)
for quick access.

Then, when searching for a directory name or file name, you can use
the "locate" command and the collection of separate "locatedb"
databases, which will return results very quickly

Note - whenever files are added, deleted, or renamed in any of the
hypothetical "somedir_0n" directories, the "updatedb" command will
have to be run again to create the locatedb databases.

The "locate -S -d /var/locatedb_somedir_0n" command outputs statistics
on the database just created ,showing the number of filenames in each
and other stats.


Example:

1. Suppose the following directories exist on your SAN, and each of
"somedir_nn" contains about 100,000 files each.

	/san_main_dir/corp_files/somedir_01
	/san_main_dir/corp_files/somedir_02
	/san_main_dir/corp_files/somedir_03
	/san_main_dir/corp_files/somedir_04
		[ .....ad nauseum..... ]
	/san_main_dir/corp_files/somedir_nn


2. Put the following lines into a script (e.g.
create_san_locatedb.sh,) which will create a separate "locatedb"
database for each subdirectory:


	#!/bin/bash
	# Create separate locatedb databases for directories containing a
large number of files on a SAN.

	time updatedb --localpaths='/san_main_dir/corp_files/somedir_01'
--output=/var/locatedb_somedir_01 2>/dev/null
	echo "locatedb_somedir_01 created..."
	echo
	locate -S -d /var/locatedb_somedir_01
	echo
	time updatedb --localpaths='/san_main_dir/corp_files/somedir_02'
--output=/var/locatedb_somedir_02 2>/dev/null
	echo "locatedb_somedir_02 created..."
	echo
	locate -S -d /var/locatedb_somedir_02
	echo
	time updatedb --localpaths='/san_main_dir/corp_files/somedir_03'
--output=/var/locatedb_somedir_03 2>/dev/null
	echo "locatedb_somedir_03 created..."
	echo
	locate -S -d /var/locatedb_somedir_03
	echo
	time updatedb --localpaths='/san_main_dir/corp_files/somedir_04'
--output=/var/locatedb_somedir_04 2>/dev/null
	echo "locatedb_somedir_04 created..."
	echo
	locate -S -d /var/locatedb_somedir_04
	echo
	echo "Custom locatedb directories created"



3. Now that the databases are created, here are some example commands
to find the directory and filenames within:


	Show a list of files ending in "DAT2012" from the database locatedb_somedir_03:

		locate --database=/var/locatedb_somedir_03 "*DAT2012"


	Show a list of files, (ignoring cAsE sEnSiTiViTy,) with "dAt2012"
anywhere in the directory path
	or in the filename, from the database locatedb_somedir_03:

		locate -i --database=/var/locatedb_somedir_03 "dat2012"


	Show a list of files with "DAT2012" with a preceding directory name
containing "uncooked",
	from the database locatedb_somedir_01:

		locate --database=/var/locatedb_somedir_01 "*uncooked*DAT2012"


	Show a list of files with "DAT2012" with a preceding file or
directory name containing "uncooked",
	from the database locatedb_somedir_01:

		locate --database=/var/locatedb_somedir_01 "*uncooked*DAT2012"


	Search across all four hypothetical locatedb_somedir_0n databases
using four separate command lines:
		
		locate --database=/var/locatedb_somedir_01 "DAT2012"
		locate --database=/var/locatedb_somedir_02 "DAT2012"
		locate --database=/var/locatedb_somedir_03 "DAT2012"
		locate --database=/var/locatedb_somedir_04 "DAT2012"


4. If the directory names or file names below /san_main_dir/corp_files
change after the "locatedb_somedir_0n" databases are created, new
files won't be found using "locate," and items deleted since the
previous steps in (2) above will still appear until the databases are
re-created.  Re-run Step 2 if this is the case.  The "time" commands
before each updatedb command will help gauge how long it takes to
create the "locatedb" databases.


========Keith

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

- Raw text -