X-Authentication-Warning: delorie.com: mail set sender to djgpp-bounces using -f
From: "Bill Cunningham" <nospam AT nspam DOT net>
Newsgroups: comp.os.msdos.djgpp
Subject: Re: Tokenizing
Date: Sat, 15 May 2004 11:12:20 -0400
Organization: Posted via Supernews, http://www.supernews.com
Message-ID: <10accp17qpola6a@corp.supernews.com>
References: <10abt7pn4699h2a AT corp DOT supernews DOT com> <6654-Sat15May2004145451+0300-eliz AT gnu DOT org>
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 6.00.2800.1409
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1409
X-Complaints-To: abuse AT supernews DOT com
Lines: 10
To: djgpp AT delorie DOT com
DJ-Gateway: from newsgroup comp.os.msdos.djgpp
Reply-To: djgpp AT delorie DOT com


> It's also called ``lexical analysis''.  And yes, GCC, like any other C
> compiler, subdivides the source into tokens before it parses the code.

    There's a type of parsing called Thompson Parsing. I suspect it's named
for Ken Thompson. It must come after tokenizing then.

    Bill