Re: new search engine for our web pages? [was: masayuki-h@xxxxxxxxxxxxxxx: Re: ITP: namazu2]

[Taketoshi Sano <kgh12351@xxxxxxxxxxx>]

Are there any idea to improve that point ?

This issue is about to use namazu (or namazu2) for search of the whole page
of www.debian.org. This mail is sent to debian-devel@debian.or.jp, with 
Cc: to debian-www.debian.or.jp and debian-www.lists.debian.org.

In article <20000219234133.B27024@xxxxxxxxxxxxxxx>,
  at Sat, 19 Feb 2000 23:41:33 -0500,
 "James A. Treacy" <treacy@debian.org> writes:

> On Sat, Feb 19, 2000 at 12:33:37PM +0100, Josip Rodin wrote:
> > Hi everyone,
> > 
> > Could we use this namazu program for searching the web pages?
> > 
> Here is the part that stopped me cold:
>    Indexing process will take fifty minutes to index 25 MByte files
>    with Linux Box has Pentium 166 MHz + 64 MB.
> 
> That's over 8 hours just to index the main part of the site (roughly
> 200MB), which should be reindexed every day. For comparison, swish++
> takes less than 10 minutes to index the Package section of the site
> (roughly 97MB). Using this for the List Archives (around 2GB) isn't
> even funny.
> 
> -- 
> James (Jay) Treacy
> treacy@debian.org

The past (not to be updated) record of the List Archives can be
indexed step by step, maybe. But everyday re-indexing may be too much.

How do you think, Kitame, and Nokubi ? (Masayuki wrote you are
the namazu "demigods", so you can answer to this issue, I hope.)

FYI:

 The size of my local cvs copy for www.debian.or.jp:
 $ du -s /Home/sano/work/Debian/Web/www.debian.or.jp/
 5089    /Home/sano/work/Debian/Web/www.debian.or.jp

 The size of my local cvs copy for www.debian.org/english:
 $ du -s /Home/sano/work/Debian/Web/webwml/english/  
 6809    /Home/sano/work/Debian/Web/webwml/english

 The size of my local cvs copy for www.debian.org/japanese:
 $ du -s /Home/sano/work/Debian/Web/webwml/japanese/
 1679    /Home/sano/work/Debian/Web/webwml/japanese

# I don't get other language tree, but there may be many langugage trees
# other than these two trees.

 The size of my local cvs copy for www.linux.or.jp/public:
 $ du -s /Home/sano/work/JLUG/Web/main/public/      
 6220    /Home/sano/work/JLUG/Web/main/public

-- 
  Taketoshi Sano: <sano@debian.org>,<sano@debian.or.jp>,<kgh12351@xxxxxxxxxxx>