Lucene Reindex

[Migrated From Jforum.net]

Hi, i have some problems with a new reindex. i don't know what happen. I try in the Admin Panel reindex by PostID and i get this message in the Glassfish server log:
**** Total: 137 ms
i have about 40000 posts, and nothing is indexed.

Also i try with Lucene commnad line tool, but i get some errors:
..............
Exception in thread "main" java.lang.NoClassDefFoundError: jargs.gnu.CmdLineParser$OptionException
at net.jforum.tools.search.LuceneCommandLineReindexer.parseCmdArgs(LuceneCommandLineReindexer.java:163)
at net.jforum.tools.search.LuceneCommandLineReindexer.init(LuceneCommandLineReindexer.java:55)
at net.jforum.tools.search.LuceneCommandLineReindexer.main(LuceneCommandLineReindexer.java:34)
..............

perahps it can be classpath, but don't seem that be this.
[originally posted on jforum.net by feusebio]

[Migrated From Jforum.net]

Try reindexing by date, instead of post id. The post id implementation has some bugs.

Rafael
[originally posted on jforum.net by Rafael Steil]

[Migrated From Jforum.net]

Hi Rafael,

I can run the command tool, but always i have same problem, i try by PostID, by Date, and the result is the same. The running time process is to fast for all posts, and it seems that don't index every posts.

With the command:
./LuceneCommandLineReindexer.sh --path=/home/jforum218 --recreateIndex --type=date --fromDate=01/12/2007 --toDate=02/08/2008

Output log:
-------------------------------
*** Going to reindex using the following options:
-> Searching by date
->App path: /home/fveusebio/jforum218src
-> From date Fri Dec 01 00:00:00 WET 2006 to Sat Aug 02 00:00:00 WEST 2008
->Recreate the index? Yes
->Avoid duplicated records? No

08:29:01,002 INFO [ConfigLoader ] Loading JDBC driver net.jforum.dao.mysql.MysqlDataAccessDriver
08:29:01,263 INFO [MLog ] MLog clients using log4j logging.
08:29:01,337 INFO [C3P0Registry ] Initializing c3p0-0.9.1-pre9 [built 23-September-2006 01:47:24; debug? true; trace: 10]
08:29:01,428 INFO [AbstractPoolBackedDataSource] Initializing c3p0 pool... com.mchange.v2.c3p0.ComboPooledDataSource [ acquireIncrement -> 3, acquireRetryAttempts -> 30, acquireRetryDelay -> 1000, autoCommitOnClose -> false, automaticTestTable -> null, breakAfterAcquireFailure -> false, checkoutTimeout -> 120000, connectionCustomizerClassName -> null, connectionTesterClassName -> com.mchange.v2.c3p0.impl.DefaultConnectionTester, dataSourceName -> 4e17e4ca, debugUnreturnedConnectionStackTraces -> false, description -> null, driverClass -> com.mysql.jdbc.Driver, factoryClassLocation -> null, forceIgnoreUnresolvedTransactions -> false, identityToken -> 4e17e4ca, idleConnectionTestPeriod -> 3600, initialPoolSize -> 3, jdbcUrl -> jdbc:mysql://HOST_DB:3306/DB?user=USER&password=XXXXXXX&autoReconnect=true&useNewIO=false&zeroDateTimeBehavior=convertToNull&useServerPrepStmts=false&dumpQueriesOnException=true&jdbcCompliantTruncation=false, maxAdministrativeTaskTime -> 0, maxConnectionAge -> 0, maxIdleTime -> 0, maxIdleTimeExcessConnections -> 0, maxPoolSize -> 10, maxStatements -> 0, maxStatementsPerConnection -> 0, minPoolSize -> 5, numHelperThreads -> 3, preferredTestQuery -> null, properties -> {}, propertyCycle -> 0, testConnectionOnCheckin -> false, testConnectionOnCheckout -> false, unreturnedConnectionTimeout -> 180, usesTraditionalReflectiveProxies -> false ]
*** INITIALIZING

** At any time, press CTRL+C to stop the process

**** Total: 1586 ms
*** FINISHED
----------------------------------

I have to try to do some debug, i think it can be a problem with post_text field data or something like this.
If you have some ideia, i appreciate.
Thanks.
[originally posted on jforum.net by feusebio]

[Migrated From Jforum.net]

Hi feusebio,

The problem with the command tool is that the compiled class is not present where it should, the folder:

jforum-2.1.8-src\tools\bin\net\jforum\search

is empty, you should compile the class LuceneCommandLineReindexer and put the .class file in that folder, that should make that tool work.

Rafael, I also have exactly the same problem feusebio has, nor the command line tool neither the web reindexing tool work.

My implementation of the forum is not searchable basically, I have deployed both to Tomcat and to Oracle 10.1.3 and the index is not properly generated so the search does not return anything at all.

We have invested a lot of time and effort extending this application, I would be personally very interested in contributing to the jforum project, but without a search the whole application won't be accepted by the business here.

Seriously looking for help here, Rafael.

Obrigad�o
[originally posted on jforum.net by Runnerdave]

[Migrated From Jforum.net]

Hi Runnerdave,

now i have the reindex work fine. But i generate the index by command line tool, only by this way i can generate this. To fix some problems that i had, i change SystemGlobals.properties, where you have: lucene.indexer.db.fetch.count = 50 i put lucene.indexer.db.fetch.count = 1000 or more. With this change Reindex work for all records in database.
[originally posted on jforum.net by feusebio]

[Migrated From Jforum.net]

Hi feusebio, thanks a lot for the tip, now mine works as well

Also the web interface works to reindex, in the meantime I found a bug for the oracle sql query (in jforum\WEB-INF\config\database\oracle\oracle.sql) it says:

SearchModel.lastPostIdByDate = SELECT * FROM (\
	SELECT post_id, ROW_NUMBER() OVER(ORDER BY post_id DESC) -1 LINENUM \
	FROM jforum_posts WHERE post_time < ? ORDER BY post_id DESC) \
	WHERE LINENUM = 0

<br /> <br /> but it should be: <br /> <br />

SearchModel.lastPostIdByDate = SELECT post_id FROM jforum_posts WHERE post_time > ? and rownum <= 1

[originally posted on jforum.net by Runnerdave]