This week's book giveaway is in the Reactive Progamming forum.
We're giving away four copies of Reactive Streams in Java: Concurrency with RxJava, Reactor, and Akka Streams and have Adam Davis on-line!
See this thread for details.
Win a copy of Reactive Streams in Java: Concurrency with RxJava, Reactor, and Akka Streams this week in the Reactive Progamming forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Liutauras Vilda
  • Junilu Lacar
  • Jeanne Boyarsky
  • Bear Bibeault
Sheriffs:
  • Knute Snortum
  • Tim Cooke
  • Devaka Cooray
Saloon Keepers:
  • Ron McLeod
  • Stephan van Hulst
  • Tim Moores
  • Tim Holloway
  • Carey Brown
Bartenders:
  • Piet Souris
  • Frits Walraven
  • Ganesh Patekar

Lucene and Google App Engine

 
Ranch Hand
Posts: 697
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
To Dear Lucene in Action Authors,

GAE has been getting a lot of buzz lately but one of the major complains is the lack of full-text search capabilities which is what Lucene provides. Is there a chance that Lucene (either Java or Python version) will be usable in GAE?

Thanks.
 
author
Posts: 20
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
There's this project:

http://code.google.com/p/gaelucene/

But I haven't used it personally.
 
Author
Posts: 111
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
And there's this excellent post: http://www.ultrasaurus.com/sarahblog/2010/06/full-text-search-on-app-engine/
 
blacksmith
Posts: 979
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
...why mix the two solutions?

Gian
 
Paul Michael
Ranch Hand
Posts: 697
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Michael: The GAELucene project shows very low activity.
Erik: The JRuby approach doesn't seem to be a complete solution.

I guess we still have to wait for some time for a more complete and mature solution to come out.

Anyway, thanks for the suggestions, I'll try to consider them in my next projects!
 
Michael McCandless
author
Posts: 20
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Or, you can help to push to a more complete and mature solution for Lucene running on GAE ;)

Do you have a sense of what challenges/limitations GAE imposes?
 
Paul Michael
Ranch Hand
Posts: 697
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Haha that's a nice suggestion Michael...

I'd probably give it more consideration if I were 10 years younger and still single. I'm afraid most of my waking time is already spent on managing the house, making a living and working on ministry projects.

Anyway, here are some of the restrictions for Java according to an article from InfoQ:

* Once a request is sent to the client no further processing can be done. This includes data streaming.
* A request will be terminated if it has taken around 30 seconds without completing. At this point an exception is thrown. If not caught a 500 error is returned to the user.

Moving up the stack are several sandbox restrictions:

* Applications can not write to the file system and must use the App Engine datastore instead.
* Applications may not open sockets
* Applications can not create their own threads or use related utilities such as timer.

java.lang.System has been restricted as follows:

* exit(), gc(), runFinalization(), and runFinalizersOnExit() do nothing.
* JNI access is not allowed.

In addition to these key items there are other limitations such as a JRE class loading white list. From the documentation GAE seems to impose a good deal of its magic using custom classloaders. However, they should allow other application level classloaders as long as they can operate with the above restrictions.

 
Michael McCandless
author
Posts: 20
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Those restrictions sound fine w/ Lucene. I guess the biggest thing is not using the filesystem, ie we need a Directory impl backed by the App Engine datastore (and if I remember there's a max file size restriction (maybe 10 MB?) in this?).

You'd have to switch to SerialMergeScheduler, too, since the default ConcurrentMergeScheduler does merges with its own background threads.
 
All of the following truths are shameless lies. But what about this tiny ad:
Building a Better World in your Backyard by Paul Wheaton and Shawn Klassen-Koop
https://coderanch.com/wiki/718759/books/Building-World-Backyard-Paul-Wheaton
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!