Java Screen scraping, is it possible?

[James Elsey]

Hi,

I'm trying to look for a solution to screen scraping in Java. I've had a look on our dear friend google, but most of my searching leads to Xquery, which I believe only works for HTML pages by sifting through the source code. That doesn't really work for me.

In an ideal world, I would like to create a java application that when invoked (most likely by some button) would identify a region on the screen, pick up some value, and then call a URL or command

The first issues I can see is how can a Java app "scan" the screen. Say for example I have outlook open, I would like this application to find the outlook window, find "inbox" down the list of folders, then "read" that section and find out how many emails I have. It would return something like "Inbox(5)"

I'm reasonably confident that once I have the info scraped, I can deal with it as I please, but my main task is how to find the correct window, and OCR text.

Can anyone give me a heads up on if this is possible? A pointer to a good API would be even better!

Many thanks

[James Elsey]

Just to add,

The reason I can't use an HTML scanning solution is that I can't always guarantee the target application is browser based. I would need this to work on desktop applications such as Outlook / Word

[David Newton]

How will it "find" the Outlook window? What if the window is partially obscured? What if the user isn't using the default Outlook layout (I don't, when I'm forced to use it)? What if they're using a difficult-to-OCR font as their system font?

A simple "java +ocr" search will pull out some OCR libraries, but the arbitrary nature of what you're trying to do makes me think you're in for more work than you realize.

[James Elsey]

Hi David,

For this to work I'll have to make some assumptions

Outlook will have the default look and feel
resolution/color settings will be fixed
the target window will be on top of others, and not obscured

Would having a pre-defined screenprint of where the information is, then using that in the application to compare what is on the screen be a viable option?

Other than that, I'm struggling to think of how a java app is able to read "from the screen" with other applications.

Thanks for the help

[Campbell Ritchie]

Not a "beginning" question. Moving.

[Ernest Friedman-Hill]

As far as grabbing the bits, check out the java.awt.Robot class, and particularly the "createScreenCapture" method.

[Amni Gandhi]

HI James,
Any luck with Screen Scraping solution?

[James Elsey]

Hi Amni

Sorry been so busy haven't had a chance to try it

Will post something up when I eventually get round to it ;)

J