Mike London

Stephan van Hulst wrote:No, scraping is not illegal and you wouldn't get sued for using the data during development. Redistributing the data is another matter and falls under copyright law. Redistributing the data 'as is' is almost certainly not allowed, but processing it to gain new insights might be considered 'new work', which you can monetize.

Copyright law is a minefield. Consult with a lawyer.

Thank you!

I convinced the client to forget about this idea.

- mike

If I use a program like JSOUP for Java, and scrape daily content from the basic news sites, is that legal?

I would add these extracted data to a database for analysis and searching for a commercial application.

From what I've read on various "Is site scraping legal" websites, the "commercial" use of those data means I (during development) or the client (in production) could have legal consequences.

Since this is a potentially very litigious area and I wouldn't want any company suing me (during development) for scraping their sites, I'm trying to figure out if I should push back on the client's requirements for these data.

---

Perhaps a commercial aggregator like "newsapi.org' would be totally legal and a good alternative.

Would appreciate comments and suggestions.

Thanks in advance,

-- mike

Campbell Ritchie wrote:You mean the company are invasive rather than the actions? I think I agree with Stephan that both are invasive.

Agreed.

-- mike

Stephan van Hulst wrote:

Mike London wrote:But, we're talking about an ultra-privacy-invasive case (company).

Why is this more invasive than any other case? It's very simple: Who or whatever you are, if I haven't given you permission to access my private stuff and you do it anyway, that's an invasion of privacy, period.

I was referring to the company itself.

Stephan van Hulst wrote:Without having any experience with the Facebook API, I'm going to say yes. I can not imagine that the API would allow you to do anything that you wouldn't be able to do through Facebook itself.

Wouldn't it worry you if a rando company could read your wall without your permission?

Yes, I would. But, we're talking about an ultra-privacy-invasive case (company).

I think I'm going to have to create a dummy account so I can post on their forum and get definitive answers.

Thanks for the sanity check.

Stephan van Hulst wrote:What do you want to do exactly? Why isn't the Facebook Graph API enough?

It doesn't look like I can search a person's FB "wall" without permission or if it's private.

Is that true?

Thanks

Stephan van Hulst wrote:LinkedIn's API is restricted. You must get vetted by them before you're allowed to use their APIs.

Awesome. Thank you! That's what I thought I was seeing....

--mike

Stephan van Hulst wrote:What do you want to do exactly? Why isn't the Facebook Graph API enough?

No, that's fine. I'm just wondering what other APIs might be in use. Just being complete.

What about LinkedIn? That looks like it's been gobbled up by MS. The page that describes the services for LinkedIn doesn't seem to indicate a search API.

Thanks

Has anyone done anything with Facebook or LinkedIn public data scraping?

I haven't found any libraries and the basic capabilities via CURL (REST) seem very limited.

Thanks in advance,

- mike

Rajesh gudupally wrote:How did you solve this issue?

The problem was a permissions issue. Tomcat was logged in as a user who didn't have permissions to write to that particular folder where the log was.

So, I changed the Tomcat user and the log started writing.

- mike

Ron McLeod wrote:There should be nothing confidential in the certs, but you do need to protect the private key.  If you don't use a keystore, how would you keep the private key safe?

That is a good point and supports using keystore approach I guess.

Thanks Ron.

-- mike

Stephan van Hulst wrote:I think what Mike means is, if he has a PEM-encoded certificate file, then why should he go through the trouble of converting it into a JKS/PKCS12 certificate file?

Honestly, it doesn't really matter what you use. If you already have a PEM-encoded certificate, good for you. Use it and be done with it.

Thank you! Yes, that's all I was asking.

-- mike

My question was whether using Coytoe is a better approach than the manual process of creating a keystore.

Maybe neither approach is better, just two ways of doing the same thing: Implementing the SSL certificate.

Sorry my question was unclear.

Thanks,

-mike

Is Coyote considered a more current way to secure Tomcat than all the (painful) steps to generate a keystore file?

I like that Coyote just refers to the cert files directly.

Thanks,

- mike

LOL...thanks Tim!

--mike