Win a copy of Mastering Corda: Blockchain for Java Developers this week in the Cloud/Virtualization forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Paul Clapham
  • Ron McLeod
  • Bear Bibeault
  • Liutauras Vilda
Sheriffs:
  • Jeanne Boyarsky
  • Tim Cooke
  • Junilu Lacar
Saloon Keepers:
  • Tim Moores
  • Tim Holloway
  • Stephan van Hulst
  • Jj Roberts
  • Carey Brown
Bartenders:
  • salvin francis
  • Frits Walraven
  • Piet Souris

What dependency does Apache Spark have on Hadoop?

 
Ranch Foreman
Posts: 2343
12
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Does Apache Spark has any dependency on Hadoop.If so What dependency does Apache Spark have on Hadoop?

thanks
 
Ranch Hand
Posts: 51
1
Eclipse IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Absolutely none. The fact that Spark is able to run over Hadoop was done merely for backwards compatibility as Hadoop was the 'big thing' in distributed processing at the time. In 'standalone mode', Spark can actually run over any distributed file system... it doesn't have to be Hadoop.
 
Monica Shiralkar
Ranch Foreman
Posts: 2343
12
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks

Theo van Kraay wrote:Spark can actually run over any distributed file system



For what reason does it always require a file system. I can understand it is required for reading data from file system but in case of streaming, why would the file system surely be required?
 
Blood pressure normal? What do I change to get "magnificent"? Maybe this tiny ad?
Building a Better World in your Backyard by Paul Wheaton and Shawn Klassen-Koop
https://coderanch.com/wiki/718759/books/Building-World-Backyard-Paul-Wheaton
reply
    Bookmark Topic Watch Topic
  • New Topic