• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Tim Cooke
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Liutauras Vilda
Sheriffs:
  • Rob Spoor
  • Junilu Lacar
  • paul wheaton
Saloon Keepers:
  • Stephan van Hulst
  • Tim Moores
  • Tim Holloway
  • Carey Brown
  • Scott Selikoff
Bartenders:
  • Piet Souris
  • Jj Roberts
  • fred rosenberger

What dependency does Apache Spark have on Hadoop?

 
Ranch Hand
Posts: 2601
13
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Does Apache Spark has any dependency on Hadoop.If so What dependency does Apache Spark have on Hadoop?

thanks
 
Ranch Hand
Posts: 51
1
Eclipse IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Absolutely none. The fact that Spark is able to run over Hadoop was done merely for backwards compatibility as Hadoop was the 'big thing' in distributed processing at the time. In 'standalone mode', Spark can actually run over any distributed file system... it doesn't have to be Hadoop.
 
Monica Shiralkar
Ranch Hand
Posts: 2601
13
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks

Theo van Kraay wrote:Spark can actually run over any distributed file system



For what reason does it always require a file system. I can understand it is required for reading data from file system but in case of streaming, why would the file system surely be required?
 
Did you ever grow anything in the garden of your mind? - Fred Rogers. Tiny ad:
Free, earth friendly heat - from the CodeRanch trailboss
https://www.kickstarter.com/projects/paulwheaton/free-heat
reply
    Bookmark Topic Watch Topic
  • New Topic