• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

performance issue in mongoDB

 
Ranch Hand
Posts: 40
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I am doing sync between sql DB to vertica.
I am moving 5 giga data in the network from sql and insert it in our document structure in mongo.
i am doing it with 10 connections to the mongo and 10 threads.
everything is going well but after i copy 1 giga, the rate of the copy become slow more and more until i dont see more new docs in the collection.

again 10 threads every one has a connection to Vertica DB(sql) and to the mongo.each one of the threads connect to Vertica copy some data turn it into entities and save it in Mongo.
they run and after ~1 giga the process become very slow until i dont see any more new docs in the collection(and there are more for sure ).

do you familiar with this?
can you advise?

 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
reply
    Bookmark Topic Watch Topic
  • New Topic