• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Can I modify a list concurrently with multiple different methods ?

 
Bund De
Greenhorn
Posts: 13
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Ok firstly, I know my code is an absolute mess, sorry bout that, but I'm just trying to get this thing done so I don't feel like I've wasted time.
So, I'm taking an online course at Udacity to build a basic search engine, the class is in Python but I'm doing it in Java for the extra challenge.

The problem is, I'm trying to simultaneously add links to a list, and loop through it to get the next links until all the links are extracted, as you can see in the crawl_web() and get_all_links() methods.

I've looked through the web and found something about Synchronization, but I probably did it wrong and it's not working.

So, can this be done ? Is there a better way to do it ?

 
Steve Luke
Bartender
Posts: 4181
22
IntelliJ IDE Java Python
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The ConcurrentModificationException you are getting is not related to threaded concurrency (despite the name), so synchronization will not help you. In this case, the problem comes from the fact that you are trying to manipulate the list directly while iterating over it using an iterator (the enhanced for loop), which you can't do. Instead, you should look at the API for java.util.List (<- link) which provides a solution.
 
Bund De
Greenhorn
Posts: 13
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Is the answer "CopyOnWriteArrayList" ? Let me try that one.
 
Stephan van Hulst
Bartender
Posts: 6311
77
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
No. The answer is to give your methods clear responsibilities with pre- and postconditions to go with them.

Why is a method named get_links() adding something to your links? This is a big design flaw. Don't patch your code to solve problems, write clear code to start with.
 
Bund De
Greenhorn
Posts: 13
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I don't understand, the name is a problem ? get_links, add_to_links, isnt that the same thing ?
 
Steve Luke
Bartender
Posts: 4181
22
IntelliJ IDE Java Python
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
No, the name isn't the problem, Stephen is saying the design is.

Regardless of that, your problem (let me repeat this) is not concurrency, threading, or anything like that. Please stay away from threading and concurrency at the moment because you clearly don't understand it.

The fix I mentioned in the API is to use a ListIterator: which allows you to modify a list as you iterate over it. But you can only access, add, or remove to/from the list through that ListIterator - not through the List itself. That means a hefty redesign (which you need to do anyway).

You can do that, or you can index manually (using an integer counter), or you could us a Queue (where you use a while((link = queue.poll())!=null) loop to get the links).
 
Bund De
Greenhorn
Posts: 13
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks I'll look into that
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic