• Post Reply Bookmark Topic Watch Topic
  • New Topic

Applying shaky grasp of design patterns - preparing for multithreading & spring  RSS feed

 
Sean Funk
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I've set out on writing my first end-to-end java app that utilizes spring & mysql, in the hopes of extending this to Android development in the near future. However, I've still got a ways to go I think and there's still some concepts which are murky to me.

I've refactored the code a couple of times now, but I'm kind of confused as to how I should proceed to implement multithreading. I'm under the assumption that the preferred methodology is using a pre-determined executorpool (threadpool?) and then starting - but I'm not sure that doing this from the main method itself is the right way to do it. Here are the two classes and java file:








Am I utilizing the concept of a class properly in this example? I feel like I don't quote have a clear separation of "duties" between the DataHandler and ForumPageDataHandler (could be renamed to match the package I guess..) classes. My goal is to effectively add multithreading at the for loop in the main[] class, since that would handle individual "rows", with no individual row being dependent on any other, with each thread just adding it's completed object to forumPosts at the end. This is where I would hand the arraylist off to a batch for insertion into a mysql db (spring integration, I believe).

Thanks for reading, and I apologize if my question isn't clear enough. I might have been staring at this for too long. Also, I just found a couple of bugs but they are minor (imgMimeList is one).
 
Sean Funk
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Maybe this was in the wrong forum, but I don't want to double post, so here's an update:

Refactored some more. Removed one file completely and I think I'm much closer to being able to properly implement Runnable, but I'm still a little murky about where I create the threads and where I assign the work to the threads. Updated code:



 
Stephan van Hulst
Saloon Keeper
Posts: 7987
143
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Sean, I doubt many people are going to look at your code to find out what you want.

All we know is you have some code, and you want to make it multi-threaded. Please give us some more background on what your application is supposed to do, what it's doing now, and where you're running into troubles.
 
Sean Funk
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
My apologies, I should have been more clear :-)

The point I want to multithread is where I hand off a 'postRow' to a set of methods for processing. We always have a variable number of postRow Strings inside the postRows arraylist. So roughly, this section of code:



Currently, the code executes, but it appears to execute only one thread - and the thread never dies so the program hangs once all the rows are complete. I"m still messing around with it locally, but i'll update the thread this evening.

To be more clear, the function of the code posted in my second post is this:

- Pings end point, retrieves a page.
- Passes a certain portion of that page into an array list, separated by <br> (individual posts (or rows) into it's own slot in the list)
- Iterate through that list, grabbing data from each line, as well as creating another get request for each individual post body, whose sourceiD is in each post row (thus the need to process each row, then each body)

At the point of processing the individual rows, and subsequently the post bodies, I would like that to spawn multiple threads. Each post row and body are unique and tied only to one another, not to other post rows or bodies - so my thinking is that each thread can operate independently once it's generated, while waiting for the last thread to finish and then program can end. It's at that point that I would hand the data off for a batch insertion into a mysql db, but I haven't gotten that far yet. :-)
 
Stephan van Hulst
Saloon Keeper
Posts: 7987
143
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Sorry, I forgot to welcome you to the Ranch. Welcome Sean!

So what you want is to retrieve message headers from a page, then concurrently retrieve each message body and process that in some way? Just pass each header to a task that you submit to the thread pool.
 
Stephan van Hulst
Saloon Keeper
Posts: 7987
143
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I rewrote some of your code:







 
Stephan van Hulst
Saloon Keeper
Posts: 7987
143
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I think the Header class warrants a closer look.

It's immutable, since that will avoid a lot of problems, especially with concurrency, or when using it as the key in maps. Since it's really annoying to have a constructor that takes a bazillion of parameters, I added a Builder class that will easily let you set various attributes, and then finally you can construct an immutable header out of it. Another addition is that it has a reference to a header it's in reply to.

Note that you shouldn't store points on a timeline as DateTime objects, since DateTimes don't represent moments in time. They represent dates and times on a calendar, which are mostly useful for manipulating data that is meaningful in that particular calendar.
Instead, you should use Instants. They are calendar-agnostic, and represent a unique moment in time, wherever you are in the universe.
 
Sean Funk
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
...
 
Winston Gutkowski
Bartender
Posts: 10575
66
Eclipse IDE Hibernate Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Sean Funk wrote:Wow!
I'm walking through/debugging the code now - if I'm correct, the code doesn't actually get the post body (which is absolutely OK - I can work from there!)? It executes perfectly. There are several implementations that will take me a little time to work through that you've used here (main(String... args) for example) Good thing I have the week off, I'll be digesting this for a little bit. :-)

Could I suggest then that the main reason is that you've only thought through one way of doing this?

And I say that without, I admit, having looked in detail at your code.

But your questions are all about "how". How do I do [this]? Why is [this] going wrong? - which suggest that you've already decided on an implementation before you know whether it's the right one.

The very fact that you've decided that "multi-threading" is what you want seems like the wrong basis from which to begin a design (or a redesign).
Why do you want to multi-thread? What benefits do you think it will provide? Do you know that it will solve the reasons for redesigning in the first place?

It's perfectly possible that you have good answers to all those questions; but unless we know what they are, all we can work with is the pile of code you submitted, and your assumption that the "change to multi-threading" is required.

Personally, there are two things I try to avoid at all costs when I'm programming: Reflection, and multi-threading.
The first because it seems wonderful, and usually isn't; the latter because the second you enter into a multi-threaded world you take on a whole new set of problems - namely: every gateway to every variable or action in your class becomes a potential "leak" or "anomaly" (ie, it basically doesn't behave the way you expect it to).

Secondly: multi-threaded programs are usually very difficult to test. Properly.

Databases have to deal with multi-threading because ... well ... that's what they were designed for; and it's also why they have teams of people to deal with the results and to test every change to blue heat.

So, first question: given the above, do you still think that multi-threading is the only solution?

Second question: Why? (and please be specific)

Multi-threading can be the right answer, and Java provides a lot of support for it; but IMO you need to really want to do it, and it should be as minimal as possible in all cases.

Sorry if it isn't the answer you wanted, or doesn't help; but it's something (as you may have gathered) that I feel quite strongly about.

Winston
 
Sean Funk
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Wow, I apologize for asking this question here. If any mods are available, please move this to the beginner forum. I didn't think posting some code and questions would elicit a response like that.

 
Stephan van Hulst
Saloon Keeper
Posts: 7987
143
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I wouldn't say this was a beginner question, and don't worry about Winston's response; text doesn't convey emotion well, and he raises some great points, take them to heart. You don't have anything to apologize for, you were stuck on something, and now you have food for thought.
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!