Win a copy of Cross-Platform Desktop Applications: Using Node, Electron, and NW.js this week in the JavaScript forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

Multithreaded distribition system  RSS feed

 
Bob Matthews
Ranch Hand
Posts: 136
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Please consider the following setup:-

Several separate java programs (each being an agent trading a particular currency pair)
A multithreaded distribution system that is trading completely autonomously.
The trading agents communicate among themselves with XML messages.

A very simple negotiation protocol is sought............

1) whenever an agent starts, it informs all the other agents - similarly when it stops
2) every six hours after it makes an investment decision, it advises all agents of this decision
3) once it receives all the trading decisions (prospective trades) it computes a reduced set of optimised trades
4) if it's currency pair appears in the list produced in(3) it opens that trade; otherwise it waits for the next period

The XML messages consist of just two types:-

(a) a status messsage - which is used to communicate 'entry' and 'exit'
(b) a decision message - which consists of their trading decision (prospective trade)

The format of (a) is currency pair followed by action e.g. EUR/JPY IN
action can be IN or OUT

The format of (b) is currency pair followed by size e.g. EUR/USD 2
size can be +2, +1, 0, -1 or -2 where "+" indicates BUY and '-' indicates SELL, '2' indicates 2* standard size and '0' indicates don't trade

I have the following code so far BUT need assistance to complete the coding..........

I am very unclear as to how to code the agents to 'wait' after they have sent their prospective trade until they have received all prospective trades from the remaining agents
Clearly, no. of agents = no. of subscribed agents = no. of prospective trades required every six hours



 
Winston Gutkowski
Bartender
Posts: 10573
65
Eclipse IDE Hibernate Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Bob Matthews wrote:I am very unclear as to how to code the agents to 'wait' after they have sent their prospective trade until they have received all prospective trades from the remaining agents...

First question: Do they need to wait? Couldn't you simply spawn a Thread that "waits" instead? Better still: hand it off to a scheduler that starts it up in 6 hours time.

On the surface, this strikes me as a simple producer→consumer problem, but first let me make sure I have the problem right - at least as far as I understand it.
And I'm going to try and explain it without any of the "technical stuff" (XML and all that jazz):
1. You have a central brokering application..
2. You have a bunch of Agents that produce investment decisions (or "recommendations") for foreign currency trading, and send them to your app. How they do this I don't know (and probably don't need to).
3. Trades may then be enacted through your broker app based on a recommendation.
4. After 6 hours, the recommendation is communicated to all other agents.

The things I DON'T understand:
1. What is an "optimised" trade, and how is it relevant?
2. What is an 'entry' and 'exit' message? They seem to be related to the 'decision' (or recommendation). Are these like "start" and "stop"?
3. 6 hours seems to be quite important, but is it only the amount of time that needs to elapse before a decision is communicated to other agents, or is there more to it?
4. Are decisions sent to agents who are not active?
5. You say "once it receives all the trading decisions". I presume "it" means your "multithreaded [trading] system", but then what is "all the trading decisions"?

The model I envisage is one where individual agents send their recommendations into your "central broker" system as and when they make them, and then disseminate them to each other after 6 hours (otherwise why would you be worried about agents waiting?). My assumption would be that your broker then simply connects actual trades to decisions until such time as an agent recinds it ... or after 6 hours? This is the bit I don't understand.

Finally:
I'm not quite sure what you want of us. You've supplied a very involved - and vague (at least to me) - set of requirements, a snippet of code, and a statement about something you're not clear about.

What would you like us to do?

Winston
 
Bob Matthews
Ranch Hand
Posts: 136
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
1) they need to wait until they have received recommendations from all participating agents

2) the whole process is repeated every six hours(and the coding for this to happen already exists)

3) one takes a collection of prospective trades and reduces this to a collection of optimized trades

4) entry and exit = start and stop (i.e. being part of the system or not)

5) decisions are only sent to active agents

6) the whole process takes place every six hours (calc. prospective trades, advising same, calc. optimized trades, placing these trades)

7) I would like some guidance into exactly what I need to code for all this to work smoothly

Hope this answers some of your questions
 
Winston Gutkowski
Bartender
Posts: 10573
65
Eclipse IDE Hibernate Ubuntu
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Bob Matthews wrote:4) entry and exit = start and stop (i.e. being part of the system or not)

OK

5) decisions are only sent to active agents

Fine.

3) one takes a collection of prospective trades and reduces this to a collection of optimized trades

No help, since I still have no idea what an 'optimized trade' is, or whether it has any relevance to what we're discussing.

1) they need to wait until they have received recommendations from all participating agents

OK, so what I take from that is that every agent has to "wait" (and I'm still not quite sure what that means) until s/he has received a decision from every other agent that has been active in ... what? ... the last 6 hours? TBH, it doesn't make much sense; and it definitely DOESN'T like the sort of thing you want to do in a multi-threaded situation.
A lot of objects waiting around for each other to do something is a recipe for deadlock.

2) the whole process is repeated every six hours

Let's back up, because I'm now totally confused. What happens every 6 hours? And explain it to me in terms of agents, decisions and trades - leave out all the protocols and the XML stuff. What exactly is supposed to happen every 6 hours? List all the major tasks and explain what each one does.

7) I would like some guidance into exactly what I need to code for all this to work smoothly

I hate to say, but the main part is going to be explaining to a duffer like me precisely what you want. Right now, a lot of your descriptions are "internalized" - ie, the way you are thinking about it as someone who already knows the system - rather than intelligible to someone like me, who doesn't.

Once we're talking on the same page, then I (or we) may be able to help.

Winston
 
Bob Matthews
Ranch Hand
Posts: 136
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Some answers:-.................

3) if we have 5 agents then we have 5 prospective trades
these can be reduced to say 2 optimized trades where the resulting trading is equivalent

1) each agent has the identical code to take a list of say 5 prospective trades and reduce this list to say 2 optimized trades,but they can not commence the reduction process until they have received all 4 prospective trades from the other agents plus their own prospective trade to make 5 prospective trades

2) Every six hours:-
(a) each agent calculates a prospective trade
(b) it then advises the result to all other active agents
(c) when and only when each agent has the complete list of prospective trades are they each able to reduce this list to a smaller list of optimized trades
(d) each agent searches the new list to see if they are involved
(e) if so,they will submit a trade

code for (a), (c) and (e) I have already

It is my fault that I am not explaining my request clearly

Just to recap...............
Currently I have several independent agents which trade 'on their own'
I am seeking
(i) a simple communication system between all of them
(ii) the ability for each agent, once they have calculated a prospective trade and advised all other agents of it, to 'wait' and do nothing (other than receive messages) until all prospective trades have been received - at which point they may continue with the existing code which calculates the reduced set of optimized trades etc.

Forget the six hours - replace that with periodically, the coding of which I already have working
i.e. when I require the agents to 'stir' and do something - they do it
 
Winston Gutkowski
Bartender
Posts: 10573
65
Eclipse IDE Hibernate Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Bob Matthews wrote:Forget the six hours - replace that with periodically, the coding of which I already have working
i.e. when I require the agents to 'stir' and do something - they do it

Sorry, I've been away for a while.

TBH, I don't really like this "distribute and wait" idea. It just doesn't sound very "dynamic".

Specifically, I don't like the idea of "waiting for all other agents", because there could be any number of reasons why they don't respond:
1. They're having tea.
2. They're playing footie.
3. Their system's crashed.
and, as I said, "everybody waiting on everybody else" is a recipe for deadlock.

Now, there are any number of ways to get around this:
1. Have a single, multi-Threaded "TradingPost" whose business is to receive prospective trades, generate a single Thread for each one, which waits a specified time and then generates an optimized trade for all agents who have responded within the time limit.
2. Do the same thing for each agent independently, but have the agent free to do anything else they like once they've spawned their "waiter" Thread.
I prefer the first, because it's likely to be more flexible, in that the Trading Post can be put on a proper server, and each agent work from a thin client like a smartphone; but it's certainly not required. And there are probably lots of other paradigms for this "share a Trade" idea too.

I'd definitely get away from this idea of each agent "waiting" for all the others to do something though.

HIH

Winston
 
Bob Matthews
Ranch Hand
Posts: 136
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Specifically, I don't like the idea of "waiting for all other agents", because there could be any number of reasons why they don't respond:
1. They're having tea.
2. They're playing footie.
3. Their system's crashed.
and, as I said, "everybody waiting on everybody else" is a recipe for deadlock.


yes - but as far as (3) goes each agent sends a 'subscribe' or 'unsubscribe' message when starting or stopping.


Now, there are any number of ways to get around this:
1. Have a single, multi-Threaded "TradingPost" whose business is to receive prospective trades, generate a single Thread for each one, which waits a specified time and then generates an optimized trade for all agents who have responded within the time limit.
2. Do the same thing for each agent independently, but have the agent free to do anything else they like once they've spawned their "waiter" Thread.
I prefer the first, because it's likely to be more flexible, in that the Trading Post can be put on a proper server, and each agent work from a thin client like a smartphone; but it's certainly not required. And there are probably lots of other paradigms for this "share a Trade" idea too.


As an alternative to my setup description, I have considered the following:-

Create a Master [TradingPost] program, whose only task is to receive a 'prospective' trade from each active agent and when all incoming messages have been received, calculate the reduced set of 'optimized' trades and send that information to all active agents.
Is this similar to your 1. suggestion ?
Here - the agents do NOT calculate the list of 'prospective' trades

As far as No. 2. goes, there is nothing else for the agent to do while waiting

 
Winston Gutkowski
Bartender
Posts: 10573
65
Eclipse IDE Hibernate Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Bob Matthews wrote:yes - but as far as (3) goes each agent sends a 'subscribe' or 'unsubscribe' message when starting or stopping.

OK, but I said "crash" - ie, some idiot turned his system off at the wall without bothering to log out. Or indeed, he was sitting on a beach in Indonesia (brokers do that, from what I hear) when a tidal wave hit.
Whenever you design a system, you must allow for the fact that the "normal" never happens.

Bob Matthews wrote:As an alternative to my setup description, I have considered the following:-
Create a Master [TradingPost] program, whose only task is to receive a 'prospective' trade from each active agent and when all incoming messages have been received, calculate the reduced set of 'optimized' trades and send that information to all active agents.
Is this similar to your 1. suggestion ?

I think so, but your Master would have to be responsible for a LOT more than just dealing with "optimized" trades - at least in my way of looking at it.

It is the "glue" that keeps all agents talking to each other. It knows who is "logged on" and who isn't; it keeps track of potential quotes, who sent them, and when, and is responsible for sending them out to all other "active" agents; it also processes responses to them, acts as the timekeeper for how long they are "potential", and calculates "optimized" quotes based on responses when their 6 hours is up.
That's quite a bit more than your description.

The ONLY things an agent has to do in a set-up like that is: log on; log off; send in potential trades; look at ones from other agents; and maybe do some trades of their own. The last part is entirely optional, since it's not part of the requirements you outlined; but I'd care to bet that it's something agents might want to do.

Winston
 
Bob Matthews
Ranch Hand
Posts: 136
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
OK, I understand 'crash' comment

I think so, but your Master would have to be responsible for a LOT more than just dealing with "optimized" trades - at least in my way of looking at it.

It is the "glue" that keeps all agents talking to each other. It knows who is "logged on" and who isn't; it keeps track of potential quotes, who sent them, and when, and is responsible for sending them out to all other "active" agents; it also processes responses to them, acts as the timekeeper for how long they are "potential", and calculates "optimized" quotes based on responses when their 6 hours is up.
That's quite a bit more than your description.


It knows who is logged in and who isn't - there is both an onStart and an OnStop method in the broker's trading platform (excluding crashes)
It receives all 'potential' trades and returns the reduced list of 'optimized' trades from & to all active agents
The time it takes to do this is measured in a few seconds

For the Master, I would put a time limit on the time between receiving and outputting a response based on a few seconds
If a 'potential' trade is not incoming from an agent in the allotted time, then that agent does no trading for this particular round
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!