Service Level Agreement

[tao gu]

Hi, All

My project services different group of users. Due to the complex processing, the response time is not good especially when one user are keep firing transactions. This will drain out the resource and other users may experience long waiting-time.

So we are thinking about implementing some kind of Service Level Agreement to regulate the traffic. Each user will be given a TPS(transaction per second) value. If the value is exceeded, the successive method call may be hung.

The problem is that, the TPS will reduce the system throughput. For example, even only one user is using the system, he still can not exceed the TPS. This is wasting the resource. How to get a balance between? That is , to prevent some user from starving and to maximize the system through as the same time?


What experiences & technologies should be followed to address the problem?

Or any other good points?

[Jeanne Boyarsky]

Tao,
You could only enforce the transactions per second when the load is only above X percent.

I assume you have already tuned the system? Before imposing a limit on users, it is important to know that you are truly operating at maximum capacity.

[Ilja Preuss]

Originally posted by Jeanne Boyarsky:
I assume you have already tuned the system? Before imposing a limit on users, it is important to know that you are truly operating at maximum capacity.

I don't fully agree - if it's cheaper to impose the limit on users and the customer can live with it, it might well be the way to go. With other words, I think it's a business decision, not a technical decision.

[tao gu]

Hi, Could you further explain the follow sentence?

Tao,
You could only enforce the transactions per second when the load is only above X percent.

I believe there must be some common solutions to this kind of problem, like to ensure the response time to some VIP user but normal users may wait longer. The response time is a very import non-functional requirement for any serious J2EE project. But I am not able to find the relevant resource from either website or books. Anyone can recommend some good source of information?

[tao gu]

It's more technical. Because the method call traffic is not plain in nature. A burst will come when a customer keeps sending requests for hours while other customer sends request maybe one per minute. As a result, most customers will encounter longer repsonse time during the burst.

We must regulate the traffic before we delivery the service. Right?

I think the main cause is that, the customer calls our service through servlet, which is designed for www without Qos in mind. It only looks at requests, not the user. All clients are the same.

What we are going to do is add one more layer on top of servlet, to ensure the Qos or SLA for each user. Currently the SLA is very simple, only Transaction Per Second (TPS) will be specified. Later we will consider to add more.

Originally posted by Ilja Preuss:


I don't fully agree - if it's cheaper to impose the limit on users and the customer can live with it, it might well be the way to go. With other words, I think it's a business decision, not a technical decision.

[Jeanne Boyarsky]

I don't fully agree - if it's cheaper to impose the limit on users and the customer can live with it, it might well be the way to go. With other words, I think it's a business decision, not a technical decision.

Ilja,
My issue comes from "if the customer can live with it." They need to know there is a choice. It could be presented to them that extra performance will cost a certain amount of money. Also, if no tuning was done, the service is likely to scale poorly.

You could only enforce the transactions per second when the load is only above X percent.

Tao,
I was just trying to say that you should artificially restrict traffic. For example, suppose one user decides to hit your service with many transactions at 3am. If nobody else is on then, why should that user be limited to a smaller number of transactions per second.

[Ilja Preuss]

Jean, I think we are in agreement!

The developers should explain the options (plus costs) to the customer and help him make a decision that is best for him.