• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Liutauras Vilda
  • Jeanne Boyarsky
  • Devaka Cooray
  • Paul Clapham
Sheriffs:
  • Tim Cooke
  • Knute Snortum
  • Bear Bibeault
Saloon Keepers:
  • Ron McLeod
  • Tim Moores
  • Stephan van Hulst
  • Piet Souris
  • Ganesh Patekar
Bartenders:
  • Frits Walraven
  • Carey Brown
  • Tim Holloway

Bugs with parameters escape

 
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Currently jforum does not do proper HTML escape of form parameters:
For example go to register page and enter this

">FF TST<

as e-mail, in other fields enter anything.
You will get

FF TST<"/>

printed outside of the text field. This is just an example, there a lot more other
such form problems.

The right way to fix this problem - on template level.
To fix this problem in JSP (JForum does not use JSP, but this just an example)

I added in addition to <%= value %> construct one more:
<%~ value %>
which does same thing, but prints value in escaped form
(replace < to < " to " , etc.)
In over 90% cases when you do a web site the <%~ value %>
is what you really want. You need <%= value %> very seldomly.
(I tried to propose this to SUN, but they rejected this proposal).

In templates jforum uses I would recommend to add a new construct like
$[value] in addition to existing ${value}
so when using <input value="$[value]">
the value would be HTML-escaped, when
<input value="${value}"> it will not be escaped.

then I think about 90% cases should be converted to $[value]




[originally posted on jforum.net by Anonymous]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If you ask my opinion about template system - current template system used by jforum is ugly and complex without a reason. (I do not want to say that "standard" JSP is much better).

The reason why jforum template system is bad are:
1. it uses reflect, thus throwing away strong java strong typing.
2. it uses homegrown ugly template language. You already have a good modern language - java, use it.
3. It does not allow compile-time check.

My proposition is to use modified JSP system, from which I cleaned all garbage, put there by SUN.
Think of a template as a java code with HTML addition (not as of HTML with java addition as SUN wants you to think).
And think of JSP only as a preprocessor which converts .jsp to .java files.

An example of a page template look as following

------- example.jsp -------------
public static void printFormA(java.io.PrintWriter out,ActionServletRequest request,
String name,String value) {
headers.printHeader(out);
%>
<FORM ACTION="/test">
<INPUT NAME="<~ name %>" VALUE="<%~ value %?">
</FORM>
<%~ I18n.getMessage("some message")} %>
<% printSomethingElse(out,value); %>
<% <br /> headers.printFooter(out);
}

static printSomethingElse(java.io.PrintWriter out,String value)
{
%> print some other HTML <% <br /> }
---------- end of example.jsp -----------------


Simple JSP-like preprocessor would convert this to example.java
and when you need to call this template you just do in java main code

net.jforum.templates.example.printFormA(out,request,name,value);

This way you:
1. Have every template identified by java signtanutes of method(s).
2. You call these method directly as java code!!! No reflect any more (slow, works poorly, vary hard to debug, you may find out you have an error only after months of testing). Think "this is ugly" every time you use reflect.
3. You pass all data to template as java methods arguments.
4. If you need some code in template - then open <% <br /> and you have all power of java, without any homegrown hack-language.


If you want - I cane send you .jsp perprocessor (trivial thing).

The main idea is to use all power of java and have typecheck at compile time and that every templates is identified by java method(s).
If you subscribe yo this idea - I would send you code and examples.


[originally posted on jforum.net by Anonymous]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
And by the way, the method of internationalization used in JForum as

I18n.getMessage("User.reset")

is very limited. Soon you would find out that you need to create a message
from data and text, like.

Welcome user <%~ username %>, happy birhday <%~ format.(...) %> ....

then you would need either to agree with limited expressiveness of generated text or use a bunch of: User.messageBeforeBirthdayDate User.messageAfterBirthdayDate , etc.

the right way (from my point of view) is to have a class
class English extends I18n

static I18n messages=new English();

and where you need

messages.birtdayWelcome(username,name)

and method birtdayWelcome would print the message using all argumens.

You also get a compile time typecheck.
To create English.jsp it is very convinient to use template system described in previous two messages.


[originally posted on jforum.net by Anonymous]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
In the first posting I meant
replace < to <
" to "
etc.

JForum converted them to the symbols they represent.
[originally posted on jforum.net by Anonymous]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Somehow I can not put & lt; and & gt;

I tried to enter & amp; gt; but JForum also converted them.
(but in "preview" it looks correctly)

In this message I put a space so you got an idea.
[originally posted on jforum.net by Anonymous]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I read about freemarker tmplate engine you use - I really do not like it, see my arguments above.

For freemarker usage the easiest thing would be probably to add thing like

${~value}

instead of $[value] as I originally recommended.
The ${~value} would mean: evaluate value and then HTML escape.

But anyway, I think freemarker is the wrong answer for web application template engine. The right template system as I see it is described above.

In the end of the day - if you have all the data to check everything during web application compilation - not using it would be a big mistake.
And, don't you feel that these homegrown template languages are limited ugly creatures?

[originally posted on jforum.net by Anonymous]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
About your $[xxx] thing, there is already support for it in freemarker:



I'm aware of the problems you described in the first post. They're caused by the HTML filter (SafeHtml.java). I'm working on a fix.

Rafael
[originally posted on jforum.net by Rafael Steil]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,

you said a lot of stupid things about the template engines. It looks like a person who never used a Template Engine before.
Firstly take a look at the frameworks, to after do this kind of comments.

The "very incredible" web language that you have described above already exists, and it is called Servlets. The Servlet API is poor when used to write HTML.

The web Template engines only exists because the Servlet/JSP API is very very difficult to use.

Sorry for my anger, but I can't be in silent with this kind of bullshit.
[originally posted on jforum.net by fsamir]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
> you said a lot of stupid things about the template engines. >It looks like a person who never used a Template Engine before.
>Firstly take a look at the frameworks, to after do this kind of comments.

fsamir,

I seen a number of template engines before and frameworks.
Believe me, almost of them are build not by professionals.
Understand this:

What is the goal of template system:
Integrate data and presentation to obtain HTML.

How the system(s) you advocate work:
1. In java code you put something in hash or in some other data structure.
2. Then this object is transfered to template engine where it is extracted at best by lookup, at worst by reflect.
3. Special homegrown ugly language is interpretes a template, printing HTML and extracting the data at best by lookup, at worst by reflect.

At all steps you loose information about types. Typical problem- you changed something and you find it only during testing (at best) or never find a bug (at worst).
The solution - use the power of java language (strong types) to your advantage.

I did not write you anything about any framework above, nothing about servlets, I wrote about printing html (about templates).
Just for your, fsamir, education: A good framework works usually like this (very briefly):

Web application consists of a set of
1. Tophalf processors.
Tophalf processors process posted data, create objects.
The end result of tophalf processor is the information what bottomhalfprocessor to use and a set of java dataobjects it created. Depending on condition different bottomhalfprocessors can be called.

2. Bottomhalf processors.
Bottomhalf processor prints a web page. It usually is a java function which receive dataobjects (created by tophalf processor) as arguments. The JSP-like language I described above usually used for creation of bottomhalf processors, i.e. printing HTML.
Think of a bottomhalf as a template.

3. How the system works: A request is processed by tophalf processor, it creates dataobjects and choses what bottomhalf processor to use. Then bottomhalf is called using standard java call.

This thing look to you probably like most other so called "frameworks", but it is way better, because it has the knowledge about datatypes and any signature mismatch would be caut at compile time.


If you do not understand what I write here I would put it in a way you would probably understand:

Take your favorite framework and based on it try to build a system which does same thing but
DOES NOT DISCARD INFORMATION ABOUT DATATYPES.
Very soon you would come to the conclusion that:
1. Templates should be compiled, not interpreted. The evident target to compile to is java. You need to compile templates (.jsp .htm or whatever) to java files.
2. The easiest way to preserve information about datatype is to transfer data to template using FUNCTION ARGUMENTS.
3. If you think some more you would find out how to obtain data without reflect and hashtable lookup. This is not hard.

P.S. Yes, fsamir, there seldom cases where you have to sacrifise datatypes. But these cases are extremely rare. And I strongly recommend you to read any book about strongly typed languages.



[originally posted on jforum.net by Anonymous]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Irregardles of the validity or not of your claims, I just think a little bit of good manners and correct spelling would do wonders to help you express yourself.


[originally posted on jforum.net by GreenEyed]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

GreenEyed wrote:Irregardles of the validity or not of your claims, I just think a little bit of good manners and correct spelling would do wonders to help you express yourself.


After some thinking I agree with both your statements. In addition to that - spelling problem is a common problem with web forms when typing a long text. Small typing area and font size make everything much harder than in a text processor. I did not put enough effort by typing text elsewere and then paste it to the form. My last message was just a personal answer to fsamir, it was not considered as an important document. It was some kind general statement that you can get much more from compiled templates than from interpreted ones. This is the main reason I do not like SUN's JSP, where they started with a good idea (compile to java) and then ruined it by mass using of reflect and other unnecessary complexity.

In regards with technicalities of jforum - I just think
HTML escape should be done in templates, on lower possible level - during form output.


[originally posted on jforum.net by Anonymous]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Well, spelling is, as you say, a problem in all kind of web forms, and it shows how much we have come up to depend on spell checkers ;).

On the other hand, right now there are lots of options for the presentation layer, and there's no clear winner, in spite of some solutions being better in some regards than others.

That also means that there's no optimum solution, and in many cases it depends not only on the technical merits but on the available resources and knowledge of the people involved.

In my case, I prefer not to use Java at the presentation layer, which discards JSP or any kind of Java based template mechanism, and prefer to use an independent standard like XSLT. It has several drawbacks, but in our case it has shown to have many advantages as well, and that's why we keep using it after 5-6 years of doing so. But that does not mean it would work so well for other teams or other kind of projects, and that's somethign one has to realise, there's no silver bullet to cover all the cases and what works for you might not work for others. The beauty of Java is that you have the freedom to choose the best path for you & your project.

And that means one should also show respect for other people choices, as even when they might seem wrong to us, they might have their explanation. And even when they are wrong, people will listen to you better if you explain it correctly, as they might "close their mind" if they feel "atacked".

My 2ec ;)
[originally posted on jforum.net by GreenEyed]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
As I just found out the problem with posted parameters escape (which I proposed to fix on template level for example by replacing all (almost all) ${xxxx} to ${xxxx?html} is a very serious one.

Using specially crafted form parameters one can do such nasty things as:
1. Inject a link to some other web site, thus increasing google rank of that site. thus jforum site will have a link to that site (link HTML is embedded in form parameters)
2. Create css div object and show some other site as it were the site where jforum is running. This simplifies web-phishing a lot.

The problem looks to be a serious one.
[originally posted on jforum.net by Anonymous]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,

please, send me a pratical example, so I can work better on a fix.

Rafael
[originally posted on jforum.net by Rafael Steil]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
This is an example of scam link I got on one of my jforum sites.
A scammer created a link on his web site, and such link propagated to google:
A link created by scammer, the page of which is treated by google as one of pages on my web site.

Jforum page with a link to some.scam.site.com


google picked this link content (this is jforum output content, so google thinks this is my content) and google thinks that I have a link to the scammer's site:
http://some.scam.site/

In a similar way I can put almost anything to a site running jforum, just change URL parameters. I can put any text on jforum web site by creating a special link and posting this link to search engines.

In 99% cases all variables printed in templates should be HTML-escaped.
This is what I was writing in my original post.
The best way to do this- on lowest (template) level.


[originally posted on jforum.net by Anonymous]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
An example to show third party site (CNN) on your web siet

Also often used by scammers when printed form parameters are not HTML-encoded.
[originally posted on jforum.net by Anonymous]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Also a scammer can inject special javascript code which would steal session ID cookie and post it on some other site. An example is trivial.

This is a common problem with most of template system. JSP is no different. This is what I was writing above (in jsp terms) about

<%= variable %>
and
<%~ variable %>

in 99% cases the <%~ variable %> is what you actually want.
[originally posted on jforum.net by Anonymous]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Ok, although I agree with you that the registration page shows the CNN iframe - and that I will fix it -, I want to have some example in this page - it can be a simple iframe or a javascript alert(). I ask that because, as far as I've tested, the code can filter this kind of piece of code, but as you're saying that it doesn't, the trivial example will help to improve the test case .

Rafael
[originally posted on jforum.net by Rafael Steil]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
>Ok, although I agree with you that the registration page shows the CNN iframe - and that I
>will fix it -, I want to have some example in this page - it can be a simple iframe or a
>javascript alert(). I ask that because, as far as I've tested, the code can filter this kind of
>piece of code, but as you're saying that it doesn't, the trivial example will help to improve the test case .

You can filter user input when some HTML tags submitter by user input to become part of HTML output.. This filtering is necessary. Fortunatelly there is few such places: message posting, user signature and few other where user entering HTML. But this is filtering input, not output.

Filtering every page HTML final output is worthless. For this task a filter should parse every page HTML output, and because output is a combination of internal data and posted parameters it is impossible to do 100% correctly, one of the reasons is an ambiguidy you may get with such HTML. And this filtering should not be done. Template system should output correct HTML from the beginning. Post-fixing broken HTML by a filter is worthless.
This second filtering of final HTML should be avoided. Instead in all templates in places where you print text (and in most placese you print data as text, not data as HTML) you should do HTML-espace of this string.
And it is clear why: when you print strings you do HTML espace, when you print HTML you do not. (but just very few places where you actually print HTML).
Think about this (imaginary example) that you have two different classes
TextString extends String
HTMLString extends String
And they must be printed differently. Very reasonable assumption about data.
(And now imagine how easier many things would be if your templates system would be java-datatype-aware, but I am not critisizing freemarker now, just an example)

To the point: An example I gave here was just one of many cases when form parameter is treated as HTML when it must be treated as text, (i.e. HTML escaped).
All form parameters, all URLs, and many other things must be treated as text string (i.e. HTML escaped).

An example I gave you (there many other cases of similar usage) is in registration FORM
put in e-mail address the follwing:

"> some HTML there (iframe, A or other)

where "> symbols are used to close INPUT tag of the form and the rest is malicious HTML to inject.
It will be put to the form. This is how I put there link and iframe. Other form parameters are arbitrary. I just need e-mail being re-printed.

This is e-mail. It may well have " > < and other such symbols.
Again, it should be printed in template as a string, not as HTML.
Same thing with hundreds other string values which you print in templates: They must be HTML encoded, because they are just that: strings.


[originally posted on jforum.net by Anonymous]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If you want alert() -in registration form type (increase parameters width, or put via URL get parameters.

e-mail (exactly the string below)
"><SCRIPT LANGUAGE="javascript">alert("XXX");</SCRIPT>

other registration parameters arbitrary. You will get alert() message
[originally posted on jforum.net by Anonymous]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
By the way.
phpbb properly escapes output parameters on template level:
phpbb HTML output example:


<input type="text" class="post" style="width:200px" name="username" size="25" maxlength="25" value=""> FF <B> TST <" />
[originally posted on jforum.net by Anonymous]
 
Migrated From Jforum.net
Ranch Hand
Posts: 17424
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I can not post string in jforum. I would add a space after & to show phpbb output


<input type="text" class="post" style="width:200px" name="username" size="25" maxlength="25" value="& quot;& gt; FF & lt;B& gt; TST & lt;" />
[originally posted on jforum.net by Anonymous]
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!