Index: openacs-4/packages/acs-templating/www/doc/no-quote-upgrade.adp =================================================================== RCS file: /usr/local/cvsroot/openacs-4/packages/acs-templating/www/doc/no-quote-upgrade.adp,v diff -u -r1.3.2.4 -r1.3.2.5 --- openacs-4/packages/acs-templating/www/doc/no-quote-upgrade.adp 22 Jun 2016 07:48:43 -0000 1.3.2.4 +++ openacs-4/packages/acs-templating/www/doc/no-quote-upgrade.adp 5 Jul 2016 12:14:22 -0000 1.3.2.5 @@ -30,37 +30,38 @@ This means that the only way your code can fail is if the new code quotes a variable which is not meant to be quoted. Which is where ;noquote - needs to be added. That's all porting effort that -is required. Actually, the variables are subject to HTML-quoting -and internationalization. The suffix ;noquote - means that -the variable's content will be internationalized, but not -HTML-quoted, while ;no18n - means quote, but don't -internationalize. Finally ;literal - means: don't quote and -don't internationalize. + needs to be added. That's all porting +effort that is required. Actually, the variables are subject to +HTML-quoting and internationalization. The suffix +;noquote + means that the variable's content will be +internationalized, but not HTML-quoted, while ;no18n + +means quote, but don't internationalize. Finally +;literal + means: don't quote and don't +internationalize.

This is not hard because most variables will not be affected by this change. Most variables either need to be quoted (those containing textual data that comes from the database or from the user) or are unaffected by quoting (numerical database IDs, etc.) -The variables where this behavior is undesired are those that -contain HTML which is expected to be included as part of the -page, and those that are already quoted by Tcl code. Such -variables should be protected from quoting by the ;noquote -modifier.

+The variables where this behavior is undesired are those +that contain HTML which is expected to be included as part +of the page, and those that are already quoted by +Tcl code. Such variables should be protected from quoting by the +;noquote modifier.

The Most Common Cases.

The most common cases where you need to add ;noquote to the variable name are easy to recognize and identify.

Hidden form variables.
-Also known as "hidden input fields", hidden form variables are form -fields with pre-defined values which are not shown to the user. -These days they are used for transferring internal state across -several form pages. In HTML, hidden form variables look like -this:

+Also known as "hidden input fields", hidden form +variables are form fields with pre-defined values which are not +shown to the user. These days they are used for transferring +internal state across several form pages. In HTML, hidden form +variables look like this:

 <form>
   <input name=var1 value="value1">
@@ -111,15 +112,15 @@
 widgets
 .
Normally we try to fit all HTML code into the ADP template and have -the Tcl code handle the "logic" of the program. And yet, sometimes -pieces of relatively convoluted HTML need to be included in many -templates. In such cases, it makes sense to generate the -widget programmatically and include it into the template as -a variable. A typical widget is a date entry widget which provides -the user the input and selection boxes for year, month, and day, -all of which default to the current date.

-

Another example of widgets is the context bar often found -on top of ACS pgages.

+the Tcl code handle the "logic" of the program. And yet, +sometimes pieces of relatively convoluted HTML need to be included +in many templates. In such cases, it makes sense to generate the +widget programmatically and include it into the template +as a variable. A typical widget is a date entry widget which +provides the user the input and selection boxes for year, month, +and day, all of which default to the current date.

+

Another example of widgets is the context bar often +found on top of ACS pgages.

Obviously, all widgets should be treated as HTML and therefore adorned with the ;noquote qualifier. This also assumes that the routines that build the widget are correctly @@ -137,12 +138,12 @@ Transfer of parameters between included ADPs often requires manual addition of ;noquote -. Let's review why. -

The property tag is used to pass a piece of information -to the master template. This is used by the ADP whose writer -consciously chose to let the master template handle a variable -given by the Tcl code. Typically page titles, headings, and context -bars are handled this way. For example:

+. Let's review why. +

The property tag is used to pass a piece of +information to the master template. This is used by the ADP whose +writer consciously chose to let the master template handle a +variable given by the Tcl code. Typically page titles, headings, +and context bars are handled this way. For example:

master:
 <head>
@@ -163,30 +164,30 @@
 
The obvious intention of the master is to allow its slave templates -to provide a "title" and a "heading" of the page in a standardized -fashion. The obvious intention of our slave template is to allow -its corresponding Tcl code to set a single variable, -title -, which will be used for both title and heading. -What's wrong with this code? +to provide a "title" and a "heading" of the +page in a standardized fashion. The obvious intention of our slave +template is to allow its corresponding Tcl code to set a single +variable, title +, which will be used for both title and +heading. What's wrong with this code?

The problem is that title gets quoted twice, once by the slave template, and once by the master template. This is the result of how the templating system works: every occurrence of \@variable\@ is converted to -[ad_quotehtml $variable], even when it is -used only to set a property and you would expect the quoting to be -suppressed.

+[ad_quotehtml $variable], even when it +is used only to set a property and you would expect the quoting to +be suppressed.

Implementation note: Ideally, the templating system should avoid this pitfall by quoting the variable (or not) only once, at the point where the value is passed from the Tcl code to the templating system. However, no such point in time exists because what in fact happens is that the template gets -compiled into code that simply takes what it needs from the -environment and then does the quoting. Properties are passed -to the master so that all the property variables are shoved into an -environment; by the time the master template is executed, all -information on which variable came from where and whether it might -have already been quoted is lost.
+compiled into code that simply takes what it needs from +the environment and then does the quoting. Properties are +passed to the master so that all the property variables are shoved +into an environment; by the time the master template is executed, +all information on which variable came from where and whether it +might have already been quoted is lost.

This occurrence is often referred to as over-quoting. Over-quoting is sometimes hard to detect because things seem to work fine in most cases. To notice the problem in the example above @@ -195,16 +196,17 @@ &. If it does, they will appear quoted to the user instead of appearing as-is.

Over-quoting is resolved by adding ;noquote to one of -the variables. We strongly recommend that you add ;literal -inside the property tag rather than in the master. The -reason is that, first, it makes sense to do so because conceptually -the master is the one that "shows" the variable, so it makes sense -that it gets to quote it. Secondly, a property tag is -supposed to merely transfer a piece of text to the master; -it is much cleaner and more maintainable if this transfer is -defined to be non-lossy. This becomes important in practice when -there is a hierarchy of master templates -- e.g. one for -the package and one for the whole site.

+the variables. We strongly recommend that you add +;literal inside the property tag rather than +in the master. The reason is that, first, it makes sense to do so +because conceptually the master is the one that "shows" +the variable, so it makes sense that it gets to quote it. Secondly, +a property tag is supposed to merely transfer a +piece of text to the master; it is much cleaner and more +maintainable if this transfer is defined to be non-lossy. This +becomes important in practice when there is a hierarchy of +master templates -- e.g. one for the package and one for +the whole site.

To reiterate, a bug-free version of the slave template looks like this:

@@ -265,15 +267,15 @@ property tag.
  • Add ;noquote to textual variables whose values are attributes to the include tag.
  • Audit the template for occurrences of <%= [ns_quotehtml \@variable\@] => -and replace them with \@variable\@.
  • Audit the Tcl code for occurrences of ns_quotehtml. If -it is used to build an HTML component, leave it, but take note of -the variable the result gets saved to. Otherwise, remove the -quoting.
  • Add ;noquote to the "HTML component" variables noted -in the previous step.
  • +and replace them with \@variable\@.
  • Audit the Tcl code for occurrences of ns_quotehtml. +If it is used to build an HTML component, leave it, but take note +of the variable the result gets saved to. Otherwise, remove the +quoting.
  • Add ;noquote to the "HTML component" +variables noted in the previous step.
  • -After that, test that the template behaves as it should, and you're -done. +After that, test that the template behaves as it should, and +you're done.

    Testing.

    Fortunately, most of the problems with automatic quoting are very @@ -287,31 +289,33 @@
  • HTML junk appearing in the page.
    Literal HTML visible to the user typically comes from an -"export_form_vars" or a widget variable that lacks -;noquote. To fix the problem, simply add ;noquote -to the variable.
  • +"export_form_vars" or a widget variable that +lacks ;noquote. To fix the problem, simply add +;noquote to the variable.
  • Over-quoting and under-quoting.
    To detect quoting defects, you need to assume an active role in naming your objects. The best way to do it is to create objects (bboard forums, messages, news items, etc.) with names that contain -the representation of an entity, e.g. "&copy;". This -looks like the copyright SGML entity, and intentionally so. The -testing consists of checking that the browser prints exactly what -you typed in as the name. Thus if your forum/message/etc. is listed -as "&copy;", everything is OK. If it is listed as -"&amp;copy;", it means that the string was quoted -twice, i.e. over-quoted. Finally, if the entity is interpreted -(shown as ©), it means that the string lacks quoting, i.e. it -is under-quoted. -

    To get rid of over-quoting, make sure that the variables don't -get quoted in transport, such as in the property -tag or as an attribute of the include tag. Also, make sure -that your Tcl code is not quoting the variable name.

    To get rid of under-quoting, make sure that your variable gets +the representation of an entity, e.g. +"&copy;". This looks like the copyright +SGML entity, and intentionally so. The testing consists of checking +that the browser prints exactly what you typed in as the name. Thus +if your forum/message/etc. is listed as +"&copy;", everything is OK. If it is +listed as "&amp;copy;", it means that the +string was quoted twice, i.e. over-quoted. Finally, if the entity +is interpreted (shown as ©), it means that the string lacks +quoting, i.e. it is under-quoted. +

    To get rid of over-quoting, make sure that the variables +don't get quoted in transport, such as in the +property tag or as an attribute of the +include tag. Also, make sure that your Tcl code is not +quoting the variable name.

    To get rid of under-quoting, make sure that your variable gets quoted exactly once. This can be achieved either by removing a -(presumably overzealous) ;noquote or by quoting the string -from Tcl. The latter is necessary when building HTML components, -such as a context bar, from strings that come from the database or -from the user.

    +(presumably overzealous) ;noquote or by quoting the +string from Tcl. The latter is necessary when building HTML +components, such as a context bar, from strings that come from the +database or from the user.