| Prev | Chapter 1. High level information: What is OpenACS? | Next |
|---|
The release of OpenACS 5.10.0 contains the 93 packages of the oacs-5-10 branch. These packages include the OpenACS core packages, the major application packages (e.g. most the ones used on OpenACS.org), and DotLRN 2.10.0.
Functional improvements
Features:
Support for range types in .xql files:
PostgreSQL supports range types since 9.5. When using range types, square braces have to be used in SQL statements. Since OpenACS uses always Tcl substitution in .xql files, and OpenACS does NOT allow backslash substitution in these files, square brackets could not be escaped and therefore not be used in .xql files so far. This change allows now a developer to deactivate the substitution by passing e.g. -subst none to the db_* command using the .xql file. Valid values for -subst are all, none, vars, and commands, default is all which is exactly the behavior of previous releases. Therefore, this change is fully backward compatible.
Registry for .js and .css libraries: allow besides classical URLs symbolic names for loading external resources (e.g. jquery), this makes it easier to upgrade libraries in multiple packages (without running into problems with duplicate versions) and supports switching between CDN and local pathsURN. The existing implementation is based on URNs and extends the existing template-head API to support registration for URNs. A URN provides an abstraction and a single place for e.g. updating references to external resources when switching between a CDN and a locally stored resource, or when a resource should be updated. Instead of adding e.g. a CDN URL via template::head::add_script, one can add an URN and control its content from a single place. Use common namespaces for OpenACS such as urn:ad:css:* and urn:ad:js:*.
Register URNs:
Example provider (e.g. in some theme):
template::register_urn \
-urn urn:ad:js:jquery \
-resource /resources/xowiki/jquery/jquery.min.js
The registered URN can be used like classical URL after registration.
Example consumer:
template::head::add_javascript -src urn:ad:js:jquery
Declare composite files: Provide an interface to define that a .js file or a .css file contains multiple other .js/.css files in order to reduce the number of requests.
template::head::includes -container urn:js::style.js -parts {urn:ad:js:jquery ...}
Improved API browser: Visualization for code dependencies (which procs calls what, from where is a proc being called) and test-coverage
Warn site administrators about expiring certificates
Added text/markdown to the accepted text formats or rich-text widget
Additional input types (and widgets) for ad_form:
checkbox_text
color
tel
url
number
file (multiple)
h5date and h5time: date and time fields using native HTML5 visualization and input normalization
Added additional page_contract filter: oneof(red|green|blue)
template::add_event_listener and template::add_confirm_handler now can target elements by CSS selector
Improved support for streaming HTML: The new API function template::collect_body_scripts can be used to get the content of template::script or CSP calls (template::add_body_script, template::add_event_listener, template::add_body_handler, template::add_script) when streaming HTML (incremental HTML) is used. Before, these call could bot be used for streaming HTML.
Reforms:
Login:
Get rid of bugging "login page expired" messages. The 17 years old construct was replaced by newer means to avoid caching of form values from the login form. Admins of existing sites should set the kernel parameter LoginPageExpirationTime to 0
Forums:
Removed hard-coded dependency with registered_users group when checking forum permissions
Don't rely so heavily on acs_permissions to model forum configuration, as this can have unexpected consequences in convoluted multi-group/multi-subsite scenarios. Prefer simpler table attributes instead
New style of attachments to the forums, allowing multiple attachments to a single message directly from the message post page, using the multiple file input widget. Retain compatibility with old style attachments, using the new 'AttachmentStyle' package instance parameter. Currently, this supports two values: 'simple' (new behavior) and 'complex' previous behavior.
Chat:
Revamping of the GUI
Responsiveness
Full screen mode
Skins support (minimal, classic and bubbles, included): Skins are located in the new /packages/xowiki/www/resources/chat-skins/ directory. New skins can be created by just adding the css and js files in the skins directory, and naming them accordingly (chat-$SKIN_NAME.{js|css}).
Avatars (can be enabled per room)
Number of active users in chat
Tab notifications of new messages
Web Notifications:
https://www.w3.org/TR/notifications/
https://developer.mozilla.org/en-US/docs/Web/API/Notifications_API/Using_the_Notifications_API
acs-lang:
admin pages:
Added the option to unregister (delete permanently the message key from all locales) a message key that has been already marked as deleted. Useful for cleaning up old message keys.
Added the option to undelete, using the new ::message::undelete proc.
Made number and category (untranslated/deleted/...) of messages coherent in all pages.
Added the columns 'total' and 'deleted' to the index page.
object_id reference: it is now possible to associate a message key to an object_id in a way that e.g. when the object is deleted, so is the message key. This addresses cases such as the message keys generated by group creation or by the new XoWiki localized fields
Notifications:
Improved scalability for notifications: One of the most expensive operations in large site is the cleanup for notification_requests in situations, where the user has lost permissions on an object, on which the user wanted to receive notifications. This check was performed previously in notification::sweep::cleanup_notifications via a permission check over all notification requests, which can be very costly on large sites. This change moves this cleanup into the actual notification sending, where the permissions have to be sent anyhow.
When sending a notification on behalf of a person, if the system is not configured to process replies to notification, do not set the reply-to address to anything different than the sender
Notifications: proper cleanup of acs_objects resulting from the deletion of dynamic notification requests
User/Person/Party API: rework and rationalize caching of all party, person and user API, create separate caches for each of these types, make the API and return dicts. acs_user::get will not fail anymore with non-existing user.
User Portrait: created API to retrieve and create, store and delete the user's portrait. Also address leftover child relationships from the past and delete them properly.
Non-functional Changes
Improved automated regression test infrastructure and test coverage
All packages in the oacs-5-10 branch pass regression test
Web testing was separated from non-maintained tcltest and was built on the standard OpenACS infrastructure
Include web testing per default in standard regression testing
Introduced new test authentication authority, allowing to run many user administration tests outside the context of a "real authority": in cases where the real authority depends on external services to proof identity, (e.g. Kerberos), those tests would just fail.
Introduce the display of warnings in the UI of automated testing
Added test coverage information in the automated testing pages, using the new proc-coverage API and providing test coverage information for packages and system wide.
Increased overall coverage of public API
New tests checking various data-model properties and smells
Improved scalability:
Provided lock-free implementation of ad_page_contract_filters and ad_page_contract_filter_rules. This change improves parallel processing of requests and is primarily interesting for sites with a few mio page views per days. These locks were among the most frequent nsv locks
Reduced locks on util_memoize_cache my more invariants values into per-thread caching (acs_lookup_magic_object, ad_acs_version, .... ) and by avoiding specialized calls, which can be realized by already optimized ones (apm_package_installed_p_not_cached ref-timezones was most frequently used util_memoize_cache entry). These changes are necessary to avoid full still-stand on the unfortunate long-time locks on util_memoize_cache stemming from permission and user management with wild-card flush operations, which require to iterate over all cache entries (which might be on a busy server several hundred thousands)
Added new interface for cache partitioning to reduce lock latencies on high load websites
Added new interface for lock-free per-thread and per-request caching to avoid scattered ad-hoc implementations
Better reuse of DB handles (reduced expiring/reopen/etc.), faster access to handles
Improved startup time:
When the package acs-automated-testing is disabled, startup time is reduced by avoiding loading of support functions and tests; the size of the blueprint is reduced
xowf: loading of at-jobs is significantly improved.
Security improvements:
Strengthened page contracts
CSP support for application packages
CSP fine tuning
Better exception handling based on Tcl 8.6 exception handlers (try and throw, also available in Tcl 8.5)
Provided a new ad_try implementation based on Tcl's try replaces now the old ad_try, with_catch and with_finally, which are marked as deprecated
The new ad_try is in essence Tcl's try but with predefined handling of ad_script_abort and should be also used instead of catch, when the OpenACS API is used (which might use script aborts)
All core packages use the new ad_try instead of the deprecated versions.
Connection close reform:
NaviServer/AOLserver continue after connection closing commands to execute a script. This is in many situations not desired, especially, when for the page as well a .adp file exists, which will try to deliver this on the already closed connection. This can lead to errors in the error.log file, which are sometimes hard to analyze
Due to this cleanup, developers should use in most such cases cases ad_script_abort
Connection closing commands are e.g. ad_returnredirect, ad_redirect_for_registration, cr_write_content, ad_page_contract_handle_datasource_error, ad_return_string_as_file, ad_return_complaint, ad_return_error, ad_return_forbidden, ad_return_warning, ad_return_exception_page, ns_returnredirect, ns_return, ns_returnerror
The new version has made on most occasions explicit, when the script should abort.
API changes (new and extended API calls):
New API call category::get to obtain category description for a category_id and locale
New utility ad_pad emulating both lpad and rpad typically available in DBMSs
New proc lc_content_size_pretty, prettify data size given in bytes. It supports three different standards (SI base-10, IEC base-2 and the old JEDEC base-2), default is SI base-10.
New flag -export for ad_form: this flag uses export_vars under the hood and supports all of this API's features (e.g. :multiple, :sign, :array). This addresses a long standing TODO
util::pdfinfo: simple poppler-utils wrapper to extract pdf information
util::http: leverage new ns_http features such as request file spooling. Native implementation will now be used only on NaviServer >= 4.99.15.
Database API:
db_foreach: queries executed inside of a db_foreach will not be issued using a different handle and will therefore be safe to use in a transaction
db_list_of_lists: new -with_headers flag, which will make the first element of the returned list to be the column names as defined in the query
Groups API:
Logics to delete a group type have now been included in the API
Allow to filter group members by member_state in the API
Deprecated commands:
Many deprecated API calls were included in the code (and sometimes still in use) sometimes more than 10 years after these calls have been deprecated. In case a site modification still uses deprecated code, the user is warned about this. The OpenACS 5.10 code base does not depend on deprecated code.
Move deprecated code into separate files
Made loading of deprecated code optional (can be controlled via parameter "WithDeprecatedCode" in section "ns_section ns/server/${server}/acs" of the config file. By default, deprecated procs are still loaded
When deprecated code is not loaded, the blueprint of the interpreter is smaller. The following number of lines of code can be omitted when loading without the deprecated procs:
acs-tcl: 3178
acs-templating: 450
xotcl-core http-client-procs: 830
acs-content-repository: 1717 (including .xql files)
Bugfix and Code Maintenance:
Made sure all party emails are stored as lowercase through the API
Fixed long standing regression in template::list: we were looping through the list "elements", rather than the "display_elements". This prevents specifying different sets of columns to be returned depending on the -formats and -selected_format options in template::list::create.
acs-content-repository: New HEIC and HEIF mimetypes
acs-mail-lite: handle to_addr specified as "DisplayName <email>" without errors
Fixed invalidating of all existing user logins, (aka) "Logout from everywhere" feature, useful e.g. to make sure no device still holds a valid login when we change our password on a device
Don't lose the return URL when one tries to join a subsite before being logged in
Added doc(base_href) and doc(base_target) for setting <base> element via blank-baster (see issue #3435)
Groups:
When a new group is created, flush all the group::get_id caches with the same name so that the new group can be fetched correctly in case it replaces a previously deleted one
Cleanup message keys coming from groups in acs-translations when a group is deleted
acs-lang:
lang::util::convert_to_i18n: do not always register a en_US translation, which would be always overridden. Instead, let lang::message::register make sure that a en_US message exists and create one only as a fallback.
lc_time_fmt: leverage Tcl clock to address shortcomings such as handling of dates in Julian/Gregorian calendar and impossible dates such as 1999-02-29, implement missing formats, support previously undocumented formats explicitly
search: make sure objects in the search indexer queue still exist by the time they are swept by the indexer (e.g. items deleted before the indexer could sweep them)
attribute::delete: fix proc so it leverages stored procedure capability of dropping the database table as well
util::http: fix UTF-8 encoding issues for some cornercases
Localization: Complete Italian and Spanish localization for the whole .LRN set of packages (including themes). Message keys for new and previously localized packages have also been updated
General cleanup/maintenance
Improved handling of server implementation-specific code: server-specific code can be optionally loaded via specifying the server family in the filename. Provided *-procs-aolserver.tcl and *-procs-naviserver.tcl similar to *.postgresql.xql and *.oracle.xql where appropriate
Modernization of Tcl idioms.
Compliance of files, proc names, ... to the naming conventions.
White space cleanup, indentation changes.
Improvement of public API documentation
Adjustment of proc protection levels (public, private)
Adjustment of log severity
Cleanup of obsolete files
Replacement of handcrafted forms by ad_form
Typo fixing
Editor hints
Replacement of deprecated calls
Addition of missing contracts
...
SQL cleanup:
Cleanup of obsolete nonportable SQL constructs in a way Oracle and PostgreSQL code base divergency is reduced:
"nvl" -> "coalesce"
"sysdate" / "now()" -> standard "current_date" or "current_timestamp"
Use standard-compliant "dual" table where appropriate (required by Oracle, supported by PostgreSQL)
Use non-dialectal cast idioms when appropriate
Adopt CTE idioms in Oracle codebase as well (e.g. connect -> with recursive)
... (reference Oracle version will be 11gr2 as is oldest version officially supported by Oracle (See here and here)
Reduced superfluous .xql queries
acs-subsite: delete 21 files with un-referenced .xql queries
acs-tcl: delete 4 files
news: 3 files
file-storage: 1 file
dotlrn: 9 files
New Packages:
cookie-consent: alerting users about the use of cookies on a website
boomerang: performance of your website from your end user’s point of view
xooauth: OAuth implementation, including LTI (Learning Tools Interoperability)
dotlrn-bootstrap3-theme: Bootstrap 3 theme for DotLRN
xowf-monaco-plugin: Integration of Monaco editor with for code exercise types in xowf
proctoring-support: utilities and user interfaces to implement proctoring of the user session, mainly intended in the context of distance education and online exams. The main proctoring feature relies only on web technologies and does not require any plugin or additional software. Optional support for the Safe Exam Browser has also been introduced. The package is currently at the core of WU Online Exam infrastructure and is integrated in the inclass exam implementation for xowf
Require Tcl 8.6, XOTcl 2.1, PostgreSQL 9.6 (PostgreSQL 9.5 EOL: February 2021), tdom 0.9
Altogether, OpenACS 5.10.0 differs from OpenACS 5.9.1 by the following statistics
3445 files changed, 215464 insertions(+), 193642 deletions(-)
contributed by 7 committers (Antonio Pisano, Gustaf Neumann, Günter Ernst, Hector Romojaro, Michael Aram, Stefan Sobernig, Thomas Renner) and additional 13 patch/bugfix providers (Felix Mödritscher, Florian Mosböck, Frank Bergmann, Franz Penz, Hanifa Hasan, Keith Paskett, Markus Moser, Maurizio Martignano, Monika Andergassen, Nathan Coulter, Rainer Bachleitner, Stephan Adelsberger, Tony Kirkham). All packages of the release were tested with PostgreSQL 13.* and Tcl 8.6.*.
For more details, consult the raw ChangeLog.
The release of OpenACS 5.9.1 contains the 88 packages of the oacs-5-9 branch. These packages include the OpenACS core packages, the major application packages (e.g. most the ones used on OpenACS.org), and DotLRN 2.9.1.
Summary of changes:
Refactoring of rich-text editor integration
Driving force: Debian packaging (e.g. js minified code is not allowed)
Moved out code from acs-templating, provided interfaces to add many different richtext editors as separate packages
New OpenACS packages:
richtext-xinha
richtext-tinymce
richtext-ckeditor4 (has ability to choose between CDN and local installation via web interface)
Improving admin interface
New theme manager:
Goals:
Make it easier to keep track of themes with local modifications
Make it easier to create local modification a new themes and to update these
Show differences between default theme parameter (in DB) and actual settings (in subsite parameters)
Allow to delete unused themes
Give site admin hints, which theme is used at which subsite
Ease theme switching
Added a subsite::theme_changed callback to be able to handle theme changes in custom themes (was also necessary for proper integration with DotLRN theming)
Added support for these features under subsite admin (/admin/)
Improved support for themed templates via [template::themed_template]
Improved (broken) interface to define/manage groups over web interface
Allow to send as well mail, when membership was rejected
New functions [membership_rel::get_user_id], [membership_rel::get] and [membership_rel::get_group_id] to avoid code duplication
Added support to let user include %forgotten_password_url% in self-registration emails (e.g. in message key acs-subsite.email_body_Registration_password)
Improved subsite/www/members
Make it possible to manage members of arbitrary groups
Improved performance for large groups
Improved configurability: when ShowMembersListTo is set to "3", show list to members only, when this is not the whole subsite
Improved user interface for /admin/applications for large number of applications
Various fixes for sitewide-admin pages (under /acs-admin)
Update blueprint in "install from repository" (currently just working in NaviServer)
SQL
Further cleanup of .xql files (like what as done for acs-subsite in OpenACS 5.9.0):
36 files deleted
Removed more than 100 obsolete named queries
Stripped misleading SQL statements
Marked redundant / uncalled SQL functions as deprecated
Replaced usages of obsolete view "all_object_party_privilege_map" by "acs_object_party_privilege_map"
Removed type discrepancy introduced in 2002:
acs_object_types.object_type has type varchar(1000), while
acs_object_types.supertype has type varchar(100)
... several more data types are involved, using acs_object_types.object_type as foreign key
Simplified core SQL functions by using defaults:
Number of functions reduced by a factor of 2 compared to OpenACS 5.9.0 (while providing compatibility for clients using old versions),
Reduced code redundancy
Affected functions:
Reduced content_item__new from 12 versions to 6,
Reduce content_revision__new from 7 to 4
Similar in image__new, image__new_revision, content_item__copy, content_item__get_title, content_item__move
PostgreSQL 9.5 supports named parameter in the same syntax as in Oracle. Further reduction of variants will be possible, once OpenACS requires at least PostgreSQL 9.5
Reduced usage of deprecated versions of SQL functions (mostly content repository calls)
Reduced generation of dead tuples by combining multiple DML statements to one (reduces costs of checkpoint cleanups in PostgreSQL)
Permission queries:
Improved performance
Support PACKAGE.FUNCTION notation for PostgreSQL to allow calls permission queries exactly the same way as in Oracle (e.g. "acs_permission.permission_p()"). This helps to reduce the number of postgres specific .xql files.
Modernize SQL:
Use real Boolean types instead of character(1) (done for new-portal, forums, faq, attachments, categories, dotlrn, dotlrn-forums, evaluation)
Use real enumeration types rather than check constraints (done for storage_type text/file/lob)
CR hygienics (reduce cr bloat)
Provided means to avoid insert/update/delete operations in the search queue:
OpenACS adds for every new revision often multiple entries to the search_queue, without providing any means to prevent this. This requires for busy sites very short intervals between queue sweeps (otherwise too many entries pile up). Another consequence is that this behavior keeps the PostgreSQL auto-vacuum daemons permanently active. Many of these operations are useless in cases where the content repository is used for content that should not be provided via search. The changed behavior should honors a publish-date set to the future, since it will not add any content with future publish dates to the search-queue.
Reduced number of insert cr_child_rels operations, just when needed:
cr_child_rels provide only little benefit (allow one to use roles in a child-rel), but the common operation is a well available in cr_items via the parent_id. cr_child_rels do not help for recursive queries either. One option would be to add an additional argument for content_item__new to omit child-rel creation (default is old behavior) and adapt the other cases.
Security improvements
Added support against CSRF (cross site request forgery)
OpenACS maintains a per-request CSRF token that ensures that form replies are coming just from sites that received the form
CSRF support is optional for packages where CSRF is less dangerous, and such requests are wanted (e.g. search and API-browser)
Added Support for W3C "Upgrade-Insecure-Headers" (see https://www.w3.org/TR/upgrade-insecure-requests/):
For standard compliant upgrade for requests from HTTP to HTTPS
Added support for W3C "Subresource Integrity" (SRI; see https://www.w3.org/TR/SRI/)
Added support for W3C "Content Security Policy" (CSP; see https://www.w3.org/TR/CSP/)
Removed "javascript:*" links (all such URLs are removed from the 90 packages in oacs-5-9, excluding js libraries (ajaxhelper) and richtext code)
Removed "onclick", "onfocus", "onblur", "onchange" handlers from all .adp and .tcl files in the 90 packages in oacs-5-9 (excluding js libraries (ajaxhelper) and richtext code)
Added optional nonces to all <script> elements with literal JavaScript content
Removed "generic downloader", which allowed to download arbitrary content items, when item_id was known (bug-fix)
Improved protection against XSS and SQL-injection (strengthen page contracts, add validators, added page_contract_filter "localurl", improve HTML escaping, and URI encoding)
Fixed for potential traversal attack (acs-api-documentation-procs)
Improvements for "host-node mapped" subsites
Fixed links from host-node mapped subsite pages to swa-functions (must be always on main subsite)
Made "util_current_directory" aware of host-node-mapped subsites
Added ability to pass "-cookie_domain" to make it possible to use the same cookie for different domains
Fixed result of affected commands "util_current_location", "ad_return_url", "ad_get_login_url" and "ad_get_logout_url" for HTTP and HTTPS, when UseHostnameDomainforReg is 0 or 1.
Improved UI for host-node maps when a large number of site nodes exists
Reform of acs-rels
Made acs-rels configurable to give the developer the option to specify, whether these are composable or not (default fully backward compatible). This is required to control transitivity in rel-segments
The code changes are based on a patch provided by Michael Steigmann. For details, see:
Improved status code handlers for AJAX scenarios
Don't report data source errors with status code 200 (use 422 instead)
Let "permission::require_permission" return forbidden (403) in AJAX calls (determined via [ad_conn ajaxp])
Improved Internationalization
Extended language catalogs for
Russian (thanks to v v)
Italian (thanks to Antonio Pisano)
Spanish (thanks to Hector Romojaro)
German (thanks to Markus Moser)
Added (missing) message keys
Improved wording of entries
Added message keys for member_state changes, provide API via group::get_member_state_pretty
Improved online documentation (/doc)
Fixed many broken links
Removed fully obsolete sections
Improved markup (modernize HTML)
Updated various sections
Misc code improvements:
18 issues from the OpenACS-bug-tracker fixed
Made code more robust against invalid/incorrect input (page_contracts, validators, values obtained from header fields such as Accept-Language)
Fixed quoting of message keys on many places
Improved exception handling (often, a "catch" swallows one too much, e.g. script_aborts), introducing "ad_exception".
Generalized handling of leading zeros:
Fixed cases where leading zeros could lead to unwanted octal interpretations
Switch to use of " util::trim_leading_zeros" instead of "template::util::leadingTrim", "dt_trim_leading_zeros" and "template::util::leadingTrim", marked the latter as deprecated
URL encoding
"ad_urlencode_folder_path": new function to perform an urlencode operation on the segments of the provided folder path
"export_vars": encode path always correctly, except -no_base_encode is specified
Fixed encoding of the URL path in "ad_returnredirect"
Improvements for "ad_conn":
Added [ad_conn behind_proxy_p] to check, whether the request is coming from behind a proxy server
Added [ad_conn behind_secure_proxy_p] to check, whether the request is coming from behind a secure proxy server
Added [ad_conn ajax_p] to check, whether the request is an AJAX requests (assumption: AJAX request sets header-field Requested-With: XMLHttpRequest")
Added [ad_conn vhost_url] to obtain the url of host-node-mapped subsites
Added various missing upgrade scripts (missing since many years) of changes that were implemented for new installs to reduce differences between "new"-and "old" (upgraded) installations
Templating
Get rid of various pesky "MISSING FORMWIDGET: ...formbutton:ok" messages
Improved support for JavaScript event handlers in template::head
New functions "template::add_event_listener" and "template::add_confirm_handler"
Fix handling, when "page_size_variable_p" is set (was broken since ages)
Improved location and URL handling:
Refactored and commented "util_current_location" to address security issues, handle IPv6 addresses, IP literal notation, multiple drivers, "
Improved "security::get_secure_location" (align with documentation)
New functions:
"util::configured_location"
"util::join_location", "util::split_location"
for working on HTTP locations to reduce scattered regexps handling URL components
Improved IPv6 support
Use native "ns_parseurl" when available, provide backward compatible version for AOLserver
MIME types:
Added more Open XML formats for MS-Office to allowed content types
Modernized entries to IANA recommendations
New function "cr_check_mime_type" centralizing the retrieval of the mime_type from uploaded content
Finalized cleanup of permissions (started in OpenACS 5.9.0):
Get rid of "acs_object_context_index " (and therefore on "acs_object_party_privilege_map " as well) on PostgreSQL.
Reasons:
huge table,
expensive maintenance, used only in a few places,
Misc new functions:
"lang::util::message_key_regexp": factor out scattered regexp for detecting message keys
"ns_md5" and "ns_parseurl": improve compatibility between AOLserver and NaviServer
"ad_dom_sanitize_html": allow one to specify different sets of tags, attributes and protocols and "ad_dom_fix_html", which is a light weight tidy variant.
Improved HTML rendering (acs-api-browser), provide width and height to speed up rendering
Improved ADP files (e.g. missing doc(title))
Added usage of "ad_include_contract" on more occasions
Modernize Tcl and HTML coding
Reduced dependency on external programs (use Tcl functions instead)
Improved robustness of "file delete" operations all over the code
Improved documentation, fix demo pages
Aligned usages of log notification levels (distinction between "error", "warning" and "notice") with coding-standards
Cleaned up deprecated calls:
Removed usage of deprecated API functions (e.g. "cc_lookup_email_user", "cc_email_from_party", "util_unlist", ...)
Moved more deprecated procs to acs-outdated
Marked remaining (and unused) "cc_*" functions as well as deprecated.
Improved Oracle and windows support
Fixed common spelling errors and standardize spelling of product names all over the code (comments, documentation, ...)
Many more small bug fixes
Packages:
New Package Parameters
acs-kernel:
MaxUrlLength: remove hard-coded constant in request processor for max accepted url paths
SecureSessionCookie: Let site admin determine, whether or not to use secured session cookies (useful, when not all requests are over HTTPS)
CSPEnabledP: activate/deactivate CSP
acs-kernel (recommended to be set via config file in section "ns/server/${server}/>acs"
NsShutdownWithNonZeroExitCode: tell NaviServer to return with a nonzero return code to cause restart (important under windows)
LogIncludeUserId: include user_id in access log
acs-api-browser:
ValidateCSRFP: make checking of CSRF optional (default 1)
acs-content-repository:
AllowMimeTypeCreationP: Decides whether we allow unknown mime types to be automatically registered (default: 0}
news-portlet:
display_item_lead_p: Should we display news leads in the portlet? (default 0)
search:
ValidateCSRFP: make checking of CSRF optional (default 1)
xotcl-request-monitor:
do_track_activity: turn activity monitoring on or off (default 0)
New OpenACS packages:
richtext-xinha
richtext-tinymce
richtext-ckeditor4 (has ability to choose between CDN and local installation via GUI)
openacs-bootstrap3-theme (as used on openacs.org)
dotlrn-bootstrap3-theme
xotcl-core:
Improved XOTcl 2.0 and NX support (e.g. api-browser)
Added "-debug", "-deprecated" to ad_* defined methods (such as e.g. "ad_instproc")
Make use of explicit "create" statements when creating XOTcl/NX objects (makes it easier to grab intentions and to detect typos)
Added parameter to "get_instance_from_db" to specify, whether the loaded objects should be initialized
Added support for PostgreSQL prepared statements of SQL interface in ::xo::dc (nsdb driver)
xowiki:
Named all web-callable methods www-NAME (to make it clear, what is called, what has to be checked especially carefully)
Moved templates from www into xowiki/resources to avoid naming conflicts
Improved ckeditor support
Added usage of prepared statements for common queries
Improved error handling
Better value checking for query parameter, error reporting via ad_return_complaint
Added option "-path_encode" to methods "pretty_link" and "folder_path" to allow one to control, whether the result should be encoded or not (default true)
Form fields:
Improved repeatable form fields (esp. composite cases), don't require preallocation (can be costly in composite cases)
Added signing of form-fields
Added HTML5 attributes such as "multiple" (for "file") or "autocomplete"
Fixed generation of "orderby" attribute based on form-field names
richtext: allow one to specify "extraAllowedContent" via options
Improved layout of horizontal check boxes
Menu bar:
Added dropzone (requires bootstrap): drag and drop file upload
Added mode toggle (requires bootstrap)
Extended default policies for handling e.g. dropzone (file-upload method)
Distinguish between "startpage" (menu.Package.Startpage) and "table of contents" (menu.Package.Toc)
Notifications:
Added support for better tailorable notifications: introduced method "notification_render" (similar to "search_render")
Added support for tailorable subject lines (method "notification_subject")
Improved bootstrap support, use "bootstrap" as PreferredCSSToolkit
Switched to ckeditor4 as PreferredRichtextEditor
Improved handling of script-abort from within the payload of ::xowiki::Object payloads
Added parameter to "get_all_children" to specify, whether the child objects should be initialized
xowf:
Added property "payload" to "WorkflowConstruct" in order to simplify customized workflow "allocate" actions
Internationalized more menu buttons
xotcl-request-monitor
Added class "BanUser" (use. e.g. ip address to disallow requests from a user via request monitor)
Added support for optional user tracking in database
Added support for monitoring response-time for certain URLs via munin
Increased usage of XOTcl 2.0 variable resolver (potentially speed improvement 4x)
Performed some refactoring of response-time handling to allow site-admin to make e.g. use of NaviServer's dynamic connection pool management (not included in CVS)
Added support for partial times in long-calls.tcl to ease interpretation of unexpected slow calls
last100.tcl: Don't report hrefs to URLs, except to SWAs
chat:
Introduced new options to set chat rooms so login and/or logout message are not issued every time a user enters/exits a chat-room (important for chats with huge number of participants)
Parameterized viewing of chat-logs
Fixed cases of over-/under-quoting
Fixed JavaScript for IE, where innerHTML can cause problems
file-storage:
Don't show action keys, when user has no permissions
Added support for copying of same-named files into a folder (adding suffix)
Fixed old bugs in connection with "views" package
Altogether, OpenACS 5.9.1 differs from OpenACS 5.9.1 by the following statistics
3548 files changed, 113292 insertions(+), 90507 deletions(-)
contributed by 5 committers (Michael Aram, Gustaf Neumann, Antonio Pisano, Hector Romojaro, Thomas Renner) and 8 patch/bugfix providers (Frank Bergmann, Günter Ernst, Brian Fenton, Felix Mödritscher, Marcus Moser, Franz Penz, Stefan Sobernig, Michael Steigman). All packages of the release were tested with PostgreSQL 9.6.* and Tcl 8.5.*.
For more details, consult the raw ChangeLog.
The release of OpenACS 5.9.0 contains the 78 packages of the oacs-5-9 branch. These packages include the OpenACS core packages, the major application packages (e.g. most the ones used on OpenACS.org), and DotLRN 2.9.0.
Summary of changes:
SQL:
Improved performance of acs-object deletion.
Replaced many manual referential integrity calls by built-in handing in PostgreSQL.
Removed various manual bookkeeping and deletion operations in the content repository by built-in handing in PostgreSQL.
Removed tree_sortkey on acs-objects to reduce its size and to speedup operations, where the context-id is changed (could take on large installation several minutes in earlier versions)
Removed several uncalled / redundant SQL statements and functions.
Cleanup of .xql files in acs-subsite:
Some cleanup of .xql files: removed misleading sql-statements from db_* calls, which were ignored due .xql files
Removed bug where same query-name was used in different branches of an if-statement for different SQL statements, but the query-name lead to the wrong result.
Removed multiple entries of same query name from .xql files (e.g. the entry "package_create_attribute_list.select_type_info" was 7 (!) times in a single .xql file)
Web Interface:
Improve Performance of WebSites created with OpenACS: e.g. move core.js to a body requests, provide kernel parameter ResourcesExpireInterval to specify expiration times for resources.
Much better protection against XSS attacks.
Improved HTML validity (especially for admin pages)
Improved admin interface:
Placed all installation options to a single page.
Added pagination to /admin/applications (was unusable for large sites)
New admin pages for subsites linked from site-wide-admin package (/acs-admin).
Added explanatory text to several admin pages.
Add lightweight support for ckeditor4 for templating::richtext widget (configurable via package parameter "RichTextEditor" of acs-templating. ckeditor4 supports mobile devices (such as iPad, ...)
Templating:
Improved theme-ability: Moved more information into theme packages in order to create responsive designs, reduce hard-coding of paths, HTML etc.
Improved include-handling: All includes are now theme-able, interfaces of includes can be defined with "ad_include_contract" (similar to ad_page_contract).
Improved them-ability for display_templates. One can now provide a display_template_name (similar to the SQL statement name) to refer to display templates. This enables reusability and is theme-able.
Dimensional slider reform (ad_dimensional): Removed hard-coded table layout from dimensional slider. Add backwards compatible templates Move hard-coded styles into theme styling
Notification chunks are now theme-able as well (using ad_include_contract)
Complete template variable suffixes (adding noi18n, addressing bug #2692, full list is now: noquote, noi18n, literal)
Added timeout and configurable secrets for signed url parameters to export_vars/page_contracts. This can be used to secure sensitive operations such as granting permissions since a link can be set to timeout after e.g. 60 seconds; after that, the link is invalid. A secret (password) can be set in section ns/server/$server/acs parameter "parametersecret". For example, one can use now "user_id:sign(max_age=60)" in export_vars to let the exported variable expire after 60 seconds.
Misc:
Added ability to show ns_log statements of current request to developer support output when developer support is activated (controlled via package parameter "TclTraceLogServerities" in the acs-tcl package parameters)
Added ability to save data sent by ns_return in files on the filesystem. This can be used to validate HTML content also for password protected pages (controlled via package parameter "TclTraceSaveNsReturn" in the acs-tcl package parameters)
New API function "ad_log" having the same interface as ns_log, but which logs the calling information (like URL and call-stack) to ease tracking of errors.
Use per-thread caching to reduce number of mutex lock operations and lock contention on various caches (util-memoize, xo_site_nodes, xotcl_object_types) and nsvs (e.g. ds_properties)
Improved templating of OpenACS core documentation
Improved Russian Internationalization
Make pretty-names of acs-core packages more consistent
Mark unused functions of acs-tcl/tcl/table-display-procs.tcl as deprecated
Many more bug fixes (from bug tracker and extra) and performance improvements.
Version numbers:
Require PostgreSQL 9.0 (End Of Life of PostgreSQL 8.4 was July 2014)
Require XOTcl 2.0 (presented at the Tcl conference in 2011).
Changes in application packages:
Various bug fixes and improvements for e.g. file-storage, forums, news, notifications, xowiki.
Altogether, OpenACS 5.9.0 differs from OpenACS 5.8.1 by the following statistics
3658 files changed, 120800 insertions(+), 97617 deletions(-)
contributed by 4 committers (Michael Aram, Victor Guerra, Gustaf Neumann, Antonio Pisano) and patch/bugfix providers (Frank Bergmann, Andrew Helsley, Felix Mödritscher, Markus Moser, Franz Penz, Thomas Renner). These are significantly more changes as the differences in the last releases. All packages of the release were tested with PostgreSQL 9.4.* and Tcl 8.5.*.
For more details, consult the raw ChangeLog.
The release contains the 78 packages of the oacs-5-8 branch. These packages contain the OpenACS core packages, major application packages (e.g. most the ones used on OpenACS.org), and DotLRN.
All packages have the following properties:
SQL:
All packages are PostgreSQL 9.1+ compatible (tested with PostgreSQL 9.3)
All SQL files with stored procedures use the recommended $$ quoting
All SQL-functions have regular function arguments instead of the old-style aliases
The function_args() (query-able meta-data) are completed and fixed
Incompatible functions (e.g. for sequences) are replaced.
Tcl:
All packages were brought up Tcl 8.5, including the actual Tcl idioms where appropriate (e.g. using the safer expand operator, range indices, dict, lassign, etc.)
The code was updated to prefer byte-compiled functions instead of legacy functions from ancient Tcl versions.
The code works with NaviServer and AOLserver.
API:
All packages are free from calls to deprecated code (157 functions are marked as deprecated and will be moved into an "outdated" package in the 5.9 or 6.0 release)
General overhaul of package management
Install-from-local and install-from-repository can be used to install the provided packages based on a acs-core installation. This means that also DotLRN can be installed from repository or from local into an existing OpenACS instance.
Install-from-repository offers filtering functions, allows to install optionally from head-channel (for packages not in the base channel of the installed instance). Install-from-repository works more like an app-store, showing as well vendor information
Packages can be equipped with xml-based configuration files (e.g. changing parameters for style packages)
Package developers can upload .apm packages via workflow for review by core members and for inclusion to the repository. The option is integrated with package management, the link is offered for local packages. We hope to attract additional vendors (universities, companies) to make their packages available on this path.
New management-functions for package instances (list, create, delete package instances)
Substantially improved API browser:
Show just relevant parts of .xql files for a function
Provide syntax-highlighting for www scripts as well
Handle more special cases like e.g. util_memoize
Provide links to Tcl functions depending on the installed Tcl version
Provide links to NaviServer or OpenACS functions depending on installed version
Syntax highlighter uses CSS rather than hard-coded markup
Significant performance improvement for large installations
Altogether, OpenACS 5.8.1 differs from OpenACS 5.8.0 in about 100,000 modifications (6145 commits) contributed by 5 committers.
Compatibility with PostgreSQL 9.2: The new version installs without any need for special parameter settings in new PostgreSQL versions. This makes it easier to use e.g. shared or packaged PostgreSQL installations.
Compatibility with NaviServer 4.99.5 or newer
Performance and scalability improvements
Various bug fixes
Altogether, OpenACS 5.8.0 differs from OpenACS 5.7.0 in more than 18.000 modifications (781 commits) contributed by 7 committers.
Made changes that extend acs-kernel's create_type and create_attribute procs, so they're optionally able to create SQL tables and columns. Optional metadata params allow for the automatic generation of foreign key references, check exprs, etc.
Added new package dependency type, "embeds". This is a variant of the "extends" package dependency type added in OpenACS 5.5.0. It allows one to write embeddable packages, with scripts made visible in client packages using URLs which include the embedded package's package key. An example embeddable package might be a rewritten "attachments" package. The current implementation requires a global instance be mounted, and client packages generate URLs to that global instance. Among other things, this leads to the user navigating to the top-level subsite, losing any subsite theming that might be associated with a community. Using "embeds", a rewritten package would run in the client package's context, maintaining theming and automatically associating attachments with the client package.
Added global package parameters - parameters can now have scope "local" or "global", with "local" being the default..
Fixes for ns_proxy handling
Significant speedup for large sites
Optional support for selenium remote control (acs-automated-tests)
New administration UI to manage mime types and extension map
Added acs-mail-lite package params for rollout support
Support for 3-chars language codes in acs-lang
Added OOXML mime types in acs-content-repository
PostgreSQL 8.3 is now fully supported, including the use of the built-in standard version of tsearch2.
TinyMCE has been upgraded to 3.2.4.1 with language pack support.
acs-mail-lite now correctly implements rollout support.
Added new package dependency type, "extends". Implements a weak form of package inheritance (parameters and, optionally, templates). Multiple inheritance is supported. For instance, the non-core "layout-managed-subsite" extends the "acs-subsite" and "layout-manager" packages, resulting in a package that combines the semantics of both.
Added new package attribute "implements-subsite-p" (default "f"). If true, this package may be mounted as a subsite and is expected to implement subsite semantics. Typically used by packages which extend acs-subsite.
Added new package attribute "inherit-templates-p" (default "t"). If true, the package inherits templates defined in the packages it extends. This means that the package only needs to specify templates where the UI of an extended package is modified or extended. This greatly reduces the need to fork base packages when one needs to customize it. Rather than modify the package directly, use "extends" rather than "requires" then rewrite those templates you need to customize.
Added a simple mechanism for defining subsite themes, removing the hard-wired choices implemented in earlier versions of OpenACS. The default theme has been moved into a new package, "openacs-default-theme". Simplifies the customization of the look and feel of OpenACS sites and subsites.
The install xml facility has been enhanced to allow the calling of arbitrary Tcl procedures and includes various other enhancements written by Xarg. Packages can extend the facility, too. As an example of what can be done, the configuration of .LRN communities could be moved from a set of interacting parameters to a cleaner XML description of how to build classes and clubs, etc.
Notifications now calls lang::util::localize on the message subject and body before sending the message out, using the recipient locale if set, the site-wide one if not. This will cause message keys (entered as <span>#</span>....# strings) to be replaced with the language text for the chosen locale.
This is a minor bugfix release.
Site node caching was removed as doesn't work correctly
Critical issues with search on oracle were fixed
More html strict work etc
New Templating API added to add scripts, css, etc to the HTML HEAD and BODY sections of the generated HTML document. Please see /packages/acs-templating/tcl/head-procs.tcl or visit the template::head procs in the API browser for details.
Templates have been modified to comply with HTML strict
The Search package's results page has been improved
TinyMCE WYSIWYG support has been added, RTE and HTMLArea support dropped
acs-mail-lite's send has been cleaned up to properly encode content, to handle file attachments, etc. "complex-send" will disappear from acs-core in a future release.
The ChangeLogs include an annotated list of changes (???) since the last release and in the entire 5.9 release sequence ???.
Bug fixes.
New TIPs implemented.
All Core Automated Tests for Postgres pass.
New Site and Blank master templates and CSS compatible with the .LRN Zen work. Compatibility master templates are provided for existing sites.
The ChangeLogs include an annotated list of changes (???) since the last release and in the entire 5.9 release sequence ???.
Bug fixes.
The missing CR Tcl API has been filled in, thanks to Rocael and his team and Dave Bauer.
This release does not include new translations.
Bug fixes, primarily for .LRN compatibility in support of upcoming .LRN 2.1.0 releases. This release does not include new translations since 5.1.2.
Translations synchronized with the translation server. Basque and Catalan added.
For a complete change list, see the Change list since 5.1.0 in ???.
This is the first release using the newest adjustment to the versioning convention. The OpenACS 5.1.1 tag will apply to OpenACS core as well as to the most recent released version of every package, including .LRN.
Translations synchronized with the translation server.
Bug 1519 fixed. This involved renaming all catalog files for ch_ZH, TH_TH, AR_EG, AR_LB, ms_my, RO_RO, FA_IR, and HR_HR. If you work with any of those locales, you should do a full catalog export and then import (via /acs-lang/admin) after upgrading acs-lang. (And, of course, make a backup of both the files and database before upgrading.)
Other bug fixes since 5.1.0: 1785, 1793, and over a dozen additional bug fixes.
For a complete change list, see the Change list since 5.0.0 in ???.
Lots of little tweaks and fixes
Complete Change list since 5.0.0 in Changelog
Bug fixes: #1495. Croatian enabled by default, #1496. APM automated install fails if files have spaces in their names, #1494. automated upgrade crashes (halting the upgrade process)
Complete Change list since 5.0.0 in Changelog
File tagging scheme in CVS changed to follow TIP #46: (Approved) Rules for Version Numbering and CVS tagging of Packages
All work on the translation server from 7 Nov 2003 to 7 Feb 2004 is now included in catalogs.
One new function in acs-tcl, util::age_pretty
Complete Change list since 5.0.0 in Changelog
Many documentation updates and doc bug fixes
This is OpenACS 5.0.0. This version contains no known security, data loss, or crashing bugs, nor any bugs judged release blockers. This version has received manual testing. It has passed current automated testing, which is not comprehensive. This release contains work done on the translation server http://translate.openacs.org through 7 Nov 2003.
Please report bugs using our Bug Tracker at the OpenACS website.
You may want to begin by reading our installation documentation for the section called “a Unix-like system”. Note that the Windows documentation is not current for OpenACS 5.9.0, but an alternative is to use John Sequeira's Oasis VM project.
After installation, the full documentation set can be found by visiting
http://yourserver/doc.
New features in this release:
Internationalization support. A message catalog to store translated text, localization of dates, number formatting, timezone conversion, etc. Allows you to serve your users in their language.
External authentication. Integrate with outside user databases through e.g. LDAP, RADIUS, Kerberos, MS Active Directory. Imports user information through IMS Enterprise 1.1 format. Easily extended to support other authentication, password management, account creation, and account import mechanisms. This includes improvements to the basic cookie handling, so logins can be expired without the user's identity being completely lost. You can set login to expire after a certain period (e.g. 8 hours, then password must be refreshed), or you can have all issues login cookies expired at once, e.g. if you have left a permanent login cookie on a public machine somewhere.
User interface enhancements. All pages, including site-wide and subsite admin pages, will be templated, so they can be styled using master template and site-wide stylesheets. We have a new default-master template, which includes links to administration, your workspace, and login/logout, and is rendered using CSS. And there's a new community template (/packages/acs-subsite/www/group-master), which provides useful navigation to the applications and administrative UI in a subsite. In addition, there's new, simpler UI for managing members of a subsite, instantiating and mounting applications, setting permissions, parameters, etc. Site-wide admin as also seen the addition of a new simpler software install UI to replace the APM for non-developer users, and improved access to parameters, internationalization, automated testing, service contracts, etc. The list builder has been added for easily generating templated tables and lists, with features such as filtering, sorting, actions on multiple rows with checkboxes, etc. Most of all, it's fast to use, and results in consistently-looking, consistently-behaving, templated tables.
Automated testing. The automated testing framework has been improved significantly, and there are automated tests for a number of packages.
Security enhancements. HTML quoting now happens in the templating system, greatly minimizing the chance that users can sneak malicious HTML into the pages of other users.
Oracle 9i support.
Who's online feature.
Spell checking.
Potential incompatibilities:
With the release of OpenACS 5, PostgreSQL 7.2 is no longer supported. Upgrades are supported from OpenACS 4.6.3 under Oracle or PostgreSQL 7.3.
The undocumented special handling of ~ and +variable+ in
formtemplates, found in packages/acs-templating/resources/*,
has been removed in favor of using <noparse> and
\@variable\@ (the standard templating mechanisms). Locally
provided formtemplate styles still using these mechanisms
will break.
Serving backup files and files from the CVS directories is turned off by default via the acs-kernel parameter ExcludedFiles in section request-processor (The variable provides a string match glob list of files and is defaulted to "*/CVS/* *~")