The Assessment Package unites the work and needs of various members
of the OpenACS community for data collection functionality within the
OpenACS framework. We're using the term "Assessment" instead of
"Survey" or "Questionnaire" (or "Case Report Form" a.k.a. CRF, the term
used in clinical trials) because it is a term used by IMS and because
it connotes the more generic nature of the data collection system we're
There has been considerable recent interest in expanding the
capabilities of generic data collection packages within OpenACS.
Identified applications include:
- Educational settings. The dotLRN project has updated the
Simple-Survey package to the Survey
package now in the current distribution. A number of groups in the
OpenACS community are interested in adding capabilities defined in the IMS Global Learning
Consortium's specs for Question
and Test Interoperability and Simple Sequencing.
- Clinical research settings. The
has created an enhanced version of the Simple-Survey package that adds
a variety of scoring and scheduling tools for use in health-related
quality-of-life assessments. This Questionnaire package has not
been ported to OpenACS 4.x yet, however, and it also lacks a wide
variety of other features that are necessary for use in formal clinical
trial data collection applications, certainly for those that intend to
create data sets acceptable for new drug applications to the US Food
and Drug Administration and equivalent European regulatory agencies.
Of note, there are large and well-funded vendors of clinical
trials data management systems. Phase Forward,
Outcome Sciences, and PHT Corporation
among others. A standards body called CDISC
(Clinical Data Interchange Standards Consortium) formed a few years ago
and is developing data models for clinical trials data derived from
schema contributed primarily by Phase Forward and PHT. These vendors
provide "electronic data capture" (EDC) services at considerable cost
-- a 18 month study of 2500 patients including about 500 data elements
costs nearly $500,000. There is clearly interest and opportunity to
craft systems that bring such costs "in house" for organizations doing
Data collection services for other OpenACS packages. Most other
OpenACS packages invoke some form of data collection from users. While
developments such as ad_form
and the templating system in OpenACS 4.x ease the construction of data
collection forms, it may be possible to expose a focused data
collection package via acs_service_contract mechanisms to other
packages. In particular, incorporating Workflow and a new data
collection package would be key to creation of new vertical-application
tools like dotWRK. Such integration would also be immensely useful for
a clinical trials management toolkit.
Historical Considerations (Work Done So Far)
Several OpenACS efforts form the context for any future work. These
- Survey. This package (largely written/revised by Dave Bauer) doesn't currently have any
documentation in the documentation section of the OpenACS.org site,
but it is in any current OpenACS installation at /doc/survey/. Dave has
added internationalization capabilities (in the version of Survey in
CVS HEAD) and cleaned up the administrative UIs very nicely. This
package was thoroughly debugged prior to the 4.6.1 release. It supports
simple one-section surveys, though the data model has as-yet
unimplemented provisions for multiple sections within a survey.
- Exam. This package (written by Ernie Ghiglione and Malte Sussdorff) is currently an Oracle-only tool
with capabilities not much different from Survey.
- Surveys. This package was written a while ago by
Buddy Dennis, and the source code package has dropped from view.
However, we've posted it here. Presumably this package has been further
developed, since it appears to be in use at the iQ&A
site, though current source doesn't appear to be available there.
Surveys included several important enhancements to the data model:
- Conditional branching within a survey (though how well
worked out this is remains unclear)
- "Folder" based repositories of questions and sections
However, Surveys has some important limitations:
- Surveys are "published" as static HTML files which are
served out to users when they complete the survey
- The package doesn't use a templating system
Still, this package adopts some naming conventions consistent
the IMS spec and definitely represents the closest effort to a "complex
survey" done to date.
- "Complex Survey". This is the descendant of
"Survey" and Buddy's "Surveys" written by Malte Sussdorf. It currently
is in the /contrib branch of the OpenACS 5 distro and represents the
currently most advanced package for OpenACS 5+. If you want to start
looking at surveys in OpenACS right now, this is the package to get. It
incorporates a number of the features of Surveys. We discuss it in
greater detail here.
- Questionnaire. This is a 3.2.5 module developed by
at The Epimetrics Group in order to
support complex scoring of a particular type of clinical measure. (You
can see a demo of this here,
and if you register at the site and join the Bay Area OpenACS Users
Group, you can play with the intuitive administrative pages for
creating and editing questionnaires, defining scoring mechanisms,
setting up user scheduling and reminder features, and configuring
results reporting/graphing capabilities.) This module runs within
OpenACS 3.2.5, though, and will need a substantial rewrite to work
within the new 5.x infrastructure.
- Simple-survey. This package remains in the OpenACS distribution
but it is now obsolete, supplanted by Survey
The number of competing products in this area is *huge*. Starting with
the usual suspects Blackboard and WebCT you can go on to clinical trial
software like Oracle Clinical or specialised survey systems. When
writing the specifications we tried to incorporate as many ideas as
possible from the various systems we had a look at and use that
experience. A detailed analysis would be too much for the moment.
An overview of the functional requirements can be found here. It is highly encouraged to be read
first, as it contains the use cases along with a global overview of the
functionality contained within assessment. Additional requirements can
be found in the specific pages for the user interface.
The assessment system has been designed with a large flexibility and
reuse of existing functionality in mind. This might result in larger
complexity for simple uses (e.g. a plain poll system on it's own will
be more performant than running a poll through assessment), but
provides the chance to maintain one code base for all these separate
The API will be defined during the development phase.
The data model is described in detail in the design descriptions.
The UI for Assessment divides into a number of primary functional
areas, with the entry page located here.
It is split up into multiple sections:
Authoring: all the pages involved in creating,
editing, and deleting the Assessments themselves
- Section Authoring:
all the pages involved in creating,
editing, and deleting the Sections themselves. Includes the page to
browse for items to include in sections
- Item Authoring and
Catalogue: all the pages involing the item creation and the item
- Assessment Delivery:
all the pages involved in
deploying a given Assessment to users for completion, processing those
results, etc; these are user pages
- Section on Tests:
Currently still split away, some notes on additional user interface for
test. Shall be integrated with the rest of the pages.
- Assessment Review: all the pages involved in select
data extracts and displaying them in whatever formats indicated; this
includes "grading" of an Assessment -- a special case of data review;
these are admin pages, though there also needs to be some access to
data displays for general users as well (eg for anonymous surveys etc).
Also, this is where mechanisms that return information to "client"
packages that embed an Assessment would run.
- Session Management: pages that set up the timing and
other "policies" of an Assessment. This area needs to interact with the
next one in some fashion, though exactly how this occurs needs to be
further thought through, depending on where the Site Management
- Triggers and Action Execution
The Page Flow
page is diagrammed below and should give a very rough and outdated
overview, but still good for getting an impression.
The specifications for the assessment system have been written by
Stan Kaufman and
with help from numerous people within and
outside the OpenACS community.