+
+
+Introduction
The Assessment Package unites the work and needs of various
+members of the OpenACS community for data collection functionality
+within the OpenACS framework. We're using the term "Assessment"
+instead of "Survey" or "Questionnaire" (or "Case Report Form" aka
+CRF, the term used in clinical trials) because it is a term used by
+IMS and because it connotes the more generic nature of the data
+collection system we're focusing on.
There has been considerable recent interest in expanding the
+capabilities of generic data collection packages within OpenACS.
+Identified applications include:
+- Educational settings. The dotLRN project has updated the
+Simple-Survey package to the Survey package now in the current
+distribution. A number of groups in the OpenACS community are
+interested in adding capabilities defined in the IMS Global Learning
+Consortium's specs for Question and Test Interoperability and Simple Sequencing.
- Clinical research settings. The Epimetrics Group has created
+an enhanced version of the Simple-Survey package that adds a
+variety of scoring and scheduling tools for use in health-related
+quality-of-life assessments. This Questionnaire package has not
+been ported to OpenACS 4.x yet, however, and it also lacks a wide
+variety of other features that are necessary for use in formal
+clinical trial data collection applications, certainly for those
+that intend to create data sets acceptible for new drug
+applications to the US Food and Drug Administration and equivalent
+European regulatory agencies.
+
Of note, there are large and well-funded vendors of clinical
+trials data management systems. Phase Forward,
+Outcome Sciences, and PHT Corporation among
+others. A standards body called CDISC (Clinical Data
+Interchange Standards Consortium) formed a few years ago and is
+developing data models for clinical trials data derived from schema
+contributed primarily by Phase Forward and PHT. These vendors
+provide "electronic data capture" (EDC) services at considerable
+cost -- a 18 month study of 2500 patients including about 500 data
+elements costs nearly $500,000. There is clearly interest and
+opportunity to craft systems that bring such costs "in house" for
+organizations doing clinical research.
+ Data collection services for other OpenACS packages. Most other
+OpenACS packages invoke some form of data collection from users.
+While developments such as ad_form and the templating system in
+OpenACS 4.x ease the construction of data collection forms, it may
+be possible to expose a focused data collection package via
+acs_service_contract mechanisms to other packages. In particular,
+incorporating Workflow and a new data collection package would be
+key to creation of new vertical-application tools like dotWRK. Such
+integration would also be immensely useful for a clinical trials
+management toolkit.
Historical Considerations (Work Done So Far)
Several OpenACS efforts form the context for any future work.
+These include:
+- Survey. This package (largely written/revised by Dave Bauer) doesn't currently have any documentation
+in the documentation section of the OpenACS.org site, but it
+is in any current OpenACS installation at /doc/survey/. Dave has
+added internationalization capabilities (in the version of Survey
+in CVS HEAD) and cleaned up the administrative UIs very nicely.
+This package was thoroughly debugged prior to the 4.6.1 release. It
+supports simple one-section surveys, though the data model has
+as-yet unimplemented provisions for multiple sections within a
+survey.
- Exam. This package (written by Ernie Ghiglione and Malte Sussdorff) is currently an Oracle-only tool with
+capabilities not much different from Survey.
- Surveys. This package was written a while ago by Buddy Dennis,
+and the source code package has dropped from view. However, we've
+posted it here. Presumably this package has been further
+developed, since it appears to be in use at the iQ&A site,
+though current source doesn't appear to be available there. Surveys
+included several important enhancements to the data model:
+
+- Conditional branching within a survey (though how well worked
+out this is remains unclear)
- "Folder" based repositories of questions and sections
+
However, Surveys has some important limitations:
+- Surveys are "published" as static HTML files which are served
+out to users when they complete the survey
- The package doesn't use a templating system
- Oracle-only
+
Still, this package adopts some naming conventions consistent
+with the IMS spec and definitely represents the closest effort to a
+"complex survey" done to date.
+ - "Complex Survey". This is the descendant of "Survey" and
+Buddy's "Surveys" written by Malte Sussdorf. It currently is in the
+/contrib branch of the OpenACS 5 distro and represents the
+currently most advanced package for OpenACS 5+. If you want to
+start looking at surveys in OpenACS right now, this is the package
+to get. It incorporates a number of the features of Surveys. We
+discuss it in greater detail
+here.
- Questionnaire. This is a 3.2.5 module developed by Stan
+Kaufman at The Epimetrics
+Group in order to support complex scoring of a particular type
+of clinical measure. (You can see a demo of this here, and if you register at the site and join
+the Bay Area OpenACS Users Group, you can play with the intuitive
+administrative pages for creating and editing questionnaires,
+defining scoring mechanisms, setting up user scheduling and
+reminder features, and configuring results reporting/graphing
+capabilities.) This module runs within OpenACS 3.2.5, though, and
+will need a substantial rewrite to work within the new 5.x
+infrastructure.
- Simple-survey. This package remains in the OpenACS distribution
+but it is now obsolete, supplanted by Survey
+
Competitive Analysis
+The number of competing products in this area is *huge*. Starting
+with the usual suspects Blackboard and WebCT you can go on to
+clinical trial software like Oracle Clinical or specialised survey
+systems. When writing the specifications we tried to incorporate as
+many ideas as possible from the various systems we had a look at
+and use that experience. A detailed analysis would be too much for
+the moment.
Functional Requirements
+An overview of the functional requirements can be found here. It is highly encouraged to be read
+first, as it contains the use cases along with a global overview of
+the functionality contained within assessment. Additional
+requirements can be found in the specific pages for the user
+interface.
Design Tradeoffs
+The assessment system has been designed with a large flexibility
+and reuse of existing functionality in mind. This might result in
+larger complexity for simple uses (e.g. a plain poll system on it's
+own will be more performant than running a poll through
+assessment), but provides the chance to maintain one code base for
+all these seperate modules.
API
+The API will be defined during the development phase.
Data model
+The data model is described in detail in the design descriptions.
+The UI for Assessment divides into a number of primary functional
+areas, with the entry page located here. It is split up into multiple
+sections:
+-
+Assessment
+Authoring: all the pages involved in creating, editing, and
+deleting the Assessments themselves
-
+Section
+Authoring: all the pages involved in creating, editing, and
+deleting the Sections themselves. Includes the page to browse for
+items to include in sections
-
+Item Authoring and
+Catalogue: all the pages involing the item creation and the
+item catalogue.
-
+Assessment
+Delivery: all the pages involved in deploying a given
+Assessment to users for completion, processing those results, etc;
+these are user pages
-
+Section on Tests:
+Currently still split away, some notes on additional user interface
+for test. Shall be integrated with the rest of the pages.
- Assessment Review: all the pages involved in select data
+extracts and displaying them in whatever formats indicated; this
+includes "grading" of an Assessment -- a special case of data
+review; these are admin pages, though there also needs to be some
+access to data displays for general users as well (eg for anonymous
+surveys etc). Also, this is where mechanisms that return
+information to "client" packages that embed an Assessment would
+run.
- Session Management: pages that set up the timing and other
+"policies" of an Assessment. This area needs to interact with the
+next one in some fashion, though exactly how this occurs needs to
+be further thought through, depending on where the Site Management
+mechanisms reside.
- Triggers and Action
+Execution
+
+The Page Flow page is diagrammed
+below and should give a very rough and outdated overview, but still
+good for getting an impression.
Authors
+The specifications for the assessment system have been written by
+Stan
+Kaufman and Malte
+Sussdorff with help from numerous people within and outside the
+OpenACS community.
+