Index: openacs-4/packages/acs-automated-testing/www/doc/xml/requirements.xml =================================================================== RCS file: /usr/local/cvsroot/openacs-4/packages/acs-automated-testing/www/doc/xml/requirements.xml,v diff -u -r1.4 -r1.5 --- openacs-4/packages/acs-automated-testing/www/doc/xml/requirements.xml 29 Jan 2004 12:57:18 -0000 1.4 +++ openacs-4/packages/acs-automated-testing/www/doc/xml/requirements.xml 16 Feb 2004 16:47:28 -0000 1.5 @@ -36,13 +36,13 @@ 1.1 partial - A + Done Execute HTTP tests. Execute tests that can interact with a the webserver via the external, HTTP interface, including retrieving pages, following links, and submitting forms. (This is partially done in the sense that we can make http calls from tcl api, but there is no framework for doing anything complicated.) 1.1.1 - A + Done Execute tclwebtest scripts. A test can contain tclwebtest commands. If tclwebtest is not installed, those commands fail gracefully. @@ -54,34 +54,11 @@ 2 - - A - Tests have categories. Individual tests can be marked as belonging to zero, one, or many of these categories. The UI provides for running only tests in selected categories, and for viewing only results of tests in selected categories. -Testing Mode - Regression - Smoke - Stress - Default-Only (for tests, such as front page UI tests, that will break - once the default site is modified and can be ignored on - non-default sites) -Layer - Web Page - TCL Page Contract - TCL API - SQL -Severity (derives from ns_log values) - Notice (use this for informational items that do not imply a problem) - Warning (use this for submitted tests w/o fixes; hygiene tests such as deprecated function sweeps) - Error (default severity) - Test Validity Bug (use this for problems that suggest the test itself in invalid or broken) -Test Data - Self-contained Requires no test data, leaves no test data. - Populate Generates and leaves test data, for other tests or for end users. -Package Version - 5.0.0 - etc - - + Done + Done + Tests have categories. Individual tests can be marked as belonging to zero, one, or many of these categories. The UI provides for running only tests in selected categories, and for viewing only results of tests in selected categories. + + 2.1 @@ -129,7 +106,38 @@ - + + Because the current test package uses in-memory variables instead of database objects to track its tests, it is incompatible with the standard category package. It uses an internal, single-dimension category field. Should this eventually get extended, a more complete list of categories to implement could be: +Testing Mode + Regression + Smoke + Stress + Default-Only (for tests, such as front page UI tests, that will break + once the default site is modified and can be ignored on + non-default sites) + production-safe + security_risk +Layer + Web Page + TCL Page Contract + TCL API + SQL +Severity (derives from ns_log values) + Notice (use this for informational items that do not imply a problem) + Warning (use this for submitted tests w/o fixes; hygiene tests such as deprecated function sweeps) + Error (default severity) + Test Validity Bug (use this for problems that suggest the test itself in invalid or broken) +Test Data + Self-contained Requires no test data, leaves no test data. + Populate Generates and leaves test data, for other tests or for end users. +Package Version + 5.0.0 + etc + + + + + References