OpenACS &version; Package Manager Design
by Bryan QuinnEssentialsOpenACS Administrator directoryER diagramTcl API
apm-procs.tcl
apm-install-procs.tcl (Supports installation of packages)
20-apm-load-procs.tcl (Bootstraps APM for server startup)
apm-admin-procs.tcl (Supports APM UI)PL/SQL file apm-create.sqlIntroduction
In general terms, a package is a unit of software that
serves a single well-defined purpose. That purpose may be to provide a
service directly to one or more classes of end-user, (e.g., discussion forums
and file storage for community members, user profiling tools for the site
publisher), or it may be to act as a building block for other packages (e.g.,
an application programming interface (API) for storing and querying access
control rules, or an API for scheduling email alerts). Thus, packages fall
into one of two categories:
OpenACS Applications: a "program or group of programs
designed for end users" (the Webopedia
definition); also known as modules, for historical reasons.
Examples of applications include Bboard and News.
OpenACS Services: the aforementioned building blocks.
Examples of services include the OpenACS
Content Repository, the OpenACS Templating
System, and the OpenACS Kernel, which includes
APM.An installation of the OpenACS includes the OpenACS Kernel, some services that
extend the kernel's functionality, and some applications intended for
end-users. Packages function as individual pieces of subsites. A subsite can contain multiple
application and service instances that provide the end-user with capabilities
and content customized to the particular subsite.This architecture supports the growth of collaborative commerce. For
example, Jane User starts a forum focusing on the merits of View Cameras by
creating an instance of the Bboard application for her personal subsite on an
OpenACS Installation. Jack User discovers Jane's forum and includes a link to
it in his subsite. As interest in Jane's forum grows, she creates a
subsite specializing in providing information about View cameras. This
subsite now includes several package instances beyond Bboard; it could
potentially include its own Ecommerce capabilities (ala Yahoo! Shopping). This could include a
knowledge management application that allows users to spread expertise about
view cameras and a portal application that links to reliable camera models
and resellers. Any subsite enabled package that is added to the OpenACS
installation through APM is another potential package instance that can
become part of Jane's View Camera subsite.The APM provides an architecture for packaging software, making instances
of that software available to subsites, specifying configuration parameters
for each instance, and managing the creation and release of new packages.Historical Considerations
Prior to ACS 3.3, all packages were lumped together into one monolithic
distribution without explicit boundaries; the only way to ascertain what
comprised a given package was to look at the top of the corresponding
documentation page, where, by convention, the package developer would specify
where to find:
the data modelthe Tcl proceduresthe user-accessible pagesthe administration pagesExperience has shown us that this lack of explicit boundaries causes a
number of maintainability problems for pre-3.3 installations:Package interfaces were not guaranteed to be stable in any formal way, so
a change in the interface of one package would often break dependent packages
(which we would only discover through manual regression testing). In this
context, any of the following could constitute an interface change:
renaming a file or directory that appears in a URLchanging what form variables are expected as input by a pagechanging a procedural abstraction, e.g., a PL/SQL or Java stored
procedure or a Tcl procedurechanging a functional abstraction, e.g., a database view or a PL/SQL or
Java stored functionchanging the data modelThis last point is especially important. In most cases, changing the data
model should not affect dependent packages. Rather, the package
interface should provide a level of abstraction above the data model (as well
as the rest of the package implementation). Then, users of the package can
take advantage of implementation improvements that don't affect the
interface (e.g., faster performance from intelligent denormalization of the
data model), without having to worry that code outside the package will now
break.A typical ACS-backed site only uses a few of the modules included in the
distribution, yet there was no well-understood way to pick only what you
needed when installing the ACS, or even to uninstall what you didn't
need, post-installation. Unwanted code had to be removed manually.
Releasing a new version of the ACS was complicated, owing again to the
monolithic nature of the software. Since we released everything in the ACS
together, all threads of ACS development had to converge on a single
deadline, after which we would undertake a focused QA effort whose scale
increased in direct proportion to the expansion of the ACS codebase.
There was no standard way for developers outside of ArsDigita to extend
the ACS with their own packages. Along the same lines, ArsDigita programmers
working on client projects had no standard way to keep custom development
cleanly separated from ACS code. Consequently, upgrading an already installed
ACS was an error-prone and time-consuming process.Consistent use of the APM format and tools will go a long way toward
solving the maintainability problems listed above. Moreover, APM is the
substrate that will enable us to soon establish a central package repository,
where both ArsDigita and third-party developers will be able publish their
packages for other ACS users to download and install.For a simple illustration of the difference between ACS without APM
(pre-3.3) and ACS with APM (3.3 and beyond), consider a hypothetical ACS
installation that uses only two of the thirty-odd modules available circa ACS
3.2 (say, bboard and e-commerce):OpenACS without APM vs. with APMAPM itself is part of a package, the OpenACS Kernel, an OpenACS
service that is the only mandatory component of an OpenACS installation.Competitive AnalysisThe OpenACS is a platform for web-based application software, and any software
platform has the potential to develop problems like those described above.
Fortunately, there are many precedents for systematic solutions,
including:Debian GNU/Linux and the Debian
Packaging manualFreeBSD has the Ports
collectionRed Hat Linux has the Red Hat Package Manager (RPM)Borrowing from all of the above, OpenACS 3.3 introduces its own package
management system, the OpenACS Package Manager (APM), which consists of:a standard format for APM packages (also called
"OpenACS packages"), including: version numbering, independent of any other package and the OpenACS as a
wholespecification of the package interfacespecification of dependencies on other packages (if any)attribution (who wrote it) and ownership (who maintains it)web-based tools for package management:obtaining packages from a remote distribution pointinstalling packages, if and only if: all prerequisite packages are installedno conflicts will be created by the installationconfiguring packages (obsoleting the monolithic OpenACS configuration
file)upgrading packages, without clobbering local modificationsuninstalling unwanted packagesa registry of installed packages, database-backed and
integrated with filesystem-based version control
web-based tools for package development:creating new packages locallyreleasing new versions of locally-created packagesDesign Tradeoffs
The design chosen for APM was meant to satisfy the following constraints:
The process of authoring a package must be as simple as possible.Strict conventions must be established that provide a set of canonical
locations and names for files and patterns, for OpenACS application
development.The processes of installing, upgrading, and using packages must be
straightforward and accessible through a web-based UI.Package instances must be able to have subsite-specific content available
at an easily configurable URL.All of these requirements were met, but at the cost of development
simplicity. As demonstrates, a set of strict directory conventions are
required in order for a package to use APM. This contrasts with the apparent
simplicity available to developers of the OpenACS 3.3 system. However, while the
system has become more complex for developers to build packages, this
complexity is easily managed and is compensated for by additional
capabilities.For example, to make a new application available to the system, a
developer must:Create the necessary files to support the data model, Tcl API, and UI
pages.Put the files in the correct locations for APM to be aware of them.Use APM to create a new package and enable it.Use the Site Map facility to create an instance of the package, mount it
on an appropriate URL, and set parameters for that particular instance.While this is complex, especially to a new OpenACS developer, the
documentation walks the developer through each of these steps. Moreover, from
following these steps, the package can be subsite specific, available to
subsites across the system, and be available for distribution to other OpenACS
installations without doing a monolithic upgrade or reinstall.APIThe APM is composed of systems for accomplishing a set of package-related
tasks. Each of these tasks comprise a feature area that has an API, data
model, and a UI:Authoring a PackageMaintaining Multiple Versions of a PackageCreating Instances of the PackageSpecifying Configuration Parameters for each InstanceAuthoring a PackageFull instructions on how to prepare an OpenACS package are available in . The API here can be invoked manually by a package's data model
creation script, but need not to be used. This API is part of the APM PL/SQL
package.
-- Informs the APM that this application is available for use.
procedure register_application (
package_key in apm_package_types.package_key%TYPE,
pretty_name in apm_package_types.pretty_name%TYPE,
pretty_plural in apm_package_types.pretty_plural%TYPE,
package_uri in apm_package_types.package_uri%TYPE,
singleton_p in apm_package_types.singleton_p%TYPE
default 'f',
spec_file_path in apm_package_types.spec_file_path%TYPE
default null,
spec_file_mtime in apm_package_types.spec_file_mtime%TYPE
default null
);
The procedure above registers an OpenACS application in the APM. It creates a
new OpenACS object and stores information about the package, such as its name, in
the APM data model. There is an analogous procedure for OpenACS services, called
apm.register_service.To remove an application from the system, there are the calls
apm.unregister_application and
apm.unregister_service.
-- Remove the application from the system.
procedure unregister_application (
package_key in apm_package_types.package_key%TYPE,
-- Delete all objects associated with this application.
cascade_p in char default 'f'
);
Use the cascade_p only if you want to completely remove the
package from the OpenACS.In order to determine if a particular package exists in the system, use
the register_p predicate. It returns 1 if the specified
package_key exists in the system, 0 otherwise.
function register_p (
package_key in apm_package_types.package_key%TYPE
) return integer;
Maintaining Multiple Versions of a PackageWhile the package authoring API provides a means for registering a
package, some information about a package is version dependent. For example,
between versions, the owner of a package, its vendor, its URI, and its
dependency information may change. The API for package versions allows this
information to be specified. All of these APIs are part of the apm_package_version PL/SQL package.To create a new package version, use the
apm_package_version.new constructor function.
function new (
version_id in apm_package_versions.version_id%TYPE
default null,
package_key in apm_package_versions.package_key%TYPE,
version_name in apm_package_versions.version_name%TYPE
default null,
version_uri in apm_package_versions.version_uri%TYPE,
summary in apm_package_versions.summary%TYPE,
description_format in apm_package_versions.description_format%TYPE,
description in apm_package_versions.description%TYPE,
release_date in apm_package_versions.release_date%TYPE,
vendor in apm_package_versions.vendor%TYPE,
vendor_uri in apm_package_versions.vendor_uri%TYPE,
installed_p in apm_package_versions.installed_p%TYPE
default 'f',
data_model_loaded_p in apm_package_versions.data_model_loaded_p%TYPE
default 'f'
) return apm_package_versions.version_id%TYPE;
In order to use this function, an existing package_key must
be specified. The version_name parameter must follow a strict
convention:A major version numberat least one minor version number. Although any number of minor version
numbers may be included, three minor version numbers is sufficient and is the
convention of software developers.One of the following: The letter d, indicating a development-only versionThe letter a, indicating an alpha releaseThe letter b, indicating a beta releaseNo letter at all, indicating a final production releaseIn addition, the letters d, a, and
b may be followed by another integer, indicating a version
within the release.For those who like regular expressions:
version_number := ^[0-9]+((\.[0-9]+)+((d|a|b|)[0-9]?)?)$
So the following is a valid progression for version numbers:
To delete a given version of a package, use the
apm_package_version.delete procedure:
procedure delete (
package_id in apm_packages.package_id%TYPE
);
After creating a version, it is possible to edit the information
associated with it using apm_package_version.edit.
function edit (
new_version_id in apm_package_versions.version_id%TYPE
default null,
version_id in apm_package_versions.version_id%TYPE,
version_name in apm_package_versions.version_name%TYPE
default null,
version_uri in apm_package_versions.version_uri%TYPE,
summary in apm_package_versions.summary%TYPE,
description_format in apm_package_versions.description_format%TYPE,
description in apm_package_versions.description%TYPE,
release_date in apm_package_versions.release_date%TYPE,
vendor in apm_package_versions.vendor%TYPE,
vendor_uri in apm_package_versions.vendor_uri%TYPE,
installed_p in apm_package_versions.installed_p%TYPE
default 'f',
data_model_loaded_p in apm_package_versions.data_model_loaded_p%TYPE
default 'f'
) return apm_package_versions.version_id%TYPE;
Versions can be enabled or disabled. Enabling a version instructs APM to
source the package's libraries on startup and to make the package
available to the OpenACS.
procedure enable (
version_id in apm_package_versions.version_id%TYPE
);
procedure disable (
version_id in apm_package_versions.version_id%TYPE
);
Files associated with a version can be added and removed. The path is
relative to the package-root which is
acs-server-root/packages/package-key.
-- Add a file to the indicated version.
function add_file(
file_id in apm_package_files.file_id%TYPE
default null,
version_id in apm_package_versions.version_id%TYPE,
path in apm_package_files.path%TYPE,
file_type in apm_package_file_types.file_type_key%TYPE
) return apm_package_files.file_id%TYPE;
-- Remove a file from the indicated version.
procedure remove_file(
version_id in apm_package_versions.version_id%TYPE,
path in apm_package_files.path%TYPE
);
Package versions need to indicate that they provide interfaces for other
software. An interface is an API that other packages can access and utilize.
Interfaces are identified as a URI and a version name, that comply with the
specification of a version name for package URIs.
-- Add an interface provided by this version.
function add_interface(
interface_id in apm_package_dependencies.dependency_id%TYPE
default null,
version_id in apm_package_versions.version_id%TYPE,
interface_uri in apm_package_dependencies.service_uri%TYPE,
interface_version in apm_package_dependencies.service_version%TYPE
) return apm_package_dependencies.dependency_id%TYPE;
procedure remove_interface(
interface_id in apm_package_dependencies.dependency_id%TYPE,
version_id in apm_package_versions.version_id%TYPE
);
procedure remove_interface(
interface_uri in apm_package_dependencies.service_uri%TYPE,
interface_version in apm_package_dependencies.service_version%TYPE,
version_id in apm_package_versions.version_id%TYPE
);
The primary use of interfaces is for other packages to specify required
interfaces, known as dependencies. A package cannot be correctly installed
unless all of its dependencies have been satisfied.
-- Add a requirement for this version. A requirement is some interface that this
-- version depends on.
function add_dependency(
requirement_id in apm_package_dependencies.dependency_id%TYPE
default null,
version_id in apm_package_versions.version_id%TYPE,
requirement_uri in apm_package_dependencies.service_uri%TYPE,
requirement_version in apm_package_dependencies.service_version%TYPE
) return apm_package_dependencies.dependency_id%TYPE;
procedure remove_dependency(
requirement_id in apm_package_dependencies.dependency_id%TYPE,
version_id in apm_package_versions.version_id%TYPE
);
procedure remove_dependency(
requirement_uri in apm_package_dependencies.service_uri%TYPE,
requirement_version in apm_package_dependencies.service_version%TYPE,
version_id in apm_package_versions.version_id%TYPE
);
As new versions of packages are created, it is necessary to compare their
version names. These two functions assist in that task.
-- Given a version_name (e.g. 3.2a), return
-- something that can be lexicographically sorted.
function sortable_version_name (
version_name in apm_package_versions.version_name%TYPE
) return varchar;
-- Given two version names, return 1 if one > two, -1 if two > one, 0 otherwise.
-- Deprecate?
function compare(
version_name_one in apm_package_versions.version_name%TYPE,
version_name_two in apm_package_versions.version_name%TYPE
) return integer;
Creating Instances of a PackageOnce a package is registered in the system, it is possible to create
instances of it. Each instance can maintain its own content and
parameters.
create or replace package apm_application
as
function new (
application_id in acs_objects.object_id%TYPE default null,
instance_name in apm_packages.instance_name%TYPE
default null,
package_key in apm_package_types.package_key%TYPE,
object_type in acs_objects.object_type%TYPE
default 'apm_application',
creation_date in acs_objects.creation_date%TYPE default sysdate,
creation_user in acs_objects.creation_user%TYPE default null,
creation_ip in acs_objects.creation_ip%TYPE default null,
context_id in acs_objects.context_id%TYPE default null
) return acs_objects.object_id%TYPE;
procedure delete (
application_id in acs_objects.object_id%TYPE
);
end apm_application;
Just creating a package instance is not sufficient for it to be served
from the web server. A corresponding site node must be created for it. As an
example, here is how the OpenACS API Documentation service
makes itself available on the OpenACS main site:
declare
api_doc_id integer;
begin
api_doc_id := apm_service.new (
instance_name => 'OpenACS API Browser',
package_key => 'acs-api-browser',
context_id => main_site_id
);
apm_package.enable(api_doc_id);
api_doc_id := site_node.new (
parent_id => site_node.node_id('/'),
name => 'api-doc',
directory_p => 't',
pattern_p => 't',
object_id => api_doc_id
);
commit;
end;
/
show errors
Specifying Configuration Parameters for each InstanceA parameter is a setting that can be changed on a package instance basis.
Parameters are registered on each package_key, and the values
are associated with each instance. Parameters can have default values and can
be of type 'string' or 'number.' There is support with this
API for setting a number of minimum and maximum values for each parameter,
but for most instances, the minimum and maximum should be 1. It is useful to
allow or require multiple values for packages that need to store multiple
pieces of information under one parameter. Default values are automatically
set when instances are created, but can be changed for each instance.All of the functions below are in the APM PL/SQL
package.
-- Indicate to APM that a parameter is available to the system.
function register_parameter (
parameter_id in apm_parameters.parameter_id%TYPE
default null,
parameter_name in apm_parameters.parameter_name%TYPE,
description in apm_parameters.description%TYPE
default null,
package_key in apm_parameters.package_key%TYPE,
datatype in apm_parameters.datatype%TYPE
default 'string',
default_value in apm_parameters.default_value%TYPE
default null,
section_name in apm_parameters.section_name%TYPE
default null,
min_n_values in apm_parameters.min_n_values%TYPE
default 1,
max_n_values in apm_parameters.max_n_values%TYPE
default 1
) return apm_parameters.parameter_id%TYPE;
function update_parameter (
parameter_id in apm_parameters.parameter_id%TYPE,
parameter_name in apm_parameters.parameter_name%TYPE,
description in apm_parameters.description%TYPE
default null,
package_key in apm_parameters.package_key%TYPE,
datatype in apm_parameters.datatype%TYPE
default 'string',
default_value in apm_parameters.default_value%TYPE
default null,
section_name in apm_parameters.section_name%TYPE
default null,
min_n_values in apm_parameters.min_n_values%TYPE
default 1,
max_n_values in apm_parameters.max_n_values%TYPE
default 1
) return apm_parameters.parameter_name%TYPE;
-- Remove any uses of this parameter.
procedure unregister_parameter (
parameter_id in apm_parameters.parameter_id%TYPE
default null
);
The following functions are used to associate values with parameters and
instances:
-- Return the value of this parameter for a specific package and parameter.
function get_value (
parameter_id in apm_parameter_values.parameter_id%TYPE,
package_id in apm_packages.package_id%TYPE
) return apm_parameter_values.attr_value%TYPE;
function get_value (
package_id in apm_packages.package_id%TYPE,
parameter_name in apm_parameters.parameter_name%TYPE
) return apm_parameter_values.attr_value%TYPE;
-- Sets a value for a parameter for a package instance.
procedure set_value (
parameter_id in apm_parameter_values.parameter_id%TYPE,
package_id in apm_packages.package_id%TYPE,
attr_value in apm_parameter_values.attr_value%TYPE
);
procedure set_value (
package_id in apm_packages.package_id%TYPE,
parameter_name in apm_parameters.parameter_name%TYPE,
attr_value in apm_parameter_values.attr_value%TYPE
);
Data Model DiscussionThe central piece of the data model is the apm_package_types
table where each package is registered. When a new application or service is
installed on an OpenACS instance, a corresponding row in this table is inserted
with information about the type of package, e.g. if the bboard package is installed on your OpenACS server, a row
in apm_package_types will be created, noting that it's an
application package type.The apm_packages table is used to contain information about
the instances of packages currently created in the system. The
package_key column references the apm_package_types
table to ensure that no package instance can be created for a type that does
not exist.The apm_package_versions table contains information specific
to a particular version of a package. Several tables reference this one to
provide further information about the particular version:apm_package_owners
Stores information about the owners of a particular version of a package.
apm_package_files
Stores information about the files that are part of a version.
apm_package_dependencies
Stores information about what interfaces the package provides and
requires.Parameter information is maintained through two tables:apm_parameters
This table contains the definition of each of the parameters for a package.
apm_parameter_values
This table holds all of the values of parameters for specific package
instances.
A number of views are available for obtaining information about packages
registered in the APM.apm_package_version_info
Provides information about all of the versions in the system with
information available from the apm_package_types table.
apm_enabled_package_versions
A view (subset) of the above table with only enabled versions.
apm_file_info
Provides a public interface for querying file information.User InterfaceThe APM's user interface is part of the
OpenACS Administration Service. The UI is the primary
point of contact with APM by developers and administrators. It is part of OpenACS
Administration, because only the site-wide administrator should be able to
access it. Thus in order to develop a package, the developer must be granted
site-wide administration.Configuration/ParametersAPM has two parameters for configuring how it interacts with the UNIX
filesystem, accessible via the Site Map admin
page. These parameters need not be changed under most circumstances, but may
need to be tweaked for Windows compatibility.GzipExecutableDirectory
This directory points to where the gunzip program can be found
for uncompressing gzip archives. This is needed for the
installation of .apm files which are simply gziped
tarballs. Default is /usr/local/binInfoFilePermissionsMode
This sets the default UNIX permissions used when creating files using the
APM. Default is 775.Future Improvements/Areas of Likely ChangeAPM has been in production since OpenACS 3.3, and as of version 4.0 offers a
stable set of features. One major feature planned is integration with the OpenACS
Package Repository for automatic dependency satisfaction. When a user tries
to install a package that depends on other packages, the APM will contact the
package repository, determine what packages depend on it, and offer the user
a chance to download and install them all. This improvement offers value to
end users by facilitating the extension of their OpenACS systems.Architecturally, minor improvements to the data model and the
specification file are planned to increase modularity. The current
implementation puts all package specification information in a single file.
This approach has certain advantages, such as centralization, but splitting
this information into several files allows for flexible extensions to the APM
architecture over time.APM packages currently lack provisions to verify security information.
There are plans to add MD5 time stamps and PGP signatures to packages to
enable secure authentication of packages. These steps are necessary for APM
to be usable as a scalable method to distribute packages on multiple
repositories worldwide.Another anticipated change is to split the APM UI into separate systems
for authoring, maintaining, and installing packages. The current UI presents
all of this functionality in one interface and it can be confusing from a
usability perspective.AuthorsSystem creator: Bryan Quinn, Jon Salz, Michael Yoon, Lars Pind, Todd
Nightingale.System owner: Bryan QuinnDocumentation author: Bryan Quinn, building from earlier versions by Jon
Salz, Michael Yoon, and Lars Pind.Revision HistoryDocument Revision #Action Taken, NotesWhen?By Whom?0.1Creation9/25/2000Bryan Quinn0.8Ready for QA9/29/2000Bryan Quinn0.9Edited for ACS 4 Beta release10/02/2000Kai Wu1.0Edited for OpenACS &version; Beta release03/02/2002Roberto Mello