server-cluster-procs.tcl

  • last updated 1 hour ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Cluster reform

- Support for dynamic cluster nodes:

Previous versions of OpenACS required to know the IP addresses of

the cluster nodes in advance, which is a show-stopper for many cloud

applications.

The new functionality allows arbitrary nodes to register as client

nodes at the canonical server, provided that these know a shared

secret. All messages of the intra-cluster talk are now

cryptographically signed using this shared secret.

In the current version, the shared secret key has to be specified in

the NaviServer configuration file (ClusterSecret). Later versions

will support the use of other measures such as generated keys, kept

as files.

Also in the new version it is still possible to use static peer

addresses (which do not have to register during startup).

- Support for multiple protocols for intra-cluster talk

Previous version of OpenACS required the communication of the

intra-cluster talk over HTTP. With these changes, multiple protocols

are supported. Most prominently HTTPS can be used, which is required

by some organization for all web communication, especially for cloud

applications. Other protocols might be added later (e.g. UDP).

- Nodes (such as the canonical server) can be specified via URL

locations. These location specifier have the following advantages

over classical IP address and port.

* support different protocols (http, https, udp)

* support of IP addresses or DNS names

* when DNS names map to multiple IP addresses

(e.g. multiple IPv4 and/or multiple IPv6 addresses)

the new parameter PreferredLocationRegexp can be use

to select the wanted. When this regexp does not match

any of the addresses, the first address is used.

* ports are now optional (except UDP)

* for location specifier: https://openacs.org

- Support for cluster communication statistics

(requires xotcl-request-monitor)

These statistics show the amount of intra-server

communication. The following statistics are collected

per node:

- number of broadcast operations

- number of send operations (active flushes)

- number of receive operation (passive flushes)

For N cluster nodes, the send operations are

basically the number of broadcast operations times

the number of nodes minus one.

These statistics can also be used prior to cluster

enabling, to obtain the number of potential broadcast

operations.

Kernel Parameters

CanonicalServer : location of the canonical server

ClusterAuthorizedIP : obsolete

ClusterEnabledP : unchanged

ClusterPeerIP : obsolete

EnableLoggingP : unchanged

PreferredLocationRegexp : new

DynamicClusterPeers : new, no not edit

Differences between old and new value for "CanonicalServer":

old:

- specify IP address with port

- example: 137.208.116.31:443

new:

- specify URL location

- support different protocols (http, https, udp)

- support of IP address or domain name

- when domain name maps to different IP addresses

(e.g. multiple IPv4 and/or multiple IPv6 addresses)

PreferredLocationRegexp can be use to select the wanted.

When the provided regexp does not match any of

the addresses, take the first address.

- port is optional

- example: https://openacs.org

Sample setup

Old:

CanonicalServer : 137.208.116.31:443

ClusterAuthorizedIP : 137.208.116.31

ClusterPeerIP : 137.208.116.31:443 137.208.116.31:8443

ClusterEnabledP : 1

EnableLoggingP : 1

New:

CanonicalServer : https://openacs.org

PreferredLocationRegexp : https://

ClusterEnabledP : 1

EnableLoggingP : 1

#

# Cluster secret for intra-cluster communications in NaviServer

# configuration file. Clustering will not be enabled if no value is

# provided.

#

ns_section ns/server/$server/acs {

# ...

ns_param ClusterSecret "please change me"

}

#-----------------------------------------------------------------------

# Support for munin statistics

#

# 1) install munin plugins for naviserver (munin-plugins-ns.git)

# 2) add link to the munin plugins (replace "openacs" by the name

# of your server in the munin configuration)

# /etc/munin/plugins/naviserver_openacs_count_cluster -> /usr/share/munin/plugins/naviserver_count

# 3) in the plugin plugin configuration (e.g., etc/munin/plugin-conf.d/naviserver)

# add a section like the following (again, replace "openacs" by the server name you used)

#

# [naviserver_openacs_count_cluster]

# env.title Cluster

# env.vars cluster:broadcast cluster:sent cluster:received

#

# 4) restart munin

  1. … 4 more files in changeset.
Support the usage of HTTP locations in kernel parameter ClusterPeerIP

The change extends the values specified in ClusterPeerIP in two respects

- one can now specify the protocol (defaults to http)

- while ClusterPeerIP required an IP address, it is now possible to specify an DNS name

The DNS name is resolved for the time being at the start time of the OpenACS instance.

  1. … 2 more files in changeset.
set the global variable ::acs::kernel_id early enough during boostrap of new instances, such it can be used in a uniform way

  1. … 2 more files in changeset.
Prefer api to retrieve the acs-kernel id, as this will work also in corercases such as bootstraping of a new OpenACS instance

  1. … 1 more file in changeset.
More work on refactoring and streamlining cluster infrastructure

- removed more cases of ad-hoc parsing of IP address and port

- reduced code redundancy

- fixed incorrect comments

- removed obsolete private functions server_cluster_httpget_from_peers

and server_cluster_do_httpget (replaced by acs::clusterwide)

  1. … 1 more file in changeset.
Refactored and streamlined of cluster infrastructure

- base the API new new cluster infrastructure (including memoize-flush, server_cluster_peer_hosts, etc.)

- improved handling for multiple IP addresses and IPv6

- removed assumptions of local vs. global network drivers

- server_cluster_httpget_from_peers not used anymore

  1. … 5 more files in changeset.
white space changes, improve line breaks

merged changes from the oacs-5-9 branch and resolved conflicts

  1. … 7834 more files in changeset.
Standardize spelling of names of products (Tcl, AOLserver, PostgreSQL, NaviServer)

  1. … 43 more files in changeset.
- add editor hints to keep spaces/tabs in the furture more consistent

  1. … 754 more files in changeset.
Merging back to HEAD branch oacs-5-8 (using tag vg-merge-oacs-5-8-from-20141027).

  1. … 2547 more files in changeset.
- use util::http::get instead of ns_httpget

  1. … 2 more files in changeset.
- don't use backslash in multiline expressions

- make long expressions more uniform and more readable

- use "in" or not in (ni) operator instead of lsearch

- brace expressions to allow for tcl compilation

- bump version number

  1. … 23 more files in changeset.
change all core proc_doc calls to ad_proc calls

  1. … 7 more files in changeset.
remove deprecated calls to ad_parameter

  1. … 11 more files in changeset.
don't use numeric comparison operators on string comparisons

solve problem in cluster with ports

Normalizing string comparison operations into tcl 8.4 operations, enabling byte-code compilation for expressions and many conditionals

  1. … 367 more files in changeset.
Merged acs-core oacs-5-2 (currently 5.2.3) -> HEAD.

TODO:

1. Bump acs-core package version numbers to 5.3.0d1

2. Would be nice to do a test install, wouldn't it?

  1. … 721 more files in changeset.
Scheduling server_cluster_do_httpget in all servers in order to

get clustering working well. Setting to true all_servers switch

when scheduling server_cluster_do_httpget.

merge of changes between oacs-5-0-0b4 and oacs-5-0-0-final to HEAD

  1. … 286 more files in changeset.
add -public and -private for some procs, minor bug in acs_object::get some @see notes

  1. … 12 more files in changeset.
OpenACS 4.x

  1. … 1321 more files in changeset.
Initial revision

    • -0
    • +104
    ./server-cluster-procs.tcl
  1. … 1321 more files in changeset.