OpenACS

gustafn
committed
MAIN:gustafn:20090527065321:4
on 27 May 09
file metrics.adp was initially added on branch openacs-org.
MAIN:gustafn:20090527065321:4
openacs-4/www/robots.txt (+15 -5)
1 1 User-agent: *
2   Disallow: /
3   Disallow: /doc/
4   Disallow: /api-doc/
  2 Disallow: /register
5 3 Disallow: /register/
6 4 Disallow: /comments/
7 5 Disallow: /notifications/
8 6 Disallow: /test/
9 7 Disallow: /workflow/
10 8 Disallow: /acs-service-contract/
11 9 Disallow: /acs-lang/
12 10 Disallow: /acs-admin/
13 11 Disallow: /RPC2/
14 12
15  
16  
  13 # googlebot knows wildcards so block all register
  14 User-agent: googlebot
  15 Disallow: /register
  16 Disallow: /register/
  17 Disallow: /*/register
  18 Disallow: /*/register/
  19 Disallow: /comments/
  20 Disallow: /notifications/
  21 Disallow: /test/
  22 Disallow: /workflow/
  23 Disallow: /acs-service-contract/
  24 Disallow: /acs-lang/
  25 Disallow: /acs-admin/
  26 Disallow: /RPC2/