From OrganicDesign
Organic Design 2 | OD/Wikia | Wikia.php | Templates | Backup | Extensions
Our Wikia is being designed to manage a group of wikis stored on a single server, and uses an extension running on one of the wikis which is designated as the "master wiki" using Treeview to exhibit an article for each of the wiki's configurations.
1 Running Concurrent MediaWiki Versions
The domain-specific inclusion of LocalSettings is done by wikia.php which is included on the end of default LocalSettings.php file in the mediawiki version's source.
|
| include("$IP/extensions/wikia.php"); |
|
each mediawiki version instance should have a database template, which contains the basic articles required by the default extensions (like a WikiSkin article and CSS etc). Also a template of the image structure image structure with tree images etc would be useful. The wikia setup process can create a new database and pipe the template into it (renaming the tables to the appropriate prefixes first).
- See 01 May 2007 regarding the dpl_clview column which should be supplied in the template-wiki databases
- Interwiki list including all our wiki's should be included in the template-wiki's
2 MediaWiki Instances
These are separate instances of the mediawiki database accessed by separate domains pointing to the same machine and then mapped to separate parts of the filesystem.
Each of the actual wikis running on the server has its own directory in /var/www/, and in that directory is a www directory for any web sites or other applications, and for the wiki's images directory.
Also in the clients individual directory is a symlink called wiki which links to the template which corresponds to the MediaWiki version the wiki is using.
3 File Structure
Here's an example showing the key files and symlinks in a setup of peerix which is running version 1.6.7 and wikifs which is a 1.9.3.
|
|
/var/www
/var/www/mediawiki -> /var/www/mediawiki-1.10.0
/var/www/mediawiki-1.10.0
/var/www/mediawiki-1.10.0/extensions -> /var/www/extensions
/var/www/mediawiki-1.10.0/files -> /var/www/files
/var/www/mediawiki-1.10.0/wiki -> /var/www/mediawiki-1.10.0
/var/www/domains
/var/www/domains/peerix.org -> /var/www/settings/peerix
/var/www/domains/www.peerix.org -> /var/www/settings/peerix
/var/www/domains/wikifs.org -> /var/www/settings/wikifs
/var/www/domains/www.wikifs.org -> /var/www/settings/wikifs
/var/www/files
/var/www/files/peerix
/var/www/files/peerix/images
/var/www/files/wikifs
/var/www/files/wikifs/images
/var/www/settings
/var/www/settings/peerix
/var/www/settings/wikifs
|
|
4 Extensions
As described above, there is a single location where all the currently available extensions are stored for all MediaWiki versions. A subset of these are bundled together into a file called extensions/GeneralExtensions.php which are recommended for all wikis in OD/Wikia.
4.1 Extension Versions
Extensions are available from multiple sources, they may be writting in house or by third parties. Extensions are generally available through the mediawiki Extensions namespace, however they may be physically located on an SVN, a mediawiki site as source code in an article, or as a zip bundle.
It is recommended that all extensions are maintained in their own directory within the $IP/extensions path. If an extension is available on SVN it is straight forward to checkout the entire directory structure which can be maintained without having to change the filesystem structure. If a new version of an extension is avaialable in SVN, it would be good to maintain an archive of old svn checkouts.
If an extension is available as source code either from a mediawiki site as source code in an article, or as a zip bundle, then using symlinks pointing to directories which are named by version allows quick changes to be made from the symlink without having to change anything in LocalSettings.php.
This way different versions of extensions can be maintained with little disruption by creating a symbolic link to the extension version you want to use.
Proposed struture:
|
| ./extensions/EXTNAME-[VERSION]/FILE[s].php
./extensions/EXTNAME -> EXTNAME-[VERSION] |
|
The version should should be consistent with the versioning system the extension developer maintains, for example it could be major/minor versions (e.g. 1.0.2), or a unique QUID in SVN (e.g. R30815).
In LocalSettings.php, the extension include statement is not effected by changes to the symbolic link.
|
| include("$IP/extensions/EXTNAME/File[s].php") |
|
This structure is generic to handling of other software, mediawiki, mediawiki extensions, and third party directory structures that extensions use.
Both the mediawiki versions and the extension versions should be able to co-exist. To reduce maintenance on the instances, both should use a consistent directory naming scheme and use a current symlink. In the file structure example above, this is shown as the /var/www/mediawiki entry which has no version number and points to the most recent instance. The items in default virtual-host container can then point to the current instance without needing to be updated whenever a new version is released.
The same system can apply to extensions by using a consistent naming structure, for example:
|
|
extensions/DynamicPageList -> ./DynamicPageList-1.2.2
extensions/DynamicPageList-1.2.2/
extensions/DynamicPageList-1.2.2/DynamicPageList.php
extensions/DynamicPageList-1.2.2/DynamicPageList.i18n.php
extensions/DynamicPageList-1.1.8/
extensions/DynamicPageList-1.1.8/DynamicPageList.php
extensions/DynamicPageList-1.1.8/DynamicPageList.i18n.php
|
|
The R installation on OS X uses a similar mechanism.
4.1.1 Including extensions
There are four ways in php you can include code to extend MediaWiki;
|
| include(...) # PHP class of code to include which is non imperitive
include_once(..) # PHP class of code to include which is non imperitive, redundant loading of class will cause a warning
require(...) # PHP class that a number of extensions require, php fails if not available
require_once(...) # PHP class that a number of extensions require once, redundant loading of class will cause an exception |
|
Ideally to include extensions you should use the include(...) syntax, if a warning is provided then other extensions which are also loading the same code redundantly can be identified.
5 Todo: Better Backup & Restore
The key thing about the backups that I want to have in place is the ability for backups to be created at any time with timestamps, and for any backup to be restored back into a working wiki. The database name, htdocs and backup names should all include timestamp so that many instances of the same wiki from different backups can be run concurrently - effectively giving history to the wiki as a whole not just the individual articles. This is an important feature to have in place because it means that moving a wiki to another server would take literally seconds if the destination was already hosting wiki's using the same system.
Backups and restores may need to take into account the major version of mysql being used, as there are subtle changes to the mysqldump command, see [Svens (old) notes] for details.
- Databases will be backed up and distributed hourly, but images and files just daily unless manually initiated
- Backups of each wiki only need minimal setup information
- Server backups only need minimal files which are added to a standard LAMP setup
- Backups will still use peer, but peer will be modified to not require XmlWiki
- Peer will delete old backups (overwrite with 0) keeping daily for last week, and only 1st-of-month's before that
6 Shared content
7 Instance Configuration
Each LAMP instance is a script which can accept standard daemon commands of install, remove and conf which specifies the file containing the instance parameters. The conf file's leafname also serves as an identifier for the specific LAMP instance, and also requires an associated update command which should be executed whenever the conf file changes.
A request command which needs to be defined for each class of LAMP application, and is used to set up the environment according to which instance is being requested. The request command must be integrated into the source when it is downloaded and added to the available LAMP applications. Also template databases and skeleton files may need to be created at that time.
An upgrade command would be required for most LAMP application classes which can be executed whenever a newer version of the application source becomes available for download.
Some standard parameters should be included in the conf file, such as state which acts as the start and stop functionality. And source which specifies which LAMP application class the instance is implementing.
7.1 Handling Domains
The previous wikia method we used required adjustments to be made to the Apache virtual server configuration adjustment and a restart of Apache for new domains unless they use the default application. To allow all configuration to be defined in filesystem alone, the mod-rewrite rules can be adjusted to include the request domain (using the %{HTTP_HOST} apache environment variable). The wikia.php code then appends the domain to the $_SERVER['DOCUMENT_ROOT'] super global (I tried to use mod-vhost-alias to achieve this but it had two problems, first it still required a hack because the PHP $_SERVER['DOCUMENT_ROOT'] variable was not being set correctly, and secondly it appears to disable mod-rewrite), but luckily I worked out an equivalent solution using only mod-rewrite by using %{HTTP_HOST} in the rewrite rules.
The new dynamic method requires a slightly different structure to the files as follows:
|
|
/var/www/domains/foo.com -> /var/www/wikis/foo
/var/www/wikis/foo/
/var/www/wikis/foo/LocalSettings.php # Individual configuration
/var/www/wikis/foo/wiki -> /var/www/mediawiki-1.10.0/ # Symbolic link to common MediaWiki instance
/var/www/mediawiki-1.10.0/extensions -> /var/www/extensions # Link to common extensions directory
/var/www/mediawiki-1.10.0/skins -> /var/www/wikis/foo/skins # Link to skins common images
/var/www/wikis/foo/files # Images filesystem hash table
|
|
Note that the files for each vhost can be isolated from each other properly this way. The wiki version or instance can be switched at runtime with no adjustment to virtual hosts ever being required. The vhosts file is reduced to a very minimal state now, but we can go one step further with mod-rewrite still.
Note: An up to date copy of our vhosts file is in organicdesign.vhost
7.2 Filtering and Logging
Mod-rewrite can be configured to process the rule using an external program as the replacement term. The program is loaded once when Apache starts up, and then each time the rule is required, the replacement term is sent to the script over STDIN to be processed and the final replacement term sent back to Apache over STDOUT where it gets used for the final URL rewrite.
The virtual host file even further by delegating the existing rewrite rules to an external script which can do the domain-mapping and URL transformation according to the sites own LocalSettings file. For example, we could have other sets of rules similar to MediaWiki friendly urls but for different applications such as Joomla.
Instead of the rule being based specifically on REQUEST_URI and/or HTTP_HOST etc, the input to the script is more like a log entry containing all the relevant information pertaining to the request. The external script then uses all these parameters to log the request, filter or block it, and map to the local resource.
This method is far superior because total control of all aspects of the request information and the way it maps to a local resource is handled dynamically by a single script. All web requests are made via the script not just those which execute index.php so we can move the new traffic analysis script (Extension:RequestFilter.php which handles logging and blocking etc) into this new re-write script (rewrite.pl).
Note: the rewrite.pl script was tested for a while but proved to have some instability so is not currently in use on the main server.
8 How To
The HowTo's are being reconstructed in a new formalised Organic Design process structure
9 See also