Server transfer documentation: Difference between revisions
No edit summary |
|||
Line 1: | Line 1: | ||
This is a rough documentation of the server migration for TiTIPI that took place in December 2023. Please read carefully the urls with the official guides for migrating a server. Each server migration case differs. | This is a rough documentation of the server migration for TiTIPI that took place in December 2023. Please read carefully the urls with the official guides for migrating a server. Each server migration case differs. | ||
== Create a Virtual Private Server (VPS) | == Create a Virtual Private Server (VPS) == | ||
=== Ubuntu vs Debian === | === Ubuntu vs Debian === | ||
Line 13: | Line 11: | ||
Ubuntu is based on Debian. It has PPA (Personal Package Archive). With PPA, installing newer software or getting the latest software version becomes a bit easier and it is a lot more forgiving when it comes to including proprietary drivers and firmware in the default ISO. Ubuntu is backed by Canonical. However, it is not entirely a corporate project. It does have a community but the final decision on any matter is in Canonical’s hands. | Ubuntu is based on Debian. It has PPA (Personal Package Archive). With PPA, installing newer software or getting the latest software version becomes a bit easier and it is a lot more forgiving when it comes to including proprietary drivers and firmware in the default ISO. Ubuntu is backed by Canonical. However, it is not entirely a corporate project. It does have a community but the final decision on any matter is in Canonical’s hands. | ||
=== New VPS | === New VPS === | ||
With the root user create new users and add them to the sudoers file. | |||
Add users: | Add users: | ||
Line 25: | Line 23: | ||
<code>$ adduser helen sudo</code> | <code>$ adduser helen sudo</code> | ||
Install tmux to share terminal sessions <code>$ sudo apt install tmux</code> | Install and use tmux to share terminal sessions with servermates <code>$ sudo apt install tmux</code> | ||
== Install Apache2 == | == Install Apache2 == | ||
Line 53: | Line 51: | ||
=== Domain set-up === | === Domain set-up === | ||
Login to the domain provider | Login to the domain provider to change DNS for the domain you want. | ||
Replace old IPv4 address with the new one | Replace old IPv4 address with the new one provided by your VPS. | ||
Configure apache for domain your_domain: | Configure apache for domain your_domain: | ||
Line 143: | Line 141: | ||
How to move a mediawiki: https://www.mediawiki.org/wiki/Manual:Moving_a_wiki | How to move a mediawiki: https://www.mediawiki.org/wiki/Manual:Moving_a_wiki | ||
MediaWiki version of old server | MediaWiki version of your old server | ||
<pre> Version: | <pre> Version: [YOUR_WIKI_DOMAIN]/index.php/Special:Version | ||
MediaWiki 1.35.1 | MediaWiki 1.35.1 | ||
PHP 7.4.21 (apache2handler) | PHP 7.4.21 (apache2handler) | ||
Line 184: | Line 182: | ||
<code>$ tar -xvzf mediawiki-1.40.1.tar.gz</code> | <code>$ tar -xvzf mediawiki-1.40.1.tar.gz</code> | ||
<code>$ mkdir /var/www/ | <code>$ mkdir /var/www/[YOUR_DOMAIN]/wiki</code> | ||
<code>$ mv mediawiki-1.40.1/* /var/www/ | <code>$ mv mediawiki-1.40.1/* /var/www/[YOUR_DOMAIN]/wiki/</code> | ||
Install requirements: | Install requirements: | ||
Line 202: | Line 200: | ||
<code>$ mariadb -u root -p -h localhost</code> | <code>$ mariadb -u root -p -h localhost</code> | ||
<code>$ cp /home/femke/LocalSettings.php /var/www/ | <code>$ cp /home/femke/LocalSettings.php /var/www/[YOUR_DOMAIN]/wiki/</code> | ||
<code>$ ls /var/www/ | <code>$ ls /var/www/[YOUR_DOMAIN]/wiki/</code> | ||
database info old server | database info from old server: | ||
<pre> $wgDBname = "name of wiki"; | <pre> $wgDBname = "name of wiki"; | ||
Line 218: | Line 216: | ||
How to back up a MediaWiki: https://www.mediawiki.org/wiki/Manual:Backing_up_a_wiki | How to back up a MediaWiki: https://www.mediawiki.org/wiki/Manual:Backing_up_a_wiki | ||
<code>$ cd /var/www/html/ | <code>$ cd /var/www/html/[YOUR_DOMAIN]/backups/</code> | ||
<code>$ mysqldump -u admin -pbugreport new_wiki -c > wikidb.sql</code> | <code>$ mysqldump -u admin -pbugreport new_wiki -c > wikidb.sql</code> | ||
Line 234: | Line 232: | ||
How to upgrade a MediaWiki: https://www.mediawiki.org/wiki/Manual:Upgrading | How to upgrade a MediaWiki: https://www.mediawiki.org/wiki/Manual:Upgrading | ||
<pre>new server | <pre>new server: | ||
MediaWiki 1.40.1 | MediaWiki 1.40.1 | ||
PHP 8.2.7 (apache2handler) | PHP 8.2.7 (apache2handler) | ||
Line 240: | Line 238: | ||
MariaDB 10.11.4-MariaDB-1~deb12u1 | MariaDB 10.11.4-MariaDB-1~deb12u1 | ||
Old server | Old server: | ||
MediaWiki 1.35.1 | MediaWiki 1.35.1 | ||
PHP 7.4.21 (apache2handler) | PHP 7.4.21 (apache2handler) | ||
Line 291: | Line 289: | ||
With filezilla we replaced: themes/default/css/ and themes/default/index.twig | With filezilla we replaced: themes/default/css/ and themes/default/index.twig | ||
Make ssh key in the new server for git clone the wiki-to-pdf. | |||
Make ssh key in | |||
We create a user for managing access to the world such as git called 'expert'. | We create a user for managing access to the world such as git called 'expert'. | ||
Line 305: | Line 300: | ||
Made the user 'expert' in gitlab and added the ssh key in it from the 'expert' user in the new server. | Made the user 'expert' in gitlab and added the ssh key in it from the 'expert' user in the new server. | ||
Create a webhook.php file in '/var/www/ | Create a webhook.php file in '/var/www/<code>[YOUR_DOMAIN]</code>' and insert: | ||
<pre><?php | <pre><?php | ||
setlocale(LC_ALL,'en_US.UTF-8'); | setlocale(LC_ALL,'en_US.UTF-8'); | ||
$mirrordir='/var/www/ | $mirrordir='/var/www/[YOUR_DOMAIN]/content'; | ||
$gitdir=$mirrordir."/.git"; | $gitdir=$mirrordir."/.git"; | ||
Revision as of 14:53, 30 January 2024
This is a rough documentation of the server migration for TiTIPI that took place in December 2023. Please read carefully the urls with the official guides for migrating a server. Each server migration case differs.
Create a Virtual Private Server (VPS)
Ubuntu vs Debian
We choose Debian because of these reasons (ref: https://itsfoss.com/debian-vs-ubuntu/):
Debian is older and more stable. Debian’s focus on stability means that it does not always aim for the latest versions of the software. It supports both 32-bit and 64-bit architecture and focuses primarily on FOSS (free and open source software). This means that the kernel provided by Debian does not include proprietary drivers and firmware. Debian is a true community project. The long-term support and reliability of Debian's stable release make the distro a preferred option for servers that require uninterrupted, surprise-free operation. Debian's package manager (the Advanced Package Tool, or APT for short) simplifies software installations and updates.
Ubuntu is based on Debian. It has PPA (Personal Package Archive). With PPA, installing newer software or getting the latest software version becomes a bit easier and it is a lot more forgiving when it comes to including proprietary drivers and firmware in the default ISO. Ubuntu is backed by Canonical. However, it is not entirely a corporate project. It does have a community but the final decision on any matter is in Canonical’s hands.
New VPS
With the root user create new users and add them to the sudoers file.
Add users:
$ useradd -m helen -s /bin/bash
or adduser helen
(both of these commands create a home directory and load the bash features on the user)
$ passwd helen
$ adduser helen sudo
Install and use tmux to share terminal sessions with servermates $ sudo apt install tmux
Install Apache2
How to: https://www.digitalocean.com/community/tutorials/how-to-install-the-apache-web-server-on-debian-11
$ apt update
$ apt install apache2
$ apt install ufw
$ ufw app list
$ ufw allow OpenSSH
$ ufw allow 'WWW'
$ ufw enable
$ ufw status
$ systemctl status apache2
$ hostname -I
Domain set-up
Login to the domain provider to change DNS for the domain you want.
Replace old IPv4 address with the new one provided by your VPS.
Configure apache for domain your_domain:
$ cd /var/www/
$ mkdir your_domain
$ mkdir your_domain/log
$ chmod -R 755 /var/www/your_domain/
$ nano your_domain/index.html
$ nano /etc/apache2/sites-available/your_domain.conf
Insert:
<VirtualHost *:80> ServerAdmin webmaster@localhost ServerName your_domain ServerAlias your_domain_ip DirectoryIndex index.html index.php DocumentRoot /var/www/your_domain ErrorLog /var/www/your_domain/log/error.log CustomLog /var/www/your_domain/log/access.log combined </VirtualHost>
$ a2ensite your_domain.conf
$ systemctl reload apache2
Disable default domain apache configuration
$ a2dissite 000-default.conf
$ systemctl reload apache2
$ apache2ctl configtest
$ systemctl restart apache2
Access logs
Reduce the size of the domain logs automatically:
Install logrotate: https://linux.die.net/man/8/logrotate
$ apt-get install logrotate
$ nano /etc/logrotate.d/apache2.conf
/var/log/apache2/* { weekly rotate 3 size 10M compress delaycompress } /var/www/titipi.org/log/* { daily rotate 3 size 10M compress delaycompress } /var/www/ddivision.xyz/log/* { daily rotate 3 size 10M compress delaycompress }
$ logrotate -d /etc/logrotate.d/apache2.conf
$ nano /var/log/apache2/access.log
$ systemctl reload apache2
Install and migrate MediaWiki
How to move a mediawiki: https://www.mediawiki.org/wiki/Manual:Moving_a_wiki
MediaWiki version of your old server
Version: [YOUR_WIKI_DOMAIN]/index.php/Special:Version MediaWiki 1.35.1 PHP 7.4.21 (apache2handler) MySQL 5.7.34-0ubuntu0.18.04.1
How to: https://www.mediawiki.org/wiki/Manual:Moving_a_wiki
Steps
- Install the latest mediawiki version on the new server https://www.mediawiki.org/wiki/Manual:Installing_MediaWiki and https://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Debian_or_Ubuntu#Get_latest_MediaWiki
- Set up database for new mediawiki on the new server https://www.mediawiki.org/wiki/Manual:MariaDB/en (We choose MariaDB because MySql uses proprietary source code)
- Install wiki-to-pdf on new server (see #wiki-to-pdf)
- Make a backup of the wiki on the old server (more in detail: https://sharkysoft.com/wiki/how_to_move_a_MediaWiki_wiki_from_one_server_to_another.html)
- upgrade the MediaWiki on the old server
- mysqldump + xml dump old wiki (backup again)
- Transfer backups from the old server to the new one
- Restore the backup on the new server
Prepare the ground on the new server and download MediaWiki:
$ apt upgrade
$ apt update
$ apt-get install php php-mysql libapache2-mod-php php-xml php-mbstring
$ apt-get install mariadb-server
$ apt-get install imagemagick
$ systemctl reload apache2
$ cd /tmp/
$ wget https://releases.wikimedia.org/mediawiki/1.40/mediawiki-1.40.1.tar.gz
$ apt-get install wget
$ tar -xvzf mediawiki-1.40.1.tar.gz
$ mkdir /var/www/[YOUR_DOMAIN]/wiki
$ mv mediawiki-1.40.1/* /var/www/[YOUR_DOMAIN]/wiki/
Install requirements:
$ apt install php-intl
$ systemctl reload apache2
$ apt-get install php-apcu inkscape php-gd php-cli php-curl php-bcmath git
$ systemctl reload apache2
Set up database. Check here: https://www.mediawiki.org/wiki/Manual:Installing_MediaWiki
$ mariadb -u root -p -h localhost
$ cp /home/femke/LocalSettings.php /var/www/[YOUR_DOMAIN]/wiki/
$ ls /var/www/[YOUR_DOMAIN]/wiki/
database info from old server:
$wgDBname = "name of wiki"; $wgDBuser = "user"; $wgDBpassword = "password";
Back up the wiki on the old server
How to back up a MediaWiki: https://www.mediawiki.org/wiki/Manual:Backing_up_a_wiki
$ cd /var/www/html/[YOUR_DOMAIN]/backups/
$ mysqldump -u admin -pbugreport new_wiki -c > wikidb.sql
$ tar czvf images.tgz images/
$ mv images.tgz ../../backups/
$ cd ../../
$ cd backups/
$ mysqldump -u admin -pbugreport new_wiki --xml > wiki.xml
Upgrade mediawiki in old server
How to upgrade a MediaWiki: https://www.mediawiki.org/wiki/Manual:Upgrading
new server: MediaWiki 1.40.1 PHP 8.2.7 (apache2handler) ICU 72.1 MariaDB 10.11.4-MariaDB-1~deb12u1 Old server: MediaWiki 1.35.1 PHP 7.4.21 (apache2handler) MySQL 5.7.34-0ubuntu0.18.04.1
$ sudo tmux attach -t webserver
check storage space, which files take more space:
$ du -h -d 1 .
$ df
$ df -h
let's vacuum it!
$ journalctl --vacuum-size=100M
$ apt clean
Python wants us to have virtual environments but we go for installing through Debian packages. We tried but it was very complex. We continued with a virtual environment.
Install Wiki-to-pdf
Don't forget to copy the pub/ folder in the new server (it includes the publications from the old server)
How to: https://titipi.org/wiki/index.php/Wiki-to-pdf-manual and https://gitlab.constantvzw.org/titipi/wiki-to-pdf
Install Pico
Install pico and check: https://picocms.org/download/
$ tar --exclude='public_html/wiki' --exclude='public_html/wiki-to-pdf' --exclude='public_html/wiki-to-pdf-webhook.php' -zcvf backups/site.tgz public_html/
Check the php modules dom and mbstring installed for pico to run
$ php -m
From /titipi.org we update pico
$ rm -r vendor/
$ wget https://github.com/picocms/Pico/releases/download/v3.0.0-alpha.2/pico-release-v3.0.0-alpha.2.tar.gz
$ tar -xvzf pico-release-v3.0.0-alpha.2.tar.gz
$ rm pico-release-v3.0.0-alpha.2.tar.gz
With filezilla we replaced: themes/default/css/ and themes/default/index.twig
Make ssh key in the new server for git clone the wiki-to-pdf.
We create a user for managing access to the world such as git called 'expert'.
Webhook Pico
Pico's edit interface is outdated. In order to create and edit web pages we decide to put the content of the website in a git repository, in a way we create an CMS (Content management system) for ourselves. The web pages can be edited in markdown format through the git.
With a webhook git pull and git push happens automatically, thus the editors don't have to access the server to edit and push back to the git.
Made the user 'expert' in gitlab and added the ssh key in it from the 'expert' user in the new server.
Create a webhook.php file in '/var/www/[YOUR_DOMAIN]
' and insert:
<?php setlocale(LC_ALL,'en_US.UTF-8'); $mirrordir='/var/www/[YOUR_DOMAIN]/content'; $gitdir=$mirrordir."/.git"; $access_token = 'cloudpass'; $headers = apache_request_headers(); $headd = array(); foreach ($headers as $header => $value) { $head["$header"] = $value; } $json= file_get_contents('php://input'); $client_token = $headers['X-Gitlab-Token']; $jsarr=json_decode($json,true); if ($client_token !== $access_token) { echo $jsarr['repository']['name']; header('HTTP/1.0 403 Forbidden'); echo "error 403"; exit(0); } $branch = $jsarr['ref']; $repo = $jsarr['repository']['name']; chdir($mirrordir); $cmd="git pull"; exec($cmd); echo 'working' ;
Check if webhook is working by testing it in gitlab (test push events):
https://gitlab.constantvzw.org/titipi/titipi-website/-/hooks
With nano /etc/apache2/envvars
change export APACHE_RUN_USER=www-data
to export APACHE_RUN_USER=expert
so 'expert' is able to use the webhook.
Install and configure Certbot
How to: https://certbot.eff.org/instructions and https://www.digitalocean.com/community/tutorials/how-to-secure-apache-with-let-s-encrypt-on-debian-11
In Gandi create web forward for http://www.your_domain to http://your_domain