Installing Laws-Africa Indigo on a Production LXC Container running Debian 11

Laws-Africa Indigo Logo

How to install the Indigo Content Management System for Legislation as maintained by Laws-Africa and the Akomo Ntoso Foundation.
A fully fledged version-controlled system to manage and update beautifully rendered legislation in line with legal formatting and reference.

All credit and atribution to Laws-Africa for their excellent work on the Indigo Platform.

What is Indigo

If you are asking that here, I think you are starting at the wrong place and I suggest you start at their official Github page to gain a proper understanding of how it works. If you are just looking to deploy it for development work, follow their official instructions. This How-to is written specifically to have a production setup running in a Debian 11 LXC Container (However other distros may work too) running on Proxmox.

What are we installing

Debian 11 (GNU/Linux OS)
Postgres (SQL Database Server)
Python 3 (Scripting Language)
WKHMTLtoPDF (PDF Creation Suite)
Ghostscript (Postscript interpreter for PDF Files)
Poppler (PDF Rendering Library)
Ruby (Another popular Scripting Langauge)
Django 3.2 (A rich web platform)
Indigo (A specialized document management system)

Getting started

We should always start on a clean and updated Debian 11 install, as this includes nano by default, I will be using nano to edit files, you are free to use your favorite. It is important to know that for some reason doing this via ssh causes issues with executibles so I run this through a tty connection in Proxmox. If I figure out the ssh issue I will fix it. Everything here is done as root, so the normal changes as relate to non-root users apply.

Update Debian:
```
apt update && apt upgrade -y
```

Install the prerequisite packages from apt:

apt install git build-essential poppler-utils fontconfig xfonts-base xfonts-75dpi postgresql ruby ruby-dev python3-pip python3-dev ghostscript --no-install-recommends

Install WKHTMLtoPDF from the .deb release file (Using apt does not result in a working system):

wget https://github.com/wkhtmltopdf/packaging/releases/download/0.12.6.1-2/wkhtmltox_0.12.6.1-2.bullseye_amd64.deb

dpkg -i wkhtmltox_0.12.6.1-2.bullseye_amd64.deb

We now have the underlying apt packages installed in Debian, now we can start with the additional scirpts and binaries for the scripting languages.

Configure Pythons pip and install some required PyPI packages:

Update pip:
```
pip install -U pip
```
Update the setuptools and wheel PyPI packages:
```
pip install -U setuptools wheel
```
Install some additional PyPI packages necessary for the production installation (Note the frozen versions for compatibility:
```
pip install gevent==21.8.0 gunicorn==20.1.0 psycopg2-binary==2.9.4
```

Create the postgres SQL database and user:

For this part we will use the database named indigodb and user named indigouser with password 123Password. Make sure you use a secure username and password, and pass the correct data to the ENV variable below, the correct format for DATABASE_URL is postgres://USER:PASSWORD@HOST:PORT/DBNAME so make sure you update this string accordingly before exporting it to the bashrc file.

Create the database named indigodb, as root:
```
su - postgres -c 'createdb indigodb'
```
Create the database user named indigouser, as root:
```
su - postgres -c 'createuser -d -P indigouser'
```
The interactive command prompt will ask you for a password, in the example we use 123Password.
Set the ENV variable for the database settings we used above:
If your database name, username and password are not the same as the example above, make sure to change the below DATABASE_URL ENV variable accordingly before setting it as this is where Indigo will look for the credentials.
```
echo 'export DATABASE_URL=postgres://indigouser:123Password@localhost:5432/indigodb' >> ~/.bashrc
```
Enable the changes to the .bashrc file:
```
source ~/.bashrc
```

Now let’s install Indigo

Let’s clone into the current Indigo Production Branch (v17.0.0 at the time of this update):
```
git clone --branch v17.0.0 --single-branch https://github.com/laws-africa/indigo
```
Change into the indigo folder (/root/indigo/):
```
cd indigo
```

Make some changes to the settings.py file (/root/indigo/indigo/settings.py):

First we are going to enable Emails as Background Tasks:

sed -i "s/'NOTIFICATION_EMAILS_BACKGROUND': False,/'NOTIFICATION_EMAILS_BACKGROUND': True,/" ./indigo/settings.py

Additionally, if you need to use encryption for your email server (SSL/TLS), you can enable either of them with the following commands:
For SSL:

sed -i "/^EMAIL_PORT = int(os.environ.get('DJANGO_EMAIL_PORT', 25))/a EMAIL_USE_SSL = True" ./indigo/settings.py

For TLS:

sed -i "/^EMAIL_PORT = int(os.environ.get('DJANGO_EMAIL_PORT', 25))/a EMAIL_USE_TLS = True" ./indigo/settings.py

Install Indigo’s Python dependencies, including Django from the setup.py file:
```
pip install -e .
```
Install Indigo’s Ruby gem dependencies:
1. First we need to fix the Gemfile and Gemfile.lock due to an issue with incorrect slaw version (13.0.0 at the time of this update):
```
sed -i "s/slaw (12.0.0)/slaw (13.0.0)/" ./Gemfile.lock
```
```
sed -i "s/slaw (~> 12.0)/slaw (~> 13.0)/" ./Gemfile.lock
```
```
sed -i "s/gem 'slaw', '~> 12.0'/gem 'slaw', '~> 13.0'/" ./Gemfile
```
2. Now we can install the bundler Ruby gem:
```
gem install bundler
```
3. And finally we can install the required Ruby gems:
```
bundle install
```

Configure your Indigo Installation:

We have now prepared our system and installed all the underlying requisite packages, we can now configure our Indigo installation:

Create the initial Database schema:
```
python3 manage.py migrate
```

Pass your specific settings to Indigo via ENV Variables:

Use nano to edit the bashrc file:
```
nano ~/.bashrc
```

Add the following lines to the bottom of the file:

export DJANGO_DEBUG=false
export DJANGO_SECRET_KEY={Some Random Characters}
export AWS_ACCESS_KEY_ID={Your AWS Key}
export AWS_SECRET_ACCESS_KEY={Your AWS Access ID}
export AWS_S3_BUCKET={The name of your AWS Bucket}
export SUPPORT_EMAIL={Your admin email address}
export DJANGO_DEFAULT_FROM_EMAIL={The email address Indigo will send mail from}
export DJANGO_EMAIL_HOST={Your SMTP Configaration}
export DJANGO_EMAIL_HOST_USER={Your SMTP Configaration}
export DJANGO_EMAIL_HOST_PASSWORD={Your SMTP Configaration}
export DJANGO_EMAIL_PORT={Your SMTP Configaration}
export INDIGO_ORGANISATION='{Your Organization Name}'
export INDIGO_URL={Indogo Official URL}
export RECAPTCHA_PUBLIC_KEY={Your Google Recaptcha Key}
export RECAPTCHA_PRIVATE_KEY={Your Google Recaptcha Key}
export GOOGLE_ANALYTICS_ID={Your Google Analytics ID}
export DJANGO_EMAIL_USE_TLS=true

Enable these ENV variables:
```
source ~/.bashrc
```

Generate some SSL Certificates (Gunicorn and Indigo Production require HTTPS) using openssl:
```
openssl req -new -x509 -days 365 -nodes -out /root/server.crt -keyout /root/server.key
```
Enter some appropriate settings as per your requirements.
Make sure that the models.py file accurately captures the correct database migrations that production Indigo will use:
```
python3 manage.py makemigrations
```
Apply any additional migrations to the database schema:
```
python3 manage.py migrate
```
Import Countries from Django-Countries-Plus:
```
python3 manage.py update_countries_plus
```

Import a Language file provided by Indigo:

python3 manage.py loaddata languages_data.json.gz

Update the Database with any added Migrations (Our changes to settings.py above):

Compile static files for the Gunicorn webserver to use:

python3 manage.py compilescss

python3 manage.py collectstatic --noinput -i docs -i \*scss 2>&1

Create a Superuser account:
```
python3 manage.py createsuperuser
```
Complete these settings as per your needs with a username, email address and secure password, remember this is a superuser and will have access to all of Indigo.

Run the Produciton Server and complete intial configuration

Start the production server with gunicorn:
```
gunicorn indigo.wsgi:application -k=gevent -t 600 --certfile=/root/server.crt --keyfile=/root/server.key -b=0.0.0.0:8000 -w=16 --threads 16 --forwarded-allow-ips=* --proxy-allow-from=* --limit-request-line 0
```
To understand each of these arguments:
1. –worker-class or -k : What kind of worker (use gevent)
2. –certfile : Where is the SSL certificate stored
3. –keyfile : where is the SSL key stored
4. -t : timeout (use 600ms for now)
5. –bind or -b : Bind to address:port (use 0.0.0.0:8000)
6. –workers or -w : How many workers (use 8 – 2x cores)
7. –threads : How many threads per worker (Use 8 – 2x cores)
8. -D : Run as daemon (Background Service)
9. –limit-request-line : Length of request line (set to 0)
Access your development server via https://your-ip:8000 (or whichever port you assgined in step one above if it differs).
Login with the superuser account created earlier.
In the top-right corner click your username and on the dropdown menu select “Site sttings”.
Under the Indigo API section, add a language.
Also under the Indigo API section, add a country, making sure to select a default language.
Edit your superuser account and add the country created above as the user’s default country.

Use Supervisor to automate server start-up – Optional Qualify of Life Improvement

Superuser is used to run certain commands at specified times, quite useful in this case.

Install Supervisor:

apt update && apt install supervisor --no-install-receommends -y

Return to the root folder for your user (/root/ in our case):
```
cd
```
Create the gunicorn_configuration file that Supervisor will use:
```
touch gunicorn_configuration
```
Then edit the file with nano (or your preferred text editor):
```
nano gunicorn_configuration
```

Now make the file resemble this, substituting any relevant paths in your installation as necessary, remember that the below export variables need to be the same as you set earlier in your bashrc file, including the DATABASE_URL variable, as Supervisor does not use ENV variables the same way as the underlying OS:

#!/bin/bash
NAME="Indigo"  #Django application name
DIR=/root/indigo   #Directory where project is located
USER=root   #User to run this script as
GROUP=root  #Group to run this script as
DJANGO_SETTINGS_MODULE=indigo.settings   #Which Django setting file should use
DJANGO_WSGI_MODULE=indigo.wsgi           #Which WSGI file should use
LOG_LEVEL=debug
cd $DIR
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DIR:$PYTHONPATH
export DATABASE_URL=postgres://indigouser:123Password@localhost:5432/indigodb
export DJANGO_DEBUG=false
export DJANGO_SECRET_KEY={Some Random Characters}
export AWS_ACCESS_KEY_ID={Your AWS Key}
export AWS_SECRET_ACCESS_KEY={Your AWS Access ID}
export AWS_S3_BUCKET={The name of your AWS Bucket}
export SUPPORT_EMAIL={Your admin email address}
export DJANGO_DEFAULT_FROM_EMAIL={The email address Indigo will send mail from}
export DJANGO_EMAIL_HOST={Your SMTP Configaration}
export DJANGO_EMAIL_HOST_USER={Your SMTP Configaration}
export DJANGO_EMAIL_HOST_PASSWORD={Your SMTP Configaration}
export DJANGO_EMAIL_PORT={Your SMTP Configaration}
export INDIGO_ORGANISATION='{Your Organization Name}'
export INDIGO_URL={Indogo Official URL}
export RECAPTCHA_PUBLIC_KEY={Your Google Recaptcha Key}
export RECAPTCHA_PRIVATE_KEY={Your Google Recaptcha Key}
export GOOGLE_ANALYTICS_ID={Your Google Analytics ID}
export DJANGO_EMAIL_USE_TLS=true
#Command to run the progam under supeperisor
exec gunicorn --chdir /root/indigo indigo.wsgi:application -k=gevent -t 600 --certfile=/root/server.crt --keyfile=/root/server.key -b=0.0.0.0:8000 -w=16 --threads 16 --forwarded-allow-ips=* --proxy-allow-from=* --limit-request-line 0 --log-level=debug --log-file=-

Make this file executable:
```
chmod u+x gunicorn_configuration
```
Configure Supervisor to start with systemd and start it now:
```
systemctl enable --now supervisor
```
Create a config file for Supervisor to use:
```
touch /etc/supervisor/conf.d/indigo.conf
```
Then edit the file with nano (or your preferred text editor):
```
nano /etc/supervisor/conf.d/indigo.conf
```

And make it look like this, again substituting any paths you may have altered from the examples:

[program:indigo]
command=/root/gunicorn_configuration
user=root
autostart=true
autorestart=true
redirect_stderr=true
stdout_logfile=/root/gunicorn-error.log

Enable and start your new Supervisor app:

supervisorctl reread

supervisorctl update

supervisorctl restart indigo

And voila, Supervisor will now start your Indigo Gunicorn server at bootup.

Use Cron to automate the running of background tasks – Optional Qualify of Life Improvement

Indigo used the django-background-tasks module to perform some maintenance and other tasks, it is ideal to automate this behaviour using cron to run the command at regular intervals, the following steps take you through that process:

Make sure that you have cron installed:

apt install cron --no-install-recommends -y

Next we need to determine the python3 absolute path:
```
which python
```
Which should return something along the lines of:
```
/usr/bin/python3
```
With that, we can now determine our cron command as follows:
```
*/30 * * * * /usr/bin/python3 /root/indigo/manage.py process_tasks --duration 60
```
Where */30 * * * * means that cron will execute the command every 30 minutes, being python3 at “/usr/bin/python3” running the script “/root/indigo/manage.py” with the instruction “process_tasks” for a duration of 60 seconds.
Now we need to launch the crontab editor:
```
crontab -e
```
And choosing your text editor of choice, I prefer nano for this.

Setting the ENV variables for Cron:
At the top of the Cron file, we need to set the same environmental variables we used for gunicorn_config and ~/.bashrc, however we don’t need to include the “export ” bit, so it should resemble the following:

DATABASE_URL=postgres://indigouser:123Password@localhost:5432/indigodb
DJANGO_DEBUG=false
DJANGO_SECRET_KEY={Some Random Characters}
AWS_ACCESS_KEY_ID={Your AWS Key}
AWS_SECRET_ACCESS_KEY={Your AWS Access ID}
AWS_S3_BUCKET={The name of your AWS Bucket}
SUPPORT_EMAIL={Your admin email address}
DJANGO_DEFAULT_FROM_EMAIL={The email address Indigo will send mail from}
DJANGO_EMAIL_HOST={Your SMTP Configaration}
DJANGO_EMAIL_HOST_USER={Your SMTP Configaration}
DJANGO_EMAIL_HOST_PASSWORD={Your SMTP Configaration}
DJANGO_EMAIL_PORT={Your SMTP Configaration}
INDIGO_ORGANISATION='{Your Organization Name}'
INDIGO_URL={Indogo Official URL}
RECAPTCHA_PUBLIC_KEY={Your Google Recaptcha Key}
RECAPTCHA_PRIVATE_KEY={Your Google Recaptcha Key}
GOOGLE_ANALYTICS_ID={Your Google Analytics ID}

At the bottom of the cron file, we need to also add the cron command from step 3:
```
*/30 * * * * /usr/bin/python3 /root/indigo/manage.py process_tasks --duration 60
```

Now simply save with ctrl+x and you are good to go, cron should run this command every 30 minutes for a duration of 60 seconds.

Installing Laws-Africa Indigo on a Production LXC Container running Debian 11

Comments

Leave a Reply Cancel reply