Skip to content

Latest commit

 

History

History
284 lines (188 loc) · 10.4 KB

TESTING.md

File metadata and controls

284 lines (188 loc) · 10.4 KB
Raw

The easy way: use Docker

  1. Install Docker and Docker Compose. https://docs.docker.com/get-docker/
  2. docker compose up --build
  3. Run ./docker/createdb.sh to create your first user, [email protected] with password password
  4. Visit the site at http://localhost:8080/admin. (http://localhost:8080/ will show a "Forbidden" error.)
  5. Run admin CLI commands (adduser, edituser, addupload, cleanup) with ./docker/admin.sh.

You'll need to restart the instance every time you edit the Python modules in tinyapp and adminlib.

Stop Docker with Ctrl-C. You can launch it again with docker compose up. (The database will be preserved.)

You'll need to docker compose down (deleting the database) and docker compose up --build again if you modify the Dockerfile, the docker-compose.yml, or any of the files that we COPY in the Dockerfile: sample.config or ifarchive-admintool.conf.

After you docker compose down and docker compose up --build, you'll need to rerun the ./docker/createdb.sh script to recreate the DB. If you forget to create the DB, you'll see this error at http://localhost:8080/admin:

sqlite3.OperationalError: no such table

Development notes

If you edit the Python modules in tinyapp and adminlib, the changes will not be visible until you restart httpd (restart Docker).

# On MacOS:
% brew services restart httpd

# On Linux:
% sudo apachectl restart

If you change the config file (ifarch.config or test.config), you should touch admin.wsgi to reload the config or restart Docker.

The admin.wsgi file itself is automatically restarted if it changes. Other file changes (templates, for example) are picked up immediately and require no restart.

This test environment does not include the "Rebuild Indexes" script. (That would require more test files which have nothing to do with the admin tool per se.) So hitting that button in the admin interface will fail.

Uploading Files

The upload script is a separate project, so that's not available either.

In Docker, you can use the docker/upload-file.sh script.

% docker/upload-file.sh --name "Andrew Plotkin" --email [email protected] --tuid 0000000000000000 myfile.txt

The script copies your file directly into the incoming directory, and then creates an upload database entry using the admin CLI, python3 admin.wsgi addupload. (You can do the same yourself if you're not using Docker.)

The hard way: creating a test environment

A note on Python versions: This software requires Python 3.7 or later. I have tested on 3.7 and 3.11.

3.7 is rather archaic, but it's what the Archive server has available. Some of the code could be simplified a bit if I installed a later version, and I will do that someday, but right now everything needs to be compatible with 3.7.

On MacOS

Clone this repo. For the purposes of these instructions, I will assume the repo is in /Users/zarf/src/ifarchive-admintool. Substitute your work directory as needed.

Install httpd and Python3 via homebrew. (Macs come with httpd installed, but it's heavily restricted. It's easier to install a separate user-level httpd for this project.)

brew install httpd [email protected]

Homebrew installs into /usr/local/bin/ (Intel Macs) or /opt/homebrew/bin (ARM Macs), so make sure the correct directory is in your $PATH.

This installation of httpd will be available at http://localhost:8080/. We will set up the admintool at http://localhost:8080/admintest.

Install the Python modules mod-wsgi and Jinja2.

pip3 install mod-wsgi Jinja2

Update /usr/local/etc/httpd/httpd.conf or /opt/homebrew/etc/httpd/httpd.conf to include WSGI support and the admintool script, as follows.

In the LoadModule section (near the top), add one of these lines:

# For Intel Macs:
LoadModule wsgi_module /usr/local/lib/python3.11/site-packages/mod_wsgi/server/mod_wsgi-py311.cpython-311-darwin.so

# For ARM Macs:
LoadModule wsgi_module /opt/homebrew/lib/python3.11/site-packages/mod_wsgi/server/mod_wsgi-py311.cpython-311-darwin.so

In the <IfModule alias_module> stanza, add:

# For Intel Macs:
SetEnv IFARCHIVE_CONFIG "/Users/zarf/src/ifarchive-admintool/test.config"
WSGIScriptAlias /admintest "/usr/local/var/www/wsgi-bin/test.wsgi"
WSGIPythonPath "/usr/local/var/www/wsgi-bin"

# For ARM Macs:
SetEnv IFARCHIVE_CONFIG "/Users/zarf/src/ifarchive-admintool/test.config"
WSGIScriptAlias /admintest "/opt/homebrew/var/www/wsgi-bin/test.wsgi"
WSGIPythonPath "/opt/homebrew/var/www/wsgi-bin"

Then, below:

<Directory "/usr/local/var/www/wsgi-bin">
    AllowOverride None
    Options FollowSymLinks
    Require all granted
</Directory>

(Or /opt/homebrew for ARM Macs. I'm gonna stop giving both versions, sorry, keep substituting as needed.)

Also make sure the IFARCHIVE_CONFIG env var is set in your own environment, with the same value as above.

# sh-style:
% export IFARCHIVE_CONFIG=/Users/zarf/src/ifarchive-admintool/test.config

# csh-style:
% setenv IFARCHIVE_CONFIG /Users/zarf/src/ifarchive-admintool/test.config

Create these directories that you just configured, and symlink in the appropriate files from your repo:

% mkdir /usr/local/var/www/wsgi-bin
% ln -s /Users/zarf/src/ifarchive-admintool/admin.wsgi /usr/local/var/www/wsgi-bin/test.wsgi
% ln -s /Users/zarf/src/ifarchive-admintool/adminlib /usr/local/var/www/wsgi-bin/adminlib
% ln -s /Users/zarf/src/ifarchive-admintool/tinyapp /usr/local/var/www/wsgi-bin/tinyapp
% cp /Users/zarf/src/ifarchive-admintool/css/admintool.css /Users/zarf/src/ifarchive-admintool/css/cssbullet.png /usr/local/var/www

(You could put the /Users/zarf/src/ifarchive-admintool/... paths directly into your <Directory> stanzas rather than creating symlinks. This is just how I did it.)

(I copied admintool.css, rather than symlinking it, because Apache's config for /usr/local/var/www might not support symlinks.)

At this point you need to restart httpd to pick up the config changes:

% brew services restart httpd

Now you need to create an test.config file for the admintool. The sample.config file in the repo is configured for the live IF Archive. You will need to change most of the paths:

[DEFAULT]

DBFile = /Users/zarf/src/ifarchive-admintool/admin.db

IncomingDir = /Users/zarf/src/ifarchive-admintool/testincoming
TrashDir    = /Users/zarf/src/ifarchive-admintool/testtrash
ArchiveDir  = /Users/zarf/src/ifarchive-admintool/testarchive

# Must be false if your test server is on http:
SecureSite = false

[AdminTool]

AppRoot = /admintest
AppCSSURI = /admintool.css

TemplateDir = /Users/zarf/src/ifarchive-admintool/templates

LogFile = /Users/zarf/src/ifarchive-admintool/out.log

BuildScriptFile = /Users/zarf/src/ifarchive-admintool/testbuild-bg
BuildLockFile = /Users/zarf/src/ifarchive-admintool/build.lock
BuildOutputFile = /Users/zarf/src/ifarchive-admintool/build.out

# Ten days (in seconds)
MaxSessionAge = 864000

# Thirty days (in seconds)
MaxTrashAge = 2592000

Create the directories mentioned above:

% mkdir testincoming testtrash testarchive testarchive/unprocessed

Create the SQLite database (which, as configured above, will be in /Users/zarf/src/ifarchive-admintool/admin.db). Then create an admin user for yourself:

% python3 admin.wsgi createdb
% python3 admin.wsgi adduser zarf [email protected] password --roles admin

(If you see "Config file not found: /var/ifarchive/lib/ifarch.config", you forgot to set the IFARCHIVE_CONFIG env variable.)

You should now be able to visit http://localhost:8080/admintest and log in (zarf / password, as set up above).

If the login page does not appear, or logging in fails, check both the Apache error log (/usr/local/var/log/httpd/error_log) and the admintool log (/Users/zarf/src/ifarchive-admintool/out.log).

On Linux

We will set up the environment as closely as possible to the real IF Archive. The admintool will be at http://localhost/admin.

Create a directory /var/ifarchive to store all Archive data. For testing purposes, we will make it world-writable.

% sudo mkdir /var/ifarchive
% sudo chmod 777 /var/ifarchive
% cd /var/ifarchive
% mkdir lib lib/sql lib/admintool
% mkdir incoming trash logs wsgi-bin
% mkdir htdocs htdocs/misc htdocs/if-archive htdocs/if-archive/unprocessed

Clone the repo into /var/ifarchive/ifarchive-admintool.

Copy the files into position:

% cp -r ifarchive-admintool/tinyapp wsgi-bin
% cp -r ifarchive-admintool/adminlib wsgi-bin
% cp -r ifarchive-admintool/templates lib/admintool
% cp ifarchive-admintool/admin.wsgi wsgi-bin
% cp ifarchive-admintool/css/admintool.css ifarchive-admintool/css/cssbullet.png htdocs/misc
% cp ifarchive-admintool/sample.config lib/ifarch.config

In lib/ifarch.config, change the SecureSite entry to false. (This must be false if your test server is on http:.)

Install apache2, python3, and libapache2-mod-wsgi-py3 via your package manager.

Install the Python modules mod-wsgi and Jinja2.

pip3 install mod-wsgi Jinja2

Enable the Apache WSGI module:

sudo a2enmod wsgi

Update /etc/apache2/sites-available/000-default.conf to include WSGI support and the admintool script, as follows.

Inside the <VirtualHost> section, change the DocumentRoot line to:

DocumentRoot /var/ifarchive/htdocs/

Inside the <VirtualHost> section, add the lines:

WSGIScriptAlias /admin /var/ifarchive/wsgi-bin/admin.wsgi

<Directory "/var/ifarchive/htdocs/">
    Order allow,deny
    Allow from all
    Require all granted
</Directory>

<Directory "/var/ifarchive/wsgi-bin/">
    Require all granted
    SetEnv LANG en_US.UTF-8
</Directory>

After the <VirtualHost> section, add the line:

WSGIPythonPath /var/ifarchive/wsgi-bin

At this point you need to restart httpd to pick up the config changes:

% sudo apachectl restart

Create the SQLite database (which, as configured above, will be in /var/ifarchive/lib/sql/admin.db). Then create an admin user for yourself:

% python3 wsgi-bin/admin.wsgi createdb
% python3 wsgi-bin/admin.wsgi adduser zarf [email protected] password --roles admin

You should now be able to visit http://localhost/admin and log in (zarf / password, as set up above).

If the login page does not appear, or logging in fails, check both the Apache error log (/var/log/apache2/error_log) and the admintool log (/var/ifarchive/logs/admintool.log).