spothole

(S)pothole is a utility to aggregate "spots" from amateur radio DX clusters and xOTA spotting sites, and provide an open JSON API as well as a website to browse the data.

While there are several other web-based interfaces to DX clusters, and sites that aggregate spots from various outfoor activity programmes for amateur radio, (S)pothole differentiates itself by supporting a large number of data sources, and by being "API first" rather than just providing a web front-end. This allows other software to be built on top of it.

The API is deliberately well-defined with an OpenAPI specification and auto-generated API documentation. The API delivers spots in a consistent format regardless of the data source, freeing developers from needing to know how each individual data source presents its data.

(S)pothole itself is also open source, Public Domain licenced code that anyone can take and modify.

Supported data sources include DX Clusters, the Reverse Beacon Network (RBN), the APRS Internet Service (APRS-IS), POTA, SOTA, WWFF, GMA, WWBOTA, HEMA, and Parks 'n' Peaks.

Accessing the public version

You can access the public version's web interface at https://spothole.m0trt.radio, and see https://spothole.m0trt.radio/apidocs for the API details.

Please note this URL is not necessarily final, and the API is likely to change as the project works its way towards v1.0. Please do not build anything on the Spothole API yet!

Running your own copy

To download and set up Spothole on a Debian server, run the following commands. Other operating systems will likely be similar.

git clone ssh://git@git.ianrenton.com/ian/spothole.git
cd spothole
python3 -m venv ./.venv
source .venv/bin/activate
pip install -r requirements.txt
deactivate
cp config-example.yml config.yml

Then edit config.yml in your text editor of choice to set up the software as you like it.

Then, to run the software this time and any future times you want to run it directly from the command line:

source .venv/bin/activate
python3 spothole.py

The software can take a few seconds to start up, mostly because it is downloading an updated file to match callsigns to countries. This is normal, don't panic!

If you see some errors on startup, check your configuration, e.g. in case you have specified a port for the web server that is already in use by something else.

systemd configuration

Create a file at /etc/systemd/system/spothole.service. Give it the following content, adjusting for the user you want to run it as and the directory in which you have installed it:

Unit]
Description=Spothole
After=syslog.target network.target

[Service]
Type=simple
User=spothole
WorkingDirectory=/home/spothole/spothole
ExecStart=/home/spothole/spothole/.venv/bin/python /home/spothole/spothole/spothole.py --serve-in-foreground
Restart=on-abort

[Install]
WantedBy=multi-user.target

Run the following:

sudo systemctl daemon-reload
sudo systemctl enable spothole
sudo systemctl start spothole

Check the service has started up correctly with sudo journalctl -u spothole -f.

nginx Reverse Proxy configuration

TODO nginx

TODO certbot

Writing your own client

TODO

Extending the server

(S)pothole is designed to be easily extensible. If you want to write your own provider, simply add a module to the providers package containing your class. (Currently, in order to be loaded correctly, the module (file) name should be the same as the class name, but lower case.)

Your class should extend "Provider"; if it operates by polling an HTTP Server on a timer, it can instead extend "HTTPProvider" where some of the work is done for you.

The class will need to implement a constructor that takes in the provider_config and provides it to the superclass constructor, while also taking any other config parameters it needs.

If you're extending the base Provider class, you will need to implement start() and stop() methods that start and stop a separate thread which handles the provider's processing needs. The thread should call submit() or submit_batch() when it has one or more spots to report.

If you're extending the HTTPProvider class, you will need to provide a URI to query and an interval to the superclass constructor. You'll then need to implement the http_response_to_spots() method which is called when new data is retrieved. Your implementation should then call submit() or submit_batch() when it has one or more spots to report.

When constructing spots, use the comments in the Spot class and the existing implementations as an example. All parameters are optional, but you will at least want to provide a time (which must be timezone-aware) and a dx_call.

Finally, simply add the appropriate config to the providers section of config.yml, and your provider should be instantiated on startup.

Third Party Libraries

The project contains a self-hosted copy of Font Awesome's free library, in the /webasset/fa/ directory. This is subject to Font Awesome's licence and is not covered by the overall licence declared in the LICENSE file. This approach was taken in preference to using their hosted kits due to the popularity of this project exceeding the page view limit for their free hosted offering.

The software uses a number of Python libraries as listed in requirements.txt, and a number of JavaScript libraries such as jQuery and moment.js. This project would not have been possible without these libraries, so many thanks to their developers.