Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
d9ce8db48b | ||
|
|
f08ff030e3 | ||
|
|
3ab2375a32 | ||
|
|
985646bb94 | ||
|
|
8d28a29f82 | ||
|
|
4f9ec65d9a | ||
|
|
f68b244e38 | ||
|
|
56a29b6203 | ||
|
|
5699627069 | ||
|
|
66081f32f9 | ||
|
|
4f8f748d88 | ||
|
|
c2ecef1003 | ||
|
|
266468f938 | ||
|
|
fffd230633 | ||
|
|
3c1b1821f5 | ||
|
|
77d7fd70d9 | ||
|
|
c3bcc1190d | ||
|
|
ecab8d6d16 | ||
|
|
0cea777616 | ||
|
|
3c4c595288 | ||
|
|
555a8418a9 | ||
|
|
7ee08e2fcd | ||
|
|
4353c9ebf9 | ||
|
|
4a9ff2c636 | ||
|
|
e7253c0916 |
@@ -0,0 +1,7 @@
|
|||||||
|
/.venv
|
||||||
|
__pycache__
|
||||||
|
*.pyc
|
||||||
|
/config.yml
|
||||||
|
/cache/
|
||||||
|
.git
|
||||||
|
.gitignore
|
||||||
@@ -4,7 +4,7 @@
|
|||||||
<content url="file://$MODULE_DIR$">
|
<content url="file://$MODULE_DIR$">
|
||||||
<sourceFolder url="file://$MODULE_DIR$" isTestSource="false" />
|
<sourceFolder url="file://$MODULE_DIR$" isTestSource="false" />
|
||||||
<excludeFolder url="file://$MODULE_DIR$/.venv" />
|
<excludeFolder url="file://$MODULE_DIR$/.venv" />
|
||||||
<excludeFolder url="file://$MODULE_DIR$/webassets/vendor" />
|
<excludeFolder url="file://$MODULE_DIR$/static/vendor" />
|
||||||
</content>
|
</content>
|
||||||
<orderEntry type="jdk" jdkName="Python 3.13 virtualenv at ~/code/spothole/.venv" jdkType="Python SDK" />
|
<orderEntry type="jdk" jdkName="Python 3.13 virtualenv at ~/code/spothole/.venv" jdkType="Python SDK" />
|
||||||
<orderEntry type="sourceFolder" forTests="false" />
|
<orderEntry type="sourceFolder" forTests="false" />
|
||||||
|
|||||||
@@ -0,0 +1,8 @@
|
|||||||
|
FROM python:3.13-slim
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
COPY . .
|
||||||
|
RUN pip install --no-cache-dir -r requirements.txt
|
||||||
|
EXPOSE 8080
|
||||||
|
|
||||||
|
CMD ["python3", "spothole.py"]
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
# 
|
# 
|
||||||
|
|
||||||
Spothole is a utility to aggregate "spots" from amateur radio DX clusters and xOTA spotting sites, and provide an open
|
Spothole is a utility to aggregate "spots" from amateur radio DX clusters and xOTA spotting sites, and provide an open
|
||||||
JSON API as well as a website to browse the data.
|
JSON API as well as a website to browse the data.
|
||||||
@@ -33,484 +33,17 @@ see [https://spothole.app/apidocs](https://spothole.app/apidocs) for the API det
|
|||||||
This is a Progressive Web App, so you can also "install" it to your Android or iOS device by accessing it in Chrome or
|
This is a Progressive Web App, so you can also "install" it to your Android or iOS device by accessing it in Chrome or
|
||||||
Safari respectively, and following the menu-driven process for installing PWAs.
|
Safari respectively, and following the menu-driven process for installing PWAs.
|
||||||
|
|
||||||
You are more than welcome to use the data and the API that Spothole provides to power your own software. There are many
|
You are more than welcome to use the data and the API that Spothole provides to power your own software, to run your
|
||||||
ways to do this; see below.
|
own Spothole server, or to modify it in any way. More details can be found in the following sections:
|
||||||
|
|
||||||
## Embedding Spothole in another website
|
* [Embedding Spothole in another website](docs/embedding.md)
|
||||||
|
* [Writing your own client](docs/clients.md)
|
||||||
You can embed Spothole's web interface in another website, e.g. for use as part of a ham radio custom dashboard.
|
* [Running your own copy](docs/running.md)
|
||||||
|
* [nginx reverse proxy configuration](docs/nginx.md)
|
||||||
URL parameters can be used to trigger an "embedded" mode which hides the headers, footers and settings. In this mode,
|
* [systemd service file](docs/systemd.md)
|
||||||
you provide configuration for the various filter and display options via additional URL parameters. Any settings that
|
* [Running in Docker](docs/docker.md)
|
||||||
the user has set for Spothole are ignored. This is so that the embedding site can select, for example, their choice of
|
* [Use of multiple cluster logins](docs/multicluster.md)
|
||||||
dark mode or SIG filters, which will not impact how Spothole appears when the user accesses it directly. Effectively, it
|
* [Modifying the source code](docs/modifying.md)
|
||||||
becomes separate to their normal Spothole settings.
|
|
||||||
|
|
||||||
Setting `embedded` to true is important for the rest of the settings to be applied; otherwise, the user's defaults will
|
|
||||||
be used in preference to the URL params.
|
|
||||||
|
|
||||||
These are supplied with the URL to the page you want to embed, for example for an embedded version of the band map in
|
|
||||||
dark mode, use `https://spothole.app/bands?embedded=true&dark-mode=true`. For an embedded version of the main spots/home
|
|
||||||
page in the system light/dark mode, use `https://spothole.app/?embedded=true`. For dark mode showing 70cm TOTA spots
|
|
||||||
only, use `https://spothole.app/?embedded=true&dark-mode=true&sig=TOTA&band=70cm`. Providing no URL params causes the
|
|
||||||
page to be loaded in the normal way it would when accessed directly in the user's browser.
|
|
||||||
|
|
||||||
The supported parameters are as follows. Generally these match the equivalent parameters in the real Spothole API, where
|
|
||||||
a mapping exists.
|
|
||||||
|
|
||||||
| Name | Allowed Values | Default | Example | Description |
|
|
||||||
|------------------|-------------------------|---------|-------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
|
||||||
| `embedded` | `true`, `false` | `false` | `?embedded=true` | Enables embedded mode. |
|
|
||||||
| `color-scheme` | `light`, `dark`, `auto` | `auto` | `?color-scheme=dark` | Forces light or dark mode in preference to the operating system default. |
|
|
||||||
| `time-zone` | `UTC`, `local` | `UTC` | `?time-zone=local` | Sets times to be in UTC or local time. |
|
|
||||||
| `limit` | 10, 25, 50, 100 | 50 | `?limit=50` | Sets the number of spots that will be displayed on the main spots page |
|
|
||||||
| `limit` | 25, 50, 100, 200, 500 | 100 | `?limit=100` | Sets the number of alerts that will be displayed on the alerts page |
|
|
||||||
| `max_age` | 300, 600, 1800, 3600 | 1800 | `?max_age=1800` | Sets the maximum age of spots displayed on the map and bands pages, in seconds. |
|
|
||||||
| `band` | Comma-separated list | (all) | `?band=20m,40m` | Sets the list of bands that will be shown on the spots, bands and map pages. Available options match the labels of the buttons in the standard web interface. |
|
|
||||||
| `sig` | Comma-separated list | (all) | `?sig=POTA,SOTA,NO_SIG` | Sets the list of SIGs that will be shown on the spots, bands and map pages. Available options match the labels of the buttons in the standard web interface. |
|
|
||||||
| `source` | Comma-separated list | (all) | `?source=Cluster` | Sets the list of sources that will be shown on any spot or alert pages. Available options match the labels of the buttons in the standard web interface. |
|
|
||||||
| `mode_type` | Comma-separated list | (all) | `?mode_type=PHONE,CW` | Sets the list of mode types that will be shown on the spots, bands and map pages. Available options match the labels of the buttons in the standard web interface. |
|
|
||||||
| `dx_continent` | Comma-separated list | (all) | `?dx_continent=NA,SA` | Sets the list of DX Continents that will be shown on any spot or alert pages. Available options match the labels of the buttons in the standard web interface. |
|
|
||||||
| `de_continent` | Comma-separated list | (all) | `?de_continent=EU` | Sets the list of DE Continents that will be shown on the spots, bands and map pages. Available options match the labels of the buttons in the standard web interface. |
|
|
||||||
| `map-center-lat` | Numeric (decimal) | (auto) | `?map-center-lat=51.5` | Sets the initial latitude of the map centre on the map page. If omitted, the map auto-fits to the loaded spots. |
|
|
||||||
| `map-center-lon` | Numeric (decimal) | (auto) | `?map-center-lon=-0.1` | Sets the initial longitude of the map centre on the map page. If omitted, the map auto-fits to the loaded spots. |
|
|
||||||
| `map-zoom` | Numeric (integer) | (auto) | `?map-zoom=6` | Sets the initial zoom level of the map on the map page. If omitted, the map auto-fits to the loaded spots. |
|
|
||||||
|
|
||||||
See the comment at the end of the next section regarding reliability and uptime of the "main" server.
|
|
||||||
|
|
||||||
## Writing your own client
|
|
||||||
|
|
||||||
One of the key strengths of Spothole is that the API is well-defined and open to anyone to use. This means you can build
|
|
||||||
your own software that uses data from Spothole.
|
|
||||||
|
|
||||||
As well as the main API endpoints to fetch spots and alerts, with various possible query parameters, there are also
|
|
||||||
Server-Sent Events (SSE) API endpoints to receive a live feed, plus various utility lookup endpoints for things like
|
|
||||||
callsign and park data.
|
|
||||||
|
|
||||||
Various approaches exist to writing your own client, but in general:
|
|
||||||
|
|
||||||
* Refer to the API docs. These are built on an OpenAPI definition file (`/webassets/apidocs/openapi.yml`), which you can
|
|
||||||
automatically use to generate a client skeleton using various software.
|
|
||||||
* Call the main "spots" or "alerts" API endpoints to get the data you want. For example, your app could call
|
|
||||||
`https://spothole.app/api/v1/spots` once every few minutes. Apply filters if necessary.
|
|
||||||
* Call the "options" API to get an idea of which bands, modes etc. the server knows about. You might want to do that
|
|
||||||
first before calling the spots/alerts APIs, to allow you to populate your filters correctly.
|
|
||||||
* Refer to the provided HTML/JS interface for a reference on different approaches. For example, the "alerts"/"upcoming"
|
|
||||||
page simply query the main spot API on a timer, whereas the spots, map and bands pages combine this approach with
|
|
||||||
using the Server-Sent Events (SSE) endpoint to update live.
|
|
||||||
* Let me know if you get stuck, I'm happy to help.
|
|
||||||
|
|
||||||
Please don't hammer the API with an unnecessarily high request rate. For example, Spothole only queries the POTA API
|
|
||||||
once every two minutes, so if your client is interested in POTA data there's no need to poll Spothole any more often
|
|
||||||
than that.
|
|
||||||
|
|
||||||
If you absolutely must be informed within seconds of a spot arriving in Spothole, please use the SSE endpoints instead,
|
|
||||||
e.g. `https://spothole.app/api/v1/spots/stream`.
|
|
||||||
|
|
||||||
If you want to handle different types of spot or alert differently within your client, please consider making a single
|
|
||||||
request to the Spothole API to retrieve all the data, then filtering on your side. For example, call
|
|
||||||
`https://spothole.app/api/v1/spots?sig=POTA,SOTA` rather than making two separate calls to
|
|
||||||
`https://spothole.app/api/v1/spots?sig=POTA` and `https://spothole.app/api/v1/spots?sig=SOTA`.
|
|
||||||
|
|
||||||
Remember, here at Spothole Inc. we offer an industry-standard "five nines" uptime on our server, with our own unique
|
|
||||||
twist: we don't tell you which side of the decimal point the nines start! (Translation: This is a hobby project.
|
|
||||||
`spothole.app` runs on the same server as my blog and other stuff. It might go down without warning. By all means base
|
|
||||||
your own project on data from the main server if you like, but if you want any control over reliability and downtime,
|
|
||||||
please run your own copy instead.)
|
|
||||||
|
|
||||||
## Running your own copy
|
|
||||||
|
|
||||||
If you want to run a copy of Spothole with different configuration settings than the main instance, you can download it
|
|
||||||
and run it on your own local machine or server.
|
|
||||||
|
|
||||||
You will require Python version 3.8 or later. If you encounter an error about `gdal-config` during the following
|
|
||||||
process, you will also need `libgdal-dev` installed.
|
|
||||||
|
|
||||||
To download and set up Spothole on a Debian server, run the following commands. Other operating systems will likely be
|
|
||||||
similar.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git clone ssh://git@git.ianrenton.com/ian/spothole.git
|
|
||||||
cd spothole
|
|
||||||
python3 -m venv ./.venv
|
|
||||||
source .venv/bin/activate
|
|
||||||
pip install -r requirements.txt
|
|
||||||
deactivate
|
|
||||||
cp config-example.yml config.yml
|
|
||||||
```
|
|
||||||
|
|
||||||
Then edit `config.yml` in your text editor of choice to set up the software as you like it. Mostly, this will involve
|
|
||||||
enabling or disabling the various providers of spot and alert data.
|
|
||||||
|
|
||||||
By default, all outdoor programme providers are enabled, as is one cluster node and the NG3K DXpedition data. The RBN
|
|
||||||
spot providers are turned off by default due to the volume of traffic from CW/RTTY/FT8 skimmers, and the APRS and Packet
|
|
||||||
spot providers are off by default on the assumption that Spothole users want a spot with a human at the other end of it,
|
|
||||||
but all can be easily re-enabled.
|
|
||||||
|
|
||||||
Other parameters you will want to update include the base URL to your instance, and whether you want to serve a full
|
|
||||||
web-based DX cluster interface or just the API endpoints for client software to use.
|
|
||||||
|
|
||||||
`config.yml` has an entry for a Clublog API key. If provided, this will allow Spothole to retrieve some more information
|
|
||||||
about DX spots. The software will work just fine without it, but you may find a few country flags etc. are less accurate
|
|
||||||
or missing. Clublog API keys are free, but you'll need to get your own by submitting a helpdesk ticket and explaining
|
|
||||||
what you'll use it for. The admin team are happy with the rate of requests made by my Spothole server, so unless you
|
|
||||||
change the source code of yours to radically increase the rate of querying Clublog, I'm sure they will be fine with your
|
|
||||||
server too.
|
|
||||||
|
|
||||||
Once you're happy with the content of `config.yml`, you can proceed to running the software.
|
|
||||||
|
|
||||||
To run the software this time and any future times you want to run it directly from the command line:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
source .venv/bin/activate
|
|
||||||
python3 spothole.py
|
|
||||||
```
|
|
||||||
|
|
||||||
The software can take a few seconds to start up, mostly because it is downloading an updated file to match callsigns to
|
|
||||||
countries. This is normal, don't panic!
|
|
||||||
|
|
||||||
If you see some errors on startup, check your configuration, e.g. in case you have specified a port for the web server
|
|
||||||
that is already in use by something else.
|
|
||||||
|
|
||||||
### Multiple cluster nodes with different settings
|
|
||||||
|
|
||||||
Dan, S50U has written in with his Spothole cluster settings. He is using a cluster node which provides RBN spots, and
|
|
||||||
uses different SSIDs on his callsign to get different settings when logged into the same cluster node. For example:
|
|
||||||
|
|
||||||
```
|
|
||||||
-
|
|
||||||
class: "DXCluster"
|
|
||||||
name: "S50CLX"
|
|
||||||
enabled: true
|
|
||||||
host: "s50clx.si"
|
|
||||||
port: 41112
|
|
||||||
login_prompt: "login: "
|
|
||||||
login_callsign: "callsign-10"
|
|
||||||
```
|
|
||||||
|
|
||||||
Telnet to DXSpider and log in with "callsign-10" and execute the following commands:
|
|
||||||
|
|
||||||
`CLEAR/SPOTS ALL` (delete all previous filters)<br/>
|
|
||||||
`UNSET/ANN` (stop announce messages)<br/>
|
|
||||||
`UNSET/WCY` (stop wcy messages)<br/>
|
|
||||||
`UNSET/WWV` (stop wwv messages)<br/>
|
|
||||||
`SET/DX` (enable human DX spots)
|
|
||||||
|
|
||||||
```
|
|
||||||
-
|
|
||||||
class: "DXCluster"
|
|
||||||
name: "RBN CW"
|
|
||||||
enabled: true
|
|
||||||
host: "s50clx.si"
|
|
||||||
port: 41112
|
|
||||||
login_prompt: "login: "
|
|
||||||
login_callsign: "callsign-11"
|
|
||||||
allow_rbn_spots: true
|
|
||||||
enabled-by-default-in-web-ui: false
|
|
||||||
```
|
|
||||||
|
|
||||||
Telnet to DXSpider and log in with "callsign-11" and execute the following commands:
|
|
||||||
|
|
||||||
`CLEAR/SPOTS ALL` (delete all previous filters)<br/>
|
|
||||||
`UNSET/ANN` (stop announce messages)<br/>
|
|
||||||
`UNSET/WCY` (stop wcy messages)<br/>
|
|
||||||
`UNSET/WWV` (stop wwv messages)<br/>
|
|
||||||
`UNSET/DX` (stop human DX spots)<br/>
|
|
||||||
`SET/SKIMMER CW` (enable CW RBN spots)
|
|
||||||
|
|
||||||
```
|
|
||||||
-
|
|
||||||
class: "DXCluster"
|
|
||||||
name: "RBN RTTY"
|
|
||||||
enabled: true
|
|
||||||
host: "s50clx.si"
|
|
||||||
port: 41112
|
|
||||||
login_prompt: "login: "
|
|
||||||
login_callsign: "callsign-12"
|
|
||||||
allow_rbn_spots: true
|
|
||||||
enabled-by-default-in-web-ui: false
|
|
||||||
```
|
|
||||||
|
|
||||||
Telnet to DXSpider and log in with "callsign-12" and execute the following commands:
|
|
||||||
|
|
||||||
`CLEAR/SPOTS ALL` (delete all previous filters)<br/>
|
|
||||||
`UNSET/ANN` (stop announce messages)<br/>
|
|
||||||
`UNSET/WCY` (stop wcy messages)<br/>
|
|
||||||
`UNSET/WWV` (stop wwv messages)<br/>
|
|
||||||
`UNSET/DX` (stop human DX spots)<br/>
|
|
||||||
`SET/SKIMMER RTTY` (enable RTTY RBN spots)
|
|
||||||
|
|
||||||
```
|
|
||||||
-
|
|
||||||
class: "DXCluster"
|
|
||||||
name: "RBN FT4/8"
|
|
||||||
enabled: true
|
|
||||||
host: "s50clx.si"
|
|
||||||
port: 41112
|
|
||||||
login_prompt: "login: "
|
|
||||||
login_callsign: "callsign-13"
|
|
||||||
allow_rbn_spots: true
|
|
||||||
enabled-by-default-in-web-ui: false
|
|
||||||
```
|
|
||||||
|
|
||||||
Telnet to DXSpider and log in with "callsign-13" and execute the following commands:
|
|
||||||
|
|
||||||
`CLEAR/SPOTS ALL` (delete all previous filters)<br/>
|
|
||||||
`UNSET/ANN` (stop announce messages)<br/>
|
|
||||||
`UNSET/WCY` (stop wcy messages)<br/>
|
|
||||||
`UNSET/WWV` (stop wwv messages)<br/>
|
|
||||||
`UNSET/DX` (stop human DX spots)<br/>
|
|
||||||
`SET/SKIMMER FT` (enable FT RBN spots)
|
|
||||||
|
|
||||||
For each callsign-SSID, we also specify our basic information with commands:
|
|
||||||
|
|
||||||
`SET/NAME Spothole10`, Spothole11... etc.<br/>
|
|
||||||
`SET/QTH Cerkno`<br/>
|
|
||||||
`SET/QRA JN66XD`<br/>
|
|
||||||
`SET/HOME S50CLX`
|
|
||||||
|
|
||||||
### systemd configuration
|
|
||||||
|
|
||||||
If you want Spothole to run automatically on startup on a Linux distribution that uses `systemd`, follow the
|
|
||||||
instructions here. For distros that don't use `systemd`, or Windows/OSX/etc., you can find generic instructions for your
|
|
||||||
OS online.
|
|
||||||
|
|
||||||
Create a file at `/etc/systemd/system/spothole.service`. Give it the following content, adjusting for the user you want
|
|
||||||
to run it as and the directory in which you have installed it:
|
|
||||||
|
|
||||||
```
|
|
||||||
[Unit]
|
|
||||||
Description=Spothole
|
|
||||||
After=syslog.target network.target
|
|
||||||
|
|
||||||
[Service]
|
|
||||||
Type=simple
|
|
||||||
User=spothole
|
|
||||||
WorkingDirectory=/home/spothole/spothole
|
|
||||||
ExecStart=/home/spothole/spothole/.venv/bin/python /home/spothole/spothole/spothole.py --serve-in-foreground
|
|
||||||
Restart=on-abort
|
|
||||||
|
|
||||||
[Install]
|
|
||||||
WantedBy=multi-user.target
|
|
||||||
```
|
|
||||||
|
|
||||||
Run the following:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo systemctl daemon-reload
|
|
||||||
sudo systemctl enable spothole
|
|
||||||
sudo systemctl start spothole
|
|
||||||
```
|
|
||||||
|
|
||||||
Check the service has started up correctly with `sudo journalctl -u spothole -f`.
|
|
||||||
|
|
||||||
### nginx Reverse Proxy configuration
|
|
||||||
|
|
||||||
Web servers generally serve their pages from port 80. However, it's best not to serve Spothole's web interface directly
|
|
||||||
on port 80, as that requires root privileges on a Linux system. It also and prevents us using HTTPS to serve a secure
|
|
||||||
site, since Spothole itself doesn't directly support acting as an HTTPS server. The normal solution to this is to use
|
|
||||||
a "reverse proxy" setup, where a general web server handles HTTP and HTTP requests (to port 80 & 443 respectively), then
|
|
||||||
passes on the request to the back-end application (in this case Spothole). nginx is a common choice for this general web
|
|
||||||
server.
|
|
||||||
|
|
||||||
To set up nginx as a reverse proxy that sits in front of Spothole, first ensure it's installed e.g.
|
|
||||||
`sudo apt install nginx`, and enabled e.g. `sudo systemd enable nginx`.
|
|
||||||
|
|
||||||
Create a file at `/etc/nginx/sites-available/` called `spothole`. Give it the following contents, replacing
|
|
||||||
`spothole.app` with the domain name on which you want to run Spothole. If you changed the port on which Spothole runs,
|
|
||||||
update that on the "proxy_pass" line too.
|
|
||||||
|
|
||||||
```nginx
|
|
||||||
server {
|
|
||||||
server_name spothole.app;
|
|
||||||
|
|
||||||
# Wellknown area for Lets Encrypt
|
|
||||||
location /.well-known/ {
|
|
||||||
alias /var/www/html/.well-known/;
|
|
||||||
}
|
|
||||||
|
|
||||||
# SSE endpoints
|
|
||||||
location ~ ^/api/v1/(spots|alerts)/stream {
|
|
||||||
proxy_pass http://127.0.0.1:8080;
|
|
||||||
|
|
||||||
# Allow keep-alive
|
|
||||||
proxy_http_version 1.1;
|
|
||||||
proxy_set_header Connection "";
|
|
||||||
|
|
||||||
# Set correct content type for SSE API calls
|
|
||||||
add_header Content-Type text/event-stream always;
|
|
||||||
|
|
||||||
# Set remove buffering, remove caching, add suitable timeouts for SSE API calls
|
|
||||||
proxy_buffering off;
|
|
||||||
proxy_cache off;
|
|
||||||
proxy_read_timeout 24h;
|
|
||||||
proxy_connect_timeout 10s;
|
|
||||||
proxy_send_timeout 24h;
|
|
||||||
proxy_set_header X-Accel-Buffering no;
|
|
||||||
add_header Cache-Control no-store always;
|
|
||||||
|
|
||||||
# Allow cross-origin requests to API
|
|
||||||
proxy_hide_header Access-Control-Allow-Origin;
|
|
||||||
add_header Access-Control-Allow-Origin * always;
|
|
||||||
|
|
||||||
# Pass on IP address and host information to Spothole, in case logging this information is required
|
|
||||||
proxy_set_header X-Real-IP $remote_addr;
|
|
||||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
|
||||||
proxy_set_header Host $http_host;
|
|
||||||
proxy_set_header X-Forwarded-Proto $scheme;
|
|
||||||
}
|
|
||||||
|
|
||||||
# Other API endpoints
|
|
||||||
location /api/ {
|
|
||||||
proxy_pass http://127.0.0.1:8080;
|
|
||||||
|
|
||||||
# Allow keep-alive
|
|
||||||
proxy_http_version 1.1;
|
|
||||||
proxy_set_header Connection "";
|
|
||||||
|
|
||||||
# Set up buffering, remove caching, add suitable timeouts for API calls
|
|
||||||
proxy_buffering on;
|
|
||||||
proxy_cache off;
|
|
||||||
proxy_read_timeout 30s;
|
|
||||||
proxy_connect_timeout 10s;
|
|
||||||
add_header Cache-Control no-store always;
|
|
||||||
|
|
||||||
# Allow cross-origin requests to API
|
|
||||||
proxy_hide_header Access-Control-Allow-Origin;
|
|
||||||
add_header Access-Control-Allow-Origin * always;
|
|
||||||
|
|
||||||
# Pass on IP address and host information to Spothole, in case logging this information is required
|
|
||||||
proxy_set_header X-Real-IP $remote_addr;
|
|
||||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
|
||||||
proxy_set_header Host $http_host;
|
|
||||||
proxy_set_header X-Forwarded-Proto $scheme;
|
|
||||||
}
|
|
||||||
|
|
||||||
# Static assets
|
|
||||||
location / {
|
|
||||||
proxy_pass http://127.0.0.1:8080;
|
|
||||||
|
|
||||||
# Allow keep-alive
|
|
||||||
proxy_http_version 1.1;
|
|
||||||
proxy_set_header Connection "";
|
|
||||||
|
|
||||||
# Set up buffering and caching, add suitable timeouts for static asset requests
|
|
||||||
proxy_buffering on;
|
|
||||||
proxy_read_timeout 30s;
|
|
||||||
proxy_connect_timeout 10s;
|
|
||||||
add_header Cache-Control "public, max-age=3600, must-revalidate" always;
|
|
||||||
|
|
||||||
# Pass on IP address and host information to Spothole, in case logging this information is required
|
|
||||||
proxy_set_header X-Real-IP $remote_addr;
|
|
||||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
|
||||||
proxy_set_header Host $http_host;
|
|
||||||
proxy_set_header X-Forwarded-Proto $scheme;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
One further change you might want to make to the file above is the `add_header Access-Control-Allow-Origin` statements.
|
|
||||||
These are what's used on
|
|
||||||
my own Spothole server to make sure that other third-party web-based software can get the data from my instance, and
|
|
||||||
applies to any endpoint underneath `/api`. If you want
|
|
||||||
*your* Spothole instance to be set up the same way, so that others can write software in JavaScript that can access it,
|
|
||||||
leave this intact. But if you want your Spothole instance to only be usable by scripts running on the web server you
|
|
||||||
write,
|
|
||||||
you can remove these lines. (Note that this doesn't stop other people writing *non-web-based* software that accesses
|
|
||||||
your
|
|
||||||
Spothole API—the enforcement of cross-origin headers only happens within the user's browser. If you need to lock
|
|
||||||
your
|
|
||||||
instance down so that no-one else can access it with *any* software, that's an aspect of nginx or firewall config that
|
|
||||||
you will need
|
|
||||||
to find help with elsewhere.)
|
|
||||||
|
|
||||||
Now, make a symbolic link to enable the site:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd /etc/nginx/sites-enabled
|
|
||||||
sudo ln -sf ../sites-available/spothole
|
|
||||||
```
|
|
||||||
|
|
||||||
Test that your nginx config isn't broken using `nginx -t`. If it works, restart nginx with
|
|
||||||
`sudo systemctl restart nginx`.
|
|
||||||
|
|
||||||
If you haven't already done so, set up a DNS entry to make sure requests for your domain name end up at the server
|
|
||||||
that's running Spothole.
|
|
||||||
|
|
||||||
You should now be able to access the web interface by going to the domain from your browser.
|
|
||||||
|
|
||||||
Once that's working, [install certbot](https://certbot.eff.org/instructions?ws=nginx&os=snap) onto your server. Run it
|
|
||||||
as root, and when prompted pick your domain name from the list. After a few seconds, it should successfully provision a
|
|
||||||
certificate and modify your nginx config files automatically. You should then be able to access the site via HTTPS.
|
|
||||||
|
|
||||||
## Modifying the source code
|
|
||||||
|
|
||||||
Spothole is Public Domain licenced, so you can grab the source code and start modifying it for your own needs.
|
|
||||||
Contributions of code back to the main repository are encouraged, but completely optional.
|
|
||||||
|
|
||||||
### Code structure
|
|
||||||
|
|
||||||
To navigate your way around the source code, this list may help.
|
|
||||||
|
|
||||||
*Python back-end code*
|
|
||||||
|
|
||||||
* `/core` - Core classes and scripts
|
|
||||||
* `/data` - Data storage classes
|
|
||||||
* `/spotproviders` - Classes providing spots by accessing the APIs of other services
|
|
||||||
* `/alertproviders` - Classes providing alerts by accessing the APIs of other services
|
|
||||||
* `/solarconditionsproviders` - Classes providing solar and propagation by accessing the APIs of other services
|
|
||||||
* `/server` - Classes for running Spothole's own web server
|
|
||||||
|
|
||||||
*Templates*
|
|
||||||
|
|
||||||
* `/templates` - Templates used for constructing Spothole's user-targeted HTML pages
|
|
||||||
|
|
||||||
*HTML/JS/CSS front-end code*
|
|
||||||
|
|
||||||
* `/webassets` - Root for static files served by the web server
|
|
||||||
* `/webassets/apidocs` - Contains the OpenAPI spec (`openapi.yml`)
|
|
||||||
* `/webassets/css` - CSS files used by the web front-end
|
|
||||||
* `/webassets/img` - image files used by the web front-end
|
|
||||||
* `/webassets/js` - JavaScript used by the web front-end
|
|
||||||
* `/webassets/vendor` - Third-party libraries (CSS, JS, fonts and images)
|
|
||||||
|
|
||||||
*Miscellaneous*
|
|
||||||
|
|
||||||
* `/` - Main script (`spothole.py`), pip `requirements.txt`, config, README, etc.
|
|
||||||
* `/images` - Image sources
|
|
||||||
* `/datafiles` - Local data sources (differentiated from the majority of data files which are loaded from URLs and
|
|
||||||
cached in `/cache`)
|
|
||||||
* `/cache` - Directory where static-ish data downloaded from the internet is cached to avoid rapid re-requests, and
|
|
||||||
where spot/alert data is cached so that it survives a software restart. Created on first run.
|
|
||||||
|
|
||||||
### Extending the server
|
|
||||||
|
|
||||||
Spothole is designed to be easily extensible. If you want to write your own spot provider, for example, simply add a
|
|
||||||
module to the `spotproviders` package containing your class. (Currently, in order to be loaded correctly, the module (
|
|
||||||
file) name should be the same as the class name, but lower case.)
|
|
||||||
|
|
||||||
Your class should extend "SpotProvider"; if it operates by polling an HTTP Server on a timer, it can instead extend "
|
|
||||||
HTTPSpotProvider" where some of the work is done for you.
|
|
||||||
|
|
||||||
The class will need to implement a constructor that takes in the `provider_config` and provides it to the superclass
|
|
||||||
constructor, while also taking any other config parameters it needs.
|
|
||||||
|
|
||||||
If you're extending the base `SpotProvider` class, you will need to implement `start()` and `stop()` methods that start
|
|
||||||
and stop a separate thread which handles the provider's processing needs. The thread should call `submit()` or
|
|
||||||
`submit_batch()` when it has one or more spots to report.
|
|
||||||
|
|
||||||
If you're extending the `HTTPSpotProvider` class, you will need to provide a URI to query and an interval to the
|
|
||||||
superclass constructor. You'll then need to implement the `http_response_to_spots()` method which is called when new
|
|
||||||
data is retrieved. Your implementation should then call `submit()` or `submit_batch()` when it has one or more spots to
|
|
||||||
report.
|
|
||||||
|
|
||||||
When constructing spots, use the comments in the Spot class and the existing implementations as an example. All
|
|
||||||
parameters are optional, but you will at least want to provide a `time` (which must be timezone-aware) and a `dx_call`.
|
|
||||||
|
|
||||||
Finally, simply add the appropriate config to the `spot_providers` section of `config.yml`, and your provider should be
|
|
||||||
instantiated on startup.
|
|
||||||
|
|
||||||
The same approach as above is also used for alert providers.
|
|
||||||
|
|
||||||
## Thanks
|
## Thanks
|
||||||
|
|
||||||
@@ -524,14 +57,14 @@ my knowledge, created by HA8TKS for his CQ and ITU zone layers for Leaflet. `/da
|
|||||||
maps of conference centres.
|
maps of conference centres.
|
||||||
|
|
||||||
The project contains a set of flag icons generated using the "Noto Color Emoji" font on a Debian system, in the
|
The project contains a set of flag icons generated using the "Noto Color Emoji" font on a Debian system, in the
|
||||||
`/webassets/img/flags/` directory.
|
`/static/img/flags/` directory.
|
||||||
|
|
||||||
The software uses a number of Python libraries as listed in `requirements.txt`, and a number of JavaScript libraries.
|
The software uses a number of Python libraries as listed in `requirements.txt`, and a number of JavaScript libraries.
|
||||||
This project would not have been possible without these libraries, so many thanks to their developers.
|
This project would not have been possible without these libraries, so many thanks to their developers.
|
||||||
|
|
||||||
### Third Party Libraries
|
### Third Party Libraries
|
||||||
|
|
||||||
A number of third-party libraries are self-hosted in the `/webassets/vendor/` directory. These files are subject to
|
A number of third-party libraries are self-hosted in the `/static/vendor/` directory. These files are subject to
|
||||||
their own licences and are not covered by the overall licence declared in the `LICENSE` file.
|
their own licences and are not covered by the overall licence declared in the `LICENSE` file.
|
||||||
|
|
||||||
Particular thanks go to country-files.com for providing country lookup data for amateur radio, to K0SWE
|
Particular thanks go to country-files.com for providing country lookup data for amateur radio, to K0SWE
|
||||||
|
|||||||
@@ -8,10 +8,10 @@ from core.config import MAX_ALERT_AGE
|
|||||||
class AlertProvider:
|
class AlertProvider:
|
||||||
"""Generic alert provider class. Subclasses of this query the individual APIs for alerts."""
|
"""Generic alert provider class. Subclasses of this query the individual APIs for alerts."""
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, name, provider_config):
|
||||||
"""Constructor"""
|
"""Constructor"""
|
||||||
|
|
||||||
self.name = provider_config["name"]
|
self.name = name
|
||||||
self.enabled = provider_config["enabled"]
|
self.enabled = provider_config["enabled"]
|
||||||
self.last_update_time = datetime.min.replace(tzinfo=pytz.UTC)
|
self.last_update_time = datetime.min.replace(tzinfo=pytz.UTC)
|
||||||
self.status = "Not Started" if self.enabled else "Disabled"
|
self.status = "Not Started" if self.enabled else "Disabled"
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ class BOTA(HTTPAlertProvider):
|
|||||||
ALERTS_URL = "https://www.beachesontheair.com/"
|
ALERTS_URL = "https://www.beachesontheair.com/"
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config, self.ALERTS_URL, self.POLL_INTERVAL_SEC)
|
super().__init__("BOTA", provider_config, self.ALERTS_URL, self.POLL_INTERVAL_SEC)
|
||||||
|
|
||||||
def _http_response_to_alerts(self, http_response):
|
def _http_response_to_alerts(self, http_response):
|
||||||
new_alerts = []
|
new_alerts = []
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ from threading import Thread, Event
|
|||||||
|
|
||||||
import pytz
|
import pytz
|
||||||
import requests
|
import requests
|
||||||
|
from requests.exceptions import ConnectionError, ReadTimeout, ConnectTimeout
|
||||||
|
|
||||||
from alertproviders.alert_provider import AlertProvider
|
from alertproviders.alert_provider import AlertProvider
|
||||||
from core.constants import HTTP_HEADERS
|
from core.constants import HTTP_HEADERS
|
||||||
@@ -13,8 +14,8 @@ class HTTPAlertProvider(AlertProvider):
|
|||||||
"""Generic alert provider class for providers that request data via HTTP(S). Just for convenience to avoid code
|
"""Generic alert provider class for providers that request data via HTTP(S). Just for convenience to avoid code
|
||||||
duplication. Subclasses of this query the individual APIs for data."""
|
duplication. Subclasses of this query the individual APIs for data."""
|
||||||
|
|
||||||
def __init__(self, provider_config, url, poll_interval):
|
def __init__(self, name, provider_config, url, poll_interval):
|
||||||
super().__init__(provider_config)
|
super().__init__(name, provider_config)
|
||||||
self._url = url
|
self._url = url
|
||||||
self._poll_interval = poll_interval
|
self._poll_interval = poll_interval
|
||||||
self._thread = None
|
self._thread = None
|
||||||
@@ -58,6 +59,8 @@ class HTTPAlertProvider(AlertProvider):
|
|||||||
|
|
||||||
except ConnectionError:
|
except ConnectionError:
|
||||||
logging.warning(f"Connection error when accessing {self.name} alerts API.")
|
logging.warning(f"Connection error when accessing {self.name} alerts API.")
|
||||||
|
except (ConnectTimeout, ReadTimeout):
|
||||||
|
logging.warning(f"Timeout when accessing {self.name} alerts API.")
|
||||||
except Exception:
|
except Exception:
|
||||||
self.status = "Error"
|
self.status = "Error"
|
||||||
logging.exception("Exception in HTTP JSON Alert Provider (" + self.name + ")")
|
logging.exception("Exception in HTTP JSON Alert Provider (" + self.name + ")")
|
||||||
|
|||||||
@@ -18,7 +18,7 @@ class NG3K(HTTPAlertProvider):
|
|||||||
AS_CALL_PATTERN = re.compile("as ([a-z0-9/]+)", re.IGNORECASE)
|
AS_CALL_PATTERN = re.compile("as ([a-z0-9/]+)", re.IGNORECASE)
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config, self.ALERTS_URL, self.POLL_INTERVAL_SEC)
|
super().__init__("NG3K", provider_config, self.ALERTS_URL, self.POLL_INTERVAL_SEC)
|
||||||
|
|
||||||
def _http_response_to_alerts(self, http_response):
|
def _http_response_to_alerts(self, http_response):
|
||||||
new_alerts = []
|
new_alerts = []
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ class ParksNPeaks(HTTPAlertProvider):
|
|||||||
ALERTS_URL = "https://parksnpeaks.org/api/ALERTS/"
|
ALERTS_URL = "https://parksnpeaks.org/api/ALERTS/"
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config, self.ALERTS_URL, self.POLL_INTERVAL_SEC)
|
super().__init__("ParksNPeaks", provider_config, self.ALERTS_URL, self.POLL_INTERVAL_SEC)
|
||||||
|
|
||||||
def _http_response_to_alerts(self, http_response):
|
def _http_response_to_alerts(self, http_response):
|
||||||
new_alerts = []
|
new_alerts = []
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ class POTA(HTTPAlertProvider):
|
|||||||
ALERTS_URL = "https://api.pota.app/activation"
|
ALERTS_URL = "https://api.pota.app/activation"
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config, self.ALERTS_URL, self.POLL_INTERVAL_SEC)
|
super().__init__("POTA", provider_config, self.ALERTS_URL, self.POLL_INTERVAL_SEC)
|
||||||
|
|
||||||
def _http_response_to_alerts(self, http_response):
|
def _http_response_to_alerts(self, http_response):
|
||||||
new_alerts = []
|
new_alerts = []
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ class SOTA(HTTPAlertProvider):
|
|||||||
ALERTS_URL = "https://api-db2.sota.org.uk/api/alerts/365/all/all"
|
ALERTS_URL = "https://api-db2.sota.org.uk/api/alerts/365/all/all"
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config, self.ALERTS_URL, self.POLL_INTERVAL_SEC)
|
super().__init__("SOTA", provider_config, self.ALERTS_URL, self.POLL_INTERVAL_SEC)
|
||||||
|
|
||||||
def _http_response_to_alerts(self, http_response):
|
def _http_response_to_alerts(self, http_response):
|
||||||
new_alerts = []
|
new_alerts = []
|
||||||
|
|||||||
@@ -18,7 +18,7 @@ class WOTA(HTTPAlertProvider):
|
|||||||
RSS_DATE_TIME_FORMAT = "%a, %d %b %Y %H:%M:%S %z"
|
RSS_DATE_TIME_FORMAT = "%a, %d %b %Y %H:%M:%S %z"
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config, self.ALERTS_URL, self.POLL_INTERVAL_SEC)
|
super().__init__("WOTA", provider_config, self.ALERTS_URL, self.POLL_INTERVAL_SEC)
|
||||||
|
|
||||||
def _http_response_to_alerts(self, http_response):
|
def _http_response_to_alerts(self, http_response):
|
||||||
new_alerts = []
|
new_alerts = []
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ class WWFF(HTTPAlertProvider):
|
|||||||
ALERTS_URL = "https://spots.wwff.co/static/agendas.json"
|
ALERTS_URL = "https://spots.wwff.co/static/agendas.json"
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config, self.ALERTS_URL, self.POLL_INTERVAL_SEC)
|
super().__init__("WWFF", provider_config, self.ALERTS_URL, self.POLL_INTERVAL_SEC)
|
||||||
|
|
||||||
def _http_response_to_alerts(self, http_response):
|
def _http_response_to_alerts(self, http_response):
|
||||||
new_alerts = []
|
new_alerts = []
|
||||||
|
|||||||
@@ -19,66 +19,55 @@ base-url: "http://localhost:8080"
|
|||||||
|
|
||||||
# Spot providers to use. This is an example set, tailor it to your liking by commenting and uncommenting.
|
# Spot providers to use. This is an example set, tailor it to your liking by commenting and uncommenting.
|
||||||
# RBN and APRS-IS are supported but have such a high data rate, you probably don't want them enabled.
|
# RBN and APRS-IS are supported but have such a high data rate, you probably don't want them enabled.
|
||||||
# Each provider needs a class, a name, and an enabled/disabled state. Some require more config such as hostnames/IP
|
# Each provider needs a class and an enabled/disabled state. Some require more config such as hostnames/IP
|
||||||
# addresses and ports. You can duplicate them if you like, e.g. to support several DX clusters. RBN uses two ports, 7000
|
# addresses and ports. You can duplicate them if you like, e.g. to support several DX clusters. RBN uses two ports, 7000
|
||||||
# for CW/RTTY and 7001 for FT8, so if you want both, you need two entries, as shown below.
|
# for CW/RTTY and 7001 for FT8, so if you want both, you need two entries, as shown below.
|
||||||
# Feel free to write your own provider classes! There are details in the README.
|
# Feel free to write your own provider classes! There are details in the README.
|
||||||
spot-providers:
|
spot-providers:
|
||||||
- class: "POTA"
|
- class: "POTA"
|
||||||
name: "POTA"
|
|
||||||
enabled: true
|
enabled: true
|
||||||
|
|
||||||
- class: "SOTA"
|
- class: "SOTA"
|
||||||
name: "SOTA"
|
|
||||||
enabled: true
|
enabled: true
|
||||||
|
|
||||||
- class: "WWFF"
|
- class: "WWFF"
|
||||||
name: "WWFF"
|
|
||||||
enabled: true
|
enabled: true
|
||||||
|
|
||||||
- class: "WWBOTA"
|
- class: "WWBOTA"
|
||||||
name: "WWBOTA"
|
|
||||||
enabled: true
|
enabled: true
|
||||||
|
|
||||||
- class: "GMA"
|
- class: "GMA"
|
||||||
name: "GMA"
|
|
||||||
enabled: true
|
enabled: true
|
||||||
# GMA requires an API key to fetch spots. After creating an account on cqgma.org, email support and request one.
|
# GMA requires an API key to fetch spots. After creating an account on cqgma.org, email support and request one.
|
||||||
api-key: ""
|
api-key: ""
|
||||||
|
|
||||||
- class: "HEMA"
|
- class: "HEMA"
|
||||||
name: "HEMA"
|
|
||||||
enabled: true
|
enabled: true
|
||||||
|
|
||||||
- class: "ParksNPeaks"
|
- class: "ParksNPeaks"
|
||||||
name: "ParksNPeaks"
|
|
||||||
enabled: true
|
enabled: true
|
||||||
|
|
||||||
- class: "ZLOTA"
|
- class: "ZLOTA"
|
||||||
name: "ZLOTA"
|
|
||||||
enabled: true
|
enabled: true
|
||||||
|
|
||||||
- class: "WOTA"
|
- class: "WOTA"
|
||||||
name: "WOTA"
|
|
||||||
enabled: true
|
enabled: true
|
||||||
|
|
||||||
- class: "LLOTA"
|
- class: "LLOTA"
|
||||||
name: "LLOTA"
|
|
||||||
enabled: true
|
enabled: true
|
||||||
|
|
||||||
- class: "Towers"
|
- class: "Towers"
|
||||||
name: "Towers"
|
|
||||||
enabled: true
|
enabled: true
|
||||||
|
|
||||||
- class: "Tiles"
|
- class: "Tiles"
|
||||||
name: "Tiles"
|
|
||||||
enabled: true
|
enabled: true
|
||||||
|
|
||||||
- class: "APRSIS"
|
- class: "APRSIS"
|
||||||
name: "APRS-IS"
|
|
||||||
enabled: false
|
enabled: false
|
||||||
|
|
||||||
- class: "DXCluster"
|
- class: "DXCluster"
|
||||||
|
# Clusters have a "name" property in case you want to say which cluster node the data is from, or connect to
|
||||||
|
# several clusters and name them separately.
|
||||||
name: "HRD Cluster"
|
name: "HRD Cluster"
|
||||||
enabled: true
|
enabled: true
|
||||||
host: "hrd.wa9pie.net"
|
host: "hrd.wa9pie.net"
|
||||||
@@ -99,18 +88,12 @@ spot-providers:
|
|||||||
enabled: false
|
enabled: false
|
||||||
host: "w3lpl.net"
|
host: "w3lpl.net"
|
||||||
port: 7373
|
port: 7373
|
||||||
# Prompt the cluster node gives when asking for a callsign to log in. Varies between cluster node software.
|
|
||||||
login_prompt: "Please enter your call:"
|
login_prompt: "Please enter your call:"
|
||||||
# Callsign Spothole will use to log into this cluster. Ensure the SSID (e.g. -99) is different to any personal
|
|
||||||
# connection you might make to this cluster node.
|
|
||||||
login_callsign: "N0CALL-99"
|
login_callsign: "N0CALL-99"
|
||||||
# Whether to allow RBN spots that come via this cluster. If you don't want RBN spots or you are making a separate
|
|
||||||
# connection to RBN directly, leave this as False. If you want RBN spots from this cluster, set this to True. (Make
|
|
||||||
# sure you aren't also separately connecting to RBN directly, otherwise you may get duplicate spots.) Note that not
|
|
||||||
# all clusters sent RBN spots anyway.
|
|
||||||
allow_rbn_spots: false
|
allow_rbn_spots: false
|
||||||
|
|
||||||
- class: "RBN"
|
- class: "RBN"
|
||||||
|
- # RBN sources have a "name" property in case you want to differentiate between the CW/RTTY and FT8 sources.
|
||||||
name: "RBN CW/RTTY"
|
name: "RBN CW/RTTY"
|
||||||
enabled: false
|
enabled: false
|
||||||
port: 7000
|
port: 7000
|
||||||
@@ -127,11 +110,12 @@ spot-providers:
|
|||||||
enabled-by-default-in-web-ui: false
|
enabled-by-default-in-web-ui: false
|
||||||
|
|
||||||
- class: "UKPacketNet"
|
- class: "UKPacketNet"
|
||||||
name: "UK Packet Radio Net"
|
|
||||||
enabled: false
|
enabled: false
|
||||||
enabled-by-default-in-web-ui: false
|
enabled-by-default-in-web-ui: false
|
||||||
|
|
||||||
- class: "XOTA"
|
- class: "XOTA"
|
||||||
|
# xOTA sources have a "name" property so you can define what programme it is for, since xOTA is generic software
|
||||||
|
# for running "something-on-the-air" programmes.
|
||||||
name: "39C3 TOTA"
|
name: "39C3 TOTA"
|
||||||
enabled: false
|
enabled: false
|
||||||
url: "wss://39c3.totawatch.de/api/spot/live"
|
url: "wss://39c3.totawatch.de/api/spot/live"
|
||||||
@@ -152,31 +136,24 @@ spot-providers:
|
|||||||
# Alert providers to use. Same setup as the spot providers list above.
|
# Alert providers to use. Same setup as the spot providers list above.
|
||||||
alert-providers:
|
alert-providers:
|
||||||
- class: "POTA"
|
- class: "POTA"
|
||||||
name: "POTA"
|
|
||||||
enabled: true
|
enabled: true
|
||||||
|
|
||||||
- class: "SOTA"
|
- class: "SOTA"
|
||||||
name: "SOTA"
|
|
||||||
enabled: true
|
enabled: true
|
||||||
|
|
||||||
- class: "WWFF"
|
- class: "WWFF"
|
||||||
name: "WWFF"
|
|
||||||
enabled: true
|
enabled: true
|
||||||
|
|
||||||
- class: "ParksNPeaks"
|
- class: "ParksNPeaks"
|
||||||
name: "ParksNPeaks"
|
|
||||||
enabled: true
|
enabled: true
|
||||||
|
|
||||||
- class: "WOTA"
|
- class: "WOTA"
|
||||||
name: "WOTA"
|
|
||||||
enabled: true
|
enabled: true
|
||||||
|
|
||||||
- class: "BOTA"
|
- class: "BOTA"
|
||||||
name: "BOTA"
|
|
||||||
enabled: true
|
enabled: true
|
||||||
|
|
||||||
- class: "NG3K"
|
- class: "NG3K"
|
||||||
name: "NG3K"
|
|
||||||
enabled: true
|
enabled: true
|
||||||
|
|
||||||
|
|
||||||
@@ -184,19 +161,15 @@ alert-providers:
|
|||||||
# conditions, etc.) and make it available via the /api/v1/solar endpoint.
|
# conditions, etc.) and make it available via the /api/v1/solar endpoint.
|
||||||
solar-condition-providers:
|
solar-condition-providers:
|
||||||
- class: "HamQSL"
|
- class: "HamQSL"
|
||||||
name: "HamQSL"
|
|
||||||
enabled: true
|
enabled: true
|
||||||
|
|
||||||
- class: "NOAA3dayForecast"
|
- class: "NOAA3dayForecast"
|
||||||
name: "NOAA 3-day Forecast"
|
|
||||||
enabled: true
|
enabled: true
|
||||||
|
|
||||||
- class: "GIROIonosonde"
|
- class: "GIROIonosonde"
|
||||||
name: "GIRO Ionosonde Data"
|
|
||||||
enabled: true
|
enabled: true
|
||||||
|
|
||||||
- class: "KC2GProp"
|
- class: "KC2GProp"
|
||||||
name: "KC2G Propagation Data"
|
|
||||||
enabled: true
|
enabled: true
|
||||||
|
|
||||||
# Maximum time to keep spots and alerts in the system before deleting them. By default, one hour for spots and one week
|
# Maximum time to keep spots and alerts in the system before deleting them. By default, one hour for spots and one week
|
||||||
|
|||||||
@@ -3,7 +3,7 @@ from data.band import Band
|
|||||||
from data.sig import SIG
|
from data.sig import SIG
|
||||||
|
|
||||||
# General software
|
# General software
|
||||||
SOFTWARE_VERSION = "1.4-pre"
|
SOFTWARE_VERSION = "1.5"
|
||||||
|
|
||||||
# HTTP headers used for spot providers that use HTTP
|
# HTTP headers used for spot providers that use HTTP
|
||||||
HTTP_HEADERS = {"User-Agent": "Spothole v" + SOFTWARE_VERSION + " (operated by " + SERVER_OWNER_CALLSIGN + ")"}
|
HTTP_HEADERS = {"User-Agent": "Spothole v" + SOFTWARE_VERSION + " (operated by " + SERVER_OWNER_CALLSIGN + ")"}
|
||||||
|
|||||||
@@ -11,6 +11,7 @@ from pyhamtools import LookupLib, Callinfo, callinfo
|
|||||||
from pyhamtools.exceptions import APIKeyMissingError
|
from pyhamtools.exceptions import APIKeyMissingError
|
||||||
from pyhamtools.frequency import freq_to_band
|
from pyhamtools.frequency import freq_to_band
|
||||||
from pyhamtools.locator import latlong_to_locator
|
from pyhamtools.locator import latlong_to_locator
|
||||||
|
from requests.exceptions import ConnectionError, ReadTimeout, ConnectTimeout
|
||||||
from requests_cache import CachedSession
|
from requests_cache import CachedSession
|
||||||
|
|
||||||
from core.cache_utils import SEMI_STATIC_URL_DATA_CACHE
|
from core.cache_utils import SEMI_STATIC_URL_DATA_CACHE
|
||||||
@@ -155,6 +156,8 @@ class LookupHelper:
|
|||||||
|
|
||||||
except ConnectionError:
|
except ConnectionError:
|
||||||
logging.warning(f"Connection error when downloading Clublog cty.xml.")
|
logging.warning(f"Connection error when downloading Clublog cty.xml.")
|
||||||
|
except (ConnectTimeout, ReadTimeout):
|
||||||
|
logging.warning(f"Timeout when downloading Clublog cty.xml.")
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logging.error("Exception when downloading Clublog cty.xml", e)
|
logging.error("Exception when downloading Clublog cty.xml", e)
|
||||||
return False
|
return False
|
||||||
@@ -179,6 +182,8 @@ class LookupHelper:
|
|||||||
|
|
||||||
except ConnectionError:
|
except ConnectionError:
|
||||||
logging.warning(f"Connection error when downloading dxcc.json.")
|
logging.warning(f"Connection error when downloading dxcc.json.")
|
||||||
|
except (ConnectTimeout, ReadTimeout):
|
||||||
|
logging.warning(f"Timeout when downloading dxcc.json.")
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logging.error("Exception when downloading dxcc.json", e)
|
logging.error("Exception when downloading dxcc.json", e)
|
||||||
return False
|
return False
|
||||||
@@ -501,9 +506,12 @@ class LookupHelper:
|
|||||||
|
|
||||||
# Try the call as given, then fall back to the base call (strips /P, /M etc.)
|
# Try the call as given, then fall back to the base call (strips /P, /M etc.)
|
||||||
calls_to_try = [call]
|
calls_to_try = [call]
|
||||||
home_call = callinfo.Callinfo.get_homecall(call)
|
try:
|
||||||
if home_call != call:
|
home_call = callinfo.Callinfo.get_homecall(call)
|
||||||
calls_to_try.append(home_call)
|
if home_call != call:
|
||||||
|
calls_to_try.append(home_call)
|
||||||
|
except ValueError:
|
||||||
|
logging.debug("Could not look up home call for callsign %s", call)
|
||||||
|
|
||||||
for lookup_call in calls_to_try:
|
for lookup_call in calls_to_try:
|
||||||
try:
|
try:
|
||||||
@@ -533,6 +541,9 @@ class LookupHelper:
|
|||||||
except ConnectionError:
|
except ConnectionError:
|
||||||
logging.warning(f"Connection error when looking up callsign %s using QRZ", lookup_call)
|
logging.warning(f"Connection error when looking up callsign %s using QRZ", lookup_call)
|
||||||
continue
|
continue
|
||||||
|
except (ConnectTimeout, ReadTimeout):
|
||||||
|
logging.warning(f"Timeout when looking up callsign %s using QRZ.", lookup_call)
|
||||||
|
continue
|
||||||
except Exception:
|
except Exception:
|
||||||
logging.error("Exception when looking up callsign %s using QRZ", lookup_call, exc_info=True)
|
logging.error("Exception when looking up callsign %s using QRZ", lookup_call, exc_info=True)
|
||||||
continue
|
continue
|
||||||
@@ -573,9 +584,12 @@ class LookupHelper:
|
|||||||
|
|
||||||
# Try the call as given, then fall back to the base call (strips /P, /M etc.)
|
# Try the call as given, then fall back to the base call (strips /P, /M etc.)
|
||||||
calls_to_try = [call]
|
calls_to_try = [call]
|
||||||
home_call = callinfo.Callinfo.get_homecall(call)
|
try:
|
||||||
if home_call != call:
|
home_call = callinfo.Callinfo.get_homecall(call)
|
||||||
calls_to_try.append(home_call)
|
if home_call != call:
|
||||||
|
calls_to_try.append(home_call)
|
||||||
|
except ValueError:
|
||||||
|
logging.debug("Could not look up home call for callsign %s", call)
|
||||||
|
|
||||||
for lookup_call in calls_to_try:
|
for lookup_call in calls_to_try:
|
||||||
try:
|
try:
|
||||||
@@ -594,6 +608,9 @@ class LookupHelper:
|
|||||||
except ConnectionError:
|
except ConnectionError:
|
||||||
logging.warning(f"Connection error when looking up callsign %s using HamQTH", lookup_call)
|
logging.warning(f"Connection error when looking up callsign %s using HamQTH", lookup_call)
|
||||||
continue
|
continue
|
||||||
|
except (ConnectTimeout, ReadTimeout):
|
||||||
|
logging.warning(f"Timeout when looking up callsign %s using HamQTH", lookup_call)
|
||||||
|
continue
|
||||||
except Exception:
|
except Exception:
|
||||||
logging.error("Exception when looking up callsign %s using HamQTH", lookup_call, exc_info=True)
|
logging.error("Exception when looking up callsign %s using HamQTH", lookup_call, exc_info=True)
|
||||||
continue
|
continue
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ import csv
|
|||||||
import logging
|
import logging
|
||||||
|
|
||||||
from pyhamtools.locator import latlong_to_locator, locator_to_latlong
|
from pyhamtools.locator import latlong_to_locator, locator_to_latlong
|
||||||
|
from requests.exceptions import ConnectionError, ReadTimeout, ConnectTimeout
|
||||||
|
|
||||||
from core.cache_utils import SEMI_STATIC_URL_DATA_CACHE
|
from core.cache_utils import SEMI_STATIC_URL_DATA_CACHE
|
||||||
from core.constants import SIGS, HTTP_HEADERS
|
from core.constants import SIGS, HTTP_HEADERS
|
||||||
@@ -11,6 +12,14 @@ from core.geo_utils import wab_wai_square_to_lat_lon
|
|||||||
# file in Spothole and load it on startup.
|
# file in Spothole and load it on startup.
|
||||||
with open("datafiles/MUNICIPIOS.csv", encoding="latin-1") as _f:
|
with open("datafiles/MUNICIPIOS.csv", encoding="latin-1") as _f:
|
||||||
_DME_INDEX = {row["COD_INE"][:5]: row for row in csv.DictReader(_f, delimiter=";")}
|
_DME_INDEX = {row["COD_INE"][:5]: row for row in csv.DictReader(_f, delimiter=";")}
|
||||||
|
# Caches for data for the SIGs where we have to download a whole global reference list, rather than looking up a single
|
||||||
|
# reference. These get populated from the SEMI_STATIC_URL_DATA_CACHE only if the data actually came
|
||||||
|
# live from the internet, to avoid repopulating them every time we pull the same data from the cache.
|
||||||
|
_WWFF_INDEX_CACHE = {}
|
||||||
|
_SIOTA_INDEX_CACHE = {}
|
||||||
|
_WOTA_INDEX_CACHE = {}
|
||||||
|
_ZLOTA_INDEX_CACHE = {}
|
||||||
|
_LLOTA_INDEX_CACHE = {}
|
||||||
|
|
||||||
|
|
||||||
def get_ref_regex_for_sig(sig):
|
def get_ref_regex_for_sig(sig):
|
||||||
@@ -103,8 +112,18 @@ def populate_sig_ref_info(sig_ref):
|
|||||||
sig_ref.name = data["name"] if "name" in data else None
|
sig_ref.name = data["name"] if "name" in data else None
|
||||||
sig_ref.url = "https://www.cqgma.org/zinfo.php?ref=" + ref_id
|
sig_ref.url = "https://www.cqgma.org/zinfo.php?ref=" + ref_id
|
||||||
sig_ref.grid = data["locator"] if "locator" in data else None
|
sig_ref.grid = data["locator"] if "locator" in data else None
|
||||||
sig_ref.latitude = data["latitude"] if "latitude" in data else None
|
|
||||||
sig_ref.longitude = data["longitude"] if "longitude" in data else None
|
# For some things (just IOTA?) the GMA actually returns a box where "latitude" and "longitude" are
|
||||||
|
# the zeroest corner of the box, then "lat2" and "lng2" provide the other corner. We detect this
|
||||||
|
# and provide a single lat/lon for the centre. Otherwise if we don't have these extra parameters,
|
||||||
|
# just use the single point we have.
|
||||||
|
if data.get("latitude") is not None and data.get("longitude") is not None and data.get(
|
||||||
|
"lat2") is not None and data.get("lng2") is not None:
|
||||||
|
sig_ref.latitude = (float(data["latitude"]) + float(data["lat2"])) / 2.0
|
||||||
|
sig_ref.longitude = (float(data["longitude"]) + float(data["lng2"])) / 2.0
|
||||||
|
else:
|
||||||
|
sig_ref.latitude = float(data["latitude"]) if data.get("latitude") is not None else None
|
||||||
|
sig_ref.longitude = float(data["longitude"]) if data.get("longitude") is not None else None
|
||||||
elif not response.from_cache:
|
elif not response.from_cache:
|
||||||
logging.warning("Malformed response looking up %s ref %s via GMA", sig, ref_id)
|
logging.warning("Malformed response looking up %s ref %s via GMA", sig, ref_id)
|
||||||
elif not response.from_cache:
|
elif not response.from_cache:
|
||||||
@@ -114,8 +133,12 @@ def populate_sig_ref_info(sig_ref):
|
|||||||
response = SEMI_STATIC_URL_DATA_CACHE.get("https://wwff.co/wwff-data/wwff_directory.csv",
|
response = SEMI_STATIC_URL_DATA_CACHE.get("https://wwff.co/wwff-data/wwff_directory.csv",
|
||||||
headers=HTTP_HEADERS)
|
headers=HTTP_HEADERS)
|
||||||
if response.ok:
|
if response.ok:
|
||||||
wwff_index = {row["reference"]: row for row in csv.DictReader(response.content.decode().splitlines())}
|
global _WWFF_INDEX_CACHE
|
||||||
row = wwff_index.get(ref_id)
|
if not bool(_WWFF_INDEX_CACHE) or not response.from_cache:
|
||||||
|
# New data from WWFF, update our internal map
|
||||||
|
_WWFF_INDEX_CACHE = {row["reference"]: row for row in
|
||||||
|
csv.DictReader(response.content.decode().splitlines())}
|
||||||
|
row = _WWFF_INDEX_CACHE.get(ref_id)
|
||||||
if row:
|
if row:
|
||||||
sig_ref.name = row["name"] if "name" in row else None
|
sig_ref.name = row["name"] if "name" in row else None
|
||||||
sig_ref.url = "https://wwff.co/directory/?showRef=" + ref_id
|
sig_ref.url = "https://wwff.co/directory/?showRef=" + ref_id
|
||||||
@@ -132,9 +155,12 @@ def populate_sig_ref_info(sig_ref):
|
|||||||
response = SEMI_STATIC_URL_DATA_CACHE.get("https://www.silosontheair.com/data/silos.csv",
|
response = SEMI_STATIC_URL_DATA_CACHE.get("https://www.silosontheair.com/data/silos.csv",
|
||||||
headers=HTTP_HEADERS)
|
headers=HTTP_HEADERS)
|
||||||
if response.ok:
|
if response.ok:
|
||||||
siota_index = {row["SILO_CODE"]: row for row in
|
global _SIOTA_INDEX_CACHE
|
||||||
csv.DictReader(response.content.decode().splitlines())}
|
if not bool(_SIOTA_INDEX_CACHE) or not response.from_cache:
|
||||||
row = siota_index.get(ref_id)
|
# New data from SIOTA, update our internal map
|
||||||
|
_SIOTA_INDEX_CACHE = {row["SILO_CODE"]: row for row in
|
||||||
|
csv.DictReader(response.content.decode().splitlines())}
|
||||||
|
row = _SIOTA_INDEX_CACHE.get(ref_id)
|
||||||
if row:
|
if row:
|
||||||
sig_ref.name = row["NAME"] if "NAME" in row else None
|
sig_ref.name = row["NAME"] if "NAME" in row else None
|
||||||
sig_ref.grid = row["LOCATOR"] if "LOCATOR" in row else None
|
sig_ref.grid = row["LOCATOR"] if "LOCATOR" in row else None
|
||||||
@@ -151,19 +177,23 @@ def populate_sig_ref_info(sig_ref):
|
|||||||
if response.ok:
|
if response.ok:
|
||||||
data = response.json()
|
data = response.json()
|
||||||
if data:
|
if data:
|
||||||
for feature in data.get("features", []):
|
global _WOTA_INDEX_CACHE
|
||||||
if feature["properties"]["wotaId"] == ref_id:
|
if not bool(_WOTA_INDEX_CACHE) or not response.from_cache:
|
||||||
sig_ref.name = feature["properties"]["title"]
|
# New data from WOTA, update our internal map
|
||||||
# Fudge WOTA URLs. Outlying fell (LDO) URLs don't match their ID numbers but require 214 to be
|
_WOTA_INDEX_CACHE = {feature["properties"]["wotaId"]: feature for feature in
|
||||||
# added to them
|
data.get("features", [])}
|
||||||
sig_ref.url = "https://www.wota.org.uk/MM_" + ref_id
|
feature = _WOTA_INDEX_CACHE.get(ref_id)
|
||||||
if ref_id.upper().startswith("LDO-"):
|
if feature:
|
||||||
number = int(ref_id.upper().replace("LDO-", ""))
|
sig_ref.name = feature["properties"]["title"]
|
||||||
sig_ref.url = "https://www.wota.org.uk/MM_LDO-" + str(number + 214)
|
# Fudge WOTA URLs. Outlying fell (LDO) URLs don't match their ID numbers but require 214 to be
|
||||||
sig_ref.grid = feature["properties"]["qthLocator"]
|
# added to them
|
||||||
sig_ref.latitude = feature["geometry"]["coordinates"][1]
|
sig_ref.url = "https://www.wota.org.uk/MM_" + ref_id
|
||||||
sig_ref.longitude = feature["geometry"]["coordinates"][0]
|
if ref_id.upper().startswith("LDO-"):
|
||||||
break
|
number = int(ref_id.upper().replace("LDO-", ""))
|
||||||
|
sig_ref.url = "https://www.wota.org.uk/MM_LDO-" + str(number + 214)
|
||||||
|
sig_ref.grid = feature["properties"]["qthLocator"]
|
||||||
|
sig_ref.latitude = feature["geometry"]["coordinates"][1]
|
||||||
|
sig_ref.longitude = feature["geometry"]["coordinates"][0]
|
||||||
elif not response.from_cache:
|
elif not response.from_cache:
|
||||||
logging.warning("Malformed response looking up %s ref %s", sig, ref_id)
|
logging.warning("Malformed response looking up %s ref %s", sig, ref_id)
|
||||||
elif not response.from_cache:
|
elif not response.from_cache:
|
||||||
@@ -174,17 +204,20 @@ def populate_sig_ref_info(sig_ref):
|
|||||||
if response.ok:
|
if response.ok:
|
||||||
data = response.json()
|
data = response.json()
|
||||||
if isinstance(data, list):
|
if isinstance(data, list):
|
||||||
for asset in data:
|
global _ZLOTA_INDEX_CACHE
|
||||||
if asset["code"] == ref_id:
|
if not bool(_ZLOTA_INDEX_CACHE) or not response.from_cache:
|
||||||
sig_ref.name = asset["name"]
|
# New data from ZLOTA, update our internal map
|
||||||
sig_ref.url = "https://ontheair.nz/assets/" + ref_id.replace("/", "_")
|
_ZLOTA_INDEX_CACHE = {asset["code"]: asset for asset in data}
|
||||||
try:
|
asset = _ZLOTA_INDEX_CACHE.get(ref_id)
|
||||||
sig_ref.grid = latlong_to_locator(asset["y"], asset["x"], 6)
|
if asset:
|
||||||
except:
|
sig_ref.name = asset["name"]
|
||||||
logging.debug("Invalid lat/lon received for reference")
|
sig_ref.url = "https://ontheair.nz/assets/" + ref_id.replace("/", "_")
|
||||||
sig_ref.latitude = asset["y"]
|
try:
|
||||||
sig_ref.longitude = asset["x"]
|
sig_ref.grid = latlong_to_locator(asset["y"], asset["x"], 6)
|
||||||
break
|
except:
|
||||||
|
logging.debug("Invalid lat/lon received for reference")
|
||||||
|
sig_ref.latitude = asset["y"]
|
||||||
|
sig_ref.longitude = asset["x"]
|
||||||
elif not response.from_cache:
|
elif not response.from_cache:
|
||||||
logging.warning("Malformed response looking up %s ref %s", sig, ref_id)
|
logging.warning("Malformed response looking up %s ref %s", sig, ref_id)
|
||||||
elif not response.from_cache:
|
elif not response.from_cache:
|
||||||
@@ -201,15 +234,18 @@ def populate_sig_ref_info(sig_ref):
|
|||||||
if response.ok:
|
if response.ok:
|
||||||
data = response.json()
|
data = response.json()
|
||||||
if isinstance(data, list):
|
if isinstance(data, list):
|
||||||
for ref in data:
|
global _LLOTA_INDEX_CACHE
|
||||||
if ref["reference_code"] == ref_id:
|
if not bool(_LLOTA_INDEX_CACHE) or not response.from_cache:
|
||||||
sig_ref.name = str(ref["name"])
|
# New data from LLOTA, update our internal map
|
||||||
sig_ref.url = "https://llota.app/list/ref/" + ref_id
|
_LLOTA_INDEX_CACHE = {ref["reference_code"]: ref for ref in data}
|
||||||
sig_ref.grid = str(ref["grid_locator"])
|
ref = _LLOTA_INDEX_CACHE.get(ref_id)
|
||||||
ll = locator_to_latlong(sig_ref.grid)
|
if ref:
|
||||||
sig_ref.latitude = ll[0]
|
sig_ref.name = str(ref["name"])
|
||||||
sig_ref.longitude = ll[1]
|
sig_ref.url = "https://llota.app/list/ref/" + ref_id
|
||||||
break
|
sig_ref.grid = str(ref["grid_locator"])
|
||||||
|
ll = locator_to_latlong(sig_ref.grid)
|
||||||
|
sig_ref.latitude = ll[0]
|
||||||
|
sig_ref.longitude = ll[1]
|
||||||
elif not response.from_cache:
|
elif not response.from_cache:
|
||||||
logging.warning("Malformed response looking up %s ref %s", sig, ref_id)
|
logging.warning("Malformed response looking up %s ref %s", sig, ref_id)
|
||||||
elif not response.from_cache:
|
elif not response.from_cache:
|
||||||
@@ -261,6 +297,8 @@ def populate_sig_ref_info(sig_ref):
|
|||||||
|
|
||||||
except ConnectionError:
|
except ConnectionError:
|
||||||
logging.warning("Connection error when looking up sig_ref info for " + sig + " ref " + ref_id)
|
logging.warning("Connection error when looking up sig_ref info for " + sig + " ref " + ref_id)
|
||||||
|
except (ConnectTimeout, ReadTimeout):
|
||||||
|
logging.warning(f"Timeout when looking up sig_ref info for " + sig + " ref " + ref_id)
|
||||||
except Exception:
|
except Exception:
|
||||||
logging.error("Exception when looking up sig_ref info for " + sig + " ref " + ref_id, exc_info=True)
|
logging.error("Exception when looking up sig_ref info for " + sig + " ref " + ref_id, exc_info=True)
|
||||||
return sig_ref
|
return sig_ref
|
||||||
|
|||||||
@@ -325,6 +325,19 @@ class Spot:
|
|||||||
self.propagation_mode = mode_tag
|
self.propagation_mode = mode_tag
|
||||||
logging.info("Seen a new propagation mode tag not yet in the system: %s", mode_tag)
|
logging.info("Seen a new propagation mode tag not yet in the system: %s", mode_tag)
|
||||||
|
|
||||||
|
# Parse "de_grid -> dx_grid" structures from the comment
|
||||||
|
if self.comment:
|
||||||
|
grid_mode_grid_match = re.search(
|
||||||
|
r'\b([A-Ra-r]{2}\d{2}(?:[A-Xa-x]{2}(?:\d{2})?)?)\s*->\s*([A-Ra-r]{2}\d{2}(?:[A-Xa-x]{2}(?:\d{2})?)?)\b',
|
||||||
|
self.comment)
|
||||||
|
if grid_mode_grid_match:
|
||||||
|
# regex matches, so extract grids:
|
||||||
|
if not self.dx_grid:
|
||||||
|
self.dx_grid = grid_mode_grid_match.group(1).upper()
|
||||||
|
self.dx_location_source = "SPOT"
|
||||||
|
if not self.de_grid:
|
||||||
|
self.de_grid = grid_mode_grid_match.group(2).upper()
|
||||||
|
|
||||||
# DX Grid to lat/lon and vice versa in case one is missing
|
# DX Grid to lat/lon and vice versa in case one is missing
|
||||||
if self.dx_grid and not self.dx_latitude:
|
if self.dx_grid and not self.dx_latitude:
|
||||||
try:
|
try:
|
||||||
|
|||||||
@@ -0,0 +1,39 @@
|
|||||||
|
## Writing your own client
|
||||||
|
|
||||||
|
One of the key strengths of Spothole is that the API is well-defined and open to anyone to use. This means you can build
|
||||||
|
your own software that uses data from Spothole.
|
||||||
|
|
||||||
|
As well as the main API endpoints to fetch spots and alerts, with various possible query parameters, there are also
|
||||||
|
Server-Sent Events (SSE) API endpoints to receive a live feed, plus various utility lookup endpoints for things like
|
||||||
|
callsign and park data.
|
||||||
|
|
||||||
|
Various approaches exist to writing your own client, but in general:
|
||||||
|
|
||||||
|
* Refer to the API docs. These are built on an OpenAPI definition file (`/static/apidocs/openapi.yml`), which you can
|
||||||
|
automatically use to generate a client skeleton using various software.
|
||||||
|
* Call the main "spots" or "alerts" API endpoints to get the data you want. For example, your app could call
|
||||||
|
`https://spothole.app/api/v1/spots` once every few minutes. Apply filters if necessary.
|
||||||
|
* Call the "options" API to get an idea of which bands, modes etc. the server knows about. You might want to do that
|
||||||
|
first before calling the spots/alerts APIs, to allow you to populate your filters correctly.
|
||||||
|
* Refer to the provided HTML/JS interface for a reference on different approaches. For example, the "alerts"/"upcoming"
|
||||||
|
page simply query the main spot API on a timer, whereas the spots, map and bands pages combine this approach with
|
||||||
|
using the Server-Sent Events (SSE) endpoint to update live.
|
||||||
|
* Let me know if you get stuck, I'm happy to help.
|
||||||
|
|
||||||
|
Please don't hammer the API with an unnecessarily high request rate. For example, Spothole only queries the POTA API
|
||||||
|
once every two minutes, so if your client is interested in POTA data there's no need to poll Spothole any more often
|
||||||
|
than that.
|
||||||
|
|
||||||
|
If you absolutely must be informed within seconds of a spot arriving in Spothole, please use the SSE endpoints instead,
|
||||||
|
e.g. `https://spothole.app/api/v1/spots/stream`.
|
||||||
|
|
||||||
|
If you want to handle different types of spot or alert differently within your client, please consider making a single
|
||||||
|
request to the Spothole API to retrieve all the data, then filtering on your side. For example, call
|
||||||
|
`https://spothole.app/api/v1/spots?sig=POTA,SOTA` rather than making two separate calls to
|
||||||
|
`https://spothole.app/api/v1/spots?sig=POTA` and `https://spothole.app/api/v1/spots?sig=SOTA`.
|
||||||
|
|
||||||
|
Remember, here at Spothole Inc. we offer an industry-standard "five nines" uptime on our server, with our own unique
|
||||||
|
twist: we don't tell you which side of the decimal point the nines start! (Translation: This is a hobby project.
|
||||||
|
`spothole.app` runs on the same server as my blog and other stuff. It might go down without warning. By all means base
|
||||||
|
your own project on data from the main server if you like, but if you want any control over reliability and downtime,
|
||||||
|
please run your own copy instead.)
|
||||||
@@ -0,0 +1,150 @@
|
|||||||
|
## Running using Docker
|
||||||
|
|
||||||
|
Spothole comes with a Docker configuration to make it easy to run it in a containerised environment. To set it up using
|
||||||
|
Docker, the easiest way is to use a Docker Compose file. Create a new directory such as `/opt/docker/spothole` and
|
||||||
|
create a `compose.yaml` file inside it with the following contents:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
spothole:
|
||||||
|
container_name: spothole
|
||||||
|
build:
|
||||||
|
context: https://git.ianrenton.com/ian/spothole.git#main
|
||||||
|
restart: unless-stopped
|
||||||
|
ports:
|
||||||
|
- "8080:8080"
|
||||||
|
volumes:
|
||||||
|
- ./config.yml:/app/config.yml
|
||||||
|
- ./cache:/app/cache
|
||||||
|
```
|
||||||
|
|
||||||
|
You can replace `#main` with any other branch or tag reference, for example `#1.5` to pin the build to tagged version
|
||||||
|
1.5.
|
||||||
|
|
||||||
|
Save the file. You will still need to create a copy of `config-example.yml` and name it `config.yml`, though with the
|
||||||
|
Docker setup nothing has actually been downloaded yet, so you will have to copy the example from the repository some
|
||||||
|
other way, e.g. [from the repo in a web browser](https://git.ianrenton.com/ian/spothole/src/branch/main/config-example.yml).
|
||||||
|
|
||||||
|
With that in place, run `docker compose up` and you should be good to go. To detach, press `d` or run the command with
|
||||||
|
the `-d` flag.
|
||||||
|
|
||||||
|
### nginx Reverse Proxy with Docker
|
||||||
|
|
||||||
|
In a containerised setup, it's typical to run an nginx reverse proxy in one container, alongside certbot for renewal
|
||||||
|
of HTTPS certificates, and then applications like Spothole in a separate container. In this case, there are a couple of
|
||||||
|
variations of the docker compose file above, and the nginx reverse proxy configuration covered [here](./nginx.md), that
|
||||||
|
you will want to make.
|
||||||
|
|
||||||
|
1. A port mapping is no longer required in the docker compose file; nginx will access into the docker container directly
|
||||||
|
on e.g. `http://spothole:8080`
|
||||||
|
2. Spothole and nginx will need to be on the same docker network.
|
||||||
|
|
||||||
|
So your `compose.yaml` might look like this:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
spothole:
|
||||||
|
container_name: spothole
|
||||||
|
build:
|
||||||
|
context: https://git.ianrenton.com/ian/spothole.git#main
|
||||||
|
restart: unless-stopped
|
||||||
|
networks:
|
||||||
|
- docker-network
|
||||||
|
volumes:
|
||||||
|
- ./config.yml:/app/config.yml
|
||||||
|
- ./cache:/app/cache
|
||||||
|
|
||||||
|
networks:
|
||||||
|
docker-network:
|
||||||
|
external: true
|
||||||
|
```
|
||||||
|
|
||||||
|
In your nginx site configuration, you'll want to refer to the Spothole container directly, and drop the block that
|
||||||
|
allows nginx to access static files directly, as these will be inaccessible in another container. So you may end up
|
||||||
|
with something like:
|
||||||
|
|
||||||
|
```nginx
|
||||||
|
server {
|
||||||
|
server_name spothole.app;
|
||||||
|
|
||||||
|
# Global proxy settings
|
||||||
|
proxy_http_version 1.1;
|
||||||
|
proxy_set_header Connection "";
|
||||||
|
proxy_connect_timeout 10s;
|
||||||
|
proxy_buffering on;
|
||||||
|
|
||||||
|
# Pass on IP address and host information to Spothole, in case logging this information is required
|
||||||
|
proxy_set_header X-Real-IP $remote_addr;
|
||||||
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
|
proxy_set_header Host $http_host;
|
||||||
|
proxy_set_header X-Forwarded-Proto $scheme;
|
||||||
|
|
||||||
|
# Wellknown area for Lets Encrypt
|
||||||
|
location /.well-known/acme-challenge/ {
|
||||||
|
root /var/www/certbot;
|
||||||
|
}
|
||||||
|
|
||||||
|
# SSE endpoints
|
||||||
|
location ~ ^/api/v1/(spots|alerts)/stream/? {
|
||||||
|
proxy_pass http://spothole:8080;
|
||||||
|
|
||||||
|
# Remove buffering, remove caching, add suitable timeouts for SSE API calls
|
||||||
|
proxy_buffering off;
|
||||||
|
proxy_cache off;
|
||||||
|
proxy_read_timeout 24h;
|
||||||
|
proxy_send_timeout 24h;
|
||||||
|
proxy_set_header X-Accel-Buffering no;
|
||||||
|
add_header Cache-Control no-store always;
|
||||||
|
|
||||||
|
# Allow cross-origin requests to API
|
||||||
|
proxy_hide_header Access-Control-Allow-Origin;
|
||||||
|
add_header Access-Control-Allow-Origin * always;
|
||||||
|
}
|
||||||
|
|
||||||
|
# Other API endpoints
|
||||||
|
location /api/ {
|
||||||
|
proxy_pass http://spothole:8080;
|
||||||
|
|
||||||
|
# Remove buffering, remove caching, add suitable timeouts for API calls
|
||||||
|
proxy_buffering off;
|
||||||
|
proxy_cache off;
|
||||||
|
proxy_read_timeout 30s;
|
||||||
|
add_header Cache-Control no-store always;
|
||||||
|
|
||||||
|
# Allow cross-origin requests to API
|
||||||
|
proxy_hide_header Access-Control-Allow-Origin;
|
||||||
|
add_header Access-Control-Allow-Origin * always;
|
||||||
|
}
|
||||||
|
|
||||||
|
# Templated pages and static assets
|
||||||
|
location / {
|
||||||
|
proxy_pass http://spothole:8080;
|
||||||
|
proxy_read_timeout 30s;
|
||||||
|
add_header Cache-Control "no-cache, must-revalidate" always;
|
||||||
|
}
|
||||||
|
|
||||||
|
listen 443 ssl;
|
||||||
|
listen [::]:443 ssl;
|
||||||
|
|
||||||
|
ssl_certificate /etc/letsencrypt/live/spothole.app/fullchain.pem;
|
||||||
|
ssl_certificate_key /etc/letsencrypt/live/spothole.app/privkey.pem;
|
||||||
|
include /etc/letsencrypt/options-ssl-nginx.conf;
|
||||||
|
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
server {
|
||||||
|
if ($host = spothole.app) {
|
||||||
|
return 301 https://$host$request_uri;
|
||||||
|
}
|
||||||
|
|
||||||
|
server_name spothole.app;
|
||||||
|
listen 80;
|
||||||
|
listen [::]:80;
|
||||||
|
return 404;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
If desired, you could even change the port on which Spothole runs from 8080 to a plain 80, in which case your
|
||||||
|
`proxy_pass` statements could drop the `:8080` suffix. Since Spothole is in a container, it can serve HTTP on port 80
|
||||||
|
if desired, because it doesn't conflict with the host system.
|
||||||
@@ -0,0 +1,41 @@
|
|||||||
|
## Embedding Spothole in another website
|
||||||
|
|
||||||
|
You can embed Spothole's web interface in another website, e.g. for use as part of a ham radio custom dashboard.
|
||||||
|
|
||||||
|
URL parameters can be used to trigger an "embedded" mode which hides the headers, footers and settings. In this mode,
|
||||||
|
you provide configuration for the various filter and display options via additional URL parameters. Any settings that
|
||||||
|
the user has set for Spothole are ignored. This is so that the embedding site can select, for example, their choice of
|
||||||
|
dark mode or SIG filters, which will not impact how Spothole appears when the user accesses it directly. Effectively, it
|
||||||
|
becomes separate to their normal Spothole settings.
|
||||||
|
|
||||||
|
Setting `embedded` to true is important for the rest of the settings to be applied; otherwise, the user's defaults will
|
||||||
|
be used in preference to the URL params.
|
||||||
|
|
||||||
|
These are supplied with the URL to the page you want to embed, for example for an embedded version of the band map in
|
||||||
|
dark mode, use `https://spothole.app/bands?embedded=true&dark-mode=true`. For an embedded version of the main spots/home
|
||||||
|
page in the system light/dark mode, use `https://spothole.app/?embedded=true`. For dark mode showing 70cm TOTA spots
|
||||||
|
only, use `https://spothole.app/?embedded=true&dark-mode=true&sig=TOTA&band=70cm`. Providing no URL params causes the
|
||||||
|
page to be loaded in the normal way it would when accessed directly in the user's browser.
|
||||||
|
|
||||||
|
The supported parameters are as follows. Generally these match the equivalent parameters in the real Spothole API, where
|
||||||
|
a mapping exists.
|
||||||
|
|
||||||
|
| Name | Allowed Values | Default | Example | Description |
|
||||||
|
|------------------|-------------------------|---------|-------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||||
|
| `embedded` | `true`, `false` | `false` | `?embedded=true` | Enables embedded mode. |
|
||||||
|
| `color-scheme` | `light`, `dark`, `auto` | `auto` | `?color-scheme=dark` | Forces light or dark mode in preference to the operating system default. |
|
||||||
|
| `time-zone` | `UTC`, `local` | `UTC` | `?time-zone=local` | Sets times to be in UTC or local time. |
|
||||||
|
| `limit` | 10, 25, 50, 100 | 50 | `?limit=50` | Sets the number of spots that will be displayed on the main spots page |
|
||||||
|
| `limit` | 25, 50, 100, 200, 500 | 100 | `?limit=100` | Sets the number of alerts that will be displayed on the alerts page |
|
||||||
|
| `max_age` | 300, 600, 1800, 3600 | 1800 | `?max_age=1800` | Sets the maximum age of spots displayed on the map and bands pages, in seconds. |
|
||||||
|
| `band` | Comma-separated list | (all) | `?band=20m,40m` | Sets the list of bands that will be shown on the spots, bands and map pages. Available options match the labels of the buttons in the standard web interface. |
|
||||||
|
| `sig` | Comma-separated list | (all) | `?sig=POTA,SOTA,NO_SIG` | Sets the list of SIGs that will be shown on the spots, bands and map pages. Available options match the labels of the buttons in the standard web interface. |
|
||||||
|
| `source` | Comma-separated list | (all) | `?source=Cluster` | Sets the list of sources that will be shown on any spot or alert pages. Available options match the labels of the buttons in the standard web interface. |
|
||||||
|
| `mode_type` | Comma-separated list | (all) | `?mode_type=PHONE,CW` | Sets the list of mode types that will be shown on the spots, bands and map pages. Available options match the labels of the buttons in the standard web interface. |
|
||||||
|
| `dx_continent` | Comma-separated list | (all) | `?dx_continent=NA,SA` | Sets the list of DX Continents that will be shown on any spot or alert pages. Available options match the labels of the buttons in the standard web interface. |
|
||||||
|
| `de_continent` | Comma-separated list | (all) | `?de_continent=EU` | Sets the list of DE Continents that will be shown on the spots, bands and map pages. Available options match the labels of the buttons in the standard web interface. |
|
||||||
|
| `map-center-lat` | Numeric (decimal) | (auto) | `?map-center-lat=51.5` | Sets the initial latitude of the map centre on the map page. If omitted, the map auto-fits to the loaded spots. |
|
||||||
|
| `map-center-lon` | Numeric (decimal) | (auto) | `?map-center-lon=-0.1` | Sets the initial longitude of the map centre on the map page. If omitted, the map auto-fits to the loaded spots. |
|
||||||
|
| `map-zoom` | Numeric (integer) | (auto) | `?map-zoom=6` | Sets the initial zoom level of the map on the map page. If omitted, the map auto-fits to the loaded spots. |
|
||||||
|
|
||||||
|
See the comment at the end of the next section regarding reliability and uptime of the "main" server.
|
||||||
@@ -0,0 +1,69 @@
|
|||||||
|
## Modifying the source code
|
||||||
|
|
||||||
|
Spothole is Public Domain licenced, so you can grab the source code and start modifying it for your own needs.
|
||||||
|
Contributions of code back to the main repository are encouraged, but completely optional.
|
||||||
|
|
||||||
|
### Code structure
|
||||||
|
|
||||||
|
To navigate your way around the source code, this list may help.
|
||||||
|
|
||||||
|
*Python back-end code*
|
||||||
|
|
||||||
|
* `/core` - Core classes and scripts
|
||||||
|
* `/data` - Data storage classes
|
||||||
|
* `/spotproviders` - Classes providing spots by accessing the APIs of other services
|
||||||
|
* `/alertproviders` - Classes providing alerts by accessing the APIs of other services
|
||||||
|
* `/solarconditionsproviders` - Classes providing solar and propagation by accessing the APIs of other services
|
||||||
|
* `/server` - Classes for running Spothole's own web server
|
||||||
|
|
||||||
|
*Templates*
|
||||||
|
|
||||||
|
* `/templates` - Templates used for constructing Spothole's user-targeted HTML pages
|
||||||
|
|
||||||
|
*HTML/JS/CSS front-end code*
|
||||||
|
|
||||||
|
* `/static` - Root for static files served by the web server. These are all served from a path starting `/static/`.
|
||||||
|
* `/static/apidocs` - Contains the OpenAPI spec (`openapi.yml`)
|
||||||
|
* `/static/css` - CSS files used by the web front-end
|
||||||
|
* `/static/img` - image files used by the web front-end
|
||||||
|
* `/static/js` - JavaScript used by the web front-end
|
||||||
|
* `/static/vendor` - Third-party libraries (CSS, JS, fonts and images)
|
||||||
|
|
||||||
|
*Miscellaneous*
|
||||||
|
|
||||||
|
* `/` - Main script (`spothole.py`), pip `requirements.txt`, config, README, etc.
|
||||||
|
* `/docs` - Documentation
|
||||||
|
* `/images` - Image sources
|
||||||
|
* `/datafiles` - Local data sources (differentiated from the majority of data files which are loaded from URLs and
|
||||||
|
cached in `/cache`)
|
||||||
|
* `/cache` - Directory where static-ish data downloaded from the internet is cached to avoid rapid re-requests, and
|
||||||
|
where spot/alert data is cached so that it survives a software restart. Created on first run.
|
||||||
|
|
||||||
|
### Extending the server
|
||||||
|
|
||||||
|
Spothole is designed to be easily extensible. If you want to write your own spot provider, for example, simply add a
|
||||||
|
module to the `spotproviders` package containing your class. (Currently, in order to be loaded correctly, the module (
|
||||||
|
file) name should be the same as the class name, but lower case.)
|
||||||
|
|
||||||
|
Your class should extend "SpotProvider"; if it operates by polling an HTTP Server on a timer, it can instead extend "
|
||||||
|
HTTPSpotProvider" where some of the work is done for you.
|
||||||
|
|
||||||
|
The class will need to implement a constructor that takes in the `provider_config` and provides it to the superclass
|
||||||
|
constructor, while also taking any other config parameters it needs.
|
||||||
|
|
||||||
|
If you're extending the base `SpotProvider` class, you will need to implement `start()` and `stop()` methods that start
|
||||||
|
and stop a separate thread which handles the provider's processing needs. The thread should call `submit()` or
|
||||||
|
`submit_batch()` when it has one or more spots to report.
|
||||||
|
|
||||||
|
If you're extending the `HTTPSpotProvider` class, you will need to provide a URI to query and an interval to the
|
||||||
|
superclass constructor. You'll then need to implement the `http_response_to_spots()` method which is called when new
|
||||||
|
data is retrieved. Your implementation should then call `submit()` or `submit_batch()` when it has one or more spots to
|
||||||
|
report.
|
||||||
|
|
||||||
|
When constructing spots, use the comments in the Spot class and the existing implementations as an example. All
|
||||||
|
parameters are optional, but you will at least want to provide a `time` (which must be timezone-aware) and a `dx_call`.
|
||||||
|
|
||||||
|
Finally, simply add the appropriate config to the `spot_providers` section of `config.yml`, and your provider should be
|
||||||
|
instantiated on startup.
|
||||||
|
|
||||||
|
The same approach as above is also used for alert providers.
|
||||||
@@ -0,0 +1,96 @@
|
|||||||
|
## Multiple cluster nodes with different settings
|
||||||
|
|
||||||
|
Dan, S50U has written in with his Spothole cluster settings. He is using a cluster node which provides RBN spots, and
|
||||||
|
uses different SSIDs on his callsign to get different settings when logged into the same cluster node. For example:
|
||||||
|
|
||||||
|
```
|
||||||
|
-
|
||||||
|
class: "DXCluster"
|
||||||
|
name: "S50CLX"
|
||||||
|
enabled: true
|
||||||
|
host: "s50clx.si"
|
||||||
|
port: 41112
|
||||||
|
login_prompt: "login: "
|
||||||
|
login_callsign: "callsign-10"
|
||||||
|
```
|
||||||
|
|
||||||
|
Telnet to DXSpider and log in with "callsign-10" and execute the following commands:
|
||||||
|
|
||||||
|
`CLEAR/SPOTS ALL` (delete all previous filters)<br/>
|
||||||
|
`UNSET/ANN` (stop announce messages)<br/>
|
||||||
|
`UNSET/WCY` (stop wcy messages)<br/>
|
||||||
|
`UNSET/WWV` (stop wwv messages)<br/>
|
||||||
|
`SET/DX` (enable human DX spots)
|
||||||
|
|
||||||
|
```
|
||||||
|
-
|
||||||
|
class: "DXCluster"
|
||||||
|
name: "RBN CW"
|
||||||
|
enabled: true
|
||||||
|
host: "s50clx.si"
|
||||||
|
port: 41112
|
||||||
|
login_prompt: "login: "
|
||||||
|
login_callsign: "callsign-11"
|
||||||
|
allow_rbn_spots: true
|
||||||
|
enabled-by-default-in-web-ui: false
|
||||||
|
```
|
||||||
|
|
||||||
|
Telnet to DXSpider and log in with "callsign-11" and execute the following commands:
|
||||||
|
|
||||||
|
`CLEAR/SPOTS ALL` (delete all previous filters)<br/>
|
||||||
|
`UNSET/ANN` (stop announce messages)<br/>
|
||||||
|
`UNSET/WCY` (stop wcy messages)<br/>
|
||||||
|
`UNSET/WWV` (stop wwv messages)<br/>
|
||||||
|
`UNSET/DX` (stop human DX spots)<br/>
|
||||||
|
`SET/SKIMMER CW` (enable CW RBN spots)
|
||||||
|
|
||||||
|
```
|
||||||
|
-
|
||||||
|
class: "DXCluster"
|
||||||
|
name: "RBN RTTY"
|
||||||
|
enabled: true
|
||||||
|
host: "s50clx.si"
|
||||||
|
port: 41112
|
||||||
|
login_prompt: "login: "
|
||||||
|
login_callsign: "callsign-12"
|
||||||
|
allow_rbn_spots: true
|
||||||
|
enabled-by-default-in-web-ui: false
|
||||||
|
```
|
||||||
|
|
||||||
|
Telnet to DXSpider and log in with "callsign-12" and execute the following commands:
|
||||||
|
|
||||||
|
`CLEAR/SPOTS ALL` (delete all previous filters)<br/>
|
||||||
|
`UNSET/ANN` (stop announce messages)<br/>
|
||||||
|
`UNSET/WCY` (stop wcy messages)<br/>
|
||||||
|
`UNSET/WWV` (stop wwv messages)<br/>
|
||||||
|
`UNSET/DX` (stop human DX spots)<br/>
|
||||||
|
`SET/SKIMMER RTTY` (enable RTTY RBN spots)
|
||||||
|
|
||||||
|
```
|
||||||
|
-
|
||||||
|
class: "DXCluster"
|
||||||
|
name: "RBN FT4/8"
|
||||||
|
enabled: true
|
||||||
|
host: "s50clx.si"
|
||||||
|
port: 41112
|
||||||
|
login_prompt: "login: "
|
||||||
|
login_callsign: "callsign-13"
|
||||||
|
allow_rbn_spots: true
|
||||||
|
enabled-by-default-in-web-ui: false
|
||||||
|
```
|
||||||
|
|
||||||
|
Telnet to DXSpider and log in with "callsign-13" and execute the following commands:
|
||||||
|
|
||||||
|
`CLEAR/SPOTS ALL` (delete all previous filters)<br/>
|
||||||
|
`UNSET/ANN` (stop announce messages)<br/>
|
||||||
|
`UNSET/WCY` (stop wcy messages)<br/>
|
||||||
|
`UNSET/WWV` (stop wwv messages)<br/>
|
||||||
|
`UNSET/DX` (stop human DX spots)<br/>
|
||||||
|
`SET/SKIMMER FT` (enable FT RBN spots)
|
||||||
|
|
||||||
|
For each callsign-SSID, we also specify our basic information with commands:
|
||||||
|
|
||||||
|
`SET/NAME Spothole10`, Spothole11... etc.<br/>
|
||||||
|
`SET/QTH Cerkno`<br/>
|
||||||
|
`SET/QRA JN66XD`<br/>
|
||||||
|
`SET/HOME S50CLX`
|
||||||
@@ -0,0 +1,123 @@
|
|||||||
|
## nginx Reverse Proxy configuration
|
||||||
|
|
||||||
|
Web servers generally serve their pages from port 80. However, it's best not to serve Spothole's web interface directly
|
||||||
|
on port 80, as that requires root privileges on a Linux system. It also and prevents us using HTTPS to serve a secure
|
||||||
|
site, since Spothole itself doesn't directly support acting as an HTTPS server. The normal solution to this is to use
|
||||||
|
a "reverse proxy" setup, where a general web server handles HTTP and HTTP requests (to port 80 & 443 respectively), then
|
||||||
|
passes on the request to the back-end application (in this case Spothole). nginx is a common choice for this general web
|
||||||
|
server.
|
||||||
|
|
||||||
|
To set up nginx as a reverse proxy that sits in front of Spothole, first ensure it's installed e.g.
|
||||||
|
`sudo apt install nginx`, and enabled e.g. `sudo systemd enable nginx`.
|
||||||
|
|
||||||
|
Create a file at `/etc/nginx/sites-available/` called `spothole`. Give it the following contents, replacing
|
||||||
|
`spothole.app` with the domain name on which you want to run Spothole. If you changed the port on which Spothole runs,
|
||||||
|
update that on the "proxy_pass" line, and if you installed Spothole somewhere other than `/home/spothole/spothole`,
|
||||||
|
adjust the alias location for serving static files.
|
||||||
|
|
||||||
|
(The latter section, configuring the nginx server to serve static files directly, improves efficiency because it saves
|
||||||
|
Spothole itself from serving JS, CSS etc. files. If you can't do this for some reason, e.g. your nginx and spothole are
|
||||||
|
on different computers, you can omit the `location /static/ {}` block.)
|
||||||
|
|
||||||
|
```nginx
|
||||||
|
server {
|
||||||
|
server_name spothole.app;
|
||||||
|
|
||||||
|
# Global proxy settings
|
||||||
|
proxy_http_version 1.1;
|
||||||
|
proxy_set_header Connection "";
|
||||||
|
proxy_connect_timeout 10s;
|
||||||
|
proxy_buffering on;
|
||||||
|
|
||||||
|
# Pass on IP address and host information to Spothole, in case logging this information is required
|
||||||
|
proxy_set_header X-Real-IP $remote_addr;
|
||||||
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
|
proxy_set_header Host $http_host;
|
||||||
|
proxy_set_header X-Forwarded-Proto $scheme;
|
||||||
|
|
||||||
|
# Wellknown area for Lets Encrypt
|
||||||
|
location /.well-known/ {
|
||||||
|
alias /var/www/html/.well-known/;
|
||||||
|
}
|
||||||
|
|
||||||
|
# Load static assets directly from the Spothole static directory
|
||||||
|
location /static/ {
|
||||||
|
alias /home/spothole/spothole/static/;
|
||||||
|
expires 1h;
|
||||||
|
add_header Cache-Control "public, max-age=3600, must-revalidate";
|
||||||
|
}
|
||||||
|
|
||||||
|
# SSE endpoints
|
||||||
|
location ~ ^/api/v1/(spots|alerts)/stream/? {
|
||||||
|
proxy_pass http://127.0.0.1:8080;
|
||||||
|
|
||||||
|
# Remove buffering, remove caching, add suitable timeouts for SSE API calls
|
||||||
|
proxy_buffering off;
|
||||||
|
proxy_cache off;
|
||||||
|
proxy_read_timeout 24h;
|
||||||
|
proxy_send_timeout 24h;
|
||||||
|
proxy_set_header X-Accel-Buffering no;
|
||||||
|
add_header Cache-Control no-store always;
|
||||||
|
|
||||||
|
# Allow cross-origin requests to API
|
||||||
|
proxy_hide_header Access-Control-Allow-Origin;
|
||||||
|
add_header Access-Control-Allow-Origin * always;
|
||||||
|
}
|
||||||
|
|
||||||
|
# Other API endpoints
|
||||||
|
location /api/ {
|
||||||
|
proxy_pass http://127.0.0.1:8080;
|
||||||
|
|
||||||
|
# Remove buffering, remove caching, add suitable timeouts for API calls
|
||||||
|
proxy_buffering off;
|
||||||
|
proxy_cache off;
|
||||||
|
proxy_read_timeout 30s;
|
||||||
|
add_header Cache-Control no-store always;
|
||||||
|
|
||||||
|
# Allow cross-origin requests to API
|
||||||
|
proxy_hide_header Access-Control-Allow-Origin;
|
||||||
|
add_header Access-Control-Allow-Origin * always;
|
||||||
|
}
|
||||||
|
|
||||||
|
# Templated pages
|
||||||
|
location / {
|
||||||
|
proxy_pass http://127.0.0.1:8080;
|
||||||
|
proxy_read_timeout 30s;
|
||||||
|
add_header Cache-Control "no-cache, must-revalidate" always;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
One further change you might want to make to the file above is the `add_header Access-Control-Allow-Origin` statements.
|
||||||
|
These are what's used on
|
||||||
|
my own Spothole server to make sure that other third-party web-based software can get the data from my instance, and
|
||||||
|
applies to any endpoint underneath `/api`. If you want
|
||||||
|
*your* Spothole instance to be set up the same way, so that others can write software in JavaScript that can access it,
|
||||||
|
leave this intact. But if you want your Spothole instance to only be usable by scripts running on the web server you
|
||||||
|
write,
|
||||||
|
you can remove these lines. (Note that this doesn't stop other people writing *non-web-based* software that accesses
|
||||||
|
your
|
||||||
|
Spothole API—the enforcement of cross-origin headers only happens within the user's browser. If you need to lock
|
||||||
|
your
|
||||||
|
instance down so that no-one else can access it with *any* software, that's an aspect of nginx or firewall config that
|
||||||
|
you will need
|
||||||
|
to find help with elsewhere.)
|
||||||
|
|
||||||
|
Now, make a symbolic link to enable the site:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /etc/nginx/sites-enabled
|
||||||
|
sudo ln -sf ../sites-available/spothole
|
||||||
|
```
|
||||||
|
|
||||||
|
Test that your nginx config isn't broken using `nginx -t`. If it works, restart nginx with
|
||||||
|
`sudo systemctl restart nginx`.
|
||||||
|
|
||||||
|
If you haven't already done so, set up a DNS entry to make sure requests for your domain name end up at the server
|
||||||
|
that's running Spothole.
|
||||||
|
|
||||||
|
You should now be able to access the web interface by going to the domain from your browser.
|
||||||
|
|
||||||
|
Once that's working, [install certbot](https://certbot.eff.org/instructions?ws=nginx&os=snap) onto your server. Run it
|
||||||
|
as root, and when prompted pick your domain name from the list. After a few seconds, it should successfully provision a
|
||||||
|
certificate and modify your nginx config files automatically. You should then be able to access the site via HTTPS.
|
||||||
@@ -0,0 +1,53 @@
|
|||||||
|
## Running your own copy
|
||||||
|
|
||||||
|
If you want to run a copy of Spothole with different configuration settings than the main instance, you can download it
|
||||||
|
and run it on your own local machine or server.
|
||||||
|
|
||||||
|
You will require Python version 3.8 or later. If you encounter an error about `gdal-config` during the following
|
||||||
|
process, you will also need `libgdal-dev` installed.
|
||||||
|
|
||||||
|
To download and set up Spothole on a Debian server, run the following commands. Other operating systems will likely be
|
||||||
|
similar.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone ssh://git@git.ianrenton.com/ian/spothole.git
|
||||||
|
cd spothole
|
||||||
|
python3 -m venv ./.venv
|
||||||
|
source .venv/bin/activate
|
||||||
|
pip install -r requirements.txt
|
||||||
|
deactivate
|
||||||
|
cp config-example.yml config.yml
|
||||||
|
```
|
||||||
|
|
||||||
|
Then edit `config.yml` in your text editor of choice to set up the software as you like it. Mostly, this will involve
|
||||||
|
enabling or disabling the various providers of spot and alert data.
|
||||||
|
|
||||||
|
By default, all outdoor programme providers are enabled, as is one cluster node and the NG3K DXpedition data. The RBN
|
||||||
|
spot providers are turned off by default due to the volume of traffic from CW/RTTY/FT8 skimmers, and the APRS and Packet
|
||||||
|
spot providers are off by default on the assumption that Spothole users want a spot with a human at the other end of it,
|
||||||
|
but all can be easily re-enabled.
|
||||||
|
|
||||||
|
Other parameters you will want to update include the base URL to your instance, and whether you want to serve a full
|
||||||
|
web-based DX cluster interface or just the API endpoints for client software to use.
|
||||||
|
|
||||||
|
`config.yml` has an entry for a Clublog API key. If provided, this will allow Spothole to retrieve some more information
|
||||||
|
about DX spots. The software will work just fine without it, but you may find a few country flags etc. are less accurate
|
||||||
|
or missing. Clublog API keys are free, but you'll need to get your own by submitting a helpdesk ticket and explaining
|
||||||
|
what you'll use it for. The admin team are happy with the rate of requests made by my Spothole server, so unless you
|
||||||
|
change the source code of yours to radically increase the rate of querying Clublog, I'm sure they will be fine with your
|
||||||
|
server too.
|
||||||
|
|
||||||
|
Once you're happy with the content of `config.yml`, you can proceed to running the software.
|
||||||
|
|
||||||
|
To run the software this time and any future times you want to run it directly from the command line:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
source .venv/bin/activate
|
||||||
|
python3 spothole.py
|
||||||
|
```
|
||||||
|
|
||||||
|
The software can take a few seconds to start up, mostly because it is downloading an updated file to match callsigns to
|
||||||
|
countries. This is normal, don't panic!
|
||||||
|
|
||||||
|
If you see some errors on startup, check your configuration, e.g. in case you have specified a port for the web server
|
||||||
|
that is already in use by something else.
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
## systemd configuration
|
||||||
|
|
||||||
|
If you want Spothole to run automatically on startup on a Linux distribution that uses `systemd`, follow the
|
||||||
|
instructions here. For distros that don't use `systemd`, or Windows/OSX/etc., you can find generic instructions for your
|
||||||
|
OS online.
|
||||||
|
|
||||||
|
Create a file at `/etc/systemd/system/spothole.service`. Give it the following content, adjusting for the user you want
|
||||||
|
to run it as and the directory in which you have installed it:
|
||||||
|
|
||||||
|
```
|
||||||
|
[Unit]
|
||||||
|
Description=Spothole
|
||||||
|
After=syslog.target network.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
User=spothole
|
||||||
|
WorkingDirectory=/home/spothole/spothole
|
||||||
|
ExecStart=/home/spothole/spothole/.venv/bin/python /home/spothole/spothole/spothole.py --serve-in-foreground
|
||||||
|
Restart=on-abort
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
```
|
||||||
|
|
||||||
|
Run the following:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo systemctl daemon-reload
|
||||||
|
sudo systemctl enable spothole
|
||||||
|
sudo systemctl start spothole
|
||||||
|
```
|
||||||
|
|
||||||
|
Check the service has started up correctly with `sudo journalctl -u spothole -f`.
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
import tornado.web
|
||||||
|
|
||||||
|
from core.config import BASE_URL
|
||||||
|
|
||||||
|
|
||||||
|
class ManifestHandler(tornado.web.RequestHandler):
|
||||||
|
"""Handler for manifest.webmanifest, which needs BASE_URL inserted and a custom content-type"""
|
||||||
|
|
||||||
|
def get(self):
|
||||||
|
self.set_header("Content-Type", "application/manifest+json; charset=UTF-8")
|
||||||
|
self.render("manifest.webmanifest", baseurl=BASE_URL)
|
||||||
@@ -15,6 +15,7 @@ from server.handlers.api.options import APIOptionsHandler
|
|||||||
from server.handlers.api.solar_conditions import APISolarConditionsHandler
|
from server.handlers.api.solar_conditions import APISolarConditionsHandler
|
||||||
from server.handlers.api.spots import APISpotsHandler, APISpotsStreamHandler
|
from server.handlers.api.spots import APISpotsHandler, APISpotsStreamHandler
|
||||||
from server.handlers.api.status import APIStatusHandler
|
from server.handlers.api.status import APIStatusHandler
|
||||||
|
from server.handlers.manifesthandler import ManifestHandler
|
||||||
from server.handlers.metrics import PrometheusMetricsHandler
|
from server.handlers.metrics import PrometheusMetricsHandler
|
||||||
from server.handlers.pagetemplate import PageTemplateHandler
|
from server.handlers.pagetemplate import PageTemplateHandler
|
||||||
|
|
||||||
@@ -99,11 +100,16 @@ class WebServer:
|
|||||||
if ALLOW_SPOTTING:
|
if ALLOW_SPOTTING:
|
||||||
ui_routes += [(r"/add-spot", PageTemplateHandler, {"template_name": "add_spot", **handler_opts})]
|
ui_routes += [(r"/add-spot", PageTemplateHandler, {"template_name": "add_spot", **handler_opts})]
|
||||||
|
|
||||||
# API docs, Prometheus metrics, and finally static assets are always available regardless of API-only mode.
|
# API docs, Prometheus metrics, webapp manifest and static assets are always available regardless of API-only
|
||||||
|
# mode.
|
||||||
misc_routes = [
|
misc_routes = [
|
||||||
(r"/apidocs", PageTemplateHandler, {"template_name": "apidocs", **handler_opts}),
|
(r"/apidocs", PageTemplateHandler, {"template_name": "apidocs", **handler_opts}),
|
||||||
(r"/metrics", PrometheusMetricsHandler),
|
(r"/metrics", PrometheusMetricsHandler),
|
||||||
(r"/(.*)", StaticFileHandler, {"path": os.path.join(_HERE, "../webassets")})
|
(r"/manifest.webmanifest", ManifestHandler),
|
||||||
|
# If e.g. nginx is configured as a reverse proxy with a hard-coded path to static files, as per the README,
|
||||||
|
# this will never have to handle anything, but having it here allows Spothole to work without nginx for
|
||||||
|
# testing.
|
||||||
|
(r"/static/(.*)", StaticFileHandler, {"path": os.path.join(_HERE, "../static")})
|
||||||
]
|
]
|
||||||
|
|
||||||
app = tornado.web.Application(api_routes + ui_routes + misc_routes,
|
app = tornado.web.Application(api_routes + ui_routes + misc_routes,
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ from threading import Thread, Event
|
|||||||
|
|
||||||
import pytz
|
import pytz
|
||||||
import requests
|
import requests
|
||||||
|
from requests.exceptions import ConnectionError, ReadTimeout, ConnectTimeout
|
||||||
|
|
||||||
from core.constants import HTTP_HEADERS
|
from core.constants import HTTP_HEADERS
|
||||||
from solarconditionsproviders.ionosonde_utils import compute_band_states
|
from solarconditionsproviders.ionosonde_utils import compute_band_states
|
||||||
@@ -29,7 +30,7 @@ class GIROIonosonde(SolarConditionsProvider):
|
|||||||
data, but is less reliable and often offline."""
|
data, but is less reliable and often offline."""
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config)
|
super().__init__("GIRO Ionosonde Data", provider_config)
|
||||||
self._stations = self._load_stations()
|
self._stations = self._load_stations()
|
||||||
self._thread = None
|
self._thread = None
|
||||||
self._stop_event = Event()
|
self._stop_event = Event()
|
||||||
@@ -133,6 +134,9 @@ class GIROIonosonde(SolarConditionsProvider):
|
|||||||
logging.warning(f"HTTP {http_response.status_code} when calling Giro ionosonde API.")
|
logging.warning(f"HTTP {http_response.status_code} when calling Giro ionosonde API.")
|
||||||
return None, None, None
|
return None, None, None
|
||||||
return self._parse_all(http_response.text)
|
return self._parse_all(http_response.text)
|
||||||
|
except (ConnectTimeout, ReadTimeout):
|
||||||
|
logging.warning(f"Timeout when accessing Giro ionosonde API.")
|
||||||
|
return None, None, None
|
||||||
except ConnectionError:
|
except ConnectionError:
|
||||||
logging.warning("Connection error when accessing Giro ionosonde API.")
|
logging.warning("Connection error when accessing Giro ionosonde API.")
|
||||||
return None, None, None
|
return None, None, None
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ class HamQSL(HTTPSolarConditionsProvider):
|
|||||||
Provides solar flux index, geomagnetic indices, and HF/VHF propagation condition summaries."""
|
Provides solar flux index, geomagnetic indices, and HF/VHF propagation condition summaries."""
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config, URL, POLL_INTERVAL)
|
super().__init__("HamQSL", provider_config, URL, POLL_INTERVAL)
|
||||||
|
|
||||||
def _http_response_to_solar_conditions(self, http_response):
|
def _http_response_to_solar_conditions(self, http_response):
|
||||||
root = ElementTree.fromstring(http_response.text)
|
root = ElementTree.fromstring(http_response.text)
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ from threading import Thread, Event
|
|||||||
|
|
||||||
import pytz
|
import pytz
|
||||||
import requests
|
import requests
|
||||||
|
from requests.exceptions import ConnectionError, ReadTimeout, ConnectTimeout
|
||||||
|
|
||||||
from core.constants import HTTP_HEADERS
|
from core.constants import HTTP_HEADERS
|
||||||
from solarconditionsproviders.solar_conditions_provider import SolarConditionsProvider
|
from solarconditionsproviders.solar_conditions_provider import SolarConditionsProvider
|
||||||
@@ -13,8 +14,8 @@ class HTTPSolarConditionsProvider(SolarConditionsProvider):
|
|||||||
"""Generic solar conditions provider for providers that request data via HTTP(S). Subclasses implement
|
"""Generic solar conditions provider for providers that request data via HTTP(S). Subclasses implement
|
||||||
_http_response_to_solar_conditions() to parse the specific API response format."""
|
_http_response_to_solar_conditions() to parse the specific API response format."""
|
||||||
|
|
||||||
def __init__(self, provider_config, url, poll_interval):
|
def __init__(self, name, provider_config, url, poll_interval):
|
||||||
super().__init__(provider_config)
|
super().__init__(name, provider_config)
|
||||||
self._url = url
|
self._url = url
|
||||||
self._poll_interval = poll_interval
|
self._poll_interval = poll_interval
|
||||||
self._thread = None
|
self._thread = None
|
||||||
@@ -53,6 +54,8 @@ class HTTPSolarConditionsProvider(SolarConditionsProvider):
|
|||||||
|
|
||||||
except ConnectionError:
|
except ConnectionError:
|
||||||
logging.warning(f"Connection error when accessing {self.name} solar conditions API.")
|
logging.warning(f"Connection error when accessing {self.name} solar conditions API.")
|
||||||
|
except (ConnectTimeout, ReadTimeout):
|
||||||
|
logging.warning(f"Timeout when accessing {self.name} solar conditions API.")
|
||||||
except Exception:
|
except Exception:
|
||||||
self.status = "Error"
|
self.status = "Error"
|
||||||
logging.exception("Exception in HTTP Solar Conditions Provider (" + self.name + ")")
|
logging.exception("Exception in HTTP Solar Conditions Provider (" + self.name + ")")
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ from threading import Thread, Event
|
|||||||
|
|
||||||
import pytz
|
import pytz
|
||||||
import requests
|
import requests
|
||||||
|
from requests.exceptions import ConnectionError, ReadTimeout, ConnectTimeout
|
||||||
|
|
||||||
from core.constants import HTTP_HEADERS
|
from core.constants import HTTP_HEADERS
|
||||||
from solarconditionsproviders.ionosonde_utils import compute_band_states
|
from solarconditionsproviders.ionosonde_utils import compute_band_states
|
||||||
@@ -23,7 +24,7 @@ class KC2GProp(SolarConditionsProvider):
|
|||||||
online, but has fewer stations and does not provide LUF data."""
|
online, but has fewer stations and does not provide LUF data."""
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config)
|
super().__init__("KC2G Propagation Data", provider_config)
|
||||||
self._thread = None
|
self._thread = None
|
||||||
self._stop_event = Event()
|
self._stop_event = Event()
|
||||||
|
|
||||||
@@ -117,6 +118,8 @@ class KC2GProp(SolarConditionsProvider):
|
|||||||
|
|
||||||
except ConnectionError:
|
except ConnectionError:
|
||||||
logging.warning("Connection error when accessing KC2G ionosonde API.")
|
logging.warning("Connection error when accessing KC2G ionosonde API.")
|
||||||
|
except (ConnectTimeout, ReadTimeout):
|
||||||
|
logging.warning(f"Timeout when accessing KC2G ionosonde API.")
|
||||||
except Exception:
|
except Exception:
|
||||||
self.status = "Error"
|
self.status = "Error"
|
||||||
logging.exception("Exception in KC2G ionosonde data provider")
|
logging.exception("Exception in KC2G ionosonde data provider")
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ class NOAA3dayForecast(HTTPSolarConditionsProvider):
|
|||||||
corresponding fields in the solar conditions object.."""
|
corresponding fields in the solar conditions object.."""
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config, URL, POLL_INTERVAL)
|
super().__init__("NOAA 3-day Forecast", provider_config, URL, POLL_INTERVAL)
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _parse_percentage_table(lines, section_header, year):
|
def _parse_percentage_table(lines, section_header, year):
|
||||||
|
|||||||
@@ -7,11 +7,11 @@ class SolarConditionsProvider:
|
|||||||
"""Generic solar conditions provider class. Subclasses of this query individual APIs for space weather and
|
"""Generic solar conditions provider class. Subclasses of this query individual APIs for space weather and
|
||||||
propagation data."""
|
propagation data."""
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, name, provider_config):
|
||||||
"""Constructor"""
|
"""Constructor"""
|
||||||
|
|
||||||
self._solar_conditions_cache = None
|
self._solar_conditions_cache = None
|
||||||
self.name = provider_config["name"]
|
self.name = name
|
||||||
self.enabled = provider_config["enabled"]
|
self.enabled = provider_config["enabled"]
|
||||||
self.last_update_time = datetime.min.replace(tzinfo=pytz.UTC)
|
self.last_update_time = datetime.min.replace(tzinfo=pytz.UTC)
|
||||||
self.status = "Not Started" if self.enabled else "Disabled"
|
self.status = "Not Started" if self.enabled else "Disabled"
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ class APRSIS(SpotProvider):
|
|||||||
"""Spot provider for the APRS-IS."""
|
"""Spot provider for the APRS-IS."""
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config)
|
super().__init__("APRS-IS", provider_config)
|
||||||
self._thread = Thread(target=self._connect)
|
self._thread = Thread(target=self._connect)
|
||||||
self._thread.daemon = True
|
self._thread.daemon = True
|
||||||
self._aprsis = None
|
self._aprsis = None
|
||||||
|
|||||||
@@ -26,7 +26,8 @@ class DXCluster(SpotProvider):
|
|||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
"""Constructor requires hostname and port"""
|
"""Constructor requires hostname and port"""
|
||||||
|
|
||||||
super().__init__(provider_config)
|
name = provider_config["name"] if "name" in provider_config else "Cluster"
|
||||||
|
super().__init__(name, provider_config)
|
||||||
self._hostname = provider_config["host"]
|
self._hostname = provider_config["host"]
|
||||||
self._port = provider_config["port"]
|
self._port = provider_config["port"]
|
||||||
self._login_prompt = provider_config["login_prompt"] if "login_prompt" in provider_config else "login:"
|
self._login_prompt = provider_config["login_prompt"] if "login_prompt" in provider_config else "login:"
|
||||||
@@ -59,6 +60,10 @@ class DXCluster(SpotProvider):
|
|||||||
self._telnet.write((self._login_callsign + "\n").encode("latin-1"))
|
self._telnet.write((self._login_callsign + "\n").encode("latin-1"))
|
||||||
connected = True
|
connected = True
|
||||||
logging.info("DX Cluster " + self._hostname + " connected.")
|
logging.info("DX Cluster " + self._hostname + " connected.")
|
||||||
|
except ConnectionRefusedError:
|
||||||
|
self.status = "Error"
|
||||||
|
logging.warning("Connection refused to DX cluster " + self._hostname)
|
||||||
|
sleep(300)
|
||||||
except Exception:
|
except Exception:
|
||||||
self.status = "Error"
|
self.status = "Error"
|
||||||
logging.exception("Exception while connecting to DX Cluster Provider (" + self._hostname + ").")
|
logging.exception("Exception while connecting to DX Cluster Provider (" + self._hostname + ").")
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ class GMA(HTTPSpotProvider):
|
|||||||
provider_config["enabled"] = False
|
provider_config["enabled"] = False
|
||||||
logging.warning("GMA spot provider configured but no api key was provided, this API will not be queried.")
|
logging.warning("GMA spot provider configured but no api key was provided, this API will not be queried.")
|
||||||
|
|
||||||
super().__init__(provider_config, self.SPOTS_URL + "?key=" + self.api_key, self.POLL_INTERVAL_SEC)
|
super().__init__("GMA", provider_config, self.SPOTS_URL + "?key=" + self.api_key, self.POLL_INTERVAL_SEC)
|
||||||
|
|
||||||
def _http_response_to_spots(self, http_response):
|
def _http_response_to_spots(self, http_response):
|
||||||
new_spots = []
|
new_spots = []
|
||||||
@@ -34,6 +34,10 @@ class GMA(HTTPSpotProvider):
|
|||||||
if "RCD" in http_response.json():
|
if "RCD" in http_response.json():
|
||||||
for source_spot in http_response.json()["RCD"]:
|
for source_spot in http_response.json()["RCD"]:
|
||||||
# Convert to our spot format
|
# Convert to our spot format
|
||||||
|
# Seen GMA spots with no (or empty) lat/lon
|
||||||
|
lat = float(source_spot["LAT"]) if (source_spot["LAT"] and source_spot["LAT"] != "") else None
|
||||||
|
lon = float(source_spot["LON"]) if (source_spot["LON"] and source_spot["LON"] != "") else None
|
||||||
|
|
||||||
spot = Spot(source=self.name,
|
spot = Spot(source=self.name,
|
||||||
dx_call=source_spot["ACTIVATOR"].upper(),
|
dx_call=source_spot["ACTIVATOR"].upper(),
|
||||||
de_call=source_spot["SPOTTER"].upper(),
|
de_call=source_spot["SPOTTER"].upper(),
|
||||||
@@ -43,14 +47,12 @@ class GMA(HTTPSpotProvider):
|
|||||||
# Filter out some weird mode strings
|
# Filter out some weird mode strings
|
||||||
mode=source_spot["MODE"].upper() if "<>" not in source_spot["MODE"] else None,
|
mode=source_spot["MODE"].upper() if "<>" not in source_spot["MODE"] else None,
|
||||||
comment=source_spot["TEXT"],
|
comment=source_spot["TEXT"],
|
||||||
sig_refs=[SIGRef(id=source_spot["REF"], sig="", name=source_spot["NAME"])],
|
sig_refs=[SIGRef(id=source_spot["REF"], sig="", name=source_spot["NAME"], latitude=lat,
|
||||||
|
longitude=lon)],
|
||||||
time=datetime.strptime(source_spot["DATE"] + source_spot["TIME"], "%Y%m%d%H%M").replace(
|
time=datetime.strptime(source_spot["DATE"] + source_spot["TIME"], "%Y%m%d%H%M").replace(
|
||||||
tzinfo=pytz.UTC).timestamp(),
|
tzinfo=pytz.UTC).timestamp(),
|
||||||
# Seen GMA spots with no (or empty) lat/lon
|
dx_latitude=lat,
|
||||||
dx_latitude=float(source_spot["LAT"]) if (
|
dx_longitude=lon,
|
||||||
source_spot["LAT"] and source_spot["LAT"] != "") else None,
|
|
||||||
dx_longitude=float(source_spot["LON"]) if (
|
|
||||||
source_spot["LON"] and source_spot["LON"] != "") else None,
|
|
||||||
qrt=source_spot["QRG"] == "QRT")
|
qrt=source_spot["QRG"] == "QRT")
|
||||||
|
|
||||||
# GMA doesn't give what programme (SIG) the reference is for until we separately look it up.
|
# GMA doesn't give what programme (SIG) the reference is for until we separately look it up.
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ from datetime import datetime
|
|||||||
|
|
||||||
import pytz
|
import pytz
|
||||||
import requests
|
import requests
|
||||||
|
from requests.exceptions import ConnectionError, ReadTimeout, ConnectTimeout
|
||||||
|
|
||||||
from core.constants import HTTP_HEADERS
|
from core.constants import HTTP_HEADERS
|
||||||
from data.sig_ref import SIGRef
|
from data.sig_ref import SIGRef
|
||||||
@@ -24,7 +25,7 @@ class HEMA(HTTPSpotProvider):
|
|||||||
SPOTTER_COMMENT_PATTERN = re.compile("^\\((.*)\\) (.*)$")
|
SPOTTER_COMMENT_PATTERN = re.compile("^\\((.*)\\) (.*)$")
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config, self.SPOT_SEED_URL, self.POLL_INTERVAL_SEC)
|
super().__init__("HEMA", provider_config, self.SPOT_SEED_URL, self.POLL_INTERVAL_SEC)
|
||||||
self._spot_seed = ""
|
self._spot_seed = ""
|
||||||
|
|
||||||
def _http_response_to_spots(self, http_response):
|
def _http_response_to_spots(self, http_response):
|
||||||
@@ -58,7 +59,8 @@ class HEMA(HTTPSpotProvider):
|
|||||||
mode=freq_mode_match.group(2).upper(),
|
mode=freq_mode_match.group(2).upper(),
|
||||||
comment=spotter_comment_match.group(2),
|
comment=spotter_comment_match.group(2),
|
||||||
sig="HEMA",
|
sig="HEMA",
|
||||||
sig_refs=[SIGRef(id=spot_items[3].upper(), sig="HEMA", name=spot_items[4])],
|
sig_refs=[SIGRef(id=spot_items[3].upper(), sig="HEMA", name=spot_items[4],
|
||||||
|
latitude=float(spot_items[7]), longitude=float(spot_items[8]))],
|
||||||
time=datetime.strptime(spot_items[0], "%d/%m/%Y %H:%M").replace(
|
time=datetime.strptime(spot_items[0], "%d/%m/%Y %H:%M").replace(
|
||||||
tzinfo=pytz.UTC).timestamp(),
|
tzinfo=pytz.UTC).timestamp(),
|
||||||
dx_latitude=float(spot_items[7]),
|
dx_latitude=float(spot_items[7]),
|
||||||
@@ -67,6 +69,8 @@ class HEMA(HTTPSpotProvider):
|
|||||||
# Add to our list. Don't worry about de-duping, removing old spots etc. at this point; other
|
# Add to our list. Don't worry about de-duping, removing old spots etc. at this point; other
|
||||||
# code will do that for us.
|
# code will do that for us.
|
||||||
new_spots.append(spot)
|
new_spots.append(spot)
|
||||||
|
except (ConnectTimeout, ReadTimeout):
|
||||||
|
logging.warning(f"Timeout when accessing HEMA spots API.")
|
||||||
except ConnectionError:
|
except ConnectionError:
|
||||||
logging.warning("Connection error when accessing HEMA spots API.")
|
logging.warning("Connection error when accessing HEMA spots API.")
|
||||||
return new_spots
|
return new_spots
|
||||||
|
|||||||
@@ -4,6 +4,8 @@ from threading import Thread, Event
|
|||||||
|
|
||||||
import pytz
|
import pytz
|
||||||
import requests
|
import requests
|
||||||
|
from requests import ReadTimeout
|
||||||
|
from requests.exceptions import ConnectionError, ConnectTimeout
|
||||||
|
|
||||||
from core.constants import HTTP_HEADERS
|
from core.constants import HTTP_HEADERS
|
||||||
from spotproviders.spot_provider import SpotProvider
|
from spotproviders.spot_provider import SpotProvider
|
||||||
@@ -13,8 +15,8 @@ class HTTPSpotProvider(SpotProvider):
|
|||||||
"""Generic spot provider class for providers that request data via HTTP(S). Just for convenience to avoid code
|
"""Generic spot provider class for providers that request data via HTTP(S). Just for convenience to avoid code
|
||||||
duplication. Subclasses of this query the individual APIs for data."""
|
duplication. Subclasses of this query the individual APIs for data."""
|
||||||
|
|
||||||
def __init__(self, provider_config, url, poll_interval):
|
def __init__(self, name, provider_config, url, poll_interval):
|
||||||
super().__init__(provider_config)
|
super().__init__(name, provider_config)
|
||||||
self._url = url
|
self._url = url
|
||||||
self._poll_interval = poll_interval
|
self._poll_interval = poll_interval
|
||||||
self._thread = None
|
self._thread = None
|
||||||
@@ -58,6 +60,8 @@ class HTTPSpotProvider(SpotProvider):
|
|||||||
|
|
||||||
except ConnectionError:
|
except ConnectionError:
|
||||||
logging.warning(f"Connection error when accessing {self.name} spots API.")
|
logging.warning(f"Connection error when accessing {self.name} spots API.")
|
||||||
|
except (ConnectTimeout, ReadTimeout):
|
||||||
|
logging.warning(f"Timeout when accessing {self.name} spots API.")
|
||||||
except Exception:
|
except Exception:
|
||||||
self.status = "Error"
|
self.status = "Error"
|
||||||
logging.exception("Exception in HTTP JSON Spot Provider (" + self.name + ")")
|
logging.exception("Exception in HTTP JSON Spot Provider (" + self.name + ")")
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ class LLOTA(HTTPSpotProvider):
|
|||||||
SPOTS_URL = "https://llota.app/api/public/spots"
|
SPOTS_URL = "https://llota.app/api/public/spots"
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config, self.SPOTS_URL, self.POLL_INTERVAL_SEC)
|
super().__init__("LLOTA", provider_config, self.SPOTS_URL, self.POLL_INTERVAL_SEC)
|
||||||
|
|
||||||
def _http_response_to_spots(self, http_response):
|
def _http_response_to_spots(self, http_response):
|
||||||
new_spots = []
|
new_spots = []
|
||||||
|
|||||||
@@ -17,7 +17,7 @@ class ParksNPeaks(HTTPSpotProvider):
|
|||||||
SIOTA_LIST_URL = "https://www.silosontheair.com/data/silos.csv"
|
SIOTA_LIST_URL = "https://www.silosontheair.com/data/silos.csv"
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config, self.SPOTS_URL, self.POLL_INTERVAL_SEC)
|
super().__init__("ParksNPeaks", provider_config, self.SPOTS_URL, self.POLL_INTERVAL_SEC)
|
||||||
|
|
||||||
def _http_response_to_spots(self, http_response):
|
def _http_response_to_spots(self, http_response):
|
||||||
new_spots = []
|
new_spots = []
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ class POTA(HTTPSpotProvider):
|
|||||||
SPOTS_URL = "https://api.pota.app/spot/activator"
|
SPOTS_URL = "https://api.pota.app/spot/activator"
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config, self.SPOTS_URL, self.POLL_INTERVAL_SEC)
|
super().__init__("POTA", provider_config, self.SPOTS_URL, self.POLL_INTERVAL_SEC)
|
||||||
|
|
||||||
def _http_response_to_spots(self, http_response):
|
def _http_response_to_spots(self, http_response):
|
||||||
new_spots = []
|
new_spots = []
|
||||||
@@ -29,7 +29,8 @@ class POTA(HTTPSpotProvider):
|
|||||||
mode=source_spot["mode"].upper(),
|
mode=source_spot["mode"].upper(),
|
||||||
comment=source_spot["comments"],
|
comment=source_spot["comments"],
|
||||||
sig="POTA",
|
sig="POTA",
|
||||||
sig_refs=[SIGRef(id=source_spot["reference"], sig="POTA", name=source_spot["name"])],
|
sig_refs=[SIGRef(id=source_spot["reference"], sig="POTA", name=source_spot["name"],
|
||||||
|
latitude=source_spot["latitude"], longitude=source_spot["longitude"])],
|
||||||
time=datetime.strptime(source_spot["spotTime"], "%Y-%m-%dT%H:%M:%S").replace(
|
time=datetime.strptime(source_spot["spotTime"], "%Y-%m-%dT%H:%M:%S").replace(
|
||||||
tzinfo=pytz.UTC).timestamp(),
|
tzinfo=pytz.UTC).timestamp(),
|
||||||
dx_grid=source_spot["grid6"],
|
dx_grid=source_spot["grid6"],
|
||||||
|
|||||||
@@ -23,7 +23,8 @@ class RBN(SpotProvider):
|
|||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
"""Constructor requires port number."""
|
"""Constructor requires port number."""
|
||||||
|
|
||||||
super().__init__(provider_config)
|
name = provider_config["name"] if "name" in provider_config else "RBN"
|
||||||
|
super().__init__(name, provider_config)
|
||||||
self._port = provider_config["port"]
|
self._port = provider_config["port"]
|
||||||
self._telnet = None
|
self._telnet = None
|
||||||
self._thread = Thread(target=self._handle)
|
self._thread = Thread(target=self._handle)
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ import logging
|
|||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
import requests
|
import requests
|
||||||
|
from requests.exceptions import ConnectionError, ReadTimeout, ConnectTimeout
|
||||||
|
|
||||||
from core.constants import HTTP_HEADERS
|
from core.constants import HTTP_HEADERS
|
||||||
from data.sig_ref import SIGRef
|
from data.sig_ref import SIGRef
|
||||||
@@ -18,11 +19,9 @@ class SOTA(HTTPSpotProvider):
|
|||||||
# The actual data lookup all happens after parsing and checking the epoch.
|
# The actual data lookup all happens after parsing and checking the epoch.
|
||||||
EPOCH_URL = "https://api-db2.sota.org.uk/api/spots/epoch"
|
EPOCH_URL = "https://api-db2.sota.org.uk/api/spots/epoch"
|
||||||
SPOTS_URL = "https://api-db2.sota.org.uk/api/spots/60/all/all"
|
SPOTS_URL = "https://api-db2.sota.org.uk/api/spots/60/all/all"
|
||||||
# SOTA spots don't contain lat/lon, we need a separate lookup for that
|
|
||||||
SUMMIT_URL_ROOT = "https://api-db2.sota.org.uk/api/summits/"
|
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config, self.EPOCH_URL, self.POLL_INTERVAL_SEC)
|
super().__init__("SOTA", provider_config, self.EPOCH_URL, self.POLL_INTERVAL_SEC)
|
||||||
self._api_epoch = ""
|
self._api_epoch = ""
|
||||||
|
|
||||||
def _http_response_to_spots(self, http_response):
|
def _http_response_to_spots(self, http_response):
|
||||||
@@ -50,8 +49,12 @@ class SOTA(HTTPSpotProvider):
|
|||||||
mode=source_spot["mode"].upper(),
|
mode=source_spot["mode"].upper(),
|
||||||
comment=source_spot["comments"],
|
comment=source_spot["comments"],
|
||||||
sig="SOTA",
|
sig="SOTA",
|
||||||
sig_refs=[SIGRef(id=source_spot["summitCode"], sig="SOTA", name=source_spot["summitName"],
|
sig_refs=[SIGRef(id=source_spot["summitCode"], sig="SOTA",
|
||||||
|
name=source_spot["summitName"], latitude=source_spot["latitude"],
|
||||||
|
longitude=source_spot["longitude"],
|
||||||
activation_score=source_spot["points"])],
|
activation_score=source_spot["points"])],
|
||||||
|
dx_latitude=source_spot["latitude"],
|
||||||
|
dx_longitude=source_spot["longitude"],
|
||||||
time=datetime.fromisoformat(source_spot["timeStamp"].replace("Z", "+00:00")).timestamp())
|
time=datetime.fromisoformat(source_spot["timeStamp"].replace("Z", "+00:00")).timestamp())
|
||||||
|
|
||||||
# Add to our list. Don't worry about de-duping, removing old spots etc. at this point; other code will do
|
# Add to our list. Don't worry about de-duping, removing old spots etc. at this point; other code will do
|
||||||
@@ -59,4 +62,6 @@ class SOTA(HTTPSpotProvider):
|
|||||||
new_spots.append(spot)
|
new_spots.append(spot)
|
||||||
except ConnectionError:
|
except ConnectionError:
|
||||||
logging.warning("Connection error when accessing SOTA spots API")
|
logging.warning("Connection error when accessing SOTA spots API")
|
||||||
|
except (ConnectTimeout, ReadTimeout):
|
||||||
|
logging.warning(f"Timeout when accessing SOTA spots API.")
|
||||||
return new_spots
|
return new_spots
|
||||||
|
|||||||
@@ -8,10 +8,10 @@ from core.config import MAX_SPOT_AGE
|
|||||||
class SpotProvider:
|
class SpotProvider:
|
||||||
"""Generic spot provider class. Subclasses of this query the individual APIs for data."""
|
"""Generic spot provider class. Subclasses of this query the individual APIs for data."""
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, name, provider_config):
|
||||||
"""Constructor"""
|
"""Constructor"""
|
||||||
|
|
||||||
self.name = provider_config["name"]
|
self.name = name
|
||||||
self.enabled = provider_config["enabled"]
|
self.enabled = provider_config["enabled"]
|
||||||
self.last_update_time = datetime.min.replace(tzinfo=pytz.UTC)
|
self.last_update_time = datetime.min.replace(tzinfo=pytz.UTC)
|
||||||
self.last_spot_time = datetime.min.replace(tzinfo=pytz.UTC)
|
self.last_spot_time = datetime.min.replace(tzinfo=pytz.UTC)
|
||||||
|
|||||||
@@ -13,8 +13,8 @@ from spotproviders.spot_provider import SpotProvider
|
|||||||
class SSESpotProvider(SpotProvider):
|
class SSESpotProvider(SpotProvider):
|
||||||
"""Spot provider using Server-Sent Events."""
|
"""Spot provider using Server-Sent Events."""
|
||||||
|
|
||||||
def __init__(self, provider_config, url):
|
def __init__(self, name, provider_config, url):
|
||||||
super().__init__(provider_config)
|
super().__init__(name, provider_config)
|
||||||
self._url = url
|
self._url = url
|
||||||
self._event_source = None
|
self._event_source = None
|
||||||
self._thread = None
|
self._thread = None
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ class Tiles(HTTPSpotProvider):
|
|||||||
SPOTS_URL = "https://icneuzxitdqtofutxbla.supabase.co/functions/v1/spots?active_hours=24"
|
SPOTS_URL = "https://icneuzxitdqtofutxbla.supabase.co/functions/v1/spots?active_hours=24"
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config, self.SPOTS_URL, self.POLL_INTERVAL_SEC)
|
super().__init__("Tiles", provider_config, self.SPOTS_URL, self.POLL_INTERVAL_SEC)
|
||||||
|
|
||||||
def _http_response_to_spots(self, http_response):
|
def _http_response_to_spots(self, http_response):
|
||||||
new_spots = []
|
new_spots = []
|
||||||
@@ -31,7 +31,8 @@ class Tiles(HTTPSpotProvider):
|
|||||||
# Tiles spots can include POTA & SOTA references, but ignore those on the basis that we will get them separately from the POTA/SOTA providers anyway.
|
# Tiles spots can include POTA & SOTA references, but ignore those on the basis that we will get them separately from the POTA/SOTA providers anyway.
|
||||||
# Just take the grid reference itself as the single Tiles SIG reference.
|
# Just take the grid reference itself as the single Tiles SIG reference.
|
||||||
sig_refs=[SIGRef(id=source_spot["maidenhead_grid"], sig="Tiles",
|
sig_refs=[SIGRef(id=source_spot["maidenhead_grid"], sig="Tiles",
|
||||||
name=source_spot["maidenhead_grid"])],
|
name=source_spot["maidenhead_grid"], latitude=source_spot["latitude"],
|
||||||
|
longitude=source_spot["longitude"])],
|
||||||
time=datetime.fromisoformat(source_spot["created_at"].replace("Z", "+00:00")).timestamp(),
|
time=datetime.fromisoformat(source_spot["created_at"].replace("Z", "+00:00")).timestamp(),
|
||||||
dx_grid=source_spot["maidenhead_grid"],
|
dx_grid=source_spot["maidenhead_grid"],
|
||||||
dx_latitude=source_spot["latitude"],
|
dx_latitude=source_spot["latitude"],
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ class Towers(HTTPSpotProvider):
|
|||||||
SPOTS_URL = "https://wwtota.com/api/cluster_live.php"
|
SPOTS_URL = "https://wwtota.com/api/cluster_live.php"
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config, self.SPOTS_URL, self.POLL_INTERVAL_SEC)
|
super().__init__("Towers", provider_config, self.SPOTS_URL, self.POLL_INTERVAL_SEC)
|
||||||
|
|
||||||
def _http_response_to_spots(self, http_response):
|
def _http_response_to_spots(self, http_response):
|
||||||
new_spots = []
|
new_spots = []
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ class UKPacketNet(HTTPSpotProvider):
|
|||||||
SPOTS_URL = "https://nodes.ukpacketradio.network/api/nodedata"
|
SPOTS_URL = "https://nodes.ukpacketradio.network/api/nodedata"
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config, self.SPOTS_URL, self.POLL_INTERVAL_SEC)
|
super().__init__("UK Packet Net", provider_config, self.SPOTS_URL, self.POLL_INTERVAL_SEC)
|
||||||
|
|
||||||
def _http_response_to_spots(self, http_response):
|
def _http_response_to_spots(self, http_response):
|
||||||
new_spots = []
|
new_spots = []
|
||||||
|
|||||||
@@ -13,8 +13,8 @@ from spotproviders.spot_provider import SpotProvider
|
|||||||
class WebsocketSpotProvider(SpotProvider):
|
class WebsocketSpotProvider(SpotProvider):
|
||||||
"""Spot provider using websockets."""
|
"""Spot provider using websockets."""
|
||||||
|
|
||||||
def __init__(self, provider_config, url):
|
def __init__(self, name, provider_config, url):
|
||||||
super().__init__(provider_config)
|
super().__init__(name, provider_config)
|
||||||
self._url = url
|
self._url = url
|
||||||
self._ws = None
|
self._ws = None
|
||||||
self._thread = None
|
self._thread = None
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ class WOTA(HTTPSpotProvider):
|
|||||||
RSS_DATE_TIME_FORMAT = "%a, %d %b %Y %H:%M:%S %z"
|
RSS_DATE_TIME_FORMAT = "%a, %d %b %Y %H:%M:%S %z"
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config, self.SPOTS_URL, self.POLL_INTERVAL_SEC)
|
super().__init__("WOTA", provider_config, self.SPOTS_URL, self.POLL_INTERVAL_SEC)
|
||||||
|
|
||||||
def _http_response_to_spots(self, http_response):
|
def _http_response_to_spots(self, http_response):
|
||||||
new_spots = []
|
new_spots = []
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ class WWBOTA(SSESpotProvider):
|
|||||||
SPOTS_URL = "https://api.wwbota.net/spots/"
|
SPOTS_URL = "https://api.wwbota.net/spots/"
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config, self.SPOTS_URL)
|
super().__init__("WWBOTA", provider_config, self.SPOTS_URL)
|
||||||
|
|
||||||
def _sse_message_to_spot(self, message):
|
def _sse_message_to_spot(self, message):
|
||||||
source_spot = json.loads(message)
|
source_spot = json.loads(message)
|
||||||
@@ -20,7 +20,8 @@ class WWBOTA(SSESpotProvider):
|
|||||||
# n-fer activations.
|
# n-fer activations.
|
||||||
refs = []
|
refs = []
|
||||||
for ref in source_spot["references"]:
|
for ref in source_spot["references"]:
|
||||||
sigref = SIGRef(id=ref["reference"], sig="WWBOTA", name=ref["name"])
|
sigref = SIGRef(id=ref["reference"], sig="WWBOTA", name=ref["name"], latitude=ref["lat"],
|
||||||
|
longitude=ref["long"])
|
||||||
refs.append(sigref)
|
refs.append(sigref)
|
||||||
|
|
||||||
spot = Spot(source=self.name,
|
spot = Spot(source=self.name,
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ class WWFF(HTTPSpotProvider):
|
|||||||
SPOTS_URL = "https://spots.wwff.co/static/spots.json"
|
SPOTS_URL = "https://spots.wwff.co/static/spots.json"
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config, self.SPOTS_URL, self.POLL_INTERVAL_SEC)
|
super().__init__("WWFF", provider_config, self.SPOTS_URL, self.POLL_INTERVAL_SEC)
|
||||||
|
|
||||||
def _http_response_to_spots(self, http_response):
|
def _http_response_to_spots(self, http_response):
|
||||||
new_spots = []
|
new_spots = []
|
||||||
@@ -29,7 +29,8 @@ class WWFF(HTTPSpotProvider):
|
|||||||
mode=source_spot["mode"].upper(),
|
mode=source_spot["mode"].upper(),
|
||||||
comment=source_spot["remarks"],
|
comment=source_spot["remarks"],
|
||||||
sig="WWFF",
|
sig="WWFF",
|
||||||
sig_refs=[SIGRef(id=source_spot["reference"], sig="WWFF", name=source_spot["reference_name"])],
|
sig_refs=[SIGRef(id=source_spot["reference"], sig="WWFF", name=source_spot["reference_name"],
|
||||||
|
latitude=source_spot["latitude"], longitude=source_spot["longitude"])],
|
||||||
time=datetime.fromtimestamp(source_spot["spot_time"], tz=pytz.UTC).timestamp(),
|
time=datetime.fromtimestamp(source_spot["spot_time"], tz=pytz.UTC).timestamp(),
|
||||||
dx_latitude=source_spot["latitude"],
|
dx_latitude=source_spot["latitude"],
|
||||||
dx_longitude=source_spot["longitude"])
|
dx_longitude=source_spot["longitude"])
|
||||||
|
|||||||
@@ -21,7 +21,8 @@ class XOTA(WebsocketSpotProvider):
|
|||||||
SIG = None
|
SIG = None
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config, provider_config["url"])
|
name = provider_config["name"] if "name" in provider_config else "xOTA"
|
||||||
|
super().__init__(name, provider_config, provider_config["url"])
|
||||||
locations_csv = str(provider_config["locations-csv"]) if "locations-csv" in provider_config else None
|
locations_csv = str(provider_config["locations-csv"]) if "locations-csv" in provider_config else None
|
||||||
self.SIG = str(provider_config["sig"]) if "sig" in provider_config else None
|
self.SIG = str(provider_config["sig"]) if "sig" in provider_config else None
|
||||||
|
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ class ZLOTA(HTTPSpotProvider):
|
|||||||
LIST_URL = "https://ontheair.nz/assets/assets.json"
|
LIST_URL = "https://ontheair.nz/assets/assets.json"
|
||||||
|
|
||||||
def __init__(self, provider_config):
|
def __init__(self, provider_config):
|
||||||
super().__init__(provider_config, self.SPOTS_URL, self.POLL_INTERVAL_SEC)
|
super().__init__("ZLOTA", provider_config, self.SPOTS_URL, self.POLL_INTERVAL_SEC)
|
||||||
|
|
||||||
def _http_response_to_spots(self, http_response):
|
def _http_response_to_spots(self, http_response):
|
||||||
new_spots = []
|
new_spots = []
|
||||||
|
|||||||
|
Before Width: | Height: | Size: 129 KiB After Width: | Height: | Size: 129 KiB |
|
Before Width: | Height: | Size: 125 KiB After Width: | Height: | Size: 125 KiB |
|
Before Width: | Height: | Size: 3.9 KiB After Width: | Height: | Size: 3.9 KiB |
|
Before Width: | Height: | Size: 7.0 KiB After Width: | Height: | Size: 7.0 KiB |
|
Before Width: | Height: | Size: 6.5 KiB After Width: | Height: | Size: 6.5 KiB |
|
Before Width: | Height: | Size: 348 B After Width: | Height: | Size: 348 B |
|
Before Width: | Height: | Size: 348 B After Width: | Height: | Size: 348 B |
|
Before Width: | Height: | Size: 7.2 KiB After Width: | Height: | Size: 7.2 KiB |
|
Before Width: | Height: | Size: 4.8 KiB After Width: | Height: | Size: 4.8 KiB |
|
Before Width: | Height: | Size: 14 KiB After Width: | Height: | Size: 14 KiB |
|
Before Width: | Height: | Size: 5.9 KiB After Width: | Height: | Size: 5.9 KiB |
|
Before Width: | Height: | Size: 4.1 KiB After Width: | Height: | Size: 4.1 KiB |
|
Before Width: | Height: | Size: 8.7 KiB After Width: | Height: | Size: 8.7 KiB |
|
Before Width: | Height: | Size: 4.9 KiB After Width: | Height: | Size: 4.9 KiB |
|
Before Width: | Height: | Size: 6.4 KiB After Width: | Height: | Size: 6.4 KiB |
|
Before Width: | Height: | Size: 14 KiB After Width: | Height: | Size: 14 KiB |
|
Before Width: | Height: | Size: 9.1 KiB After Width: | Height: | Size: 9.1 KiB |
|
Before Width: | Height: | Size: 4.8 KiB After Width: | Height: | Size: 4.8 KiB |
|
Before Width: | Height: | Size: 348 B After Width: | Height: | Size: 348 B |
|
Before Width: | Height: | Size: 7.3 KiB After Width: | Height: | Size: 7.3 KiB |
|
Before Width: | Height: | Size: 348 B After Width: | Height: | Size: 348 B |
|
Before Width: | Height: | Size: 4.9 KiB After Width: | Height: | Size: 4.9 KiB |
|
Before Width: | Height: | Size: 9.5 KiB After Width: | Height: | Size: 9.5 KiB |
|
Before Width: | Height: | Size: 6.5 KiB After Width: | Height: | Size: 6.5 KiB |
|
Before Width: | Height: | Size: 348 B After Width: | Height: | Size: 348 B |
|
Before Width: | Height: | Size: 9.8 KiB After Width: | Height: | Size: 9.8 KiB |
|
Before Width: | Height: | Size: 8.2 KiB After Width: | Height: | Size: 8.2 KiB |
|
Before Width: | Height: | Size: 8.9 KiB After Width: | Height: | Size: 8.9 KiB |
|
Before Width: | Height: | Size: 14 KiB After Width: | Height: | Size: 14 KiB |
|
Before Width: | Height: | Size: 7.0 KiB After Width: | Height: | Size: 7.0 KiB |
|
Before Width: | Height: | Size: 4.8 KiB After Width: | Height: | Size: 4.8 KiB |
|
Before Width: | Height: | Size: 4.4 KiB After Width: | Height: | Size: 4.4 KiB |
|
Before Width: | Height: | Size: 348 B After Width: | Height: | Size: 348 B |
|
Before Width: | Height: | Size: 348 B After Width: | Height: | Size: 348 B |
|
Before Width: | Height: | Size: 7.6 KiB After Width: | Height: | Size: 7.6 KiB |
|
Before Width: | Height: | Size: 6.4 KiB After Width: | Height: | Size: 6.4 KiB |
|
Before Width: | Height: | Size: 9.8 KiB After Width: | Height: | Size: 9.8 KiB |
|
Before Width: | Height: | Size: 7.0 KiB After Width: | Height: | Size: 7.0 KiB |
|
Before Width: | Height: | Size: 6.1 KiB After Width: | Height: | Size: 6.1 KiB |
|
Before Width: | Height: | Size: 9.3 KiB After Width: | Height: | Size: 9.3 KiB |
|
Before Width: | Height: | Size: 348 B After Width: | Height: | Size: 348 B |
|
Before Width: | Height: | Size: 8.4 KiB After Width: | Height: | Size: 8.4 KiB |