Fix mojibake in DX cluster and RBN spot comments

Both providers always decoded telnet bytes as Latin-1, which silently
mangles nodes that send UTF-8 (Latin-1 decode never raises, so mixed
encodings across cluster nodes went unnoticed). Added a shared
decode_telnet_bytes() helper that tries UTF-8 first and falls back to
Latin-1, since a byte stream that happens to be valid multi-byte UTF-8
is essentially never accidental Latin-1 text.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Ian Renton
2026-09-20 19:08:05 +01:00
co-authored by Claude Sonnet 5
parent 9272766ffb
commit e33eecbc4e
3 changed files with 16 additions and 4 deletions
+12
View File
@@ -5,6 +5,18 @@ import pytz
from core.data_store import DATA_STORE
def decode_telnet_bytes(data: bytes) -> str:
"""Decode a line of text received from a telnet connection. DX cluster and RBN nodes are inconsistent about the
character encoding they use for spot comments: most send UTF-8, but some older ones send Latin-1/CP1252. Try
UTF-8 first, since valid multi-byte UTF-8 sequences are very unlikely to occur by chance in Latin-1 text, then
fall back to Latin-1, which can decode any byte sequence without raising."""
try:
return data.decode("utf-8")
except UnicodeDecodeError:
return data.decode("latin-1")
class SpotProvider:
"""Generic spot provider class. Subclasses of this query the individual APIs for data."""