mirror of
https://git.ianrenton.com/ian/spothole.git
synced 2026-09-21 06:47:42 +00:00
Fix mojibake in DX cluster and RBN spot comments
Both providers always decoded telnet bytes as Latin-1, which silently mangles nodes that send UTF-8 (Latin-1 decode never raises, so mixed encodings across cluster nodes went unnoticed). Added a shared decode_telnet_bytes() helper that tries UTF-8 first and falls back to Latin-1, since a byte stream that happens to be valid multi-byte UTF-8 is essentially never accidental Latin-1 text. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
9272766ffb
commit
e33eecbc4e
@@ -5,6 +5,18 @@ import pytz
|
||||
from core.data_store import DATA_STORE
|
||||
|
||||
|
||||
def decode_telnet_bytes(data: bytes) -> str:
|
||||
"""Decode a line of text received from a telnet connection. DX cluster and RBN nodes are inconsistent about the
|
||||
character encoding they use for spot comments: most send UTF-8, but some older ones send Latin-1/CP1252. Try
|
||||
UTF-8 first, since valid multi-byte UTF-8 sequences are very unlikely to occur by chance in Latin-1 text, then
|
||||
fall back to Latin-1, which can decode any byte sequence without raising."""
|
||||
|
||||
try:
|
||||
return data.decode("utf-8")
|
||||
except UnicodeDecodeError:
|
||||
return data.decode("latin-1")
|
||||
|
||||
|
||||
class SpotProvider:
|
||||
"""Generic spot provider class. Subclasses of this query the individual APIs for data."""
|
||||
|
||||
|
||||
Reference in New Issue
Block a user