In relation to #224

[danmrossi]

Hello @hjelev hope you are well.

I noticed this error on a couple of machines, only copied the one here as on both machines it is the exact same line by line and only just started today:

sshing in and restarting the services "fixes" it but not overly ideal when out and about lol - If there is anything else i can provide please let me know.

● rpi-mqtt-monitor.service - RPI MQTT Monitor
Loaded: loaded (/etc/systemd/system/rpi-mqtt-monitor.service; enabled; vendor preset: enabled)
Active: active (running) since Sat 2025-06-07 19:28:34 AWST; 2 days ago
Main PID: 574 (python3)
Tasks: 4 (limit: 3933)
CPU: 38min 54.693s
CGroup: /system.slice/rpi-mqtt-monitor.service
└─574 /home/roboctrl/rpi-mqtt-monitor/rpi_mon_env/bin/python3 /home/roboctrl/rpi-mqtt-monitor/src/rpi-cpu2mqtt.py --service

Jun 10 15:43:02 roboctrl python3[574]: File "/home/roboctrl/rpi-mqtt-monitor/rpi_mon_env/lib/python3.9/site-packages/paho/mqtt/client.py", line 1556, in loop_read
Jun 10 15:43:02 roboctrl python3[574]: rc = self._packet_read()
Jun 10 15:43:02 roboctrl python3[574]: File "/home/roboctrl/rpi-mqtt-monitor/rpi_mon_env/lib/python3.9/site-packages/paho/mqtt/client.py", line 2419, in _packet_read
Jun 10 15:43:02 roboctrl python3[574]: data = self._sock_recv(self._in_packet['to_process'])
Jun 10 15:43:02 roboctrl python3[574]: File "/home/roboctrl/rpi-mqtt-monitor/rpi_mon_env/lib/python3.9/site-packages/paho/mqtt/client.py", line 640, in _sock_recv
Jun 10 15:43:02 roboctrl python3[574]: return self._sock.recv(bufsize)
Jun 10 15:43:02 roboctrl python3[574]: AttributeError: 'NoneType' object has no attribute 'recv'
Jun 10 16:43:05 roboctrl sudo[10419]: root : PWD=/home/roboctrl/rpi-mqtt-monitor ; USER=root ; COMMAND=/usr/bin/apt update
Jun 10 16:43:05 roboctrl sudo[10419]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=0)
Jun 10 16:43:11 roboctrl sudo[10419]: pam_unix(sudo:session): session closed for user root
~

[danmrossi]

another machine has randomly dropped off but no error has i can see, maybe some logging has to be added to catch any issues in the code?

I know its not networking issues as been running various utilities to test connection 24/7 between router and machines and machines to each other etc the last few days and i have manually tested all my cables with a cable tester i borrowed from work.

● rpi-mqtt-monitor.service - RPI MQTT Monitor
Loaded: loaded (/etc/systemd/system/rpi-mqtt-monitor.service; enabled; preset: enabled)
Active: active (running) since Sun 2025-06-08 10:54:50 AWST; 3 days ago
Main PID: 1105 (python3)
Tasks: 4 (limit: 9447)
CPU: 9min 30.852s
CGroup: /system.slice/rpi-mqtt-monitor.service
└─1105 /home/pbs01/rpi-mqtt-monitor/rpi_mon_env/bin/python3 /home/pbs01/rpi-mqtt-monitor/src/rpi-cpu2mqtt.py --service

Jun 11 10:05:53 pbs01 sudo[348569]: pam_unix(sudo:session): session closed for user root
Jun 11 11:05:59 pbs01 sudo[349162]: root : PWD=/home/pbs01/rpi-mqtt-monitor ; USER=root ; COMMAND=/usr/bin/apt update
Jun 11 11:05:59 pbs01 sudo[349162]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=0)
Jun 11 11:06:02 pbs01 sudo[349162]: pam_unix(sudo:session): session closed for user root
Jun 11 12:06:08 pbs01 sudo[349760]: root : PWD=/home/pbs01/rpi-mqtt-monitor ; USER=root ; COMMAND=/usr/bin/apt update
Jun 11 12:06:08 pbs01 sudo[349760]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=0)
Jun 11 12:06:12 pbs01 sudo[349760]: pam_unix(sudo:session): session closed for user root
Jun 11 13:06:18 pbs01 sudo[350398]: root : PWD=/home/pbs01/rpi-mqtt-monitor ; USER=root ; COMMAND=/usr/bin/apt update
Jun 11 13:06:18 pbs01 sudo[350398]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=0)
Jun 11 13:06:21 pbs01 sudo[350398]: pam_unix(sudo:session): session closed for user root

[danmrossi]

Chat GPT says (have found it handy with fixing arduino code of all things lol), not sure if this helps or is a hinderance:

1. MQTT reconnect logic

• Use on_disconnect callback: register an on_disconnect handler that logs disconnects and attempts reconnection.
• Leverage reconnect_delay_set(): after creating the client, call client.reconnect_delay_set(min_delay=1, max_delay=120) so Paho will automatically back off and retry.
• Remove polling is_connected() loop in create_mqtt_client(): instead rely on the background loop and callbacks to reconnect.

2. Centralize subprocess calls

You shell out in dozens of places. Consider a helper:

def run_cmd(cmd, *, shell=True, fallback=None):
    try:
        return subprocess.check_output(cmd, shell=shell, text=True).strip()
    except subprocess.CalledProcessError:
        return fallback

Then e.g. voltage = run_cmd("vcgencmd measure_volts | cut -f2 -d=") This cuts down on repeated try/except boilerplate and ensures consistent decoding.

3. Replace prints with proper logging

• Switch to Python’s logging module.
• Use different levels (INFO, WARNING, ERROR) so you can control verbosity with one config flag instead of scavenging print statements.

4. Reduce duplication in discovery/publish

• All of these blocks for discovery and publishing—move into a generic function:

def publish_sensor(client, key, value, config_json):
    topic = f"{base_topic}/{hostname}/{key}"
    if config.discovery_messages:
        client.publish(f"{disco_prefix}/sensor/{prefix}/{hostname}_{key}/config",
                       config_json(key), qos=config.qos)
    client.publish(topic, value, qos=config.qos, retain=config.retain)

You can then loop through a dict of enabled metrics rather than hard‐coding each.

5. Thread and shutdown handling

• Use a single ThreadPoolExecutor or move to asyncio—threads plus time.sleep() loops make clean shutdown tricky.
• Catch KeyboardInterrupt in the main thread, then call stop_event.set(), join threads, and only then exit.

6. Leverage psutil over shell for system info

You already have psutil—it can provide memory, swap, disk and uptime without shell calls. E.g.

psutil.disk_usage(path).percent
psutil.boot_time()

This is faster, more portable and sidesteps parsing brittle command output.

7. Configuration and type safety

• Add type hints throughout to clarify what each function accepts/returns.
• Validate critical config values (e.g. mqtt_host, numeric ports) at startup, and fail fast if something is missing.

8. Modularize and test

Break the huge monolithic script into modules:

• metrics.py (all check_* funcs)
• mqtt_client.py (connect & publish logic)
• main.py (argument parsing & orchestration)

Write unit tests against each module—mock out subprocess calls and verify JSON outputs.

9. Clean up translation logic

Currently you reload translations.ini on every launch. If you only ever need it once, read it at import‐time, then expose a simple translate(key) wrapper.

10. Graceful JSON building

Using json.dumps() on large dicts is fine, but you might consider pydantic or dataclasses.asdict for more structured config objects—especially the Home Assistant discovery payloads.

[danmrossi]

And when i feed in my issues:

"Based on the stack-trace, your MQTT client’s internal socket (client._sock) is ending up as None after the broker drops the connection, so when Paho’s loop_read calls self._sock.recv() you get the AttributeError. The root of the problem is that you’re using connect_async() + loop_start() without any on-disconnect or reconnect logic, so the client never rebuilds its socket when the broker comes back."

Here’s a targeted recipe to make it automatically reconnect cleanly:

1. Register an on_disconnect callback

Add at client setup time:

def on_disconnect(client, userdata, rc):
    if rc != 0:
        # unexpected disconnect
        print(f"MQTT disconnected (rc={rc}), attempting to reconnect…")
        try:
            client.reconnect()
        except Exception as e:
            print(f"Reconnect failed: {e}")

And in create_mqtt_client() after client.username_pw_set(...):

client.on_disconnect = on_disconnect

2. Enable exponential back-off retries

Right after creating your client, tell Paho to space out retries:

client.reconnect_delay_set(min_delay=1, max_delay=120)
This makes client.reconnect() back off automatically if the broker is down for a longer stretch.

3. Switch to loop_forever(retry_first_connection=True) in “service” mode

If you want the simplest semantics—block your main thread in the MQTT loop and let Paho handle reconnects—replace:

client.connect_async(config.mqtt_host, int(config.mqtt_port))
client.loop_start()

with

# blocks, retries internally on disconnect
client.connect(config.mqtt_host, int(config.mqtt_port), keepalive=60)
client.loop_forever(retry_first_connection=True)

You can still spin off your gather-and-send threads in parallel; just don’t call loop_start() yourself.

4. Guard against publishing on a dead socket

Inside your publish_to_mqtt() (and similar) wrap publishes with:

if not client.is_connected():
    print("Warning: MQTT client not connected, skipping publish")
    return

That way you’ll never try to .publish() on a None socket.

5. Example refactored create_mqtt_client()

def create_mqtt_client():
    def on_log(client, userdata, level, buf):
        if level == paho.MQTT_LOG_ERR:
            print("MQTT error:", buf)

    def on_connect(client, userdata, flags, rc):
        if rc == 0:
            print("MQTT connected")
        else:
            print("MQTT connect failed, rc:", rc)

    def on_disconnect(client, userdata, rc):
        if rc != 0:
            print(f"MQTT disconnected (rc={rc}), reconnecting…")
            try:
                client.reconnect()
            except Exception as e:
                print("Reconnect error:", e)

    client = paho.Client(
        client_id=f"rpi-mqtt-monitor-{hostname}-{int(time.time())}"
    )
    client.username_pw_set(config.mqtt_user, config.mqtt_password)
    client.on_log = on_log
    client.on_connect = on_connect
    client.on_disconnect = on_disconnect

    # automatic back-off
    client.reconnect_delay_set(min_delay=1, max_delay=120)

    try:
        # use blocking connect + loop_forever to simplify
        client.connect(config.mqtt_host, int(config.mqtt_port), keepalive=60)
    except Exception as e:
        print("Error connecting to MQTT broker:", e)
        return None

    return client

Then in your __main__ service path:

client = create_mqtt_client()
if not client:
    sys.exit(1)

# start your worker threads here…

# hand over control to Paho; it will reconnect automatically
client.loop_forever(retry_first_connection=True)

With those changes:

• on_disconnect will fire whenever the broker goes away.
• reconnect_delay_set ensures you don’t hammer the broker with retries.
• loop_forever(retry_first_connection=True) manages the socket lifecycle for you.

[danmrossi]

@hjelev i have tried making my own changes and for once appears to "work" but testing with system reboots of the machine im running this updated code on and rebooting the Home Asssistant VM etc: danmrossi@6d505c0

[danmrossi]

The other thing i have noticed is that even if a machine is reporting to home assistant ok, i sometimes find after a few days i cant send a shutdown or restart command until the rpi monitor service has been restarted, these changes i made in the link above don't fix that.

[danmrossi]

i managed to fix up the shutdown/reboot command issue: danmrossi@a9a0ffd

[danmrossi]

Have done one of those pull request things (all new to me) this was all done by chatgpt, one of the rare times i can get it to actually produce viable code when its not hallucinating lol.

Time will tell if this makes it more stable once and for all.

I have put this code manually onto all my machines to see how it reacts over the coming week and if after a week and stable measurements and can still send reboot commands etc ill advise here.

[danmrossi]

Hi @hjelev

I have been running the modifications for a few days now and so far so good, takes all my testing i can throw at it without failing :)

The pull request should have the 4 commits i have made in my fork to your project.

[hjelev]

Thank you, I am on vacation now for 2 weeks, will take a look and merge it when I come back.

…

On Sun, Jun 15, 2025, 3:15 PM Daniel ***@***.***> wrote: *danmrossi* left a comment (hjelev/rpi-mqtt-monitor#226) <#226 (comment)> Hi @hjelev <https://github.com/hjelev> I have been running the modifications for a few days now and so far so good, takes all my testing i can throw at it without failing :) The pull request should have the 4 commits i have made in my fork to your project. — Reply to this email directly, view it on GitHub <#226 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC5ZSSY5Z46FVY7NS35JDS33DVPUJAVCNFSM6AAAAAB67E3MBWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSNZTG42DEMBTGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[danmrossi]

@hjelev yeah no rush :)

Also so you know, i cloned your repository so i could "work" on a V2 with various online AI coders without affecting the "Fork" i submitted my changes on and so far so good it works.

I have installed, uninstalled and got it running currently on a test machine which ill leave up for a week or two to keep tabs on it.

You can check it out at: https://github.com/danmrossi/rpi-mqtt-monitor-v2

If you want to make it your main feel free to clone/copy/fork or whatever or i can make minor changes like setting back in place "hjelev" and "rpi-mqtt-monitor" instead of "rpi-mqtt-monitor-v2" etc and copy it to my fork of your project and submit a pull request that way :)

Summary

Key differences between the two repositories include:

Logging and code structure.
The new script uses Python’s logging module, event signaling (stop_event) and subprocess.run for better error handling, replacing the original’s print() statements and global exit_flag. This is visible around the updated on_message handler in the new repo compared with the original’s implementation which sets an exit_flag and uses print statements.

Improved MQTT client handling.
create_mqtt_client in the new version sets up automatic reconnect with reconnect_delay_set and logs connection status, whereas the original code manually waits for a connection and lacks this back‑off behavior

Minor configuration changes.
The default topic prefix changed to rpi-MQTT-monitor-v2 and the script version increased from 0.6.3 to 0.6.4 in config.py.example compared with the original settings.

Unit tests added.
The new repository introduces a tests directory with unit tests for helper functions, which does not exist in the original repo.

Service file and installation scripts updated.
A systemd service template named rpi-mqtt-monitor-v2.service references the virtual environment path, mirroring the original service file but under a new name and path. Installation scripts (install.sh and remote_install.sh) have been adjusted accordingly.

Readme adjustments.
The README text is largely the same but references the new command name “rpi-mqtt-monitor-v2” and updated features in the bullet list versus the original wording which references “rpi-mqtt-monitor”

Overall, “rpi-mqtt-monitor-v2” maintains the same functionality as the original project while adding logging, cleaner shutdown handling, automatic MQTT reconnect, updated naming, and test coverage.

A more "In-Depth" look at the changes made:

1. Logging and PEP 8/282 compliance

• rpi-mqtt-monitor-v2 introduces Python’s logging module and a helper function to sanitize values, improving error handling and readability:

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

def sanitize_numeric(value):
    """Return a valid numeric value or a fallback when invalid."""

The original script uses print() statements and lacks this helper:

def check_wifi_signal(format):
    try:
        full_cmd =  "ls /sys/class/ieee80211/*/device/net/"

2. Robust MQTT client initialization

• NEW version sets up a persistent session, automatic reconnect, and logs connection state:

client = paho.Client(
    client_id=f"rpi-mqtt-monitor-v2-{hostname}-{int(time.time())}",
    clean_session=False
)
client.reconnect_delay_set(min_delay=1, max_delay=120)

ORIGINAL uses client.connect_async() with manual timeout logic:

client = paho.Client(client_id="rpi-mqtt-monitor-" + hostname + str(int(time.time())))
client.socket_timeout = 5  # seconds
client.connect_async(config.mqtt_host, int(config.mqtt_port))

3. Cleaner shutdown and subprocess usage

• NEW handles commands via subprocess.run() with logging and no global exit flag:

elif msg.payload.decode() == "restart":
    logger.info("Restarting the system...")
    subprocess.run(["sudo", "reboot"], check=True)

ORIGINAL uses os.system() and sets a global exit_flag:

elif msg.payload.decode() == "restart":
    print("Restarting the system...")
    os.system("sudo reboot")

4. Thread management

• NEW relies on threading.Event (stop_event) to signal threads when shutting down.

ORIGINAL uses a boolean exit_flag.

5. CLI and service name changes

• Command name updated (prog='rpi-mqtt-monitor-v2') and a new --uninstall option was added.
• Systemd service file renamed and paths updated:

Description=RPI MQTT Monitor V2
ExecStart=/home/username/git/rpi-mqtt-monitor-v2/rpi_mon_env/bin/python ...

vs.

Description=RPI MQTT Monitor
ExecStart=/home/username/git/rpi-mqtt-monitor/rpi_mon_env/bin/python ...

6. Configuration defaults

• Version bumped from 0.6.3 to 0.6.4 and the MQTT topic prefix changed to "rpi-MQTT-monitor-v2":

version = "0.6.4"
mqtt_topic_prefix = "rpi-MQTT-monitor-v2"

ORIGINAL uses "rpi-MQTT-monitor" and an older version value:

version = "0.6.3"
mqtt_topic_prefix = "rpi-MQTT-monitor"

7. Unit tests

• NEW adds a tests/ directory with several unit tests for helper functions:

class TestFunctions(unittest.TestCase):
    def test_check_sys_clock_speed(self):
        ...

There are no tests in the original repository.

8. Installation scripts

• Both install.sh and remote_install.sh reference the new name (rpi-mqtt-monitor-v2) and virtual environment path. Logic is otherwise similar.

9. README and documentation tweaks

• Title reflects the new project name and links:

# Raspberry Pi MQTT Monitor V2

• The “To Do” section from the original README has been removed. The CLI examples and commands reference the new executable (rpi-mqtt-monitor-v2) throughout.

10. Minor fixes

• requirements.txt in NEW ends with a newline and reads cleanly:

paho-mqtt==1.6.1
requests
psutil

The original file lacks a trailing newline (shows “psutil” merged with the shell prompt).

11. PEP compliance highlights

• Use of the logging module instead of print() (PEP 282).
• Use of helper functions with clear docstrings such as sanitize_numeric() (PEP 257 style).
• Consistent snake_case naming, explicit argument parsing, and modular functions, improving overall PEP 8 readability.
• Replacement of os.system calls with subprocess.run for better security and error checking.

These refinements make rpi-mqtt-monitor-v2 more maintainable and closer to standard Python practices compared with the original project.

[danmrossi]

Have you had a chance to look at my merge? I’ve been running it all this time and is completely rock solid for me

Sent from my iPhone On 17 Jun 2025, at 2:19 pm, Masoko ***@***.***> wrote:  [https://avatars.githubusercontent.com/u/12294475?s=20&v=4]hjelev left a comment (hjelev/rpi-mqtt-monitor#226)<#226 (comment)> Thank you, I am on vacation now for 2 weeks, will take a look and merge it when I come back.

On Sun, Jun 15, 2025, 3:15 PM Daniel ***@***.***> wrote: *danmrossi* left a comment (hjelev/rpi-mqtt-monitor#226) <#226 (comment)> Hi @hjelev <https://github.com/hjelev> I have been running the modifications for a few days now and so far so good, takes all my testing i can throw at it without failing :) The pull request should have the 4 commits i have made in my fork to your project. — Reply to this email directly, view it on GitHub <#226 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC5ZSSY5Z46FVY7NS35JDS33DVPUJAVCNFSM6AAAAAB67E3MBWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSNZTG42DEMBTGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

— Reply to this email directly, view it on GitHub<#226 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AMCDRZOTS2MM45CL6TMC56D3D6XQNAVCNFSM6AAAAAB67E3MBWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSNZZGA4DKMZTGU>. You are receiving this because you authored the thread.Message ID: ***@***.***>