[ENHANCEMENT] [OctoPrint support] Printer state information is not passed to host over serial

[arekm]

Printer type - MK3.5
Printer firmware version - 5.2.0
Original or Custom firmware - Original

USB drive or USB/Octoprint
USB flash drive - that's important. Prints are started from printer LCD screen, from USB drive.

Describe the bug
MK3S+ firmware informed host (OctoPrint) over serial about what is the state of the printer.
For example if you started print from sd card then OctoPrint knew that. When print finished - the same. On pause - the same.

OctoPrint with such knowledge was able to start timelapse recording when print was started (from lcd) and finish it when it ended. And do much more like turn on LED lighting at print start, turn off at end of print. Send notification about printer state to the user phone (using OctoPod plugin) and more.

All that functionality is lost with 32bit firmware.

How to reproduce / Expected behavior
Connect Octoprint to MK3.5/MK4 printer. Start print from LCD. OctoPrint should see that print has started. Stop the print from LCD. OctoPrint should also see that fact.

Logs

Example from MK3S+ log for a print started from MK3S+ LCD:

2024-02-01 12:18:00,997 - Recv: Not SD printing
2024-02-01 12:18:00,998 - Recv: ok
2024-02-01 12:18:01,791 - Send: M27
2024-02-01 12:18:01,796 - Recv: Not SD printing
2024-02-01 12:18:01,796 - Recv: ok
2024-02-01 12:18:02,484 - Recv: echo:Now fresh file: gasket~1.gco
2024-02-01 12:18:02,492 - Recv: File opened: /GASKET_0.2mm_FLEX_MK3S_21m.gcode Size: 125448
2024-02-01 12:18:02,495 - Recv: File selected
2024-02-01 12:18:02,514 - Recv: LCD status changed
2024-02-01 12:18:02,793 - Send: M27
2024-02-01 12:18:02,943 - Recv: LCD status changed
2024-02-01 12:18:02,947 - Recv: echo:enqueing "M23 gasket~1.gco"
2024-02-01 12:18:02,947 - Recv: echo:enqueing "M24"
2024-02-01 12:18:02,983 - Recv: echo:Now fresh file: gasket~1.gco
2024-02-01 12:18:02,992 - Recv: File opened: /GASKET_0.2mm_FLEX_MK3S_21m.gcode Size: 125448
2024-02-01 12:18:02,996 - Recv: File selected
2024-02-01 12:18:03,017 - Recv: LCD status changed
2024-02-01 12:18:03,017 - Recv: T:59.5 /0.0 B:35.5 /0.0 T0:59.5 /0.0 @:0 B@:0 P:0.0 A:33.6
2024-02-01 12:18:03,020 - Recv: GASKET_0.2mm_FLEX_MK3S_21m.gcode
2024-02-01 12:18:03,021 - Recv: SD printing byte 76/125448
2024-02-01 12:18:03,021 - Changing monitoring state from "Operational" to "Starting print from SD"
2024-02-01 12:18:03,090 - Recv: 0:0
2024-02-01 12:18:03,090 - Recv: ok
2024-02-01 12:18:03,097 - Changing monitoring state from "Starting print from SD" to "Printing from SD"

Notice that OctoPrint recognized that print started - "Changing monitoring state from "Operational" to "Starting print from SD"", "Changing monitoring state from "Starting print from SD" to "Printing from SD""

MK3.5 puts zero information over serial about printer state which means OctoPrint never sees that print from USB flash drive is happening.

2024-02-12 09:11:00,746 - Recv: Cap:MOTION_MODES:0
2024-02-12 09:11:00,746 - Recv: Cap:CHAMBER_TEMPERATURE:0
2024-02-12 09:11:00,747 - Recv:  T:210.17/170.00 B:43.27/85.00 A:39.98/0.00 @:0 B@:0 W:?
2024-02-12 09:11:01,740 - Recv:  T:209.07/170.00 B:43.27/85.00 A:40.00/0.00 @:0 B@:255 W:?
2024-02-12 09:11:02,740 - Recv:  T:208.31/170.00 B:43.22/85.00 A:39.96/0.00 @:0 B@:255 W:?
2024-02-12 09:11:02,741 - Recv: echo:busy: processing
2024-02-12 09:11:03,740 - Recv:  T:207.44/170.00 B:43.25/85.00 A:40.00/0.00 @:0 B@:255 W:?
2024-02-12 09:11:04,740 - Recv:  T:206.55/170.00 B:43.29/85.00 A:40.03/0.00 @:0 B@:255 W:?
2024-02-12 09:11:04,746 - Recv: echo:busy: processing
2024-02-12 09:11:05,740 - Recv:  T:205.55/170.00 B:43.43/85.00 A:40.09/0.00 @:0 B@:255 W:?
2024-02-12 09:11:06,740 - Recv:  T:204.65/170.00 B:43.66/85.00 A:40.11/0.00 @:0 B@:255 W:?
2024-02-12 09:11:06,743 - Recv: echo:busy: processing
2024-02-12 09:11:07,740 - Recv:  T:203.72/170.00 B:43.86/85.00 A:40.17/0.00 @:0 B@:255 W:?
[...]

(that's mostly all)

Related - HOST_ACTION_COMMANDS but not sure if that was fully supported in MK3S+. Partially at least I think.

[bkerler]

In this case not a bug but a feature request, as most SD card related commands haven't yet been implemented so far (as SD card isn't used anymore but USB is used instead). As for M27, I already filed the PR #3739. It would be helpful if you can state the issued commands by octoprint with the "S" flag for auto report so I can trace back which functionality needs to be added to the xbuddy firmware. Thanks.

[arekm]

After "MK3S+" -> "MK3.5" upgrade it's a regression for me, so a bug. But naming doesn't matter for me. I get your thinking. New printer family, new software, new start. You can change label to feature request etc if you want.

OctoPrint]$ git grep " S{"
src/octoprint/plugins/action_command_prompt/__init__.py:                [f"{self._command} S{choice}"],
src/octoprint/printer/standard.py:                self.commands(f"M104 T{toolNum} S{value}", tags=tags)
src/octoprint/printer/standard.py:                self.commands(f"M104 S{value}", tags=tags)
src/octoprint/printer/standard.py:            self.commands(f"M140 S{value}", tags=tags)
src/octoprint/printer/standard.py:            self.commands(f"M141 S{value}", tags=tags)
src/octoprint/util/comm.py:                                f"M26 S{pos}",
src/octoprint/util/comm.py:            f"M155 S{interval}",
src/octoprint/util/comm.py:            f"M27 S{interval}",
src/octoprint/util/comm.py:            f"M154 S{interval}",
src/octoprint/util/comm.py:            f"M113 S{interval}",

so

• M155 Automatically send temperatures - that's already implemented
• M27 - in mentioned PR gcode: Add gcode M27 auto report support #3739
• M154 - Auto Report Position M114 / Cap:AUTOREPORT_POSITION mystery explained in [ENHANCEMENT] [OctoPrint support] Printer state information is not passed to host over serial #3748 (comment)

Unfortunately I guess that won't be enough. Old 8-bit firmware and also Marlin 1 or Marlin 2 prints various texts on serial that OctoPrint uses to figure out what is happening. All that sits in src/octoprint/util/comm.py in _monitor():

• position processing

##~~ position report processing
                if "X:" in line and "Y:" in line and "Z:" in line:

• temperature processing (that's already in 32bit firmware)

                ##~~ temperature processing
                elif (
                    " T:" in line
                    or line.startswith("T:")
                    or " T0:" in line
                    or line.startswith("T0:")
                    or ((" B:" in line or line.startswith("B:")) and "A:" not in line)

• capabilities process (that's also already in 32bit firmware)

##~~ Firmware capability report triggered by M115
                elif lower_line.startswith("cap:"):

• media handling. OctoPrint doesn't care if that's SD card or USB card or other media. It's just a some storage media.

Failure or "umounted" media. On M22 or just automatically after media failed insert/release.

                ##~~ SD Card handling
                elif self._sdEnabled:
                    if (
                        "SD init fail" in line
                        or "volume.init failed" in line
                        or "No media" in line
                        or "openRoot failed" in line
                        or "SD Card unmounted" in line
                        or "SD card released" in line
                    ):

"Mounted" / available media. On M21 or just automatically after media insert.

elif (
                        "SD card ok" in line or "Card successfully initialized" in line

• M20 files listing and printing needed texts like "Begin file list", "End file list". (that's already implemented in 32bit firmware although not in full blown form as it was in 8bit firmware - [BUG] Bring back support for long file names and timestamps in M20 L T g-code command #3054)

• M27 printing (as mentioned earlier - gcode: Add gcode M27 auto report support #3739)

elif "SD printing byte" in line:

elif (
                        "Not SD printing" in line

• M23

elif "File opened" in line and not self._ignore_select:
                        # answer to M23, at least on Marlin, Repetier and Sprinter: "File opened:%s Size:%d"

elif "File selected" in line:

• printing states

elif "Writing to file" in line and self.isStreaming():
                        self._changeState(self.STATE_PRINTING)
                    elif "Done printing file" in line and self.isSdPrinting():
                        # printer is reporting file finished printing
                        self._changeState(self.STATE_FINISHING)

("Writing to file: " == MSG_SD_WRITE_TO_FILE is called in CardReader::openFile() but no idea if this method is called by Buddy firmware)
("Done file printing" == MSG_FILE_PRINTED exists in Buddy source code but is not used)

• M73 responses

That's pretty important for monitoring progress and showing/estimating end of print time or time until filament change on host.

Old 8bit firmware did that in simple way:
https://github.com/prusa3d/Prusa-Firmware/blob/v3.13.2/Firmware/Marlin_main.cpp#L5948C2-L5960C10

Marlin 2 got this implemented not long time ago in more generic way, so that could be an inspiration, too MarlinFirmware/Marlin#24714
and the code:
https://github.com/MarlinFirmware/Marlin/blob/2.1.2.2/Marlin/src/gcode/lcd/M73.cpp#L76C3-L91C9

• action commands protocol:

Cap:HOST_ACTION_COMMANDS
(https://reprap.org/wiki/Firmware_Capabilities_Protocol#Cap:HOST_ACTION_COMMANDS)

Not sure if I found out everything needed. I could miss something.

Note that 32bit firmware doesn't have to follow exactly what 8bit firmware did. If there is cleaner and nicer way then OctoPrint can be patched to handle that, too. BUT diverging can be problematic since OctoPrint devs consider existing ways "a standard" (which it is after all) and don't like adding new variations of the same thing (and also due to maintenance and speed considerations)

[bkerler]

@arekm thanks for the detailed explanation. I cannot change the label as I'm not a prusa dev :)

I'll start having a look at "M154 - Auto Report Position / Cap:AUTOREPORT_POSITION".

[bkerler]

Ok, M154 doesn't seem to be implemented at all (not in Marlin and not in MK3 firmware)

[arekm]

Hmm

2024-01-27 13:37:30,603 - Recv: FIRMWARE_NAME:Prusa-Firmware 3.13.2 based on Marlin FIRMWARE_URL:https://github.com/prusa3d/Prusa-Firmware PROTOCOL_VERSION:1.0 MACHINE_TYPE:Prusa i3 MK3S-R EXTRUDER_COUNT:1 UUID:00000000-0000-0000-0000-000000000000
2024-01-27 13:37:30,625 - Recv: Cap:AUTOREPORT_TEMP:1
2024-01-27 13:37:30,643 - Recv: Cap:AUTOREPORT_FANS:1
2024-01-27 13:37:30,650 - Recv: Cap:AUTOREPORT_POSITION:1

Note: "Cap:AUTOREPORT_POSITION:1". Maybe it works without M154 being called. Ah, yes, M114 (and M114 is also called by OctoPrint).

Best way on what to choose to implement for better OctoPrint support would be by looking into serial.log from MK3S+ print job started from LCD. Sadly my OctoPrint rotated logs deleting all my serial.log files with jobs started from MK3S+ :/ And my printer is already upgraded to MK3.5. (well, there is https://github.com/vintagepc/MK404)

[bkerler]

I've just added M20 options in PR #3765.

[bkerler]

M114 does exist actually and it also works fine.

[arekm]

M114 does exist actually and it also works fine.

Nice. So the question now is if the firmware should (like 8bit) or should not report AUTOREPORT_POSITION capability.

Edit:
3d-gussner explained the mystery. There is a auto position reporting in 8bit firmware hidden under M155 C bitmask (prusa3d/Prusa-Firmware#2981) which predates existence of M154 in Marlin.

Octoprint isn't using M155 C.

So M154 could be implemented in 32bit firmware, OctoPrint would use this but all that would be a new thing that wasn't available (to OctoPrint) before.

[bkerler]

Yes, seems to be a bug on the MK3. So M154 doesn't exist, thus no Cap:AUTOREPORT_POS:1 is given, which is correct behaviour on the MK4.

[bkerler]

M73 is disabled as "LCD_SET_PROGRESS_MANUALLY" isn't enabled. So seems to be intended behaviour on the MK4.

[arekm]

Probably they don't want to support "M73 P" which is fine [1] but M73 report was the only way for host (OctoPrint) to get information about print progress and more importantly estimated times when job was started from sd/usb media.

Marlin2 firmware splits that into separate options:

SET_PROGRESS_PERCENT for "P" and M73_REPORT for reporting to host and M73_REPORT is what's missing mainly

https://github.com/MarlinFirmware/Marlin/blob/2.1.2.2/Marlin/src/gcode/lcd/M73.cpp#L76C3-L91C9

1. it it used for prints from OctoPrint over serial, so OctoPrint is able to tell firmware what's the progress etc. Some users might want that, too.

[bkerler]

ok, so added M73 report functionality as a PR #3767.

[arekm]

Got MK404 running, so two logs that can be useful in future:

• printing over usb(serial) from OctoPrint:

mk3s-print-from-octoprint-serial.txt

• printing from sd card started from printer LCD as seen by OctoPrint:

mk3s-print-from-sd-card.txt

[Gefionious]

@arekm Thank you very much for your hard work tracking and reporting this, much appreciated!
@bkerler Thank you for taking it seriously. Octoprint has been a godsend for managing my printers for several years now and allows all kinds of software extensions that Prusa would not want to maintain (for example, open standard camera support).

[Prusa-Support]

Good news! PR #3766 - which solves this issue - was merged, and implemented in FW 6.0.0.

Michele Moramarco
Prusa Research

[arekm]

It's not completed. This issue is meta issue gathering (hopefully) all missing stuff for OctoPrint and only one thing was (recently) merged.

Covered in this issue and not merged are:
#3739
#3765
#3767

[Prusa-Support]

PR #3739 couldn't be merged because it didn't pass the approbation control yet.
Also, I seem to understand there is already issue #3737 to cover the reported problem.

PR #3765 couldn't be merged because it didn't pass the approbation control yet.
Also, I seem to understand there is already issue #3054 to cover the reported problem.

PR #3767 - same as PR #3766 - was merged before the last firmware release so it is already implemented, theoretically.
This issue should remain closed as partly solved and in the advantage of already existing and probably more specific issues.
Please monitor the existing issues, feel free to add comments to relevant threads, and let me know if I'm missing anything else.
Thanks.

Michele Moramarco
Prusa Research