feat: receive pre-messages and adapt download on demand #7431

Simon-Laux · 2025-11-12T15:07:45Z

This PR implements receiving of pre-message, download of post-messages and the new logic for deciding what to fetch when from IMAP.

This is part of #7367

progress:

code quality / nitpicking

use tcm.section("..."); instead of comments in tests
check which log statements are important and remove the non relevant ones.

specific aspects that should become reality:

In receive_imf, the post-message can be handled similarly to Chat-Edit (grep for "ChatEdit" in receive_imf.rs).
As opposed to the current download-on-demand, we won't replace the whole msgs row, because this way, it will just work if an edit-message is received, and then later the user requests to download the attachment.
- Need to change Viewtype, probably metadata params like image size
pre-message gets Viewtype::Text in the database until downloaded
background_fetch should not download messages with the Chat-Is-Post-Message header.
~~background_fetch: If a large message without Chat-Is-Post-Message header. is received, the UI should be notified via an event.~~ postponed
A read receipt should be shown already when the user saw the pre-message. feat: pre messages can now get read receipts #7433

Later (not in this pr, not now)

metadata: thumbnails (image, no video thumbnail because it may be too hard)
metadata: webxdc name
faq/inApp: mention/explain somewhere that the download limit is now only about large attachments sent via chatmail/deltachat which trigger the pre-message/post-message split
write specs (used headers, the format of metadata header and the principles behind how this feature works)

API Changes:

all: change Message.get_filebytes - if message is not downloaded pre message, and metadata is available, then it returns the size that the file has after download, so ui can use this to show download size
duration and dimensions (width, heigth) are already set in pre-messages (if they contain that metadata)

src/message.rs

src/test_utils.rs

src/message.rs

Hocuri · 2025-12-01T11:48:37Z

background_fetch: If a large message without Chat-Is-Full-Message header. is received, the UI should be notified via an event.

rust: add Message.get_full_message_viewtype to get view type that message will have after being downloaded.

jsonrpc: add full_message_view_type to Message and MessageInfo

In the first iteration, it's not necessary to emit an event, or add any new API. (not sure how the event landed in the "steps" at #7367, but it already is in the "Future Possibilities")

src/context.rs

src/download.rs

Simon-Laux · 2025-12-01T18:15:21Z

src/imap.rs


        chat::mark_old_messages_as_noticed(context, received_msgs).await?;

+        // TODO: is there correct place for this?


address to do comment

src/mimefactory.rs

src/mimeparser.rs

Simon-Laux · 2025-12-01T18:43:27Z

src/scheduler.rs

            .context("delete_expired_imap_messages")?;
+
+        //-------
+        // TODO: verify that this is the correct position for this call


address todo comment

Guarding against lost pre-messages is optional, so, wouldn't actually be required for this first iteration of pre-messages. It's there now, but if this requires any further thought, we should just leave away.

From a logical standpoint, I would put it to the end of fetch_move_delete(), but it should be fine here, too (without me deeply thinking into this).

~~I think I put it here so it does only run on normal fetch and not on background fetch, because those post messages could be huge.~~

EDIT: I will think about it again, I'm a bit confused again.

Simon-Laux · 2025-12-01T18:59:07Z

src/message.rs

Simon-Laux · 2025-12-03T19:02:16Z

src/receive_imf.rs

+                    "Message is a Pre-Message (post_msg_exists:{post_msg_exists})."
+                );
+                post_msg_exists
+                // TODO find out if trashing affects multi device usage?


address todo comment

src/download.rs

fix python lint errors receive pre-mesages, start with changes to imap loop. refactor: move download code from `scheduler.rs` to `download.rs`, also move `get_msg_id_by_rfc724_mid` to `MsgId::get_by_rfc724_mid` `MAX_FETCH_MSG_SIZE` is no longer unused Parse if it is a pre-message or full-message start with receiving logic get rid of `MsgId::get_by_rfc724_mid` because it was a duplicate of `message::rfc724_mid_exists` docs: add hint to `MimeMessage::from_bytes` stating that it has side-effects. receiving full message send and receive `attachment_size` and set viewtype to text in pre_message metadata as struct in pre-message in header. And fill params that we can already fill from the metadata. Also add a new api to check what viewtype the message will have once downloaded. api: jsonrpc: add `full_message_view_type` to `Message` and `MessageInfo` make PreMsgMetadata.to_header_value not consume self/PreMsgMetadata add api to merge params on download full message: merge new params into old params and remove full-message metadata params move tests to `src/tests/pre_messages.rs` dynamically allocate test attachment bytes fix detection of pre-messages. (it looked for the ChatFullMessageId header in the unencrypted headers before) fix setting dl state to avaiable on pre-messages fix: save pre message with rfc724_mid of full message als disable replacement for full messages add some receiving tests and update test todo for premessage metadata test: process full message before pre-message test receive normal message some serialization tests for PreMsgMetadata remove outdated todo comment test that pre-message contains message text PreMsgMetadata: test_build_from_file_msg and test_build_from_file_msg test: test_receive_pre_message_image Test receiving the full message after receiving an edit after receiving the pre-message test_reaction_on_pre_message test_full_download_after_trashed test_webxdc_update_for_not_downloaded_instance simplify fake webxdc generation in test_webxdc_update_for_not_downloaded_instance test_markseen_pre_msg test_pre_msg_can_start_chat and test_full_msg_can_start_chat test_download_later_keeps_message_order test_chatlist_event_on_full_msg_download fix download not working log splitting into pre-message add pre-message info to text when loading from db. this can be disabled with config key `hide_pre_message_metadata_text` if ui wants to display it in a prettier way. update `download_limit` documentation more logging: log size of pre and post messages rename full message to Post-Message split up the pre-message tests into multiple files dedup test code by extracting code to create test messages into util methods remove post_message_view_type from api, now it is only used internally for tests remove `hide_pre_message_metadata_text` config option, as there currently is no way to get the full message viewtype anymore Update src/download.rs resolve comment use `parse_message_id` instead of removing `<>`parenthesis it manually fix available_post_msgs gets no entries handle forwarding and add a test for it. convert comment to log warning event on unexpected download failure add doc comment to `simple_imap_loop` more logging handle saving pre-message to self messages and test.

Immediately fully download all messages that do not have the `Chat-Is-Post-Message` header. This way, we simplify the logic for when which messages are downloaded, there are no differences at all anymore between 'background_fetch' and 'normal fetch'. Also, we prevent message reordering when reveiving a message from a legacy client. Messages larger than 1MB without `Chat-Is-Post-Message` should be rare, so, we do not expect this to really worsen things.

This was referenced Nov 12, 2025

[Tracking Issue] Pre-Messages #7367

Open

[Branch] pre-messages / next version of download on demand #7371

Draft

Simon-Laux force-pushed the simon/sending-pre-message branch 3 times, most recently from 2b900c3 to 305498d Compare November 16, 2025 20:07

Simon-Laux force-pushed the simon/receive-pre-messages branch 4 times, most recently from b94964a to ce97ba3 Compare November 23, 2025 02:27

Simon-Laux force-pushed the simon/receive-pre-messages branch from cd716c5 to 4c14566 Compare November 26, 2025 14:13

Base automatically changed from simon/sending-pre-message to pre-messages November 26, 2025 14:14

Simon-Laux force-pushed the simon/receive-pre-messages branch from 4c14566 to ca6bf65 Compare November 26, 2025 15:09

Simon-Laux commented Nov 30, 2025

View reviewed changes

src/message.rs Show resolved Hide resolved

Simon-Laux commented Nov 30, 2025

View reviewed changes

src/test_utils.rs Show resolved Hide resolved

Simon-Laux commented Nov 30, 2025

View reviewed changes

src/message.rs Outdated Show resolved Hide resolved

Hocuri reviewed Dec 1, 2025

View reviewed changes

src/context.rs Show resolved Hide resolved

Simon-Laux marked this pull request as ready for review December 1, 2025 15:46

Simon-Laux commented Dec 1, 2025

View reviewed changes

Simon-Laux mentioned this pull request Dec 2, 2025

draft: prettier pre-messages for files deltachat/deltachat-desktop#5800

Closed

r10s reviewed Dec 3, 2025

View reviewed changes

src/message.rs Outdated Show resolved Hide resolved

Simon-Laux force-pushed the simon/receive-pre-messages branch from 778dae2 to 2a3d5a3 Compare December 3, 2025 16:40

Simon-Laux commented Dec 3, 2025

View reviewed changes

Simon-Laux commented Dec 4, 2025

View reviewed changes

src/download.rs Outdated Show resolved Hide resolved

link2xt mentioned this pull request Dec 9, 2025

pre-message bugfixing branch #7560

Merged

link2xt force-pushed the pre-messages branch from 1ec5606 to bcbd75b Compare December 9, 2025 13:36

link2xt force-pushed the simon/receive-pre-messages branch 2 times, most recently from b7fb4cb to b6dc11a Compare December 9, 2025 14:20

link2xt force-pushed the pre-messages branch from 1d7b925 to 2f1c383 Compare December 9, 2025 14:29

link2xt force-pushed the simon/receive-pre-messages branch from b6dc11a to bfc58e9 Compare December 9, 2025 14:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat: receive pre-messages and adapt download on demand #7431

feat: receive pre-messages and adapt download on demand #7431

Simon-Laux commented Nov 12, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Hocuri commented Dec 1, 2025

Uh oh!

Uh oh!

Uh oh!

Simon-Laux Dec 1, 2025

Uh oh!

Uh oh!

Uh oh!

Simon-Laux Dec 1, 2025

Uh oh!

Hocuri Dec 3, 2025

Uh oh!

Simon-Laux Dec 3, 2025 •

edited

Loading

Uh oh!

Simon-Laux commented Dec 1, 2025 •

edited

Loading

Uh oh!

Uh oh!

Simon-Laux Dec 3, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants


		chat::mark_old_messages_as_noticed(context, received_msgs).await?;

		// TODO: is there correct place for this?

Uh oh!

feat: receive pre-messages and adapt download on demand #7431

Are you sure you want to change the base?

feat: receive pre-messages and adapt download on demand #7431

Conversation

Simon-Laux commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

progress:

code quality / nitpicking

specific aspects that should become reality:

Later (not in this pr, not now)

API Changes:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Hocuri commented Dec 1, 2025

Uh oh!

Uh oh!

Uh oh!

Simon-Laux Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Simon-Laux Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Hocuri Dec 3, 2025

Choose a reason for hiding this comment

Uh oh!

Simon-Laux Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Simon-Laux commented Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Progress of the tests

TODO Tests

postponed to do tests

discarded/ignored test ideas

Uh oh!

Uh oh!

Simon-Laux Dec 3, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Simon-Laux commented Nov 12, 2025 •

edited

Loading

Simon-Laux Dec 3, 2025 •

edited

Loading

Simon-Laux commented Dec 1, 2025 •

edited

Loading