Skip to content

Commit 5034449

Browse files
authored
feat!: QR codes and symmetric encryption for broadcast channels (#7268)
Follow-up for #7042, part of #6884. This will make it possible to create invite-QR codes for broadcast channels, and make them symmetrically end-to-end encrypted. - [x] Go through all the changes in #7042, and check which ones I still need, and revert all other changes - [x] Use the classical Securejoin protocol, rather than the new 2-step protocol - [x] Make the Rust tests pass - [x] Make the Python tests pass - [x] Fix TODOs in the code - [x] Test it, and fix any bugs I find - [x] I found a bug when exporting all profiles at once fails sometimes, though this bug is unrelated to channels: #7281 - [x] Do a self-review (i.e. read all changes, and check if I see some things that should be changed) - [x] Have this PR reviewed and merged - [ ] Open an issue for "TODO: There is a known bug in the securejoin protocol" - [ ] Create an issue that outlines how we can improve the Securejoin protocol in the future (I don't have the time to do this right now, but want to do it sometime in winter) - [ ] Write a guide for UIs how to adapt to the changes (see deltachat/deltachat-android#3886) ## Backwards compatibility This is not very backwards compatible: - Trying to join a symmetrically-encrypted broadcast channel with an old device will fail - If you joined a symmetrically-encrypted broadcast channel with one device, and use an old core on the other device, then the other device will show a mostly empty chat (except for two device messages) - If you created a broadcast channel in the past, then you will get an error message when trying to send into the channel: > The up to now "experimental channels feature" is about to become an officially supported one. By that, privacy will be improved, it will become faster, and less traffic will be consumed. > > As we do not guarantee feature-stability for such experiments, this means, that you will need to create the channel again. > > Here is what to do: > • Create a new channel > • Tap on the channel name > • Tap on "QR Invite Code" > • Have all recipients scan the QR code, or send them the link > > If you have any questions, please send an email to delta@merlinux.eu or ask at https://support.delta.chat/. ## The symmetric encryption Symmetric encryption uses a shared secret. Currently, we use AES128 for encryption everywhere in Delta Chat, so, this is what I'm using for broadcast channels (though it wouldn't be hard to switch to AES256). The secret shared between all members of a broadcast channel has 258 bits of entropy (see `fn create_broadcast_shared_secret` in the code). Since the shared secrets have more entropy than the AES session keys, it's not necessary to have a hard-to-compute string2key algorithm, so, I'm using the string2key algorithm `salted`. This is fast enough that Delta Chat can just try out all known shared secrets. [^1] In order to prevent DOS attacks, Delta Chat will not attempt to decrypt with a string2key algorithm other than `salted` [^2]. ## The "Securejoin" protocol that adds members to the channel after they scanned a QR code This PR uses the classical securejoin protocol, the same that is also used for group and 1:1 invitations. The messages sent back and forth are called `vg-request`, `vg-auth-required`, `vg-request-with-auth`, and `vg-member-added`. I considered using the `vc-` prefix, because from a protocol-POV, the distinction between `vc-` and `vg-` isn't important (as @link2xt pointed out in an in-person discussion), but 1. it would be weird if groups used `vg-` while broadcasts and 1:1 chats used `vc-`, 2. we don't have a `vc-member-added` message yet, so, this would mean one more different kind of message 3. we anyways want to switch to a new securejoin protocol soon, which will be a backwards incompatible change with a transition phase. When we do this change, we can make everything `vc-`. [^1]: In a symmetrically encrypted message, it's not visible which secret was used to encrypt without trying out all secrets. If this does turn out to be too slow in the future, then we can remember which secret was used more recently, and and try the most recent secret first. If this is still too slow, then we can assign a short, non-unique (~2 characters) id to every shared secret, and send it in cleartext. The receiving Delta Chat will then only try out shared secrets with this id. Of course, this would leak a little bit of metadata in cleartext, so, I would like to avoid it. [^2]: A DOS attacker could send a message with a lot of encrypted session keys, all of which use a very hard-to-compute string2key algorithm. Delta Chat would then try to decrypt all of the encrypted session keys with all of the known shared secrets. In order to prevent this, as I said, Delta Chat will not attempt to decrypt with a string2key algorithm other than `salted` BREAKING CHANGE: A new QR type AskJoinBroadcast; cloning a broadcast channel is no longer possible; manually adding a member to a broadcast channel is no longer possible (only by having them scan a QR code)
1 parent 997e821 commit 5034449

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

43 files changed

+2639
-479
lines changed

Cargo.toml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -156,6 +156,11 @@ name = "receive_emails"
156156
required-features = ["internals"]
157157
harness = false
158158

159+
[[bench]]
160+
name = "decrypting"
161+
required-features = ["internals"]
162+
harness = false
163+
159164
[[bench]]
160165
name = "get_chat_msgs"
161166
harness = false

benches/decrypting.rs

Lines changed: 200 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,200 @@
1+
//! Benchmarks for message decryption,
2+
//! comparing decryption of symmetrically-encrypted messages
3+
//! to decryption of asymmetrically-encrypted messages.
4+
//!
5+
//! Call with
6+
//!
7+
//! ```text
8+
//! cargo bench --bench decrypting --features="internals"
9+
//! ```
10+
//!
11+
//! or, if you want to only run e.g. the 'Decrypt a symmetrically encrypted message' benchmark:
12+
//!
13+
//! ```text
14+
//! cargo bench --bench decrypting --features="internals" -- 'Decrypt a symmetrically encrypted message'
15+
//! ```
16+
//!
17+
//! You can also pass a substring.
18+
//! So, you can run all 'Decrypt and parse' benchmarks with:
19+
//!
20+
//! ```text
21+
//! cargo bench --bench decrypting --features="internals" -- 'Decrypt and parse'
22+
//! ```
23+
//!
24+
//! Symmetric decryption has to try out all known secrets,
25+
//! You can benchmark this by adapting the `NUM_SECRETS` variable.
26+
27+
use std::hint::black_box;
28+
29+
use criterion::{Criterion, criterion_group, criterion_main};
30+
use deltachat::internals_for_benches::create_broadcast_secret;
31+
use deltachat::internals_for_benches::create_dummy_keypair;
32+
use deltachat::internals_for_benches::save_broadcast_secret;
33+
use deltachat::{
34+
Events,
35+
chat::ChatId,
36+
config::Config,
37+
context::Context,
38+
internals_for_benches::key_from_asc,
39+
internals_for_benches::parse_and_get_text,
40+
internals_for_benches::store_self_keypair,
41+
pgp::{KeyPair, decrypt, pk_encrypt, symm_encrypt_message},
42+
stock_str::StockStrings,
43+
};
44+
use rand::{Rng, rng};
45+
use tempfile::tempdir;
46+
47+
const NUM_SECRETS: usize = 500;
48+
49+
async fn create_context() -> Context {
50+
let dir = tempdir().unwrap();
51+
let dbfile = dir.path().join("db.sqlite");
52+
let context = Context::new(dbfile.as_path(), 100, Events::new(), StockStrings::new())
53+
.await
54+
.unwrap();
55+
56+
context
57+
.set_config(Config::ConfiguredAddr, Some("bob@example.net"))
58+
.await
59+
.unwrap();
60+
let secret = key_from_asc(include_str!("../test-data/key/bob-secret.asc")).unwrap();
61+
let public = secret.signed_public_key();
62+
let key_pair = KeyPair { public, secret };
63+
store_self_keypair(&context, &key_pair)
64+
.await
65+
.expect("Failed to save key");
66+
67+
context
68+
}
69+
70+
fn criterion_benchmark(c: &mut Criterion) {
71+
let mut group = c.benchmark_group("Decrypt");
72+
73+
// ===========================================================================================
74+
// Benchmarks for decryption only, without any other parsing
75+
// ===========================================================================================
76+
77+
group.sample_size(10);
78+
79+
group.bench_function("Decrypt a symmetrically encrypted message", |b| {
80+
let plain = generate_plaintext();
81+
let secrets = generate_secrets();
82+
let encrypted = tokio::runtime::Runtime::new().unwrap().block_on(async {
83+
let secret = secrets[NUM_SECRETS / 2].clone();
84+
symm_encrypt_message(
85+
plain.clone(),
86+
create_dummy_keypair("alice@example.org").unwrap().secret,
87+
black_box(&secret),
88+
true,
89+
)
90+
.await
91+
.unwrap()
92+
});
93+
94+
b.iter(|| {
95+
let mut msg =
96+
decrypt(encrypted.clone().into_bytes(), &[], black_box(&secrets)).unwrap();
97+
let decrypted = msg.as_data_vec().unwrap();
98+
99+
assert_eq!(black_box(decrypted), plain);
100+
});
101+
});
102+
103+
group.bench_function("Decrypt a public-key encrypted message", |b| {
104+
let plain = generate_plaintext();
105+
let key_pair = create_dummy_keypair("alice@example.org").unwrap();
106+
let secrets = generate_secrets();
107+
let encrypted = tokio::runtime::Runtime::new().unwrap().block_on(async {
108+
pk_encrypt(
109+
plain.clone(),
110+
vec![black_box(key_pair.public.clone())],
111+
Some(key_pair.secret.clone()),
112+
true,
113+
true,
114+
)
115+
.await
116+
.unwrap()
117+
});
118+
119+
b.iter(|| {
120+
let mut msg = decrypt(
121+
encrypted.clone().into_bytes(),
122+
std::slice::from_ref(&key_pair.secret),
123+
black_box(&secrets),
124+
)
125+
.unwrap();
126+
let decrypted = msg.as_data_vec().unwrap();
127+
128+
assert_eq!(black_box(decrypted), plain);
129+
});
130+
});
131+
132+
// ===========================================================================================
133+
// Benchmarks for the whole parsing pipeline, incl. decryption (but excl. receive_imf())
134+
// ===========================================================================================
135+
136+
let rt = tokio::runtime::Runtime::new().unwrap();
137+
let mut secrets = generate_secrets();
138+
139+
// "secret" is the shared secret that was used to encrypt text_symmetrically_encrypted.eml.
140+
// Put it into the middle of our secrets:
141+
secrets[NUM_SECRETS / 2] = "secret".to_string();
142+
143+
let context = rt.block_on(async {
144+
let context = create_context().await;
145+
for (i, secret) in secrets.iter().enumerate() {
146+
save_broadcast_secret(&context, ChatId::new(10 + i as u32), secret)
147+
.await
148+
.unwrap();
149+
}
150+
context
151+
});
152+
153+
group.bench_function("Decrypt and parse a symmetrically encrypted message", |b| {
154+
b.to_async(&rt).iter(|| {
155+
let ctx = context.clone();
156+
async move {
157+
let text = parse_and_get_text(
158+
&ctx,
159+
include_bytes!("../test-data/message/text_symmetrically_encrypted.eml"),
160+
)
161+
.await
162+
.unwrap();
163+
assert_eq!(text, "Symmetrically encrypted message");
164+
}
165+
});
166+
});
167+
168+
group.bench_function("Decrypt and parse a public-key encrypted message", |b| {
169+
b.to_async(&rt).iter(|| {
170+
let ctx = context.clone();
171+
async move {
172+
let text = parse_and_get_text(
173+
&ctx,
174+
include_bytes!("../test-data/message/text_from_alice_encrypted.eml"),
175+
)
176+
.await
177+
.unwrap();
178+
assert_eq!(text, "hi");
179+
}
180+
});
181+
});
182+
183+
group.finish();
184+
}
185+
186+
fn generate_secrets() -> Vec<String> {
187+
let secrets: Vec<String> = (0..NUM_SECRETS)
188+
.map(|_| create_broadcast_secret())
189+
.collect();
190+
secrets
191+
}
192+
193+
fn generate_plaintext() -> Vec<u8> {
194+
let mut plain: Vec<u8> = vec![0; 500];
195+
rng().fill(&mut plain[..]);
196+
plain
197+
}
198+
199+
criterion_group!(benches, criterion_benchmark);
200+
criterion_main!(benches);

deltachat-ffi/deltachat.h

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2563,6 +2563,7 @@ void dc_stop_ongoing_process (dc_context_t* context);
25632563

25642564
#define DC_QR_ASK_VERIFYCONTACT 200 // id=contact
25652565
#define DC_QR_ASK_VERIFYGROUP 202 // text1=groupname
2566+
#define DC_QR_ASK_VERIFYBROADCAST 204 // text1=broadcast name
25662567
#define DC_QR_FPR_OK 210 // id=contact
25672568
#define DC_QR_FPR_MISMATCH 220 // id=contact
25682569
#define DC_QR_FPR_WITHOUT_ADDR 230 // test1=formatted fingerprint
@@ -2595,8 +2596,9 @@ void dc_stop_ongoing_process (dc_context_t* context);
25952596
* ask whether to verify the contact;
25962597
* if so, start the protocol with dc_join_securejoin().
25972598
*
2598-
* - DC_QR_ASK_VERIFYGROUP with dc_lot_t::text1=Group name:
2599-
* ask whether to join the group;
2599+
* - DC_QR_ASK_VERIFYGROUP or DC_QR_ASK_VERIFYBROADCAST
2600+
* with dc_lot_t::text1=Group name:
2601+
* ask whether to join the chat;
26002602
* if so, start the protocol with dc_join_securejoin().
26012603
*
26022604
* - DC_QR_FPR_OK with dc_lot_t::id=Contact ID:
@@ -2679,7 +2681,8 @@ dc_lot_t* dc_check_qr (dc_context_t* context, const char*
26792681
* Get QR code text that will offer an Setup-Contact or Verified-Group invitation.
26802682
*
26812683
* The scanning device will pass the scanned content to dc_check_qr() then;
2682-
* if dc_check_qr() returns DC_QR_ASK_VERIFYCONTACT or DC_QR_ASK_VERIFYGROUP
2684+
* if dc_check_qr() returns
2685+
* DC_QR_ASK_VERIFYCONTACT, DC_QR_ASK_VERIFYGROUP or DC_QR_ASK_VERIFYBROADCAST
26832686
* an out-of-band-verification can be joined using dc_join_securejoin()
26842687
*
26852688
* The returned text will also work as a normal https:-link,
@@ -2720,7 +2723,7 @@ char* dc_get_securejoin_qr_svg (dc_context_t* context, uint32_
27202723
* Continue a Setup-Contact or Verified-Group-Invite protocol
27212724
* started on another device with dc_get_securejoin_qr().
27222725
* This function is typically called when dc_check_qr() returns
2723-
* lot.state=DC_QR_ASK_VERIFYCONTACT or lot.state=DC_QR_ASK_VERIFYGROUP.
2726+
* lot.state=DC_QR_ASK_VERIFYCONTACT, lot.state=DC_QR_ASK_VERIFYGROUP or lot.state=DC_QR_ASK_VERIFYBROADCAST
27242727
*
27252728
* The function returns immediately and the handshake runs in background,
27262729
* sending and receiving several messages.

deltachat-ffi/src/lot.rs

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@ impl Lot {
4545
Self::Qr(qr) => match qr {
4646
Qr::AskVerifyContact { .. } => None,
4747
Qr::AskVerifyGroup { grpname, .. } => Some(Cow::Borrowed(grpname)),
48+
Qr::AskJoinBroadcast { name, .. } => Some(Cow::Borrowed(name)),
4849
Qr::FprOk { .. } => None,
4950
Qr::FprMismatch { .. } => None,
5051
Qr::FprWithoutAddr { fingerprint, .. } => Some(Cow::Borrowed(fingerprint)),
@@ -98,6 +99,7 @@ impl Lot {
9899
Self::Qr(qr) => match qr {
99100
Qr::AskVerifyContact { .. } => LotState::QrAskVerifyContact,
100101
Qr::AskVerifyGroup { .. } => LotState::QrAskVerifyGroup,
102+
Qr::AskJoinBroadcast { .. } => LotState::QrAskJoinBroadcast,
101103
Qr::FprOk { .. } => LotState::QrFprOk,
102104
Qr::FprMismatch { .. } => LotState::QrFprMismatch,
103105
Qr::FprWithoutAddr { .. } => LotState::QrFprWithoutAddr,
@@ -124,6 +126,7 @@ impl Lot {
124126
Self::Qr(qr) => match qr {
125127
Qr::AskVerifyContact { contact_id, .. } => contact_id.to_u32(),
126128
Qr::AskVerifyGroup { .. } => Default::default(),
129+
Qr::AskJoinBroadcast { .. } => Default::default(),
127130
Qr::FprOk { contact_id } => contact_id.to_u32(),
128131
Qr::FprMismatch { contact_id } => contact_id.unwrap_or_default().to_u32(),
129132
Qr::FprWithoutAddr { .. } => Default::default(),
@@ -166,6 +169,9 @@ pub enum LotState {
166169
/// text1=groupname
167170
QrAskVerifyGroup = 202,
168171

172+
/// text1=broadcast_name
173+
QrAskJoinBroadcast = 204,
174+
169175
/// id=contact
170176
QrFprOk = 210,
171177

deltachat-jsonrpc/src/api.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1030,7 +1030,7 @@ impl CommandApi {
10301030
.await
10311031
}
10321032

1033-
/// Create a new **broadcast channel**
1033+
/// Create a new, outgoing **broadcast channel**
10341034
/// (called "Channel" in the UI).
10351035
///
10361036
/// Broadcast channels are similar to groups on the sending device,

deltachat-jsonrpc/src/api/types/qr.rs

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,26 @@ pub enum QrObject {
3535
/// Authentication code.
3636
authcode: String,
3737
},
38+
/// Ask the user whether to join the broadcast channel.
39+
AskJoinBroadcast {
40+
/// The user-visible name of this broadcast channel
41+
name: String,
42+
/// A string of random characters,
43+
/// uniquely identifying this broadcast channel across all databases/clients.
44+
/// Called `grpid` for historic reasons:
45+
/// The id of multi-user chats is always called `grpid` in the database
46+
/// because groups were once the only multi-user chats.
47+
grpid: String,
48+
/// ID of the contact who owns the broadcast channel and created the QR code.
49+
contact_id: u32,
50+
/// Fingerprint of the broadcast channel owner's key as scanned from the QR code.
51+
fingerprint: String,
52+
53+
/// Invite number.
54+
invitenumber: String,
55+
/// Authentication code.
56+
authcode: String,
57+
},
3858
/// Contact fingerprint is verified.
3959
///
4060
/// Ask the user if they want to start chatting.
@@ -208,6 +228,25 @@ impl From<Qr> for QrObject {
208228
authcode,
209229
}
210230
}
231+
Qr::AskJoinBroadcast {
232+
name,
233+
grpid,
234+
contact_id,
235+
fingerprint,
236+
authcode,
237+
invitenumber,
238+
} => {
239+
let contact_id = contact_id.to_u32();
240+
let fingerprint = fingerprint.to_string();
241+
QrObject::AskJoinBroadcast {
242+
name,
243+
grpid,
244+
contact_id,
245+
fingerprint,
246+
authcode,
247+
invitenumber,
248+
}
249+
}
211250
Qr::FprOk { contact_id } => {
212251
let contact_id = contact_id.to_u32();
213252
QrObject::FprOk { contact_id }

deltachat-rpc-client/src/deltachat_rpc_client/account.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -326,7 +326,7 @@ def create_group(self, name: str) -> Chat:
326326
return Chat(self, self._rpc.create_group_chat(self.id, name, False))
327327

328328
def create_broadcast(self, name: str) -> Chat:
329-
"""Create a new **broadcast channel**
329+
"""Create a new, outgoing **broadcast channel**
330330
(called "Channel" in the UI).
331331
332332
Broadcast channels are similar to groups on the sending device,

deltachat-rpc-client/src/deltachat_rpc_client/message.py

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,17 @@ def wait_until_delivered(self) -> None:
9393
if event.kind == EventType.MSG_DELIVERED and event.msg_id == self.id:
9494
break
9595

96+
def resend(self) -> None:
97+
"""Resend messages and make information available for newly added chat members.
98+
Resending sends out the original message, however, recipients and webxdc-status may differ.
99+
Clients that already have the original message can still ignore the resent message as
100+
they have tracked the state by dedicated updates.
101+
102+
Some messages cannot be resent, eg. info-messages, drafts, already pending messages,
103+
or messages that are not sent by SELF.
104+
"""
105+
self._rpc.resend_messages(self.account.id, [self.id])
106+
96107
@futuremethod
97108
def send_webxdc_realtime_advertisement(self):
98109
"""Send an advertisement to join the realtime channel."""

0 commit comments

Comments
 (0)