Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions examples/theremin/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ This example generates real-time audio by creating sine waves at varying frequen
## Bricks Used

- `web_ui`: Brick that provides the web interface and a WebSocket channel for real-time control of the theremin.
- `wave_generator`: Brick that generates continuous audio waveforms and streams them to the USB speaker with smooth frequency and amplitude transitions.


## Hardware and Software Requirements
Expand Down Expand Up @@ -51,30 +52,29 @@ This example generates real-time audio by creating sine waves at varying frequen

## How it Works

The application creates a real-time audio synthesizer controlled by a web interface. User interactions on the webpage are sent to the Python backend via a WebSocket. The backend then calculates the audio parameters, generates a sine wave, and streams the audio data directly to the connected **USB** audio device.
The application creates a real-time audio synthesizer controlled by a web interface. User interactions on the webpage are sent to the Python backend via a WebSocket. The backend uses the `wave_generator` brick to continuously generate and stream audio to the connected **USB** audio device with smooth transitions.

- **User Interaction**: The frontend captures mouse or touch coordinates within a designated "play area".
- **Real-time Communication**: These coordinates are sent to the Python backend in real-time using the `web_ui` Brick's WebSocket channel.
- **Audio Synthesis**: The backend maps the X-coordinate to **frequency** and the Y-coordinate to **amplitude**. It uses a sine wave generator to create small blocks of audio data based on these parameters.
- **Audio Output**: The generated audio blocks are continuously streamed to the **USB** audio device, creating a smooth and responsive sound.
- **Audio Synthesis**: The backend maps the X-coordinate to **frequency** and the Y-coordinate to **amplitude**, then updates the `wave_generator` brick's state. The brick handles smooth transitions using configurable envelope parameters (attack, release, glide).
- **Audio Output**: The `wave_generator` brick runs continuously in a background thread, generating audio blocks and streaming them to the **USB** audio device with minimal latency.

High-level data flow:
```
Web Browser Interaction → WebSocket → Python Backend → Sine Wave Generation → USB Audio Device Output
Web Browser Interaction → WebSocket → Python Backend → WaveGenerator Brick → USB Audio Device Output
```


## Understanding the Code

### 🔧 Backend (`main.py`)

The Python code manages the web server, handles real-time user input, and performs all audio generation and playback.
The Python code manages the web server, handles real-time user input, and controls the audio generation brick.

- `ui = WebUI()` – Initializes the web server that serves the HTML interface and handles WebSocket communication.
- `speaker = Speaker(...)` – Initializes the connection to the USB audio device. This will raise an error if no compatible device is found.
- `sine_gen = SineGenerator(...)` – Creates an instance of the audio synthesis engine.
- `ui.on_message('theremin:move', on_move)` – Registers a handler that fires whenever the frontend sends new coordinates. This function updates the target frequency and amplitude.
- `theremin_producer_loop()` – Core audio engine. Runs continuously, generating ~**30 ms** blocks of audio based on the current frequency and amplitude, and streams them to the audio device for playback. This non-blocking, continuous stream ensures smooth audio without cracks or pops.
- `wave_gen = WaveGenerator(...)` – Creates the wave generator brick with configured envelope parameters (attack=0.01s, release=0.03s, glide=0.02s). The brick automatically manages the USB speaker connection and audio streaming in a background thread.
- `ui.on_message('theremin:move', on_move)` – Registers a handler that fires whenever the frontend sends new coordinates. This function updates the wave generator's frequency and amplitude using `wave_gen.set_frequency()` and `wave_gen.set_amplitude()`.
- The `wave_generator` brick handles all audio generation and streaming automatically, including smooth transitions between frequency and amplitude changes, continuous audio output with ~**30 ms** blocks, and non-blocking playback without cracks or pops.

### 💻 Frontend (`main.js`)

Expand All @@ -84,7 +84,7 @@ The web interface provides the interactive play area and controls for the user.
- **Event listeners** capture `mousedown`, `mousemove`, `mouseup` (and touch equivalents) to track user interaction in the play area.
- `socket.emit('theremin:move', { x, y })` – Sends normalized (0.0–1.0) X and Y coordinates to the backend; emissions are **throttled to ~80 Hz (≈12 ms)** to avoid overload.
- `socket.on('theremin:state', ...)` – Receives state updates from the backend (like the calculated frequency and amplitude) and updates the values displayed on the webpage.
- `socket.emit('theremin:set_volume', { volume })` – Sends a **0.0–1.0** master volume value and updates a progress bar in the UI.
- `socket.emit('theremin:set_volume', { volume })` – Sends a **0-100** hardware volume value to control the USB speaker's output level.
- `socket.emit('theremin:power', { on })` – Toggles synth power (**On/Off**). After turning **On**, move/tap in the play area to resume sound.


Expand Down
1 change: 1 addition & 0 deletions examples/theremin/app.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@ icon: 🎼
description: A simple theremin simulator that generates audio based on user input.
bricks:
- arduino:web_ui
- arduino:wave_generator
8 changes: 4 additions & 4 deletions examples/theremin/assets/main.js
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@

const thereminSvg = document.getElementById('theremin-svg');

let currentVolume = 0.8; // Default volume
let currentVolume = 80; // Default volume (0-100)
let powerOn = false;
let accessOn = false;
let isGridOn = false;
Expand Down Expand Up @@ -97,9 +97,9 @@
let newVolume = currentVolume;

if (plusBtn) {
newVolume = Math.min(1.0, currentVolume + 0.1);
newVolume = Math.min(100, currentVolume + 10);
} else if (minusBtn) {
newVolume = Math.max(0.0, currentVolume - 0.1);
newVolume = Math.max(0, currentVolume - 10);
}

if (newVolume !== currentVolume) {
Expand Down Expand Up @@ -274,7 +274,7 @@
function updateVolumeIndicator(volume) {
const indicator = document.getElementById('volume-indicator');
if (indicator) {
const angle = (volume - 0.5) * 180; // -90 to +90 degrees
const angle = ((volume / 100.0) - 0.5) * 180; // -90 to +90 degrees
indicator.style.transform = `rotate(${angle}deg)`;
}
}
Expand Down
170 changes: 54 additions & 116 deletions examples/theremin/python/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,150 +2,88 @@
#
# SPDX-License-Identifier: MPL-2.0

import threading
import time

from arduino.app_bricks.web_ui import WebUI
from arduino.app_peripherals.speaker import Speaker
from arduino.app_utils import App, SineGenerator
from arduino.app_bricks.wave_generator import WaveGenerator
from arduino.app_utils import App, Logger
import logging

logger = Logger("theremin", logging.DEBUG)

# configuration
SAMPLE_RATE = 16000
# duration of each produced block (seconds).
BLOCK_DUR = 0.03

# speaker setup
speaker = Speaker(sample_rate=SAMPLE_RATE, format='FLOAT_LE')
speaker.start()
speaker.set_volume(80)

# runtime state (module-level)
current_freq = 440.0
current_amp = 0.0
master_volume = 0.8
running = True

# Sine generator instance encapsulates buffers/state
sine_gen = SineGenerator(SAMPLE_RATE)
# Configure envelope parameters: attack, release, and frequency glide (portamento)
sine_gen.set_envelope_params(attack=0.01, release=0.03, glide=0.02)


# --- Producer scheduling ---------------------------------------------------------
# The example provides a producer loop that generates audio blocks at a steady
# cadence (BLOCK_DUR). The loop is executed under the application's main
# lifecycle by passing it to `App.run()`; we avoid starting background threads
# directly from example code so the AppController can manage startup/shutdown.

# event to wake the producer when state changes (e.g. on_move updates freq/amp)
prod_wake = threading.Event()

# Producer loop
# The producer loop is executed inside App.run() by passing a user_loop callable.
# This keeps the example simple and aligns with AppController's lifecycle management.
def theremin_producer_loop():
"""Single-iteration producer loop executed repeatedly by App.run().

This function performs one producer iteration: it generates a single
block and plays it non-blocking. `App.run()` will call this repeatedly
until the application shuts down (Ctrl+C).
"""
global running
next_time = time.perf_counter()
# lightweight single-iteration producer used by the App.run() user_loop.
while running:
# steady scheduling
next_time += float(BLOCK_DUR)

# if no amplitude requested, avoid stopping the producer indefinitely.
# Instead wait with a timeout and emit a silent block while idle. This
# keeps scheduling steady and avoids large timing discontinuities when
# the producer is woken again (which can produce audible cracks).
if current_amp <= 0.0:
prod_wake.clear()
# wait up to one block duration; if woken earlier we proceed
prod_wake.wait(timeout=BLOCK_DUR)
# emit a silent block to keep audio device scheduling continuous
if current_amp <= 0.0:
data = sine_gen.generate_block(float(current_freq), 0.0, BLOCK_DUR, master_volume)
speaker.play(data, block_on_queue=False)
# maintain timing
now = time.perf_counter()
sleep_time = next_time - now
if sleep_time > 0:
time.sleep(sleep_time)
else:
next_time = now
continue

# read targets
freq = float(current_freq)
amp = float(current_amp)

# generate one block and play non-blocking
data = sine_gen.generate_block(freq, amp, BLOCK_DUR, master_volume)
speaker.play(data, block_on_queue=False)

# wait until next scheduled time
now = time.perf_counter()
sleep_time = next_time - now
if sleep_time > 0:
time.sleep(sleep_time)
else:
next_time = now

# Wave generator brick - handles audio generation and streaming automatically
wave_gen = WaveGenerator(
sample_rate=SAMPLE_RATE,
wave_type="sine",
block_duration=0.03,
attack=0.01,
release=0.03,
glide=0.02,
)

# Set initial state
wave_gen.set_frequency(440.0)
wave_gen.set_amplitude(0.0)


# --- Web UI and event handlers -----------------------------------------------------
# The WaveGenerator brick handles audio generation and streaming automatically in
# a background thread. We only need to update frequency and amplitude via its API.
ui = WebUI()


def on_connect(sid, data=None):
ui.send_message('theremin:state', {'freq': current_freq, 'amp': current_amp})
ui.send_message('theremin:volume', {'volume': master_volume})
state = wave_gen.get_state()
ui.send_message("theremin:state", {"freq": state["frequency"], "amp": state["amplitude"]})
ui.send_message("theremin:volume", {"volume": state["volume"]})


def _freq_from_x(x):
return 20.0 * ((SAMPLE_RATE / 2.0 / 20.0) ** x)


def on_move(sid, data=None):
"""Update desired frequency/amplitude and wake producer.
"""Update desired frequency/amplitude.

The frontend should only send on mousedown/move/mouseup (no aggressive
repeat). This handler updates shared state and signals the producer. The
actual audio scheduling is handled by the producer loop executed under
`App.run()`.
The WaveGenerator brick handles smooth transitions automatically using
the configured envelope parameters (attack, release, glide).
"""
global current_freq, current_amp
d = data or {}
x = float(d.get('x', 0.0))
y = float(d.get('y', 1.0))
freq = d.get('freq')
x = float(d.get("x", 0.0))
y = float(d.get("y", 1.0))
freq = d.get("freq")
freq = float(freq) if freq is not None else _freq_from_x(x)
amp = max(0.0, min(1.0, 1.0 - float(y)))
current_freq = freq
current_amp = amp
# wake the producer so it reacts immediately
prod_wake.set()
ui.send_message('theremin:state', {'freq': freq, 'amp': amp}, room=sid)

logger.debug(f"on_move: x={x:.3f}, y={y:.3f} -> freq={freq:.1f}Hz, amp={amp:.3f}")

# Update wave generator state
wave_gen.set_frequency(freq)
wave_gen.set_amplitude(amp)

ui.send_message("theremin:state", {"freq": freq, "amp": amp}, room=sid)


def on_power(sid, data=None):
global current_amp
d = data or {}
on = bool(d.get('on', False))
on = bool(d.get("on", False))
if not on:
current_amp = 0.0
prod_wake.set()
wave_gen.set_amplitude(0.0)


def on_set_volume(sid, data=None):
global master_volume
d = data or {}
v = float(d.get('volume', master_volume))
master_volume = max(0.0, min(1.0, v))
ui.send_message('theremin:volume', {'volume': master_volume})
volume = int(d.get("volume", 100))
volume = max(0, min(100, volume))
wave_gen.set_volume(volume)
ui.send_message("theremin:volume", {"volume": volume})


ui.on_connect(on_connect)
ui.on_message('theremin:move', on_move)
ui.on_message('theremin:power', on_power)
ui.on_message('theremin:set_volume', on_set_volume)
ui.on_message("theremin:move", on_move)
ui.on_message("theremin:power", on_power)
ui.on_message("theremin:set_volume", on_set_volume)

# Run the app and use the theremin_producer_loop as the user-provided loop.
App.run(user_loop=theremin_producer_loop)
# Run the app - WaveGenerator handles audio generation automatically
App.run()