Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ Access the dashboard at `http://localhost:8080`
## Features

- 📧 **IMAP Integration** - Automatically fetches DMARC reports from your email inbox
- 📬 **Multi-Inbox Support** - Fetch from multiple IMAP accounts concurrently
- 🔍 **RFC 7489 Compliant** - Fully compliant with DMARC aggregate report standards
- 📊 **Beautiful Dashboard** - Modern Vue.js 3 SPA with real-time statistics
- 🗄️ **SQLite Storage** - Lightweight embedded database, no external dependencies
Expand Down Expand Up @@ -125,6 +126,61 @@ The compiled binary will be at `./bin/parse-dmarc`.
}
```

### Multi-Inbox Support

Parse DMARC supports fetching from multiple IMAP inboxes concurrently. This is useful when you want to aggregate DMARC reports from multiple email accounts into a single dashboard.

**Using configuration file:**

Instead of using the single `imap` field, use the `imap_configs` array:

```json
{
"imap_configs": [
{
"host": "imap.gmail.com",
"port": 993,
"username": "account1@gmail.com",
"password": "your-app-password-1",
"mailbox": "INBOX",
"use_tls": true
},
{
"host": "imap.gmail.com",
"port": 993,
"username": "account2@gmail.com",
"password": "your-app-password-2",
"mailbox": "INBOX",
"use_tls": true
}
],
"database": {
"path": "~/.parse-dmarc/db.sqlite"
},
"server": {
"port": 8080,
"host": "0.0.0.0"
}
}
```

See `config.multi-inbox.example.json` for a complete example.

**Using environment variable:**

You can also configure multiple inboxes via the `IMAP_CONFIGS` environment variable with a JSON array:

```bash
export IMAP_CONFIGS='[{"host":"imap.gmail.com","port":993,"username":"account1@gmail.com","password":"pass1","mailbox":"INBOX","use_tls":true},{"host":"imap.gmail.com","port":993,"username":"account2@gmail.com","password":"pass2","mailbox":"INBOX","use_tls":true}]'
```

**Features:**
- Concurrent fetching from all configured inboxes
- Automatic deduplication of reports across inboxes
Copy link

Copilot AI Nov 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation claims 'Automatic deduplication of reports across inboxes', but this feature relies on the existing INSERT OR IGNORE behavior in SaveReport() which uses report_id as a unique constraint. This is not new functionality added by this PR. Consider clarifying that deduplication is handled by the existing database unique constraint on report_id, or remove this bullet if it's not specific to the multi-inbox feature.

Suggested change
- Automatic deduplication of reports across inboxes
- Deduplication of reports across inboxes is handled by the database's unique constraint on `report_id`

Copilot uses AI. Check for mistakes.
- Individual error handling per inbox (one failing inbox doesn't stop others)
- Progress logging shows which inbox is being processed
- Backward compatible with single inbox configuration

### Running

**Fetch reports and start dashboard:**
Expand Down
119 changes: 87 additions & 32 deletions cmd/parse-dmarc/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import (
"log"
"os"
"os/signal"
"sync"
"syscall"
"time"

Expand Down Expand Up @@ -167,50 +168,104 @@ func run(ctx context.Context, cmd *cli.Command) error {
func fetchReports(cfg *config.Config, store *storage.Storage) error {
log.Println("Fetching DMARC reports...")

// Create IMAP client
client := imap.NewClient(&cfg.IMAP)
if err := client.Connect(); err != nil {
return err
// Get all IMAP configurations (supports both single and multiple inboxes)
imapConfigs := cfg.GetIMAPConfigs()
if len(imapConfigs) == 0 {
return fmt.Errorf("no IMAP configuration found")
}
defer func() { _ = client.Disconnect() }()

// Fetch reports
reports, err := client.FetchDMARCReports()
if err != nil {
return err
}
log.Printf("Fetching from %d inbox(es)...", len(imapConfigs))

if len(reports) == 0 {
log.Println("No new reports found")
return nil
}
var wg sync.WaitGroup
var mu sync.Mutex
var errors []error
totalProcessed := 0

// Fetch from each IMAP inbox concurrently
for i := range imapConfigs {
wg.Add(1)
go func(imapCfg *config.IMAPConfig, index int) {
Copy link

Copilot AI Nov 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The goroutine receives a pointer to the slice element (&imapConfigs[i]), which is safe, but it's better practice to pass by value for the config struct to avoid potential issues if the slice is modified elsewhere. Consider changing the parameter to imapCfg config.IMAPConfig and passing imapConfigs[i] instead of &imapConfigs[i] at line 249.

Copilot uses AI. Check for mistakes.
defer wg.Done()

log.Printf("Processing %d reports...", len(reports))
mailboxName := fmt.Sprintf("%s@%s:%s", imapCfg.Username, imapCfg.Host, imapCfg.Mailbox)
log.Printf("[Inbox %d/%d] Connecting to %s", index+1, len(imapConfigs), mailboxName)

// Process each report
processed := 0
for _, report := range reports {
for _, attachment := range report.Attachments {
feedback, err := parser.ParseReport(attachment.Data)
// Create IMAP client for this inbox
client := imap.NewClient(imapCfg)
if err := client.Connect(); err != nil {
mu.Lock()
errors = append(errors, fmt.Errorf("failed to connect to %s: %w", mailboxName, err))
mu.Unlock()
return
}
defer func() { _ = client.Disconnect() }()

// Fetch reports from this inbox
reports, err := client.FetchDMARCReports()
if err != nil {
log.Printf("Failed to parse %s: %v", attachment.Filename, err)
continue
mu.Lock()
errors = append(errors, fmt.Errorf("failed to fetch from %s: %w", mailboxName, err))
mu.Unlock()
return
}

if err := store.SaveReport(feedback); err != nil {
log.Printf("Failed to save report %s: %v", feedback.ReportMetadata.ReportID, err)
continue
if len(reports) == 0 {
log.Printf("[Inbox %d/%d] No new reports found in %s", index+1, len(imapConfigs), mailboxName)
return
}

log.Printf("Saved report: %s from %s (domain: %s, messages: %d)",
feedback.ReportMetadata.ReportID,
feedback.ReportMetadata.OrgName,
feedback.PolicyPublished.Domain,
feedback.GetTotalMessages())
processed++
log.Printf("[Inbox %d/%d] Processing %d report(s) from %s", index+1, len(imapConfigs), len(reports), mailboxName)

// Process each report
processed := 0
for _, report := range reports {
for _, attachment := range report.Attachments {
feedback, err := parser.ParseReport(attachment.Data)
if err != nil {
log.Printf("[Inbox %d/%d] Failed to parse %s: %v", index+1, len(imapConfigs), attachment.Filename, err)
continue
}

if err := store.SaveReport(feedback); err != nil {
log.Printf("[Inbox %d/%d] Failed to save report %s: %v", index+1, len(imapConfigs), feedback.ReportMetadata.ReportID, err)
continue
}

log.Printf("[Inbox %d/%d] Saved report: %s from %s (domain: %s, messages: %d)",
index+1, len(imapConfigs),
feedback.ReportMetadata.ReportID,
feedback.ReportMetadata.OrgName,
feedback.PolicyPublished.Domain,
feedback.GetTotalMessages())
processed++
}
}

mu.Lock()
totalProcessed += processed
mu.Unlock()

log.Printf("[Inbox %d/%d] Successfully processed %d report(s) from %s", index+1, len(imapConfigs), processed, mailboxName)
}(&imapConfigs[i], i)
}

// Wait for all fetches to complete
wg.Wait()

// Report results
if len(errors) > 0 {
log.Printf("Completed with %d error(s):", len(errors))
for _, err := range errors {
log.Printf(" - %v", err)
}
}

log.Printf("Successfully processed %d reports", processed)
log.Printf("Successfully processed %d total report(s) across all inboxes", totalProcessed)

// Return error only if all fetches failed
if len(errors) > 0 && len(errors) == len(imapConfigs) {
return fmt.Errorf("all inbox fetches failed")
}

return nil
}
35 changes: 35 additions & 0 deletions config.multi-inbox.example.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{
"database": {
"path": "~/.parse-dmarc/db.sqlite"
},
"imap_configs": [
{
"host": "imap.gmail.com",
"mailbox": "INBOX",
"password": "your-app-password-1",
"port": 993,
"use_tls": true,
"username": "account1@gmail.com"
},
{
"host": "imap.gmail.com",
"mailbox": "INBOX",
"password": "your-app-password-2",
"port": 993,
"use_tls": true,
"username": "account2@gmail.com"
},
{
"host": "imap.example.com",
"mailbox": "DMARC",
"password": "your-password-3",
"port": 993,
"use_tls": true,
"username": "account3@example.com"
}
],
"server": {
"host": "0.0.0.0",
"port": 8080
}
}
67 changes: 58 additions & 9 deletions internal/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,10 @@ import (

// Config holds the application configuration
type Config struct {
IMAP IMAPConfig `json:"imap"`
Database DatabaseConfig `json:"database"`
Server ServerConfig `json:"server"`
IMAP IMAPConfig `json:"imap"`
IMAPConfigs []IMAPConfig `json:"imap_configs"`
Database DatabaseConfig `json:"database"`
Server ServerConfig `json:"server"`
}

// IMAPConfig holds IMAP server configuration
Expand All @@ -36,6 +37,21 @@ type ServerConfig struct {
Host string `json:"host" env:"SERVER_HOST" envDefault:"0.0.0.0"`
}

// GetIMAPConfigs returns all IMAP configurations, normalizing single and multiple configs
func (c *Config) GetIMAPConfigs() []IMAPConfig {
// If IMAPConfigs array is populated, use it
if len(c.IMAPConfigs) > 0 {
return c.IMAPConfigs
}

// Otherwise, use the single IMAP config for backward compatibility
if c.IMAP.Host != "" {
return []IMAPConfig{c.IMAP}
}

return []IMAPConfig{}
}

func defaultDBPath() (string, error) {
home, err := os.UserHomeDir()
if err != nil {
Expand Down Expand Up @@ -69,15 +85,48 @@ func Load(path string) (*Config, error) {
}
}

if err := env.Parse(&cfg); err != nil {
return nil, err
// Support IMAP_CONFIGS environment variable for multiple configs
imapConfigsJSON := os.Getenv("IMAP_CONFIGS")
if imapConfigsJSON != "" {
var imapConfigs []IMAPConfig
if err := json.Unmarshal([]byte(imapConfigsJSON), &imapConfigs); err != nil {
return nil, err
}
cfg.IMAPConfigs = imapConfigs
}
Comment on lines +88 to +96
Copy link

Copilot AI Nov 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The IMAP_CONFIGS environment variable overrides any imap_configs loaded from the config file without warning or merging. This could lead to confusion if a user has both a config file with imap_configs and the IMAP_CONFIGS environment variable set. Consider documenting this precedence order or logging a warning when the environment variable overrides the file configuration.

Copilot uses AI. Check for mistakes.

// Only parse environment variables if we're using single IMAP config
// (env.Parse incorrectly applies envDefault to array elements)
if len(cfg.IMAPConfigs) == 0 {
if err := env.Parse(&cfg); err != nil {
return nil, err
}
} else {
// Parse only non-IMAP fields to avoid envDefault overwriting array values
if err := env.Parse(&cfg.Database); err != nil {
return nil, err
}
if err := env.Parse(&cfg.Server); err != nil {
return nil, err
}
}

if cfg.IMAP.Port == 0 {
cfg.IMAP.Port = 993
// Apply defaults to all IMAP configs
allConfigs := cfg.GetIMAPConfigs()
for i := range allConfigs {
if allConfigs[i].Port == 0 {
allConfigs[i].Port = 993
}
if allConfigs[i].Mailbox == "" {
allConfigs[i].Mailbox = "INBOX"
}
}
Comment on lines +115 to 123
Copy link

Copilot AI Nov 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The defaults are being applied to a copy of the slice returned by GetIMAPConfigs(), not to the actual config values. This means that if cfg.IMAPConfigs is populated, the changes to allConfigs won't affect cfg.IMAPConfigs, and if cfg.IMAP is used, changes won't affect cfg.IMAP. The issue is partially addressed by lines 126-130, but those lines won't work correctly when a single IMAP config exists because cfg.IMAP.Host != \"\" will be true but allConfigs will contain modified copies, not the original. Consider directly iterating over cfg.IMAPConfigs when it's populated, and handling cfg.IMAP separately.

Copilot uses AI. Check for mistakes.
if cfg.IMAP.Mailbox == "" {
cfg.IMAP.Mailbox = "INBOX"

// Update the config with normalized values
if len(cfg.IMAPConfigs) > 0 {
cfg.IMAPConfigs = allConfigs
} else if cfg.IMAP.Host != "" {
cfg.IMAP = allConfigs[0]
Comment on lines +128 to +129
Copy link

Copilot AI Nov 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When using single IMAP config, if allConfigs is empty (which can only happen if cfg.IMAP.Host is empty based on GetIMAPConfigs() logic), this code will panic with an index out of bounds error when accessing allConfigs[0]. Add a length check before accessing the first element or restructure the logic to ensure allConfigs is non-empty.

Copilot uses AI. Check for mistakes.
}
if cfg.Database.Path == "" {
cfg.Database.Path, err = defaultDBPath()
Expand Down