Add IMAP/SMTP email provider for unattended bot mailboxes

[rockfordlhotka]

Motivation

Consumer Gmail accounts authenticated against an OAuth app in Testing publishing status (the only realistic state for a self-hosted server with restricted Gmail scopes) have refresh tokens revoked after 7 days. That makes consumer Gmail unusable for an unattended bot — the bot's tokens go stale every week and there is no headless way to recover.

A native IMAP/SMTP provider sidesteps OAuth entirely. Using a Gmail App Password (or any other IMAP/SMTP host's static credential), the bot operates with permanent credentials and no token-refresh treadmill. The new provider is also useful for any non-Microsoft/non-Google mailbox a user may want to attach.

Out of scope for this issue: any calendar or contacts surface — the new provider is email-only by design.

Provider definition

• Provider id: imap
• Aliases: imap, imap-smtp
• Library: MailKit (MIT, the standard .NET IMAP/SMTP client; same author as MimeKit, which the bulk-email tools may also benefit from)
• Capabilities reported by list_accounts: email only (no calendar, no contacts). Read/write — not read-only.

Configuration

Stored on AccountInfo.ProviderConfig. All keys are case-insensitive (per the recent fix in #49).

Key	Required	Default	Notes
imapHost	yes	imap.gmail.com	Form pre-fills the default; user can override for any host.
imapPort	no	993
smtpHost	yes	smtp.gmail.com
smtpPort	no	587	STARTTLS on 587, implicit TLS on 465 — SecureSocketOptions.Auto picks per port.
username	yes	(none)	Typically the email address.
password	yes	(none)	App password / IMAP password. Encrypted at rest via ASP.NET DataProtection — see below.
inboxFolder	no	INBOX	The folder treated as the default for GetEmailsAsync.
sentFolder	no	[Gmail]/Sent Mail	After SMTP send, the message is APPENDed here so it appears in the user's Sent box.
trashFolder	no	[Gmail]/Trash	DeleteEmailAsync moves to this folder rather than expunging.

Defaults are Gmail-tuned but not hard-coded in the provider — the form and the validator supply them when a user picks the IMAP option, so any IMAP host works by overriding the relevant fields.

Capability surface

ImapProviderService implements the full IProviderService interface but only the email methods do real work. Calendar and contact methods throw NotSupportedException with a clear message ("Provider 'imap' does not support calendar/contact operations"). ListAccountsTool already advertises capabilities to LLM callers, so well-behaved callers will not hit those paths.

Email methods to implement:

• GetEmailsAsync — opens inboxFolder, returns the most recent N (with optional unread filter)
• SearchEmailsAsync — uses IMailFolder.Search with SearchQuery.SubjectContains / BodyContains / etc. Date range honored.
• GetEmailDetailsAsync — fetch full message by parsed ID
• SendEmailAsync — SMTP send via SmtpClient, then APPEND a copy to sentFolder so it shows up in the user's Sent box
• DeleteEmailAsync — IMAP MOVE to trashFolder (not EXPUNGE; matches Gmail/most clients)
• MarkEmailAsReadAsync — set/unset \Seen flag
• MoveEmailAsync — IMAP MOVE to caller-supplied destination folder
• Bulk variants (BulkDeleteEmailsTool, BulkMarkEmailsAsReadTool, BulkMoveEmailsTool) call the same single-message methods today; they continue to work without IMAP-specific changes.
• UnsubscribeFromEmailTool / GetUnsubscribeInfoTool — MailKit exposes the List-Unsubscribe and List-Unsubscribe-Post headers as MIME header values; the existing UnsubscribeInfo parser in EmailMessage works as long as the provider populates those headers when fetching.

Email ID format

IMAP messages are uniquely identified by (folder, UIDVALIDITY, UID). UID alone is not stable across folder recreations. The provider encodes IDs as:

{folder}/{uidvalidity}/{uid}

e.g. INBOX/1234567890/4567. The provider parses this format on every *ById call. Using / as the separator avoids collision with the : characters that appear inside Gmail's special folder names like [Gmail]/Trash.

If parsing fails (caller sent a non-IMAP id), return a clear error rather than silently misinterpreting it.

Connection management

For the first cut: open a fresh ImapClient / SmtpClient per call, authenticate, do work, dispose. MailKit connection setup is fast (single-digit hundreds of ms over TLS), and this avoids the complexity of pooling, reconnect, and IDLE keep-alives in the initial implementation. If usage patterns make this a hotspot, a per-account pool can be added later — call this out as a follow-up rather than over-engineering up front.

SecureSocketOptions.Auto for both clients (picks STARTTLS or implicit TLS by port).

Password storage — ASP.NET DataProtection

The password field is encrypted at rest using ASP.NET DataProtection. The plaintext password never lands in appsettings.json on disk; the persisted value is the DataProtection-protected blob with an ENC: prefix marker (e.g. ENC:CfDJ8...).

Plumbing:

• New helper src/CalendarMcp.Core/Security/PasswordProtector.cs wrapping IDataProtector with the ENC: prefix convention. Methods: Protect(plaintext) -> string, Unprotect(stored) -> string. If the stored value lacks the ENC: prefix, Unprotect returns the value as-is — defensive against any pre-encryption migration data.
• New helper src/CalendarMcp.Core/Configuration/DataProtectionExtensions.cs registering services.AddDataProtection().PersistKeysToFileSystem(new DirectoryInfo(Path.Combine(ConfigurationPaths.GetDataDirectory(), "keys"))) and .SetApplicationName("CalendarMcp"). Called from each host's Program.cs (HttpServer, StdioServer, Cli) so encryption is host-portable.
• The keys directory (/app/data/keys/ in k8s, %LOCALAPPDATA%\CalendarMcp\keys\ locally) is auto-created on first protect operation. In k8s it persists on the existing PVC — survives pod restarts.
• ImapProviderService injects PasswordProtector and calls Unprotect before authenticating to IMAP/SMTP.
• AccountFormModels.BuildProviderConfig (Add) protects the password before assigning to the dictionary. EditAccountFormModel.FromAccountInfo unprotects for display in the password field; ToAccountInfo re-protects on save.

Threat model: the encryption protects the password against anyone who has read access to appsettings.json but not to the keys directory. If both are exfiltrated, encryption provides no protection — same as any local-key encryption scheme. This is a meaningful improvement over plaintext for the common k8s case (config volume vs. keys volume permission separation, accidental log/grep leaks, ad-hoc file shares) without claiming to solve full-disk compromise.

Follow-up: the existing Google ClientSecret field is still plaintext today. Migrating it to the same PasswordProtector is a separate small issue that can be done after this lands; out of scope here to keep the diff focused.

Files to add / touch

New files:

• src/CalendarMcp.Core/Security/PasswordProtector.cs — DataProtection wrapper.
• src/CalendarMcp.Core/Configuration/DataProtectionExtensions.cs — DI registration helper.
• src/CalendarMcp.Core/Services/IImapProviderService.cs — marker interface extending IProviderService.
• src/CalendarMcp.Core/Providers/ImapProviderService.cs — implementation.
• src/CalendarMcp.Tests/Providers/ImapProviderServiceTests.cs — tests (see testing notes).
• src/CalendarMcp.Tests/Security/PasswordProtectorTests.cs — round-trip + ENC-prefix detection tests.
• docs/IMAP-SETUP.md — Gmail app password walkthrough, folder reference, troubleshooting (mirrors the existing GOOGLE-SETUP.md / M365-SETUP.md pattern).

Modified — code:

• src/CalendarMcp.Core/CalendarMcp.Core.csproj — add MailKit and Microsoft.AspNetCore.DataProtection PackageReferences.
• src/CalendarMcp.Core/Providers/ProviderServiceFactory.cs — inject and dispatch imap / imap-smtp.
• src/CalendarMcp.Core/Configuration/ServiceCollectionExtensions.cs — register IImapProviderService and PasswordProtector.
• src/CalendarMcp.HttpServer/Program.cs, src/CalendarMcp.StdioServer/Program.cs, src/CalendarMcp.Cli/Program.cs — call the new DataProtection extension.
• src/CalendarMcp.Auth/AccountValidation.cs — add imap to KnownProviders; add ValidateImapConfig requiring imapHost, smtpHost, username, password.
• src/CalendarMcp.Core/Tools/ListAccountsTool.cs — capability mapping for imap returns email-only.
• src/CalendarMcp.HttpServer/BlazorAdmin/AccountFormModels.cs — IMAP fields on both CreateAccountFormModel and EditAccountFormModel; protect/unprotect password through PasswordProtector in BuildProviderConfig / FromAccountInfo; default Gmail values pre-populated when the user selects the IMAP provider.
• src/CalendarMcp.HttpServer/Components/Pages/AddAccount.razor — new case "imap" block with form fields for host/port/username/password and the optional folder overrides under an Advanced section.
• src/CalendarMcp.HttpServer/Components/Pages/EditAccount.razor — same form fields.
• src/CalendarMcp.HttpServer/Components/Shared/ProviderPicker.razor — add an IMAP card (icon bi-envelope-at, description "Any IMAP/SMTP mailbox — Gmail, Fastmail, custom, etc.").

Modified — docs:

• README.md — add IMAP to the supported-providers list near the top.
• docs/providers.md — full IMAP section: id, aliases, capabilities, config schema with defaults, folder semantics, ID format.
• docs/configuration.md — IMAP config keys in the reference table.
• docs/security.md — document password encryption-at-rest via DataProtection: keystore location, what it protects against, what it doesn't, k8s key persistence note.
• docs/authentication.md — IMAP/SMTP section explaining app passwords (with a Gmail-specific subsection) and pointing to the new IMAP-SETUP.md.
• docs/IMPLEMENTATION-STATUS.md — flip IMAP to Implemented in any provider matrix.

Testing

MailKit's classes (ImapClient, SmtpClient) are largely concrete and not designed for mocking. Two paths:

1. Unit-level: cover ID parsing/formatting, configuration validation, capability reporting, and PasswordProtector round-trip with normal MSTest tests. Deterministic, no network.
2. Integration-level (deferred): a smoke test against a local fake IMAP server (e.g., GreenMail in a Docker test container) is overkill for this initial PR. Note as a follow-up.

Net new test count: ~15-20 covering ID round-trip, config validation, factory dispatch for imap / imap-smtp aliases, capability reporting, and password protector behavior (including the ENC: prefix passthrough).

Out of scope (explicit non-goals)

• Calendar / contacts via IMAP-adjacent protocols (CalDAV/CardDAV).
• OAuth XOAUTH2 against IMAP — supported by Gmail but defeats the purpose (still subject to the 7-day refresh-token expiry).
• Connection pooling / IDLE push — mentioned as a follow-up if measurements show contention.
• TLS client certificates / Kerberos — only username/password auth in this iteration.
• Migrating an existing OAuth Google account to IMAP in place — users delete and re-add; the auth model is fundamentally different.
• Migrating Google ClientSecret to PasswordProtector — separate follow-up issue.

Definition of done

• New imap provider can be added through the admin UI with Gmail defaults filled in.
• Stored password in appsettings.json is the ENC:-prefixed protected blob, never plaintext. New tests prove round-trip correctness.
• DataProtection keys persist on the data volume (verified by restarting the container and confirming a previously-saved IMAP account still authenticates).
• All email tools (get_emails, search_emails, get_email_details, send_email, delete_email, mark_email_as_read, move_email, plus bulk variants and unsubscribe tools) work end-to-end against a Gmail mailbox using an app password.
• list_accounts reports the new account with email capability only.
• Calls to calendar/contact tools against an IMAP account return a clear NotSupportedException message rather than a confusing failure.
• All existing tests still pass; new tests cover ID handling, config validation, factory dispatch, capability reporting, and password protector behavior.
• Documentation is updated in the same PR (README, providers.md, configuration.md, security.md, authentication.md, IMPLEMENTATION-STATUS.md, new IMAP-SETUP.md).
• Docker image rebuilt; deployable to k8s without manifest changes.