Technical specification

Overview

The notgram service reads a set of Telegram channels through a dedicated Telegram account and republishes their posts as a single Atom feed, so the channels can be followed without running Telegram on personal devices. The dedicated account is registered on a separate virtual phone number and follows only the wanted channels.

The system is written in Rust and runs in Docker on a server. It connects to Telegram as that account, listens for new channel posts in real time, stores their text and metadata in SQLite, uploads attached media to S3-compatible object storage, and exposes the stored posts as a single Atom feed. The feed is read in any RSS reader from a computer or a phone.

Goals and non-goals

Goals:

Follow Telegram channels through a dedicated user account and capture new posts as they arrive.
Persist post text and metadata in SQLite and attached media in S3-compatible storage.
Serve a single Atom feed of all channels over HTTP Basic Auth, with media linked from object storage.
Run unattended in Docker with minimal operational effort.
Build, test and ship through CI/CD, delivering the system as a Docker image.

Non-goals:

No web reader or any styled web page. The Atom feed is the only reading interface.
No read-state tracking. The feed presents posts newest first, with no read or unread state.
Single user. There is no account system, no multi-tenant support.
No backfill of history in the first version. Only posts that arrive while the ingestion service is connected are captured. See out of scope.

Architecture

The system is split into two long-running services that share one SQLite database:

The ingestion service, started by the run ingest command, connects to Telegram as the dedicated account (built on the grammers library, holding one authenticated user session), receives new channel posts, downloads their media into S3-compatible storage, and writes posts and metadata to the database. It is the only writer to the database.
The feed service, started by the run feed command, is an HTTP server that reads the database and serves the Atom feed, and nothing else. It only reads from the database and issues no writes.

Both services are built from the same Rust binary and run as separate Docker containers, each started by its own command. The same binary also provides one-off commands for login and subscription management.

The only externally reachable surface is the feed service, protected by HTTP Basic Auth. Media objects live in a public-read bucket and are referenced by URLs that contain the full content hash, which makes the URLs effectively unguessable.

Telegram ingestion

Authentication

A one-time login command authenticates the dedicated account and saves its session, so the ingestion service connects on its own afterwards without asking again. The login must support two-factor password authentication.

Subscription model

The set of followed channels is the account's own server-side membership, not a list kept in notgram configuration. Subscriptions are managed out of band through commands, and the ingestion service stores posts from whatever channels the account belongs to. There is no allowlist and no startup reconciliation step.

Subscription commands

subscribe <username-or-invite-link> adds a channel: it takes a public channel's username or a private channel's invite link and joins it with the dedicated account. Joining is required because a user account only receives posts from channels it has joined.

A known edge case: some private invite links lead to a join request that needs admin approval, so membership is not immediate in that case. Joining respects Telegram's rate limits, and Telegram caps the number of channels a single account can join (around 500).

list shows the account's current subscriptions.

Capture strategy

The ingestion service keeps a live connection to Telegram and receives new posts as they are published, rather than polling. The first version captures only the posts that arrive while the ingestion service is connected.

Peer filtering

Only posts from broadcast channels are stored. Anything else that reaches the dedicated account is ignored: private messages, basic groups, and supergroups. As a result, nothing a person sends to the account privately can ever enter a feed.

Dynamic channel discovery

When a post arrives from a channel that is not yet known, the channel is recorded automatically, with its title and username taken from the post. The set of known channels is accumulated data, not configuration.

Edits and deletions

Editing a captured post updates the stored text, and deleting a post is ignored so the archive is kept.

Stored data

The store is SQLite. It must retain at least the following information.

For each channel: its Telegram identity, username, title, and when it was first seen.
For each post: which channel it belongs to, its Telegram message id, when it was posted and last edited, its text in the raw Telegram format, and which album it belongs to when it is part of one.
For each media file: a reference to the stored object, its type and size, image dimensions or duration where relevant, and the content hash.

Media is content-addressed by the hash of its contents, so identical files are stored only once. The exact schema is left to implementation. The recommended access library is sqlx, with its built-in migrations kept as versioned SQL files embedded in the binary and applied on startup by the ingestion service.

Media handling

A single configurable size threshold applies uniformly to every attached file regardless of its type or extension. Photos, videos, documents and animations are treated the same way: a file is downloaded only when its size is at or below the threshold, and is skipped otherwise. The proposed default threshold is 50 MB.

Each downloaded file is identified by a hash of its contents and stored under a sharded path derived from the hexadecimal hash:

/ab/cd/ef/<full-hex-hash>.<ext>

The first three two-character segments are the leading characters of the hash, and the file name is the full hash with the original extension. Objects are uploaded to a public-read bucket in the existing Garage instance, reached over a configurable S3-compatible endpoint. A configured public base URL lets feed items link media directly from object storage. SQLite stores only the object reference and metadata, never the file contents.

Feed output

A single Atom feed carries the posts from all subscribed channels together, newest first, without separating them by channel. Each item identifies the channel it came from. The feed is served at a plain URL, and access is protected by HTTP Basic Auth: the feed service checks the credentials the RSS client sends against the password in configuration. The URL itself does not need to be secret.

Each feed item is built from a stored post by a dedicated module that converts the raw Telegram representation in the database into the form the feed needs. The text formatting, such as bold and code, is preserved, but links and media are not shown inline. Instead, the links used in the text and the post's media files are collected into a numbered list at the end of the item. Each link used in the text is marked with a small superscript number that matches its entry in that list, and the media files are added to the list as plain links. The parts of an album are rendered as a single item. This rendering is the current intended approach and stays flexible during development, since the goal is a convenient reading experience rather than a fixed layout.

Configuration

The binary reads its settings from a configuration file. The recommended format is TOML, read into typed structures with the toml and serde crates, since TOML is the configuration format of the Rust ecosystem. The exact set of options is decided during development, once the precise needs are clear, so it is not fixed here. The one firm point is that there is no channel list in configuration, because subscriptions live in the account.

Continuous integration and delivery

The project is built and shipped through a CI/CD pipeline, run by Forgejo Actions, since the repository is hosted on Forgejo. On each change the pipeline builds the project, runs the tests, and checks formatting and lints. On a release it builds the Docker image and publishes it to a container registry, from which the containers are deployed.

Deployment

The two services ship as one Docker image holding a single binary, started as two containers from the run ingest and run feed commands. Both containers mount the same volume, which holds the SQLite database and the Telegram session, and they must run on the same host, because SQLite file locking is not reliable over a network filesystem. Only the feed service's HTTP port is exposed, ideally behind a reverse proxy that terminates TLS. The Garage instance is reached over the network. The login command is run once interactively to create the session before the services start.

Operational concerns

Logging covers connection state, captured posts, media uploads and errors.
Network and Telegram errors are retried with backoff, and Telegram's rate limits are honored.
A session that expires or is revoked is surfaced clearly so that login can be run again.
The same post is never stored twice, so reprocessing a post does not create duplicates.
The database runs in WAL mode so the ingestion service (the only writer) and the feed service (a reader) do not block each other. The ingestion service enables WAL and must start first, so it initializes the database before the feed service opens it. A busy timeout, set on every connection, lets brief lock contention retry instead of failing.

Out of scope of the MVP, planned as later features

Backfill of posts that were published while the service was disconnected. This is the first planned follow-up after the MVP.
An unsubscribe command to leave a channel.