Ingestion

From entry to content

A feed entry doesn't land in a side table — it becomes a content record with a readable body, an attributed image, and the same lifecycle every other piece of platform content follows.

Dedup & lifecycle

Fetch twice, store once

Every entry is keyed by its source and external GUID — so re-fetching a feed is idempotent, however often the schedule fires.

A new GUID creates a content record with the feed fields — title, summary, author, published date — and source attribution: source id, GUID, article link.
A known GUID updates the existing record in place; nothing is ever duplicated.
New items are brought to ready, firing the standard readiness event downstream consumers react to — updates don't re-fire it.
The dedup mapping — source plus GUID — is recorded alongside the record, so every item traces back to the exact entry it came from.

ingest · 12 entries

guid:a41f…known → updated in place

guid:b7c2…new → created · ready

guid:c9d8…new → created · ready

2 readiness events fired · 0 duplicates

Normalization

Readable bodies, originals kept

Feed HTML is messy. Every item gets a clean, structured body — and the mess is kept on file, in case you ever want to re-process it.

The entry's HTML — content:encoded, Atom content, or JSON Feed's content_html — is converted to the platform's rich-text document model.
That normalized document is the item's body, so serving and preview render it like any authored content.
The original, un-normalized HTML is preserved as a supplementary attachment under the key original — for audit or re-processing.
Every item is bound to a shared document template, so feed items carry a consistent type across the platform.

body · normalized

<!-- original — kept as `original` -->
<div><b>Breaking:</b> markets rally…</div>

// stored body — rich-text document
{ "type": "doc", "content": [
  { "type": "paragraph", … }
] }

Featured images

The right image, with its credit

Each item's most prominent image becomes its own content record — imported into platform storage, attributed, and linked to the item.

Chosen in order: media:content with image semantics, media:thumbnail, an image enclosure, JSON Feed's image or banner_image — falling back to the first image in the body HTML.
Attribution comes from the feed's declared media:credit, or the publisher's host when none is declared.
The bytes are imported into platform storage by the import job — items don't hotlink the origin.
The image links to its item as an image.featured relationship, and relationships persist across re-fetches — a forced fetch backfills images for older items.

image extraction

1media:content / media:thumbnail

2image enclosure

3JSON Feed image · banner_image

4first <img> in the body

→ imported · credited · linked as image.featured

Keep exploring

From entry to content

Fetch twice, store once

Readable bodies, originals kept

The right image, with its credit

More on Feeds

Feeds overview

Sources

Serving & subscriptions