v7.0

Breaking changes:

• jsonfeed:
  • jsonfeed_to_activities: return AS1 objects, not activities.

Non-breaking changes:

• as1:
  • activity_changed: add displayName, summary fields.
  • is_public: return False if the object/activity contains to that's empty or has only unknown aliases.
• as2:
  • Add support for the Application, Block, Flag, and Link types.
  • Generalize actor logic in to/from_as1 across all actor types, not just Person.
  • Add new link_tags function.
• atom:
  • activities_to_atom: handle image attachments without url field.
• bluesky:
  • to_as1:
    • Add support for:
      • app.bsky.embed.record
      • app.bsky.embed.recordWithMedia
      • app.bsky.feed.defs#notFoundPost
      • app.bsky.feed.generator
      • app.bsky.graph.block
      • app.bsky.graph.list
      • app.bsky.graph.listitem
      • com.atproto.admin.defs#repoRef
      • com.atproto.moderation.createReport#input
      • com.atproto.repo.strongRef
    • Add hashtag facet support.
    • Convert blobs in embeds to getBlob image URLs.
    • app.bsky.actor.profile: add HTML links for URLs in summary (snarfed/bridgy-fed#1065).
    • Escape HTML characters (<, >, &) in app.bsky.actor.profile description field.
    • Bug fix for create/update activities with bare string object.
  • from_as1:
    • Add hashtag, mention, block, and flag support. Interpret tags with missing objectType as hashtags.
    • Guess missing indices in facets based on content text. Otherwise, if we still don't know a facet's indices, discard it.
    • Extract HTML links ( tags) from HTML content and convert to link facets (snarfed/bridgy-fed#976).
    • If an output string value is longer than its maxGraphemes or maxLength in its lexicon, truncate it with an … ellipsis character at the end in order to fit. If this happens to post text, include a link embed pointing to the original post.
    • If the object has a video, include an external embed pointing to the original post and mark it as [Video] (snarfed/bridgy-fed#1078).
    • If the object has images, add the original post link to the end of the text, since Bluesky doesn't support both image and external embeds in the same post (bluesky-social/atproto#2575, snarfed/bridgy-fed#1106).
    • If a note has summary - often used for content warnings in the fediverse - add it to content as a prefix instead of overriding content (snarfed/bridgy-fed#1001).
    • Populate reply.root properly in reply posts (snarfed/bridgy#1696).
    • Add new original_fields_prefix kwarg to store original data in custom (off-Lexicon) *OriginalDescription and *OriginalUrl fields in app.bsky.actor.profile and *OriginalText and *OriginalUrl fields in app.bsky.feed.post (snarfed/bridgy-fed#1092).
    • Support lexrpc.Client as well as Bluesky for client kwarg.
  • from_as1_to_strong_ref:
    • Add value boolean kwarg.
    • Change client kwarg from Bluesky to lexrpc.Client.
• microformats2:
  • Generalize actor logic across all actor types, not just person.
  • json_to_object:
    • Strip leading # prefix (if present) from hashtag u-categorys.
    • Bug fix for when name property is an object, eg an h-card.
  • object_to_json:
    • Convert both id and url inside inReplyTo to in-reply-to.
• nostr:
  • Handle connection closing while sending initial query.
• source:
  • Source.postprocess: when extracting @-mentions, defer to existing tag if it has the same displayName and has url.

v6.2: ### Notable changes

Notable changes

• as1:
  • get_owner bug fix for post, update, delete activities.
  • activity_changed: add new inReplyTo kwarg.
  • is_public: add new unlisted kwarg.
• as2:
  • to_as1: bug fix, preserve objectType: featured for banner/header images even when mediaType is also set.
  • is_public: add new unlisted kwarg.
  • from_as1:
    • For icon field, prefer image types that are allowed by Mastodon.
    • Bug fix, handle stop-following with string object id.
• atom:
  • Add new extract_entries function.
  • activity_to_atom: default actor/author name to username.
  • atom_to_activities: support top-level entry element as well as feed.
  • atom_to_*:
    • add object.author
    • default objectType to article/note and verb to post
    • convert link rel=self/alternate to url
    • use displayName in objects instead of title
    • Interpret entry link without rel as self link.
  • If entry.author doesn't have id or url, default them to feed author's.
• bluesky:
  • Implement create and preview.
  • Fully support both record and object types in from_as1 and to_as1. Use to_as1's type kwarg and from_as1's out_type kwarg to disambiguate.
  • Implement Bluesky.post_id.
  • Add new blob_to_url function.
  • Delete as1_to_profile, switch from_as1 to return $type: app.bsky.actor.profile.
  • Convert HTML summary and content to plain text.
  • Implement Bluesky.user_to_actor, Bluesky.get_actor.
  • Don't log in (fetch an access token) eagerly in the constructor; wait until the client makes a call.
  • Prefer DID to handle in API calls that accept either.
  • at_uri_to_web_url: support lists.
  • web_url_to_at_uri: convert profile URLs like https://bsky.app/profile/snarfed.org to profile record URIs (at://snarfed.org/app.bsky.actor.profile/self) instead of repo URIs (at://snarfed.org).
  • Add from_as1_to_strong_ref.
  • Allow :s in record keys (atproto#2224).
  • to_as1:
    • Convert blobs, both new and old style, to PDS getBlob URLs.
    • Add new uri kwarg.
    • Translate handle to username, add new repo_handle kwarg.
    • Add support for app.bsky.feed.repost, app.bsky.graph.defs#listView, app.bsky.feed.defs#blockedPost.
    • Add actor/author based on repo_did.
    • Improve url field: include custom handles, only use repo_did/handle for app.bsky.actor.profile.
    • Handle bad facet indices that point inside Unicode code points (example; discussion).
    • Convert !no-unauthenticated label on profiles to AS1 @unlisted audience target (bridgy-fed#828).
  • from_as1:
    • Add out_type kwarg to specify desired output type, eg app.bsky.actor.profile vs app.bsky.actor.defs#profileViewBasic vs app.bsky.actor.defs#profileView.
    • Add blobs kwarg to provide blob objects to use for image URLs.
    • Add client kwarg to fetch and populate CIDs.
    • Handle mention tags pointing to bare DIDs.
    • Use parent as root in replies. (Technically wrong in cases where the parent isn't the root, but we don't actually know the root. 🤷)
    • Bug fix: handle bare string URLs in image field.
    • Bug fix: handle tags without url field.
    • Strip trailing slash from home page URLs in order to remove visible / from rel-me verified links on Mastodon etc.
    • Convert attributedTo to singular if it has only one element.
    • If name isn't set, fall back to preferredUsername or infer Webfinger handle from id or url.
    • Prioritize bsky.app profile URL before handle URL in url field (bridgy#1640).
    • Convert bsky.app inReplyTo URLs to at:// URIs.
    • Tighten up datetime conversion to match the ATProto recommended format.
• facebook:
  • Remove Facebook.fql_stream_to_post. Facebook turned down FQL in 2016.
• github:
  • When converting data to AS1, use displayName in objects instead of title.
• mastodon:
  • get_activities bug fix: use query params for /api/v1/notifications API call, not JSON body.
  • Convert HTTP 200 responses with error JSON field (eg from Sharkey) to 400/401 exceptions.
  • Prefer media_attachments.remote_url when available since it may be more long-lived than url for remote statuses (bridgy#1675).
• microformats2:
  • object_to_json bug fix: handle singular inReplyTo.
  • json_to_object bug fix: handle list-valued location.
• nostr:
  • get_*: return partial results when the websocket connection is closed prematurely.
  • to_as1: handle invalid NIP05 values (eg {})
• rss:
  • to_activities:
    • Use objectType: note if title isn't set or is a prefix (possibly ellipsized) of content/description.
    • Add support for images in media:content tags (#674).
• Source:
  • postprocess_activity/object: add mentions kwarg to convert @-mentions in HTML links to mention tags.

v6.1: ### Notable changes

Notable changes

Nostr, Bluesky get_activities, lots of improvements in as2 and microformats2, and more!

REST API breaking changes:

Twitter is dead, at least in the REST API.

Non-breaking changes:

• Add new nostr module!
• as1:
  • Add get_owner, targets.
  • Add accept, reject, stop-following to VERBS_WITH_OBJECT and remove repost, it's not an AS1 verb.
  • Handle url field list values (even though it's invalid AS1).
• as2:
  • to_as1:
    • Improve Video handling: support Link objects in url, extract stream URLs and types from link tags.
    • Coerce non-float latitude and longitude to float, raise ValueError on failure.
    • Put image attachments into image as well as attachments (bridgy-fed#429).
    • Handle Hubzilla's composite object attachment values.
    • Bug fix for null mediaType in attachment and tags.
  • Add new TYPES_WITH_OBJECT constant.
  • Add new get_urls, address functions.
  • Improve Content-Type compatibility with application/ld+json; profile="https://www.w3.org/ns/activitystreams".
  • Bug fix for Undo activities with bare string id objects.
  • Revise HTML in PropertyValue attachments on actors to include full URL in anchro text to be compatible with Mastodon's profile link verification.
• atom:
  • activities_to_atom etc:
    • Switch content from XHTML to HTML inside CDATA to support non-XHTML input content (bridgy-fed#624.
    • Bug fix, handle bare string URL image values.
    • Bug fix, emove incorrect type="application/atom+xml" from rel="self" link in entry.
    • Render objectType: comment attachments.
    • Remove invalid <a> element for tags.
  • Bug fixes in activity_to_atom/activities_to_atom for dict-valued url fields.
  • Render images in article/note attachments.
  • Render objectType: service attachments, eg Bluesky custom feeds.
• bluesky:
  • Implement Bluesky API class, including get_activities.
  • Drop bundled app.bsky/com.atproto lexicons, use lexrpc's instead.
  • Convert reposts, quotes, inline links, attached links, and mentions, both directions. Includes Bluesky facet (rich text) support.
  • Handle quote posts with attached images, both directions.
  • Handle likes, both directions.
  • Add new web_url_to_at_uri function.
  • from_as1: handle link tags without start/end indices.
  • to_as1:
    • Add new type kwarg.
    • Generate staging.bsky.app profile and post URLs.
    • Propagate profile did into actor id.
    • Add unimplemented stub for custom feeds, eg app.bsky.feed.defs#generatorView.
  • Add as1_to_profile.
  • Bug fix for converting follows, both directions: subject in app.bsky.graph.follow is followee, not follower. (That field is badly named!)
• jsonfeed:
  • activities_to_jsonfeed:
    • Bug fix, handle bare string values for image and stream.
    • Bug fix: handle non-object author.
• mastodon:
  • status_to_object: add/fix alt text handling for images.
• microformats2:
  • json_to_html:
    • HTML-escape tag and quote attachment names. Fixes GHSA-4w4f-g49g-3f7j; thank you @janboddez!
  • json_to_object:
    • Improve handling of items with multiple types by using post type discovery more aggressively.
    • Normalize ISO-8601 format of published and updated timestamps.
  • object_to_json:
    • Bug fix, handle bare string URL image values.
    • Normalize ISO-8601 format of published and updated timestamps.
    • Handle bare string ids for replies and shares (usually from AS2.)
  • Include objectType: service attachments, eg Bluesky custom feeds, in JSON and HTML output.
• rss:
  • from_activities: handle bare string id author.

v6.0

Breaking changes

• as2:
  • Interpret bare string object, inReplyTo, etc values as ids, convert them to bare strings or id instead of url.
• microformats2:
  • Convert simple string in-reply-to, repost-of, like-of etc values to AS1 bare strings or ids instead of urls.

Non-breaking changes

• Add new bluesky module for Bluesky/AT Protocol!
• as1:
  • Add the organization object type and ACTOR_TYPES constant (based on AS2).
  • Add new get_ids, get_object, and get_objects functions.
• activity_changed: ignore inReplyTo.author (snarfed/bridgy#1338)
• as2:
  • Support converting between AS1 stop-following and AS2 Undo Follow.
  • Support AS2 Accept and Reject for follows as well as event RSVPs.
  • Add support for the Question (ie poll), Organization, and Delete object types.
  • Convert to/cc to/from AS1 to for public and unlisted.
  • Handle type: Document video attachments like Mastodon emits.
  • from_as1: bug fix for image objects with url and value fields (for alt text).
  • from_as1: bug fix, handle bare string URL image values.
  • from_as1: convert urls.displayName to attachment.name (bridgy-fed#331).
  • from_as1: preserve inReplyTo object values as objects, inline single-element lists down down to just single element.
  • to_as1: use objectType: featured for first image in image field.
  • to_as1: populate actor into object.author for Updates as well as Creates.
  • to_as1: convert Mastodon profile metadata PropertyValue attachments to url composite objects with displayName.
  • Preserve to and cc values when converting both directions.
• atom:
  • Bug fix for rendering image attachments without image field to Atom.
  • Bug fix for published and updated in entries with objects, eg likes, reposts, RSVPs, bookmarks. Thanks @gregorlove! (#480)
  • Bug fix for content activity/ies_to_atom when object is present and empty.
  • Bug fix for objects with elements without objectType in the to field.
• flickr:
  • get_activities: add support for the count kwarg.
• github:
  • get_activities: add support for the count kwarg.
• jsonfeed:
  • Switch from white-space: pre CSS to converting newlines to <br>s because some feed readers follow it strictly and don't even line wrap (#456).
• mastodon:
  • Add compatibility support for Truth Social.
  • Handle truncated JSON API responses.
• microformats2:
  • json_to_object: drop backward compatibility support for like and repost properties. Background discussion.
  • json_to_object: add new rel_urls kwarg to allow attaching displayNames to urls based on HTML text or title attribute (bridgy-fed#331).
  • Add new json_to_activities function.
  • hcard_to_html/maybe_linked_name: when name is missing, use pretty URL as visible text.
  • Support the h-card org property.
  • json_to_object: handle composite rsvp property value.
  • json_to_object: bug fix when fetch_mf2 is True, handle when we run the authorship algorithm and fetch an author URL that has a u-photo with alt.
• rss:
  • from_activities: fix item ordering to match input activities.

v5.0: ### Breaking changes

Breaking changes

• Drop Python 3.6 support. Python 3.7 is now the minimum required version.
• Twitter, Instagram, Mastodon:
  • Drop get_activities cache kwarg's support for App Engine memcache interface. It's now only used as a plain dict. get_activities will now make many small modifications, so if you pass an object that implements those as API calls, you'll probably want to batch those separately.
• Twitter, Mastodon, Flickr, GitHub:
  • create/preview: support the AS1 favorite verb as well as like. (bridgy#1345)
• Atom:
  • Switch to converting AS1 id (instead of url) to Atom id.
• Reddit:
  • Implement get_actor.
• Mastodon:
  • create/preview: allow non-Mastodon replies, ie activities that include inReplyTo URLs even if none of them point to a toot. (bridgy#1321)
  • Raise requests.HTTPError with response.status_code 502 instead of JSONDecodeError on non-JSON responses. This is synthetic, but more helpful for error handling.
• microformats2:
  • object_to_json and related functions: handle all escaped HTML entities, not just & < >.
  • Unify microformats2.prefix_image_urls and prefix_video_urls into a new as1.prefix_urls function.
• RSS:
  • Remove itunes:category. It has to be one of Apple's explicit categories, which we aren't prepared to validate, so don't try.
• ActivityStreams 2:
  • Translate both url and urls from AS1 into multi-valued AS2 url field.
• Move a number of utility methods from the Source class to a new as1 module: object_type, merge_by_id, is_public, add_rsvps_to_event, get_rsvps_from_event, activity_changed, append_in_reply_to, actor_name, original_post_discovery.
• as1.original_post_discovery: remove deprecated cache kwarg.

Non-breaking changes

• ActivityStreams 2:
  • Fix spec compliance bug: icon and image are singly valued, not multiply valued.
  • Add new is_public method and PUBLIC_AUDIENCE constant.
  • Prefer "objectType": "featured" first in the image field when converting from AS1, last in the icon field. This matches the ActivityPub (Mastodon) convention of using icon for profile pictures and image for header images.
  • Propagate url values into new PropertyValue attachments on Person objects; these end up in Mastodon's "profile metadata" link fields.
  • to_as1: if an attachment's mediaType is image/..., override objectType and set it to image.
• Twitter
  • Trim alt text in line between post preview and creation
  • Correctly trim Twitter alt text
• Facebook
  • Scraping: extract post id and owner id from data-ft attribute and _ft_ query param more often instead of story_fbid, which is now an opaque token that changes regularly. (facebook-atom#27)
• Instagram
  • Add new Instagram.scraped_json_to_activities method.
• GitHub
  • create and preview: convert profile URLs to @-mentions, eg https://github.com/snarfed to @snarfed (bridgy#1090).
    • get_activities with activity_id now supports fetch_replies and fetch_likes.
• Reddit
  • Add cache support to get_activities.
• REST API
  • Add new /scraped endpoint that accepts POST requests with silo HTML as input. Currently only supports Instagram. Requires site=instagram, output=... (any supported output format), and HTML as either raw request body or MIME multipart encoded file in the input parameter.
• microformats2
  • Add new extra and body_class kwargs to activities_to_html.
  • When converting u-featured images to AS1, add new non-standard "objectType": "featured" field to distinguish them from u-photo.
  • Convert p-note to AS1 summary.
  • Bug fixes for converting image attachments to photo.
• Source.original_post_discovery: add new max_redirect_fetches keyword arg.

v4.0: ### Breaking changes

Breaking changes

• Drop Python 3.5 support. Python 3.6 is now the minimum required version.

Non-breaking changes

• Add new include_shares kwarg to get_activities, implemented for Twitter and Mastodon. Defaults to True. If False, shares (retweets in Twitter, boosts in Mastodon) will be discarded and not returned. Also add a corresponding shares query param to the REST API.
• RSS:
  • Add support for RSS input via new rss.to_activities function.
• Instagram (scraping):
  • Handle media items with no user object, add new fetch for comments.
  • Add Instagram.merge_scraped_comments().
• ActivityStreams 2:
  • Handle error when type isn't a string.
• Reddit:
  • Implement get_activities() to fetch posts by the current user or a user specified with user_id.
• Facebook scraping:
  • Skip "Suggested for you" posts.
  • Add log_html kwarg to get_activities; defaults to False.
  • Miscellaneous bug fixes.
• JSONFeed:
  • Handle malformed items.author element.

v3.2: ### Notable changes

Notable changes

• Source.original_post_discovery: add new include_reserved_hosts kwarg, defaults to True.
• Facebook:
  • Scraping: handle pictures, videos, link attachments, and text links in timeline/news feed posts.
• Mastodon:
  • Bug fix for get_activities() with fetch_mentions=True: handle notifications with status: null. Maybe happens when a status is deleted?
  • create/preview_create: support bookmarks. (Nothing special happens with them; their content is posted as a normal toot.)
• microformats2:
  • Stop rendering image.displayName as visible text in HTML, since it's already in the <img>'s alt attribute.
  • Add bookmark-of support.
  • Add prefix_image_urls() function.
  • Handle null content in AS1/2 objects.
  • json_to_object bug fix for composite bookmark-of properties.
• Twitter:
  • create/preview: support large videos via async upload. We now pass media_category=tweet_video to the chunked upload INIT stage, and then make blocking STATUS calls until the video is finished processing. (bridgy#1043)
  • create/preview: allow bookmarks. (bridgy#1045)
  • create/preview: allow non-Twitter replies, ie activities that include inReplyTo URLs even if none of them point to a tweet. (bridgy#1063)
  • get_activities: support list ids as well as slugs.
  • Bug fixes for removing t.co links to quoted tweets.
  • Bug fix for multiple instances of the same link in tweet text.
  • get_activities(): raise ValueError on invalid user_id.
• REST API: ported web framework from webapp2 to Flask. No user-visible behavior change expected.

v3.1

Notable changes

• Add Python 3.8 support, drop 3.3 and 3.4. Python 3.5 is now the minimum required version.
• Add Pixelfed! Heavily based on Mastodon.
• Standardize Instagram's and Facebook's scraping into new common scraped_to_activities(), scraped_to_activity(), and merge_scraped_reactions() methods.
• Atom:
  • Add the summary element (#157).
• REST API:
  • Bug fix: URL-encode Unicode characters in Link HTTP headers (eg rel=self, rel=header).
• Facebook:
  • Scraping now uses mbasic.facebook.com instead of m.facebook.com.
• Flickr:
  • Add support for adding tags to existing photos (bridgy#857).
  • get_comment(): skip fetching comments from API if activity kwarg is provided and contains the requested comment.
• GitHub:
  • Handle HTTP 451 Unavailable for Legal Reasons responses (eg for DMCA takedowns) gracefully.
  • Add create/preview support for reactions on pull review request comments (ie URLs with #discussion_r... fragments).
• HTML/microformats2:
  • Add aria-hidden="true" to empty links (bridgy#947).
  • Bug fix: escape &, <, and > characters in bare mf2 content properties (aaronpk/XRay#102).
  • json_to_object(): convert nickname to username.
• JSON Feed:
  • Gracefully handle when content_html and content_text are incorrectly lists instead of strings.
• Instagram:
  • Include threaded (ie nested) comments in scraping (bridgy#958).
• Mastodon:
  • Bug fix for alt text with image attachments (bridgy#975).
  • Omit empty limit param for compatibility with Pleroma (bridgy#977).
• Meetup:
  • create(): handle API errors and return the error message in the CreationResult (bridgy#921).
• Twitter:
  • Bug fix: URL-encode list names in API calls.
  • Bug fix: propagate alt text into AS1 photo.displayName so that it gets all the way into microformats2 JSON and HTML (#183).
• Reddit:
  • Implement post_id().
  • Cache user data fetched from the API for 5m to avoid repeating user profile API requests (bridgy#1021).
    when fetching multiple comments or posts from the same author
  • Bug fix: use 'displayName' instead of 'name' in AS1 objects for submissions.
  • Bug fix: use tag URIs for activity ids.
• ActivityStreams 2:
  • to_as1(): for Create activities, include the activity actor's data in the object's author (snarfed/bridgy-fed#75).
  • to_as1(): convert preferredUsername to username.
  • from_as1(): convert username to preferredUsername.
  • from_as1(): bug fix, make context kwarg actually work.

v3.0

Breaking changes:

• Python 2 is no longer supported! Including the App Engine Standard Python 2 runtime. On the plus side, the Python 3 runtime is now supported! See this list of differences for more details.

Non-breaking changes:

• Migrate demo app and API to the App Engine Standard Python 3 runtime.
• Instagram:
  • Scraping: fetch 50 likes instead of 24. (snarfed/bridgy#898)
  • Scraping bug fix for get_actor() with user_id.
• Twitter:
  • Add image alt text support to get_activites() etc (#183).
• RSS:
  • Add itunes:image, itunes:author, and itunes:category.
  • Strip HTML from title element (#177). Background.
  • Always include author in items (#177).
  • Bug fix: extract feed image from hfeed correctly.
  • Bug fix: don't crash on article or mention tags in items with enclosures.
• Atom:
  • Bug fix: extract feed image from hfeed correctly.
• REST API:
  • Add HTTP HEAD support.
  • Add support for URL fragments with input=html. If a fragment is provided, only that specific element is extracted and converted. (#185)
• GitHub:
  • Publish: preserve <code> tags instead of converting them to `s so that GitHub renders HTML entities like > inside them instead of leaving them escaped. Background.
• JSON Feed:
  • Handle malformed attachments better.
• microformats2:
  • Don't crash on string context fields.
  • html_to_activities(): limit to h-entry, h-event, and h-cite items (#192).
• The cache kwarg to Source.original_post_discovery() now has no effect. webutil.util.follow_redirects() has its own built in caching now.
• Added Meetup.com support for publishing RSVPs.

v2.2: ### Notable changes

Notable changes

• Add Mastodon support!
• Add Python 3.7 support, and improve overall Python 3 compatibility.
• Update a number of dependencies.
• Switch from Python's built in json module to ujson to speed up JSON parsing and encoding.
• Add duration and size support to ActivityStreams 1 and 2, RSS, and microformats2 HTML and JSON. microformats2 support is still emerging for both. Both integer seconds and ISO 8601 string durations are supported for duration. Integer bytes is used for size everywhere. microformats2 HTML also includes human-readable strings, eg 5.1 MB. (#169)
• Twitter:
  • [preview]_create(): detect attempts to upload images over 5MB and return an error.
• Facebook:
  • Add get_activities(scrape=True) for scraping HTML from m.facebook.com. Requires c_user and xs cookies from a logged in session (snarfed/bridgy#886).
  • Upgrade Graph API version from 2.10 to 4.0.
• Atom:
  • Bug fix for de-duping images in attachments.
• RSS:
  • Wrap all <description> element contents in CDATA sections.
  • Render images in <description> with HTML <img> tags (#175).
  • from_activities() bug fix: don't crash when converting multiple attachments to enclosures in a single item. (RSS only supports one enclosure per item, so we now only include the first, and log a warning if the activity has more.)