finish rustdoc

This commit is contained in:
Felix Ableitner 2023-03-02 15:18:06 +01:00
parent 19c459fc02
commit d5ecab1b61
27 changed files with 207 additions and 76 deletions

View file

@ -53,11 +53,11 @@ steps:
environment:
CARGO_HOME: .cargo
commands:
- cargo run --example simple_federation --features actix-web
- cargo run --example local_federation actix-web
- name: cargo run axum example
image: rust:1.65-bullseye
environment:
CARGO_HOME: .cargo
commands:
- cargo run --example simple_federation --features axum
- cargo run --example local_federation axum

View file

@ -49,7 +49,7 @@ hyper = { version = "0.14", optional = true }
displaydoc = "0.2.3"
[features]
default = []
default = ["actix-web", "axum"]
actix-web = ["dep:actix-web"]
axum = ["dep:axum", "dep:tower", "dep:hyper"]

View file

@ -32,10 +32,6 @@ There are two examples included to see how the library altogether:
To see how this library is used in production, have a look at the [Lemmy federation code](https://github.com/LemmyNet/lemmy/tree/main/crates/apub).
## Setup
To use this crate in your project you need a web framework, preferably `actix-web` or `axum`. Be sure to enable the corresponding feature to access the full functionality. You also need a persistent storage such as PostgreSQL. Additionally, `serde` and `serde_json` are required to (de)serialize data which is sent over the network.
## Federating users
This library intentionally doesn't include any predefined data structures for federated data. The reason is that each federated application is different, and needs different data formats. Activitypub also doesn't define any specific data structures, but provides a few mandatory fields and many which are optional. For this reason it works best to let each application define its own data structures, and take advantage of serde for (de)serialization. This means we don't use `json-ld` which Activitypub is based on, but that doesn't cause any problems in practice.
@ -101,7 +97,7 @@ pub struct Person {
`PersonType` is an enum with a single variant `Person`. It is used to deserialize objects in a typesafe way: If the JSON type value does not match the string `Person`, deserialization fails. This helps in places where we don't know the exact data type that is being deserialized, as you will see later.
Besides we also need a second struct to represent the data which gets stored in our local database. This is necessary because the data format used by SQL is very different from that used by that from Activitypub. It is organized by an integer primary key instead of a link id. Nested structs are complicated to represent and easier if flattened. Some fields like `type` don't need to be stored at all. On the other hand, the database contains fields which can't be federated, such as the private key and a boolean indicating if the item is local or remote.
Besides we also need a second struct to represent the data which gets stored in our local database (for example PostgreSQL). This is necessary because the data format used by SQL is very different from that used by that from Activitypub. It is organized by an integer primary key instead of a link id. Nested structs are complicated to represent and easier if flattened. Some fields like `type` don't need to be stored at all. On the other hand, the database contains fields which can't be federated, such as the private key and a boolean indicating if the item is local or remote.
```rust
# use url::Url;

13
examples/README.md Normal file
View file

@ -0,0 +1,13 @@
# Examples
## Local Federation
Creates two instances which run on localhost and federate with each other. This setup is ideal for quick development and well as automated tests. In this case both instances run in the same process and are controlled from the main function.
In case of Lemmy we are using the same setup for continuous integration tests, only that multiple instances are started with a bash script as different threads, and controlled over the API.
Use one of the following commands to run the example with the specified web framework:
`cargo run --example local_federation axum`
`cargo run --example local_federation actix-web`

View file

@ -14,9 +14,11 @@ use activitypub_federation::{
use actix_web::{web, web::Bytes, App, HttpRequest, HttpResponse, HttpServer};
use anyhow::anyhow;
use serde::Deserialize;
use tracing::info;
pub fn listen(config: &FederationConfig<DatabaseHandle>) -> Result<(), Error> {
let hostname = config.hostname();
info!("Listening with actix-web on {hostname}");
let config = config.clone();
let server = HttpServer::new(move || {
App::new()

View file

@ -20,9 +20,11 @@ use axum::{
use axum_macros::debug_handler;
use serde::Deserialize;
use std::net::ToSocketAddrs;
use tracing::info;
pub fn listen(config: &FederationConfig<DatabaseHandle>) -> Result<(), Error> {
let hostname = config.hostname();
info!("Listening with axum on {hostname}");
let config = config.clone();
let app = Router::new()
.route("/:user/inbox", post(http_post_user_inbox))

View file

@ -5,7 +5,10 @@ use crate::{
use activitypub_federation::config::{FederationConfig, UrlVerifier};
use anyhow::anyhow;
use async_trait::async_trait;
use std::sync::{Arc, Mutex};
use std::{
str::FromStr,
sync::{Arc, Mutex},
};
use url::Url;
pub fn new_instance(
@ -48,14 +51,31 @@ impl UrlVerifier for MyUrlVerifier {
}
}
pub fn listen(config: &FederationConfig<DatabaseHandle>) -> Result<(), Error> {
if cfg!(feature = "actix-web") == cfg!(feature = "axum") {
panic!("Exactly one of features \"actix-web\" and \"axum\" must be enabled");
pub enum Webserver {
Axum,
ActixWeb,
}
impl FromStr for Webserver {
type Err = ();
fn from_str(s: &str) -> Result<Self, Self::Err> {
Ok(match s {
"axum" => Webserver::Axum,
"actix-web" => Webserver::ActixWeb,
_ => panic!("Invalid webserver parameter, must be either `axum` or `actix-web`"),
})
}
}
pub fn listen(
config: &FederationConfig<DatabaseHandle>,
webserver: &Webserver,
) -> Result<(), Error> {
match webserver {
Webserver::Axum => crate::axum::http::listen(config)?,
Webserver::ActixWeb => crate::actix_web::http::listen(config)?,
}
#[cfg(feature = "actix-web")]
crate::actix_web::http::listen(config)?;
#[cfg(feature = "axum")]
crate::axum::http::listen(config)?;
Ok(())
}

View file

@ -1,9 +1,10 @@
use crate::{
instance::{listen, new_instance},
instance::{listen, new_instance, Webserver},
objects::post::DbPost,
utils::generate_object_id,
};
use error::Error;
use std::{env::args, str::FromStr};
use tracing::log::{info, LevelFilter};
mod activities;
@ -24,11 +25,16 @@ async fn main() -> Result<(), Error> {
.format_timestamp(None)
.init();
info!("Starting local instances alpha and beta on localhost:8001, localhost:8002");
info!("Start with parameter `axum` or `actix-web` to select the webserver");
let webserver = args()
.nth(1)
.map(|arg| Webserver::from_str(&arg).unwrap())
.unwrap_or(Webserver::Axum);
let alpha = new_instance("localhost:8001", "alpha".to_string())?;
let beta = new_instance("localhost:8002", "beta".to_string())?;
listen(&alpha)?;
listen(&beta)?;
listen(&alpha, &webserver)?;
listen(&beta, &webserver)?;
info!("Local instances started");
info!("Alpha user follows beta user via webfinger");

View file

@ -2,7 +2,7 @@ use crate::{
core::activity_queue::create_activity_queue,
error::Error,
traits::ActivityHandler,
utils::verify_domains_match,
protocol::verification::verify_domains_match,
};
use async_trait::async_trait;
use background_jobs::Manager;
@ -78,6 +78,7 @@ pub struct FederationConfig<T: Clone> {
}
impl<T: Clone> FederationConfig<T> {
/// Returns a new config builder with default values.
pub fn builder() -> FederationConfigBuilder<T> {
FederationConfigBuilder::default()
}
@ -135,6 +136,11 @@ impl<T: Clone> FederationConfig<T> {
));
}
// Urls which use our local domain are not a security risk, no further verification needed
if self.is_local_url(url) {
return Ok(());
}
self.url_verifier
.verify(url)
.await
@ -160,6 +166,10 @@ impl<T: Clone> FederationConfig<T> {
}
impl<T: Clone> FederationConfigBuilder<T> {
/// Constructs a new config instance with the values supplied to builder.
///
/// Values which are not explicitly specified use the defaults. Also initializes the
/// queue for outgoing activities, which is stored internally in the config struct.
pub fn build(&mut self) -> Result<FederationConfig<T>, FederationConfigBuilderError> {
let mut config = self.partial_build()?;
let queue = create_activity_queue(
@ -190,7 +200,7 @@ impl<T: Clone> Deref for FederationConfig<T> {
///
/// ```
/// # use async_trait::async_trait;
/// use url::Url;
/// # use url::Url;
/// # use activitypub_federation::config::UrlVerifier;
/// # #[derive(Clone)]
/// # struct DatabaseConnection();
@ -217,6 +227,7 @@ impl<T: Clone> Deref for FederationConfig<T> {
/// ```
#[async_trait]
pub trait UrlVerifier: DynClone + Send {
/// Should return Ok iff the given url is valid for processing.
async fn verify(&self, url: &Url) -> Result<(), &'static str>;
}
@ -245,9 +256,12 @@ pub struct RequestData<T: Clone> {
}
impl<T: Clone> RequestData<T> {
/// Returns the data which was stored in [FederationConfigBuilder::app_data]
pub fn app_data(&self) -> &T {
&self.config.app_data
}
/// Returns the domain that was configured in [FederationConfig].
pub fn domain(&self) -> &str {
&self.config.domain
}
@ -261,10 +275,12 @@ impl<T: Clone> Deref for RequestData<T> {
}
}
/// Middleware for HTTP handlers which provides access to [RequestData]
#[derive(Clone)]
pub struct ApubMiddleware<T: Clone>(pub(crate) FederationConfig<T>);
impl<T: Clone> ApubMiddleware<T> {
/// Construct a new middleware instance
pub fn new(config: FederationConfig<T>) -> Self {
ApubMiddleware(config)
}

View file

@ -11,7 +11,9 @@ use actix_web::{web::Bytes, HttpRequest, HttpResponse};
use serde::de::DeserializeOwned;
use tracing::debug;
/// Receive an activity and perform some basic checks, including HTTP signature verification.
/// Handles incoming activities, verifying HTTP signatures and other checks
///
/// After successful validation, activities are passed to respective [trait@ActivityHandler].
pub async fn receive_activity<Activity, ActorT, Datatype>(
request: HttpRequest,
body: Bytes,

View file

@ -29,6 +29,8 @@ where
}
}
/// Passes [FederationConfig] to HTTP handlers, converting it to [RequestData] in the process
#[doc(hidden)]
pub struct ApubService<S, T: Clone>
where
S: Service<ServiceRequest, Error = Error>,

View file

@ -1,2 +1,4 @@
/// Handles incoming activities, verifying HTTP signatures and other checks
pub mod inbox;
#[doc(hidden)]
pub mod middleware;

View file

@ -11,7 +11,9 @@ use crate::{
use serde::de::DeserializeOwned;
use tracing::debug;
/// Receive an activity and perform some basic checks, including HTTP signature verification.
/// Handles incoming activities, verifying HTTP signatures and other checks
///
/// After successful validation, activities are passed to respective [trait@ActivityHandler].
pub async fn receive_activity<Activity, ActorT, Datatype>(
activity_data: ActivityData,
data: &RequestData<Datatype>,

View file

@ -3,8 +3,7 @@ use axum::response::IntoResponse;
use http::header;
use serde::Serialize;
/// A wrapper struct to respond with [`APUB_JSON_CONTENT_TYPE`]
/// in axum handlers
/// Wrapper struct to respond with `application/activity+json` in axum handlers
///
/// ```
/// # use anyhow::Error;

View file

@ -15,6 +15,8 @@ impl<S, T: Clone> Layer<S> for ApubMiddleware<T> {
}
}
/// Passes [FederationConfig] to HTTP handlers, converting it to [RequestData] in the process
#[doc(hidden)]
#[derive(Clone)]
pub struct ApubService<S, T: Clone> {
inner: S,

View file

@ -7,8 +7,11 @@ use axum::{
};
use http::{HeaderMap, Method, Uri};
/// Handles incoming activities, verifying HTTP signatures and other checks
pub mod inbox;
/// Wrapper struct to respond with `application/activity+json` in axum handlers
pub mod json;
#[doc(hidden)]
pub mod middleware;
/// Contains everything that is necessary to verify HTTP signatures and receive an

View file

@ -23,7 +23,9 @@ static HTTP_SIG_CONFIG: OnceCell<Config> = OnceCell::new();
/// A private/public key pair used for HTTP signatures
#[derive(Debug, Clone)]
pub struct Keypair {
/// Private key in PEM format
pub private_key: String,
/// Public key in PEM format
pub public_key: String,
}

View file

@ -1,9 +1,14 @@
/// Queue for sending outgoing activities
pub mod activity_queue;
/// Everything related to creation and verification of HTTP signatures, used to authenticate activities
pub mod http_signatures;
/// Typed wrapper for Activitypub Object ID which helps with dereferencing and caching
pub mod object_id;
/// Utilities for using this library with axum web framework
#[cfg(feature = "axum")]
pub mod axum;
/// Utilities for using this library with actix-web framework
#[cfg(feature = "actix-web")]
pub mod actix_web;

View file

@ -8,7 +8,7 @@ use std::{
};
use url::Url;
/// Typed wrapper for Activitypub Object ID.
/// Typed wrapper for Activitypub Object ID which helps with dereferencing and caching.
///
/// It provides convenient methods for fetching the object from remote server or local database.
/// Objects are automatically cached locally, so they don't have to be fetched every time. Much of
@ -63,10 +63,12 @@ where
Ok(ObjectId(Box::new(url.try_into()?), PhantomData::<Kind>))
}
/// Returns a reference to the wrapped URL value
pub fn inner(&self) -> &Url {
&self.0
}
/// Returns the wrapped URL value
pub fn into_inner(self) -> Url {
*self.0
}

View file

@ -1,17 +1,19 @@
#![doc = include_str!("../README.md")]
//#![deny(missing_docs)]
#![deny(missing_docs)]
/// Configuration for this library
pub mod config;
/// Contains main library functionality
pub mod core;
/// Error messages returned by this library.
pub mod error;
/// Data structures which help to define federated messages
pub mod protocol;
/// Traits which need to be implemented for federated data types
pub mod traits;
/// Some utility functions
pub mod utils;
/// Resolves identifiers of the form `name@example.com`
pub mod webfinger;
pub use activitystreams_kinds as kinds;

View file

@ -52,6 +52,7 @@ impl<T> WithContext<T> {
WithContext { context, inner }
}
/// Returns the inner `T` object which this `WithContext` object is wrapping
pub fn inner(&self) -> &T {
&self.inner
}

View file

@ -1,4 +1,9 @@
/// Wrapper for federated structs which handles `@context` field
pub mod context;
/// Serde deserialization functions which help to receive differently shaped data
pub mod helpers;
/// Struct which is used to federate actor key for HTTP signatures
pub mod public_key;
pub mod values;
/// Verify that received data is valid
pub mod verification;

View file

@ -1,17 +1,24 @@
use serde::{Deserialize, Serialize};
use url::Url;
/// Public key of actors which is used for HTTP signatures. This needs to be federated in the
/// `public_key` field of all actors.
/// Public key of actors which is used for HTTP signatures.
///
/// This needs to be federated in the `public_key` field of all actors.
#[derive(Clone, Debug, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct PublicKey {
pub(crate) id: String,
pub(crate) owner: Url,
/// Id of this private key.
pub id: String,
/// ID of the actor that this public key belongs to
pub owner: Url,
/// The actual public key in PEM format
pub public_key_pem: String,
}
impl PublicKey {
/// Create a new [PublicKey] struct for the `owner` with `public_key_pem`.
///
/// It uses an standard key id of `{actor_id}#main-key`
pub fn new(owner: Url, public_key_pem: String) -> Self {
let id = main_key_id(&owner);
PublicKey {

View file

@ -38,6 +38,7 @@ use serde::{Deserialize, Serialize};
/// <https://www.iana.org/assignments/media-types/media-types.xhtml>
#[derive(Clone, Debug, Deserialize, Serialize)]
pub enum MediaTypeMarkdown {
/// `text/markdown`
#[serde(rename = "text/markdown")]
Markdown,
}
@ -47,6 +48,7 @@ pub enum MediaTypeMarkdown {
/// <https://www.iana.org/assignments/media-types/media-types.xhtml>
#[derive(Clone, Debug, Deserialize, Serialize)]
pub enum MediaTypeHtml {
/// `text/html`
#[serde(rename = "text/html")]
Html,
}
@ -54,8 +56,10 @@ pub enum MediaTypeHtml {
/// Media type which allows both markdown and HTML.
#[derive(Clone, Debug, Deserialize, Serialize, PartialEq, Eq)]
pub enum MediaTypeMarkdownOrHtml {
/// `text/markdown`
#[serde(rename = "text/markdown")]
Markdown,
/// `text/html`
#[serde(rename = "text/html")]
Html,
}

View file

@ -0,0 +1,38 @@
use crate::{
error::Error,
};
use url::Url;
/// Check that both urls have the same domain. If not, return UrlVerificationError.
///
/// ```
/// # use url::Url;
/// # use activitypub_federation::utils::verify_domains_match;
/// let a = Url::parse("https://example.com/abc")?;
/// let b = Url::parse("https://sample.net/abc")?;
/// assert!(verify_domains_match(&a, &b).is_err());
/// # Ok::<(), url::ParseError>(())
/// ```
pub fn verify_domains_match(a: &Url, b: &Url) -> Result<(), Error> {
if a.domain() != b.domain() {
return Err(Error::UrlVerificationError("Domains do not match"));
}
Ok(())
}
/// Check that both urls are identical. If not, return UrlVerificationError.
///
/// ```
/// # use url::Url;
/// # use activitypub_federation::utils::verify_urls_match;
/// let a = Url::parse("https://example.com/abc")?;
/// let b = Url::parse("https://example.com/123")?;
/// assert!(verify_urls_match(&a, &b).is_err());
/// # Ok::<(), url::ParseError>(())
/// ```
pub fn verify_urls_match(a: &Url, b: &Url) -> Result<(), Error> {
if a != b {
return Err(Error::UrlVerificationError("Urls do not match"));
}
Ok(())
}

View file

@ -52,37 +52,3 @@ pub async fn fetch_object_http<T: Clone, Kind: DeserializeOwned>(
res.json_limited().await
}
/// Check that both urls have the same domain. If not, return UrlVerificationError.
///
/// ```
/// # use url::Url;
/// # use activitypub_federation::utils::verify_domains_match;
/// let a = Url::parse("https://example.com/abc")?;
/// let b = Url::parse("https://sample.net/abc")?;
/// assert!(verify_domains_match(&a, &b).is_err());
/// # Ok::<(), url::ParseError>(())
/// ```
pub fn verify_domains_match(a: &Url, b: &Url) -> Result<(), Error> {
if a.domain() != b.domain() {
return Err(Error::UrlVerificationError("Domains do not match"));
}
Ok(())
}
/// Check that both urls are identical. If not, return UrlVerificationError.
///
/// ```
/// # use url::Url;
/// # use activitypub_federation::utils::verify_urls_match;
/// let a = Url::parse("https://example.com/abc")?;
/// let b = Url::parse("https://example.com/123")?;
/// assert!(verify_urls_match(&a, &b).is_err());
/// # Ok::<(), url::ParseError>(())
/// ```
pub fn verify_urls_match(a: &Url, b: &Url) -> Result<(), Error> {
if a != b {
return Err(Error::UrlVerificationError("Urls do not match"));
}
Ok(())
}

View file

@ -14,8 +14,10 @@ use std::collections::HashMap;
use tracing::debug;
use url::Url;
/// Turns a person id like `@name@example.com` into an apub ID, like `https://example.com/user/name`,
/// using webfinger.
/// Takes an identifier of the form `name@example.com`, and returns an object of `Kind`.
///
/// For this the identifier is first resolved via webfinger protocol to an Activitypub ID. This ID
/// is then fetched using [ObjectId::dereference], and the result returned.
pub async fn webfinger_resolve_actor<T: Clone, Kind>(
identifier: &str,
data: &RequestData<T>,
@ -29,7 +31,7 @@ where
let (_, domain) = identifier
.splitn(2, '@')
.collect_tuple()
.ok_or_else(|| WebfingerResolveFailed)?;
.ok_or(WebfingerResolveFailed)?;
let protocol = if data.config.debug { "http" } else { "https" };
let fetch_url =
format!("{protocol}://{domain}/.well-known/webfinger?resource=acct:{identifier}");
@ -37,6 +39,7 @@ where
let res: Webfinger = fetch_object_http(&Url::parse(&fetch_url)?, data).await?;
debug_assert_eq!(res.subject, format!("acct:{identifier}"));
let links: Vec<Url> = res
.links
.iter()
@ -60,7 +63,8 @@ where
/// Extracts username from a webfinger resource parameter.
///
/// For a parameter of the form `acct:gargron@mastodon.social` it returns `gargron`.
/// Use this method for your HTTP handler at `.well-known/webfinger` to handle incoming webfinger
/// request. For a parameter of the form `acct:gargron@mastodon.social` it returns `gargron`.
///
/// Returns an error if query doesn't match local domain.
pub fn extract_webfinger_name<T>(query: &str, data: &RequestData<T>) -> Result<String, Error>
@ -78,11 +82,23 @@ where
.to_string())
}
/// Builds a basic webfinger response under the assumption that `html` and `activity+json`
/// links are identical.
pub fn build_webfinger_response(resource: String, url: Url) -> Webfinger {
/// Builds a basic webfinger response for the actor.
///
/// It assumes that the given URL is valid both to the view the actor in a browser as HTML, and
/// for fetching it over Activitypub with `activity+json`. This setup is commonly used for ease
/// of discovery.
///
/// ```
/// # use url::Url;
/// # use activitypub_federation::webfinger::build_webfinger_response;
/// let subject = "acct:nutomic@lemmy.ml".to_string();
/// let url = Url::parse("https://lemmy.ml/u/nutomic")?;
/// build_webfinger_response(subject, url);
/// # Ok::<(), anyhow::Error>(())
/// ```
pub fn build_webfinger_response(subject: String, url: Url) -> Webfinger {
Webfinger {
subject: resource,
subject,
links: vec![
WebfingerLink {
rel: Some("http://webfinger.net/rel/profile-page".to_string()),
@ -97,23 +113,39 @@ pub fn build_webfinger_response(resource: String, url: Url) -> Webfinger {
properties: Default::default(),
},
],
aliases: vec![],
properties: Default::default(),
}
}
/// A webfinger response with information about a `Person` or other type of actor.
#[derive(Serialize, Deserialize, Debug)]
pub struct Webfinger {
/// The actor which is described here, for example `acct:LemmyDev@mastodon.social`
pub subject: String,
/// Links where further data about `subject` can be retrieved
pub links: Vec<WebfingerLink>,
/// Other Urls which identify the same actor as the `subject`
#[serde(default)]
pub aliases: Vec<Url>,
/// Additional data about the subject
#[serde(default)]
pub properties: HashMap<Url, String>,
}
/// A single link included as part of a [Webfinger] response.
#[derive(Serialize, Deserialize, Debug)]
pub struct WebfingerLink {
/// Relationship of the link, such as `self` or `http://webfinger.net/rel/profile-page`
pub rel: Option<String>,
/// Media type of the target resource
#[serde(rename = "type")]
pub kind: Option<String>,
/// Url pointing to the target resource
pub href: Option<Url>,
/// Additional data about the link
#[serde(default)]
pub properties: HashMap<String, String>,
pub properties: HashMap<Url, String>,
}
#[cfg(test)]