There has been discussion recently about bring your own algorithm (BYOA) for social networks. Here are some challenges:
At-scale, world-class consumer apps in 2024 use machine learning (AI) based feeds. This is overwhelming consumer revealed preference in terms of time spent.
Making a good machine learning feed is hard and requires significant resources to make performant and real-time.
If you want to do a feed marketplace with good UX, you'd likely need to create a backend where developers would upload their models and the client runs the model in their infra. This obviously has privacy concerns, but maybe possible.
TBD if consumers would be willing to pay for your algo, though.
The likely outcome is a marketplace of clients, where each have a different algorithm that uses inputs (both public from the protocol and private from app usage) to tailor a model for each user.
PS — here are two data points from a developer who was building a Farcaster client: 1) feed performance is a challenging engineering problem and 2) running the backend is expensive (the client had a few hundred active users).
Kartik on Warpcast
1. building a mobile app, we rebuilt it 3x before we grokked it 2. feed latency/performance, much harder problem than we initially realized, hence scrapping custom feeds beyond lists
Kartik on Warpcast
~$2k/month
re: the Jack video quote that Elon tweeted 1. At-scale, world-class consumer apps in 2024 use machine learning (AI) based feeds. This is overwhelming consumer revealed preference in terms of time spent. 2. Making a good machine learning feed is hard and requires significant resources to make performant and real-time.
3. If you want to do a feed marketplace with good UX, you'd likely need to create a backend where developers would upload their models and the client runs the model in their infra. This obviously has privacy concerns, but maybe possible. 4. TBD if consumers would be willing to pay for your algo, though.
Turned this into a @paragraph: https://dwr.email/challenges-with-byoa-and-social-networks
3. seems doable 4. can monetize through a limited form of ads instead - being able to set who gets distribution is hugely valuable
3. Doable, but does it actually grow retained DAU vs. investing that effort in your default algo. 4. What incentive does the client have to give the attention value via ads to the algo developer. If the algo is good enough, dev just builds own client or client dev builds own algo.
The key word is "at-scale". Farcaster isn't there yet. There's definitely a need to clean up feeds and surface engaging content but I don't think FC is at a place for a heavy investment in ML powered algos. PageRank and heuristic based algos should be good enough maybe with some ML sprinkles for classification.
Get Jack as a guest on MoZ 👀
Yes. You’re the expert here. But the fact we now have real Twitter alternatives is already a choice of its own. 👏
hard parsing for me “revealed preference in terms of time spent” correlated w carefully manipulated algorithms literally designed to make the user come back and engage
Pretty basic: 1. Run 50-50 A/B test, half algo vs. half reverse chronological 2. Algo will win on all of the metrics. 3. If you ask people what they prefer, a good % will tell you they prefer the algo with more control despite the data saying otherwise. 4. If control was actually desirable, would outcompete
yes i agree — we are physiological creatures easy to induce. ofc the math will show algo is better. isn’t that what facebook tiktok and everyone else has been doing? the algo is designed to reinforce that. my stated preference is to be healthy. my revealed is to eat like shit. as a user tho, need help.
AI rules bro 🤖
BYOA for social networks faces several challenges: AI-based feeds dominate user preference, building performant feeds is resource-intensive, creating a secure backend for feed models may pose privacy issues, and it's unclear if users will pay for custom algorithms. More insights from @dwr.eth
Yes, building recommendation algorithms presents numerous challenges. That's why many are eager for Warpcast, with its vast user base and strong development capabilities, to excel in this area. These algos can also combat spam and bots while uncovering hidden high-quality users, making everyone's feed more engaging.