One of the ultimate system ideas I keep coming back to is a system that personally learns what you want. In the past we've had algorithms completely controlled by corporations. In the early days, the algorithms for social media were just chronological because that was easy. Then the algorithms became about who you follow—there were different versions, some requiring mutual follows, but what won out was Twitter-style unidirectional following.
Pretty quickly it became clear that chronological feeds didn't work for the algorithms. Either too few posts or way too many, or one person going crazy during a sports game. So platforms started having their algorithms collapse threads, reorder posts, showing just one from a burst. These algorithmic changes happened really early.
Then the algorithms got sophisticated. The algorithms became less about friendship, less about recency. Once you start interacting, platforms learn everything about you and create algorithms you'll like. For a while this was magical—the golden age before monetization pressure made the algorithms worse.
But then consolidation happened and making money became crucial. The algorithms for advertising and creator monetization took over, incentivizing terrible behavior: sensationalist content, clickbait, lowest-common-denominator stuff.
I've always wanted user control over these algorithms instead of corporate-controlled algorithms. Some platforms now offer "most recent" modes under pressure, giving users slightly more algorithmic choice. Bluesky and Mastodon allow complete algorithmic control. But you're still choosing between coarse algorithmic options made by others, not by you.
I want personal software where you speak your algorithm into existence:
"Show me uplifting stuff and aesthetic landscape photography and enthusiast aerial drone footage; breaking news in my corner of science; keep me informed on the situation in Ukraine, follow the U.S. midterms, exclude XYZ person, exclude extremist viewpoints; funny clips, but only 5% of the time; the rest should be informative."
That becomes the seed of your algorithm. Then the algorithm learns from everything you do—no need to rate posts explicitly. The algorithm watches your hovers, clicks, interactions.
Using LoRA and on-policy reinforcement learning, the algorithm presents content, learns from your actions, updates immediately. Think of it as raising a digital pet algorithm. You can rewind to earlier versions if you liked the algorithm better as a "puppy."
The technical challenges for these algorithms are fascinating. How do you format sparse user feedback into RL signals for the algorithm? Maybe binary choices between posts, time-on-screen metrics, or simple A/B comparisons for the algorithm.
With online learning, each action backpropagates immediately into the algorithm. The next content comes from the updated algorithm. You can save all interactions and rewind to any point if you have deterministic training for your algorithm.
When new models emerge, transferring the learned preferences to the new algorithm is tricky. It's like your pet algorithm dies and you start the algorithm over, though maybe techniques like ORPO or mixing old data with new learning can help the algorithm.
This goes way beyond social feed algorithms. Entire apps will be built this way—you'll describe what you want, iterate with the AI algorithm, teach it your preferences. The algorithm'll create different versions and learn from which you prefer.
The exciting part is when on-device algorithms become better than anything else. Then peer-to-peer algorithms become viable, your data stays yours, and every individual becomes more important because their feedback directly shapes their algorithmic tools.
We're entering an age where apps self-adapt their algorithms to you personally.