Organic Ranking System
The platform had engagement. It didn't have signal. I built a ranking system that separated real contribution from noise and made the results feel earned.
Situation
Busy platform. Loud community. People participated in wildly different ways. Some showed up every day for years. Some spiked in hard, vanished, came back when it suited them. A smaller group figured out how to game visibility. None of this was unusual. What was unusual was that the platform had no way to tell the difference.
The metrics rewarded volume. That's it. Show up a lot, look important. Quiet, steady contributors got buried. Bursty behavior outperformed sustained effort. Gaming outperformed both. Participation was high. Meaning wasn't. The goal wasn't to rank people. It was to surface the ones who actually showed up consistently and make the system feel fair enough that people trusted it.
My Role
I owned signal design. Not scoring. Signal. The difference matters. Scoring is math. Signal is deciding what contribution actually means at scale and then building something that reflects it without falling apart under real behavior.
I defined the principles early and held to them throughout. Sat across design, engineering, and data. My job was to separate noise from signal, decide what the system rewarded, and make sure it survived contact with real users doing real things.
Key Actions
Reward consistency over bursts
Showing up every week for a year had to matter more than flooding the system for three days. Built a composite signal that tracked sustained contribution, not just volume.
Diminishing returns and recency
Grinding the same action stopped paying off. Weighted recent contribution without punishing people who'd been around a long time.
ML inferencing to optimize the algorithm
Used ML to tune the ranking model as real behavior came in. Same signals we defined, calibrated by what actually predicted contribution versus what got gamed. We stayed in the loop.
Intentional opacity
Kept parts of the system opaque on purpose. Full transparency in a ranking system is an invitation to reverse-engineer it.
Graceful degradation under load
Real-time at peak traffic was expensive and brittle. Optimistic frontend updates, backend reconciliation. Felt instant. Stayed correct.
Results
Behavior changed. Participation steadied. Noise dropped. The quiet contributors who'd been buried for months started surfacing. Trust went up because outcomes finally felt earned, not arbitrary.
Ranking systems shape behavior whether you design for it or not. This one rewarded the right things. And when people could feel that, they stopped trying to game it and started actually contributing.
Key Learnings
Ranking systems shape behavior
Whether you want them to or not. Every signal you reward, every weight you assign, every threshold you set tells users what matters. Design with that in mind or the system will decide for you.
Perfect fairness is a myth
You won't get it exactly right. You can get it meaningfully right. Clarity, consistency, and restraint matter more than precision. People don't need a perfect system. They need one they can trust.
Signal design is product design
This wasn't a feature. It was the product. How you measure and surface contribution defines what the platform becomes. Get it wrong and you optimize for noise. Get it right and the system does the hard work for you.