How AI-native companies
interview software engineers
Over the last 4-5 years, I’ve interviewed for product engineering roles at a lot of companies, from 100 people to 1,000+, pre-seed to post-IPO, across fintech, developer tools, customer service, AI labs, talent marketplaces, and more.
The emerging trend is that more and more companies are dropping contrived data structures and algorithms puzzles in favor of assessments that better model the actual job. I think this will continue as code becomes cheaper to produce.
A brief history
In the 1990s and early 2000s, engineering interviews were a mix of brainteasers, whiteboard coding, and trivia. Microsoft was famous for questions like “Why are manhole covers round?” and “How would you move Mount Fuji?” (the 2003 book How Would You Move Mount Fuji?). Google asked how many golf balls fit in a school bus. In 2013, Google’s head of people operations said:
Brainteasers were a waste of time and didn’t predict job performance.
Over time, algorithm and data structure questions became the standard. Books like Programming Interviews Exposed (2000) and Cracking the Coding Interview (2008) turned interview prep into a category of its own, and in the 2010s platforms like HackerRank and LeetCode turned it into an industry.
Underneath all of it was one idea: specialized knowledge was scarce. If you could recall how to balance a tree, implement a cache, or reverse a linked list, that signaled something valuable, because that knowledge was slow and expensive to acquire.
Superhuman coding agents
We’ve watched coding agents go from autocomplete to doing most of the job in about four years. By early 2026, most engineers I know had stopped writing code by hand. Superhuman coding models are also getting cheaper quickly. As of this writing in October 2026, the cheapest models cost about $0.10 per million input tokens and $0.50 per million output tokens, while the best model for complex coding work costs about $4 and $20. These numbers will keep falling.
When code is this cheap, being able to recall an algorithm from memory stops being the valuable skill. AI-native companies have fully moved their coding assessments away from LeetCode-style problems and toward project-based tasks. The project-based tasks I’ve seen take two common forms:
- Build, then explain: Given some requirements and constraints, build a small project, then explain your system design to an engineer.
- Debug live: Given a simple codebase with some bugs, debug it live.
I call these the drop-in tests. They measure how effective a candidate is after they’re dropped into an unfamiliar, open-ended environment.
They’re useful because they test more than writing new code from a problem description. They test comprehension: how quickly you can understand a codebase you’ve never seen and make good changes. That’s much closer to what engineers do every day.
I think evaluation will keep moving this way, toward comprehension first and then even more toward system design. In fact, I recently chatted with a YC startup that only does system-design interviews.
Hackerjoy
For engineers: The first goal is to let you practice for these drop-in tests, starting with live debugging.
For companies: I want Hackerjoy to be one of the best places to discover engineers. Google used to do this with Foobar: a coding challenge that popped up in Search when you looked up programming terms, and it felt like finding hidden talent. I’d like to bring back some of that spirit: people showing what they can do by doing it.
There’s a lot more to build. If you have thoughts on what interviews should look like, I’d love to hear them.