Anyone Can Ship a PR to a Massive Repo Now

Open weight models have been an absolute banger lately.

Be it Kimi, Qwen, or GLM, all of them are crazy good performance-wise. DeepSeek, for some reason, has never really clicked for me, though I’ve heard DeepSeek V4 is worth a shot. I’ve yet to try it properly in my own workflow.

For most of the year, if you wanted frontier coding performance, you reached for Claude or GPT and just ate the bill. Open weight releases existed, fine for self-hosting, but they weren't quite the thing devs used.

GLM-5.2 kind of broke that pattern. It dropped in June, and for a few weeks it was quietly the model a lot of builders were running. Vercel's CEO Guillermo Rauch posted that he was “genuinely impressed, almost shocked” at how good it was at coding. Not the usual reaction to an open weight release.

Then, about a month later, Kimi K3 showed up and broke the records. Biggest open weight model ever shipped, topped basically every open-weight leaderboard, and reportedly sent competitor stocks (including Z.ai's) down double digits on launch day. Crazy stuff!

On paper, they're holding their own against Opus and Fable on real coding benchmarks.

So I didn't really want to write another “which model wins” post. That question gets answered by a benchmark table anyway.

What I actually wanted to know is this: what happens if you point one of these at a massive, real, actively maintained GitHub repo, one with 80k+ stars and a real maintainer team, and just ask it to ship a feature people have genuinely been asking for?

Open Models, so almost all of it is free (if you self-host).

Because for years, the advice for breaking into tech with zero experience was “go contribute to open source.” Ship a real PR, work with maintainers, build a portfolio. That advice always assumed contributing was hard. That it took actual skill.

Let's see what happens.

TL;DR

• Both models shipped real, working code into two large, actively maintained open source repos, a new feature on Hoppscotch (80k+ stars) and a real bug fix on Mealie (12k+ stars), with zero lines written by a human.
• Both nailed the Hoppscotch feature test. Kimi K3 needed one follow-up to fix a linting issue. GLM-5.2 passed clean on the first try.
• The Mealie bug fix is the standout result of the whole post. Both models, working independently, found the exact same non-obvious root cause that the original human reporter had missed.
• GLM-5.2 played it safe, fixing the bug entirely in the frontend, for well under a dollar. Kimi K3 went further, touching backend auth code too, for a more complete fix at a noticeably higher cost.
• Kimi K3 wins on paper, higher scores across nearly every benchmark. In practice, GLM-5.2 kept pace on both real-world tests, and did it consistently cheaper.

💁 The actual point of this post: pointing a free, or nearly free, open weight model at a real GitHub issue produced code that would plausibly survive review. The “go contribute to open source to prove you can code” advice doesn't hold up the same way anymore.

Model Benchmarks

Before running our own tests, let's look at what both labs are claiming.

Kimi K3 actually benchmarked itself against GLM-5.2 directly in its own report. So we get a rare side-by-side table instead of two separate vendor pages.

⚠️ SWE-bench Verified and SWE-bench Pro aren't the same test, so that row isn't a fair 1:1. Also, this is Kimi's own testing, so don't blindly believe.

On paper, Kimi K3 is ahead almost everywhere.

The biggest gap is SWE Marathon. That's the test for staying sharp during long coding sessions, lots of steps, lots of back and forth. GLM-5.2 gets 13.0. Kimi K3 gets 42.0. That's more than 3x.

Independent scores back this up too. Artificial Analysis runs its own tests instead of trusting what labs report.

• GLM-5.2: 53 on their Intelligence Index
• Kimi K3: 60 on their Intelligence Index

Kimi K3 currently sits at #1 open weight on that board. GLM-5.2 held that spot before K3 shipped.

So going in, the expectation is simple: Kimi K3 should have the edge, especially on longer sessions. But honestly, that's not really the question I care about here.

The real question is whether either of these is good enough to walk into someone else's massive, actively maintained codebase and ship something real, without the person prompting it knowing how to code at all.

Test Workflow

For the test, we will use the following CLI coding agent:

• OpenCode (via OpenRouter)

Instead of made-up prompts, I picked two real, currently open feature requests off big, actively maintained open source repos. The kind of issues real users have been asking for. If a free model can ship these well enough to pass as a legitimate PR, that tells you something.

We will do a feature-request, and a bug fix test.

We will check the models on two different tasks:

1. Task 1: Ship a real, currently open feature request on Hoppscotch, an 80k+ star API testing tool
2. Task 2: Fix a real, currently open bug on Mealie, a self-hosted recipe manager, that's been logging real users out mid-session

💡 NOTE: I will share the source code changes for each task by each model in a repository. This way, you can easily view them on your local system by cloning the repository branch: git clone <reppo_link> -b <branch_name>.

Coding Test

1. Add Cross-Request Execution to Hoppscotch

For the first test, I wanted a real, currently open feature request, not something I made up.

I found one on Hoppscotch, a Postman-style API testing tool sitting at 80k+ stars, real maintainers, real users.

ℹ️ The issue is #5861 Common Workspace & Cross-Request Execution. Still open. Zero PRs against it.

Here's the actual ask. Let one request run another request first and use its response. Log in, grab a token, then use that token somewhere else. Right now you're stuck running a whole collection in a fixed order to make that happen.

💁 PROMPT: full task brief here Hoppscotch Cross-Request Execution Prompt

Kimi K3

Kimi K3 nailed it.

It went and explored the existing pre-request script sandbox first. Then found the actual code that sends requests. Then built a proper hopp.executeRequest() function on top of both. It looks up the request by name, catches circular dependencies so nothing loops forever, and stops cleanly if a dependency fails. It wired all of this through the sandbox, the host hooks, and the request runner. Then ran a typecheck and a build to make sure everything actually worked before writing up its report.

Only real issue was the code failed pre-commit checks on the first pass. Nothing major, just some stylistic stuff the linter didn't like. One follow-up prompt asking it to fix that, and it was clean.

• Context: 160,989 tokens (15% used)
• Cost: $5.83

You can find the code it generated here: Kimi K3: Hoppscotch Cross-Request Execution

Here's the demo:

GLM-5.2

GLM-5.2 also nailed it, with a little less token and cost usage.

Same kind of work. Found the sandbox, extended it with its own module, wired the hook all the way through RequestRunner.ts. Typecheck clean and lint clean on the first try. No follow up needed at all.

• Context: 220,947 tokens (21% used)
• Cost: $3.74

💁 Worth noting, GLM-5.2 actually burned more tokens than Kimi K3 here and still came out cheaper. That tracks with the pricing. GLM-5.2 is just the cheaper model per token by a wide margin.

You can find the code it generated here: GLM-5.2: Hoppscotch Cross-Request Execution

Here's the demo:

2. Fix Transparent Token Refresh in Mealie

For the second test, I wanted a real bug this time, not a feature.

I found one on Mealie, a self-hosted recipe manager sitting at 12k+ stars, real maintainers, real users.

ℹ️ The issue is #7835 Auth tokens not refreshed during normal use, causing logouts on token expiry. Still open. Zero PRs against it.

Here's the actual bug. Users get logged out in the middle of using the app, not because their session was actually invalid, but because nothing was ever refreshing the token before it expired.

💁 PROMPT: full task brief here Mealie Auth Refresh Prompt

GLM-5.2

Okay, this one's actually phenomenal.

GLM-5.2 didn't just fix the bug the reporter described. It found something the reporter missed entirely. Mealie's backend decodes the JWT with zero leeway, and there's no separate long-lived refresh token, so once an access token actually expires, the refresh endpoint itself starts rejecting it too..

So it built two layers. A reactive one in the central axios interceptor, on a 401 it refreshes and retries once, with concurrent 401s sharing a single refresh call and auth endpoints excluded so a wrong password doesn't loop. And a proactive one, a new file that decodes the token's expiry and schedules a refresh 5 minutes early, then keeps re-scheduling itself, even accounting for background-tab timer throttling.

No backend changes. No bumping the token lifetime as a shortcut, which is exactly the fake fix we told it not to take. No new dependencies either.

• Context: 71,512 tokens (7% used)
• Cost: $0.42
• Duration: ~11 minutes

Genuinely wild to sit with for a second. This bug's been quietly logging real self-hosters out of their own recipe app, and GLM-5.2 fixed it, properly, root cause and all, for under fifty cents, in 11 minutes? Wild!

You can find the code it generated here: GLM-5.2: Mealie Auth Refresh Fix

Kimi K3

Kimi went even deeper.

It found the exact same root cause GLM did, on its own. The refresh endpoint rejects an already-expired token, since there's no separate long-lived refresh token. Two different models, working independently, found the same non-obvious bug neither of us told them about.

But Kimi didn't stop at the frontend. It touched the backend too, extending the cookie's lifetime past the JWT's actual expiry and adding a new dependency that accepts a token for a short grace window after expiry, just for the refresh endpoint. It also caught a subtle FastAPI issue, the route sat behind a router-level auth check that would reject the request before its own fix could even run, so it moved the route to fix that.

Bigger change than GLM's, and it solves a real edge case GLM's doesn't: closing the tab for a long stretch and coming back after the token's already dead without touching TOKEN_TIME.

Before calling it done, it ran the existing test suite too, ruff, ESLint, all 151 frontend tests, a byte-identical build, plus an isolated check of the PyJWT behavior to confirm its own theory. Nothing broke.

• Context: 83,246 tokens (8% used)
• Cost: $2.28
• Duration: ~21 minutes

You can find the code it generated here: Kimi K3: Mealie Auth Refresh Fix

Conclusion

Both of these models are open weight. Both are either free to self-host or a few cents per task through an API. And both of them just shipped real, working features into two of the biggest, most actively maintained repos on GitHub, the kind with actual maintainers, actual review standards, actual users.

You didn't need to write a single line of code to watch that happen. You just needed a prompt.

Both models could pass both the tests.

For years, “go contribute to open source” was the advice you gave someone with zero experience trying to break in. Read the codebase. Understand the conventions. Ship something real. It worked as a signal because it was hard, and doing the hard thing proved something about you.

That barrier's basically gone now..

I'm not saying open source contributions are worthless now. Maintainers still review, still push back, still catch weak PRs. But “just contribute to open source, it proves you can code”? That line doesn't really hold up the same way anymore. Not when the thing doing the proving is a two-cent API call. 🤷

Wild place to be in, honestly.