Dead weight PRs

[seroperson]

The problem

As a miner I encounter situation when you do PRs in repositories, but maintainers ignore them.

There are a lot of cases when your PR could be ignored and will be just a dead weight for you. Besides cases when project is obviously inactive and you should pay attention to not contribute there, there are many other reasons why your PR could be ignored, everything depends on maintainer. Sometimes they just ignore contributions from non-members, sometimes review-loop takes too long (I mean months or even years), sometimes they are half-active and merge only simple fixes.

You can say that I can ask before contributing, but it doesn't guarantee anything. I mean, you should ask, of course, but it doesn't protect you from being ignored after you open PR. You can also say that I just can skip this project if it seems suspicious. Well, yes, but you never know whether you would be ignored in other project too.

All what I want to say that current system restricts you too much:

• You're not protected against ignoring. I see a lot of great projects, but then I look at their average PR lifetime, feedback loop, maintainers involvement and so on, and skip them, because of a risk that merging a PR would take forever. I would be happy to try contributing into some projects, but such risk often stops me.
• If you already opened a PR in such project, it will hang there for months and will take score from each consequence PRs to other projects. Closing it will punish you with credibility penalty.
• At the end, it looks not fair to enforce contributors to face credibility loss because they opened a PR in a "wrong" project with non-involved maintainers.

What I suggest

There are quite a lot of possible ways to improve this system and it requires quite a big discussion as I think. But the very first thing comes to my mind is the following approach:

• When opening a PR, it's "closing" credibility penalty factor starts declining with time (from 1 to 0). For example, after a month of "opened" state, I can safely close PR without credibility loss. If it was closed after a week, then let's say its' penalty coefficient is something around 0.7.
• We can also consider additional penalty if closed by a maintainer or/and the opposite, consider slightly decreased penalty if closed by PR's owner.
• closed_prs variable in the credibility formula (credibility = merged_prs / (merged_prs + closed_prs)) now not just a plain count of closed PRs, but sum of their's coefficients.

Something like that looks fairer than what we have today. It will negate the fear contributing to projects without previous contributions and enforce people to try more instead of sticking to the well-known "tested" projects.

[anderdc]

I like how you brought up the issue in a digestible manner, gittensor was intended to be collaborative and bringing up problems this way makes it easy to have a formal discussion on it (rather than through discord chat).

My initial thoughts are yes this has been pestering people, and yes there are countless ways to go about it. To note, it's intended to be restrictive which is why we get more lax with credibility as you progress through to higher tiers. Arguably, I'd say it makes miners cautious and less prone to explore new repositories more than anything. We recently merged a miner made feature that actually gives a fair boost to pioneers now.

Now the flip side is our lookback window is 90D, so you COULD hang onto that open PR (collateral will give you a hit possibly) and once it hits past the 90 day lookback, close it, and then it will no longer affect you at all, but then it takes up an open PR threshold and collateral until then. I've seen many people padding their closed PRs with various trivial PRs bc trivial and high-token scoring are weighted the same for credibility.

While I do think this is a problem, I don't think it's BIG problem. I like the ideas/suggestions you put forth, but I'm very wary against changing what is a 'simple' KPI, credibility. instead of it being an accuracy, you're ideating a weighted accuracy/average which makes sense but makes the algorithm murkier.

I'd almost want to go the stupid simple route and give everyone a couple mulligans... hmmm open to other ideas. I hope others chime in. @LandynDev for visibility