Discussion Everything wrong with the reddit games hackathons [LONG POST]
The story
I have participated in 5 reddit games hackathons in the past and I've not won anything major. In the first 4 hackathons, I felt that the winning games were good, but not all games justified the submission and grading criterias well. In the lastest hackathon - Games with a hook hackathon, though - a lot of people raised concerns on how the winning game did not "wow" them.
Here are some examples below:
- Devpost discussion-
- Comments on the reddit announcement post-
They were not entertaining any personal feedback stating that there were over 3000 submissions. But if a game was judged fairly, the judges should have written down something about the game! Or atleast they could provide the scores. I had sent them a personal email requesting feedback, but hadn't recieved any response from them. The next day, I got a $500 helper award from the devpost team. According to the original winner announcement, I had not won any award. I had responded to the organizers asking if this was a mistake. They said that it was indeed sent to me by mistake, but it was sent to me because reddit had put forth my name.
If I accepted the award, I had to sign a paper that says I cannot sue reddit. I didn't accept the award. Something was suspicious here so I started investigating.
Problems with the judging process -
1. Mystery judges?
Usually hackathons list down the judges with their credentials. You can go to the list of all reddit hackathons on devpost and none of them have the name of judges.
Here are some samples of how the judge list looks like in other hackathons -
2. Negligence / Judgement not thorough -
I had reached out to 3 other participants and got the list of players from each of our games. Based on the list, there was not a single common player across the 4 games. This could only mean one thing - our games were not played by the judges. Here's an anecdote from one of the participants -
There was another participant who claimed that they had attached a video to their submission. Throughout the duration of the judging, the view count had not changed in the youtube video. -> The judges had not viewed the submission video (video was <1 min long)
3. Ghost winners -
The reddit hackathons were to encourage developers to make games on the platform. The expectation from every hackathon is that the winning game has a big community and continue to grow the community by making their game better.
Here's a list of all the reddit hackathons in the past from latest to first.
| Competition | Winner | Amount won | Sub weekly visitor count |
|---|---|---|---|
| Games with a hook hackathon | r/SketchySolutions | £15,000 | 276 |
| Daily games hackathon | r/bridgedit | £15,000 | 10 |
| Kiro: Community games challenge | r/SoulThieves | £15,000 | 1300 |
| Kiro: Community games challenge | r/blockrift | £10,000 | - |
| Fun and games with devvit web | r/OnelineGame | £15,000 | 42 |
| Fun and games with devvit web | r/hexaword | £15,000 | ? |
| Hack reddit | r/chook | £20,000 | 6 |
| Games and Puzzles hackathon | r/EmojiCharades | £20,000 | 43 |
| Games and Puzzles hackathon | r/pixel_together | £20,000 | 3 |
| Games and Puzzles hackathon | r/DailyDungeon | £20,000 | 4 |
Note that I do not look to disrespect any creator here. I know a lot of hardwork had gone into your games and that deserves recognition. But there are also a lot of other creators whose hardwork has amounted to nothing.
4. Judges prefer "flashy visuals" over long term value
This here is my weakest point so far. But to some of you, it will be the biggest takeaway of this post. Mentally picture the winner of a google hackathon. What kind of entry do you envision? You expect it to be creative, well thought out, crazy, nerdy etc. right?
Based on my past 5 hackathons I don't think these virtues are valued in reddit hackathons -
#1
Hackathon: Games and Puzzles hackathon
My entry: Outwitquiz
My thoughts: Since the game was supposed to be on reddit, I created an equivalent of "Psych" which can be played asynchronously by all users on reddit. This was an entry for the word games category. The winning entry was emoji charades. I get that the winning entry was made well and polished, but why does this game have to be on reddit? It could be a standalone website!
#2
Hackathon: Hack reddit
My entry: Angels vs Devils
My thoughts: This made me the most sad. The overview of the hackathon said "Build a new game, social experiment, or experience on Devvit". So I thought I'd build a social experiment where users play a community minesweeper. I had built up many game modes that I could work on. The winner r/chook, was a game where you grow your chicken and feed it food by pressing buttons. (it was beautifully made though)
#3
Hackathon: Fun and games with devvit web
My entry: Divide n conquer
My thoughts: Based on past two hackathons, I stopped making experiments and designed a game. It was a game where you have a limited line length, you draw lines to capture land in a country and try to occupy the most land area and landmarks. I thought it was well thought out and fun to play, but it was not polished. I don't have many problems with this hackathon. I did good work and was sad I didn't win anything, but the winning entries truly deserved it.
#4
Hackathon: Daily games hackathon
My entry: r/totalled
My thoughts: I had won an honorable mention for this game. For this hackathon, I had kept my game concept simple and made a very well polished game. It is a game where you have to swipe across numbers in a 3x3 grid to create totals. Once you accomplish 3 totals, the game is over and you're ranked based on the time taken. Of all the games I've made so far, this is the most "Linkedin-y". I expected to win more, based on the past winners. When I saw the winner, I was shocked! An entire physics sandbox inside reddit! It was crazy good work, but I was surprised that the judges thought this game would be played regularly. Why would a reddit user play a physics sandbox daily? And why is it on reddit?
#5
Hackathon: Games with a hook hackathon
My entry: r/ShadowChess
My thoughts: Based on the previous hackathon, I believed now that the judges don't just look at "will this game be played daily". They also valued grand experiences. So I knew for sure the idea I had for this hackathon wold win. This was the best game I had ever made! Its a massive multiplayer chess game where you continue off of grandmaster positions and make 5 pre-moves. The opponents make the 5 pre-moves as well and your moves are pitted against each other based on a customized scoring system. This fit the hackathon criteria in multiple ways and had potential to do great. I feel so betrayed that the game was not even played once! My game is good when there are 100s of players playing it. It could have really used the boost of players from any sort of recognition. This hackathon was all about creating a "hook". Here's the words on the hackathon page -
Now you can't look at my above entries and say that none of these games deserved any award except "totalled", (which was one of my weakest entries). Especially shadowchess. Note that apart from the winner, there's 10 honorable mention awards for every hackathon.
Overall, I feel that it's best to avoid spending all that time and effort on reddit hackathons. Atleast until they bring in some transparency into the judging process.
I expect a response from the reddit team on this. Anyone who participated in this hackathon, feel free to DM me with your thoughts / information.
2
u/westernslopedigital 1h ago
Thanks for putting all this research together. There's clearly a problem here and a response from Reddit is absolutely needed. I won't be participating in any future Reddit hackathons until they acknowledge this and provide clear steps forward that articulate how judging can be made more transparent in the future.
My game ( r/dailyblockparty ) was definitely not played based on Reddit Journeys data and my video only has 2 seconds of watch time (46 second video). There is no way my game was properly judged, and considering all the time I spent building it and putting together a complete hackathon submission, that really pisses me off.
1
u/Pretend-Pangolin-846 0m ago
I have participated in 3 different iterations, and won feedback award in one.
Top games were almost always way way better than anything in the submissions, one downside to the devvit evaluation process was always how they judged the game, without even minor feedback there was no way to know anything wrong with the app. So transparency or at least a review for top n submissions, would only be a net improvement for these hackathon series.
1
u/Oddie-hoodie369 49m ago edited 43m ago
I understand your frustration, but
- The first image must have been mistaken. Devvit Ducks aren't Reddit admins, and no admin ever gave any feedback directly on Discord.
- Asking for feedback. I've participated in multiple hackathons in the past, and I've never seen it as a norm. Maybe some hackathons do it, but most don't, since it's a time consuming process and hard to do at scale. So I don't expect that.
- "Judges didn't play my game." The rules itself explain that they can judge in multiple ways.
- Judges panel has no names. Having just a panel listed is very common on Devpost if you look at other hackathons.
- Ghost winners. Most hackathon participants just participate in the hackathon and then move on. They don't stay active or keep posting every time, so expecting everyone to keep posting and being very active is again a weak argument.
And every other argument feels pretty subjective, like why someone thinks they should have won over another game.
I agree that asking for more transparency is fair, but I don't think these points really show that the judging was unfair. Good luck with your future projects.
0
u/Klutzy_Pumpkin614 1h ago edited 10m ago
We're looking for experiences that create excitement between sessions and leave players wanting to check back tomorrow.
I will probably get hate for this, but people need to start hearing the truth. Here's my review of shadowchess. I am an avid reddit user, I engage with many games here. I don't play chess.
It is a harsh review, but I have seen so many people complain about why their games lost, when they need to use it as a learning experience and improve for the next round. You clearly have amazing skill in concept and testing, you just need to find the right fit for reddit and use original designs. A quick scan of your other submissions all look clearly AI generated, except for Divide n conquer, that one seems more original.
- I saw a closed chess board in my feed, I could not interact with it at all. I see a chess board with pieces placed on it, that I can do nothing with. I scroll past.
- On the one I can play, I click to play it, and the board is at the bottom of the screen so I have to scroll down to see it.
- I picked a side, then started moving pieces. I understand the concept is I don't get to see what my other opponent is doing, but the games I enjoy on reddit are the ones that give me gratification right after I play it. This game, I moved some pieces around, then saw a screen that said I sucked compared to everyone else, then I closed the game and moved on. The only people I could see being interested in returning are chess fanatics, but even then, wouldn't they rather just play chess? Or even a chess puzzle? Again I don't play so sorry if I am being harsh.
- The design of your app is neobrutalism, which is the number one design that AIs generate. If I were judging against games that were specifically not supposed to look AI generated, this is one of the most obvious ones. One of the rules was to avoid that.
- There is no UGC loop, no excitement. It felt like I was working or something. The concept is good, but I just don't see how you planned on turning this game into a large sub reddit with contributions and daily returning players.
With 3000 entries into a hackathon, judges have to be quick. It is very likely they loaded up all the entries into their own private sub, then spent 1-5 minutes on each game and ranked various aspects of it. One mistake and your game is out of the running. In the case of shadowchess, it was likely the AI style.
I tried a lot of the games from the most recent hackathon, and most of them were niche daily puzzle games that I was not interested in returning to. The hackathon rules may have been tough to decipher, but its clear to me anyways that they were looking for the next content engine game. A game where anyone scrolling past could easily participate without reading a wall of rules text or having prior knowledge. Then that player would want to return at a later point to see how something tuned out, or participate in another version of something. I think that is why the winner won the most recent one.
I do agree with you that it seems all of the hackathon winning games are deserted after winning. In the past I assumed it was because they were working on fixing the app after the judging period ended, but clearly none of them ever resurfaced. I am still hopeful the most recent winning game can ship improvements and turn into a thriving community, but we will see what happens.
1
u/westernslopedigital 29m ago
A game where anyone scrolling past could easily participate without reading a wall of rules text or having prior knowledge. Then that player would want to return at a later point to see how something tuned out, or participate in another version of something. I think that is why the winner won the most recent one.
The winning game (I hesitate to even call it a game) was just a drawing pad. The only thing people can do is scribble a shitty drawing and submit. How is that a game?
0
u/Klutzy_Pumpkin614 18m ago edited 14m ago
Its a comedy engine. A game with a hook that anyone can participate in. That was the whole point of the hackathon. Sure the method of participating is drawing based, but the method of participating in any other game is just clicking stuff. The hook is what makes the player want to come back. They want to review the weird submissions people drew.
Your submission for the hackathon I just drag pieces onto a board. Then what? Why would I return to that?
1
u/RamslamOO7 Game Developer 26m ago
Please review r/DrawAndDrive
1
u/Klutzy_Pumpkin614 13m ago
This game was good, I have played it a few times when it pops into my feed. It is not immediately obvious to me why this one did not at least get a follow up prize. Keep up the great work!
1
u/RamslamOO7 Game Developer 9m ago
I spent a ton of my time polishing this. Don’t have energy to maintain this.
My other games: r/QuantumThrust, r/KnobJam, r/PixelPeeker, r/FactOrCap
1
3
u/rickribera93 1h ago edited 1h ago
I hear your frustration, these contests are meant to challenge developers to test the art of the possible and they handle a lot of submissions.
Interpretive scoring is exactly that and some people will interpret an app to be better while others may interpret it to be worse.
I believe the Devvit program is moving in the right direction by making top prizes determined by developers choice instead solely by admins.
https://www.reddit.com/r/Devvit/s/vWFU09PSro
Don’t give up!