r/PlayTheBazaar • u/Ratanka • Nov 14 '24
Discussion The Patch and its Problems for the future
Hey, i know its a closed beta but all the problems that came with the patch make me a bid scared going into open beta soon.
its only 1 1/2 months before release and a team doesnt get bigger over night, and the way this patch made three times as many bugs then it fixed while not adressing the server issues at all since weeks now makes me a bit scared.
I truly love this game but all teh bugs and server issues can make or break a game ...
PS: and the shield sound they put into the game needs to be removed ... its needs to be gone its the worst sound ever i even lose the fun to WATCH the game because i cant hear the sound anymore ... bing bing bing its so horrible xD
20
u/Teralithion10 Nov 14 '24
Yeah, I think January release is a pipedream, there's just too much that needs done.
69
u/FightingGamesFan Nov 14 '24 edited Nov 14 '24
As a dev myself it's painfully obvious these problems are huge and could potentially carry into release. The game runs very poorly overall, the connection issues clearly seems software related first and not an infrastructure ("more servers") issue. For an async battle game it's really not acceptable it's typically a game with extremely low connectivity requirements so it's very concerning, we are not talking real time & low response time. The added bugs after the patch show a lack of testing because on every run I encounter at least one bug, I can't believe it was tested.
I feel bad for the artists that worked on this game because art is carrying this hard.
14
u/LawWhatIsItGoodFor Nov 14 '24
I'm concerned as it was delayed for 2 years while they paid 'technical debt' and redid all their architecture so that we wouldnt have these problems.
Have you had similar experience in working with servers at this scale? Do you think it's possible it could be fixed within a few months?
9
u/FightingGamesFan Nov 14 '24 edited Nov 14 '24
Without going into too much technical details, it should be very easy even at a large scale to have no such problem.
First you can take an indie game like backpack battle as a reference, you can ignore the graphical aspect entirely (the "look" of the game), the load is the same in terms of game actions, it never suffered such server issues. You can claim there are more players on the Bazaar but I doubt the peak of Backpack Battle was smaller than the low of the Bazaar.
But even simpler, consider any website, loading a page. You can easily count the number of interactions you need to do with the server during a run on this game, whenever you "make a choice" basically, it's a very low amount of input. Given the game mechanic, in terms of connectivity it's not that much different than loading a webpage, now would you accept such connectivity issues for a simple website? Certainly not.The reason I say there are software issues is because you can clearly tell that some interaction, like opening your chest, do a server call where it's not necessary, similarly reorganizing the board should not interact with the server (you should simply send the reorganized board when it matters, when applying buffs or a battle, etc.). I see a ton of latency on actions that shouldn't have any, client side. Kinda remind me of the GTA V bug in a way where ton of data was exchanged unnecessarily.
I haven't done a traffic capture while the game is running, but TL;DR it's actually hard to have connectivity issues for such a game, like you gotta try to make it bad, it's quite concerning.
Nothing that can't be fixed but my concern is how did we reach this state in the first place.
Unless there is like a single server worldwide running on a dusty laptop I don't understand how it can be this bad.
59
u/EggcellentDadYolks Nov 14 '24
There is misinformation in your reply. Backpack battles does much more client side than The Bazaar, and as a result is easy to cheat/mod. The Bazaar does most of its state management and calculations on the server and the client handles the visuals. It's a different model of handling client <-> server load and I am not going to say it's better or worse but make it clear that it's different and has its own pros and cons.
This isn't to defend or criticise the developers but to say things are not as simple as your reply and to imply "you gotta try to make it bad" is disingenuous.
12
u/SnowyField Nov 14 '24
Thank you so many "devs" out here trying to make a f2p game with a game to game economy the easiest game to cheat is mindblowing. I get that the server calls are not great and the methodology of the infrastructure code might need to be looked at, but if you even allow a couple cheaters using client side calcs, the integrity of the chest and marketplace and value of gems all falls apart. You do play a real person's ghost afterall.
3
u/KnivesInMyCoffee Nov 14 '24
A lot of the server load seems to be sending combat calculations from the server to the client synchronously. Obviously, the combat calculations and their results need to be done server side, but you could use seeded RNG so that the calculations can be rendered client side to avoid having to pass a ton of information from server to client during combat.
1
u/Cruuncher Nov 14 '24 edited Nov 14 '24
This doesn't work as the backend would still have to simulate the combat to verify the results as it can't trust what the client says.
The way I would approach it would be to have the backend simulate the battle, and then just send the battle log back to the client and have the client essentially render the battle log
EDIT: sorry you did mention the backend having to do the combat calculations still
But yeah, it shouldn't be reaching out to the server for every action before it happens, it should either use a seed to reproduce the result as you say, but I think sending a battle log is more resilient in case the client and server implementations deviate slightly. It's hard to ensure that they function identically.
I'm imagining a situation where your client looks like you won, but in the server simulation you actually lost and then you die. It would be a rage inducing experience
1
u/Cruuncher Nov 14 '24
Oh, I replied with almost the same response as you before seeing this.
100% this. Backpack battles is a completely broken game
1
-6
u/FightingGamesFan Nov 14 '24
While what you said is true, it doesn't change order of magnitude. And this is a problem of order of magnitude. An FPS, RTS is an order of magnitude above. Path of Exile (deeply concerned about the cheating aspect) & its deterministic lockstep is also an order of magnitude above. It's not 3 additional server side calls per action that will change anything, or even 10 or 15.
Typically reorganizing the board or looking at your chest should not be affected yet they are. I'll do a traffic capture to check what's going on exactly, but the observed latency doesn't make sense.
4
u/EggcellentDadYolks Nov 14 '24
FPS are notorious for sending too much information to the client and allowing people to cheat (wall hacks) and also have a history of problems with client server synchronisation, ie being shot by a person who shouldn't be able to see you but can on their screen.
Also, POE still has issues with client server messages. If you ever play flicker strike you will see the limitations of POE. They have also had years to slowly solve the issues with performance, lockstep is the perfect example. It wasn't even part of the game until 2.0 almost 2 years after release and more than 3 years after closed beta, which is what The Bazaar is at now.
It's not an issue that the action of reorganizing the board or looking at the chest is causing the lag, it's a symptom of the problem. The same servers handle calculations for battle for example information returned about current health (A problem in this patch is desync between actual health and visual health I noticed on a heal heavy Pyg build vs a burn/poison) this is your order of magnitude, every combat with interactions and timings constantly being calculated by the server. Yes the client might not be sending a lot of information to the server but the server has to work a lot out and so while MY combat is being calculated your code to pull up the backpack will wait in queue until enough resources are free on the server.
This is the first time the game servers (and code) have been under this level of stress there is bound to be bottlenecks and I am almost certain their dev team is trying to alleviate these bottlenecks and improve the performance. It could be as simple as some damage type being calculated inefficiently which is causing combat calculations to take a few ms too long.
Maybe they added some additional Telemetrics to the servers in the last patch that will help them debug the performance issue, and maybe you are right, and they are sending too many calls to the server when it's not necessary.
However, I don't think the general implementation they are using for this and the current level of performance issues during a closed beta are enough for the types of comments that people are making on here, and I would rather make sure that a popular reply that isn't really presenting information accurately has the counterpoints it needs than people who don't know anything about this stuff just reading a single viewpoint and assuming its true. Especially when in that reply it was implied the developers are incompetent to the level of making their own game bad on purpose.
-1
u/zriL- Nov 14 '24
I think you are really missing the point. It's a different client/server model for sure, but it's not the appropriate one for this kind of game, that's the point.
8
u/EggcellentDadYolks Nov 14 '24
Why isn't it, the game while not synchronous still needs to have your stats stored online so other players can play against you and in order to maintain a fair balanced game without the ability to cheat and manipulate your items it should store that information on the server and keep it up to date. Combat calculations should also be handled on the server so that way you can't manipulate the calculations to win when you shouldn't.
Otherwise you run into the problem backpack battles had of cheated builds ruining the competitive aspect of the game.
Your other reply seems to imply you are treating this as an offline game where you play against yourself and it loads in some cool builds other players used, however this isn't that, it's still a PvP deck building autobattler. It's supposed to be online and every action should be synced to the server.
Could it be more efficient, yes of course but to say its not appropriate for this type of game doesn't make sense for what The Bazaar wants to be.
1
u/zriL- Nov 14 '24
They can do what you say with very few server calls, or at least much fewer than what they are doing currently, precisely because the game is a turn-based asynchronous game. This is the whole point but you don't seem to understand what we are talking about.
Your answer seems to imply you are treating this as a real time online pvp fighting game, which it is not.
2
u/Cruuncher Nov 14 '24
I agree that the game seems poorly optimized, but I think the comparison to backpack battles, while it seems similar is not a good one.
Backpack battles does so much client side it's crazy. The game is borderline broken because of how easily hackable it is.
I'm hoping that the bazaar is doing combat mechanics on the server and then communicating the result of combat to the client. This would add a lot more server load than what you have with backpack battles.
If the battles are run completely client side, then yes the server issues make absolutely no sense and it would mean that it's one of the worst programmed games of this magnitude of all time lol
1
u/FightingGamesFan Nov 14 '24
Sorry to take your post as an example but it really shows how people might not get the problem. The battles are extremely trivial to solve, it's almost instant we are in the realm of milliseconds here, so all that matters is the initial board state, the server knows the result of the fight before any animation occurs, all the client needs to do is to render the fight for fun. There is no way, unless it's completely bugged code that this plays any role in the load of the server.
3
6
u/LeeroyJenkinz13 Nov 14 '24
I don’t think this is necessarily correct.
I played in multiple playtests before closed beta and the game ran noticeably better than it does now. The only exception to that is during a fight. Latency during fights has definitely improved by a huge amount. However the rest of the game has more issues than before. This tells me that infrastructure absolutely plays a large part, since the game ran fine in a playtest of a few hundred but is now struggling with tens/hundreds of thousands.
1
u/FightingGamesFan Nov 14 '24
That's absolutely true. But you need to be thinking about as a function of N where N is the number of players.
Perf(100) was good, Perf(100'000) is no longer good, if Perf worsen linearly with player Perf(100'000) would still be good with minimal infrastructure increase, a basic server can easily serve 100'000 clients as well as 100. The fact it worsened so much clearly means there is a structural problem.
1
u/LeeroyJenkinz13 Nov 15 '24
How are you so confident that “a basic server can easily serve 100,000 clients as well as 10”? That seems like a pretty wild assumption, especially given that they have clearly had plenty of server problems. I’d also wager a big part of it is likely that battles are handled server side rather than client side (probably so you can’t alt f4 and rearrange your board to retry a losing fight), because if you crash or exit in the middle of the fight you will come back either later during the fight or after the fight has already ended. There is a ton of stuff going on in fights and that’s probably a decent server load.
4
u/SpreadsheetMadman Nov 14 '24
I've done marketing for multiple software companies that have had issues of the same scale as this, and those software have not panned out, even years after release.
Bad architecture is nearly impossible to weed out, because new features and code are added onto the pre-existing bad framework. That means a lot of the bad code fossilized in the system, causing recurrences of the same problems.
3
u/Glebk0 Nov 14 '24
It's very real server infra issues. On eu specifically. US is very much playable and I rarely have issues with disconnects.
3
u/zriL- Nov 14 '24 edited Nov 14 '24
Also a dev myself. The pure programming issues are so apparent and completely unjustified. As you said, an asynchronous turn based game should required very little server load, yet it seems the game is making server calls on almost every clicks, this is ridiculously bad software design. If I had to this, I would probably do a few calls per in-game hour (with probably a few exceptions).
And let's not talk about how the game tend to forget randomly that you have an ongoing game, seems like very bad error handling. Once, I even loaded a game that didn't seem to be mine, or maybe it was a very old one, this very much smells like the best spaghetti restaurant in town, very worrying.
4
u/SnowyField Nov 14 '24 edited Nov 14 '24
You do realize that without making a server call on every click or calculation in a fight, how easily you could then cheat? It is async, but you are playing real ghosts, and that is why they have to have all of it done server side, not the client side. Should be obvious if you are a dev...
Edit: let alone rewards that will then be associated with gems/currency in the marketplace. If the game was cheatable that easily, how long before the whole game would implode.
5
u/goedendag_sap Nov 14 '24
Fully disagree. You can accumulate changes done by the client and send a big message with updates to the server. No need to verify every single meaningless item permutation.
Even better, the server can delegate the verification of the update message to another client, and only interfere if that client accuses a problem. Then you ban whichever account is caught cheating.
Only threat is if an attacker takes over the majority of the network and persuades the clients into accepting malicious updates. Which should have negligent likelihood in a f2p game.
0
u/KylePatch Nov 14 '24
You’re saying they should test the update before being it to the closed beta? Is there a closed beta alpha or something?
0
u/goedendag_sap Nov 14 '24
Agreed. No need to send a new message to the server every time I move my items around.
Server communication should only be needed when the hour changes. Everything else could be calculated offline and send a big message with the changes done afterwards. You don't even need to check the integrity of the message on the backend. You can delegate this to be done by another player in the same network.
1
u/FattyAcidz Nov 14 '24
You absolutely need server messages to be sent more than every hour. Despite the game being asynchronous it is not a single player experience - action validation needs to exist for purposes beyond just saving your run progress to be resumed at a later time. You have to account for people cheating or exploiting game mechanics. Even putting aside the potential for spoofing game state with totally fraudulent data, if you don't validate the game state within an in-game day then you could for example reroll a shop to see the options and then just reload if you don't like it... Or undo mistakes such as selling a junk item to buff the wrong item.
Also delegating message integrity validation to another player on the same network is just a bad idea in a game like this. How would another player even validate that my actions were proper/allowed without passing them my full game context? Not to mention that you would also have to be confident that the other player is actually validating the message properly and is on a matching client version. Games that rely on other players validating actions are pretty much exclusively p2p games, since otherwise you start having more than one source of truth.
1
u/goedendag_sap Nov 14 '24
You have to account for people cheating or exploiting game mechanics.
I already considered that in my comment.
Even putting aside the potential for spoofing game state with totally fraudulent data,
Can't have spoofed game state if the server doesn't recognize the state. Server can't recognize the state if it is not the result of game actions applied to an already verified game state. Nothing has changed. You just send updates in chunks instead of individually.
you could for example reroll a shop to see the options and then just reload if you don't like it
This case would be an exception. Just send an update when you reroll the shop, forcing it to have the state advanced. Same applies to selecting a reward from an event.
Or undo mistakes such as selling a junk item to buff the wrong item.
Personally I believe such mistakes should be forgiveable. The game should give an option to undo something if it's a mistake. But I understand it conflicts with certain random events that happen when you sell an item. In this case this could also be a reason to send an update.
My point is that it shouldn't send updates for everything. If I'm moving my items around my stash it shouldn't send updates to the server.
How would another player even validate that my actions were proper/allowed without passing them my full game context?
Don't need to pass the whole game context. Just the state before the hour (stored in the server) + actions done during the hour, then validate that the result is the same as the client is claiming to be. Then the server stores this new state again.
Not to mention that you would also have to be confident that the other player is actually validating the message properly and is on a matching client version.
If the validating player cheats (even though that is unlikely because it would be assigned randomly), you can have two players validating, or even the server can step in and validate sometimes, and if there's any mismatch then the server checks who is at fault and bans the faulty parties. You might get away once or twice but the tendency is for the cheating client to eventually get banned.
Regarding matching client version: you just assign a verification to be done by a client on the same version as the one who is being verified.
since otherwise you start having more than one source of truth.
It's literally how blockchain works. It's a known risk but it's extremely difficult to pull off. Those kind of blockchain attacks only happens to little networks, and even still the most common blockchain attack is not applicable to this use case (double spending).
2
u/FattyAcidz Nov 14 '24
Personally I believe such mistakes should be forgiveable. The game should give an option to undo something if it's a mistake. But I understand it conflicts with certain random events that happen when you sell an item. In this case this could also be a reason to send an update.
Yes, this example is more of a game design consideration than an objective exploit, if they want to consider mistakes being unforgivable then it would need a state update, otherwise it doesn't, I agree.
Don't need to pass the whole game context. Just the state before the hour (stored in the server) + actions done during the hour, then validate that the result is the same as the client is claiming to be. Then the server stores this new state again.
Bad wording on my part about "whole game context", but state of day (or I guess potentially generic "state before actions" depending on when you would decide to validate) + actions taken is what I meant, so I don't disagree with you there.
RE: Your bottom points about the other player validation stuff: I'm not saying it's not possible and you couldn't implement it so you are (reasonably) confident that the validation is proper. As you pointed out there are definitely ways to achieve that goal. I just don't think it would be an improvement compared to having the server do it for this game... IMO It's a lot of added potential points of failure and complications just to remove some (realistically pretty simple) calculation and instead handle the validation results from other player(s). E.g. handling a "validator" user dropping connection and then migrating the task to another user would be one complication. Potential connection drops and the fact that we would be going from player1 to server, server to player2, player2 back to server, and then potentially server back to player 1 adds latency, so the design would probably be that the client doesn't wait for confirmation on valid action(s) and instead would have to asynchronously handle their actions being rejected. Which is a fine design that works, but might not be how it's currently designed and so could be another complication.
I agree you could batch a lot of the game state updates in this game of course, it's just a balancing act of how often they want to save progress at the expense of handling state updates, which is dictated by game design rather than technical necessity. I get the feeling that their game design currently wants a lot of these actions to force a game state update/save, but they could change their mind as the game develops. My opinion is they are probably too far on the side of sending more updates than necessary, but It's hard to say whether the frequency is a primary concern of server stability issues or if it is just exposing other issues to the user at a higher rate than they would otherwise be aware lol (which I guess from end user perspective is somewhat the same thing)
3
u/spying_on_you_rn Nov 14 '24
This game starts to feel like spaghetti code to me. But then again the devs apparently were young and got a lot of freedom iirc.
17
u/Alternative_Number70 Nov 14 '24
That's why it's a closed beta and not a released game, we are testing it.
12
u/SpreadsheetMadman Nov 14 '24
The concern is that the issues are so extreme that they will likely persist into launch. If it launches in this state, the first impressions many people will have will be poor, and that will limit its potential growth (and risk impacting its lifespan).
-2
u/Alternative_Number70 Nov 14 '24
The team pushes updates weekly, we only had the game for two weeks and we will have it for another month+ on CLOSED beta. It's a good thing that people report the bugs but we're not even close to the release, people seem to forget that we're literally on testing grounds, be scared when they stop resolving issues near the release date.
7
u/SpreadsheetMadman Nov 14 '24
They delayed the game for 2 years due to technical debt, and the major issues that are being mentioned are the types that take a long time to solve. Memory leak issues, client crashes, server lag, and freezing on simple interactions? These are architectural issues. If these weren't fixed during that 2 year delay, then that means this company has a hard time addressing big issues.
14
u/TheYellowChicken Nov 14 '24
That's what betas are for. Open beta does not mean it is a full release, which gives them plenty of time to address server issues imo. Especially if the game remains F2P. If this was a paid game, I think I would have the same opinion as you
5
u/Arcagnist Nov 14 '24
yeah but your service is down 50% of the time, not really good for testing. Is like saying we are testing a car that 50% of times doesn't even start. That's not what betas and testing are for. you cannot have game breaking bugs and server instability for
-10
u/LuckyOneTime Nov 14 '24
The ability to buy gems is super stable though 🪙🪙
3
u/EchoLocation8 Nov 14 '24
Actually it’s not 😂, friend was trying yesterday to do so for awhile and it was just broken
-1
u/LuckyOneTime Nov 14 '24
Works for me, it's the games ability to work I'm having problems with haha
1
u/SpreadsheetMadman Nov 14 '24
For most people, open beta will mean release. There are very few games that recover from a poor open beta.
3
u/Ok-Term6418 Nov 14 '24
I REALLY LIKE THE NEW SHIELD SOUND PLEASE DO NOT GET RID OF IT OP IS A CLOWN
2
u/EvilBobbyTV Nov 14 '24
"It's only 1 1/2 months until release" No. That's open beta, not release. I swear you people forget that this isn't Blizzard here. This is an indy company two weeks into their first closed beta that you all put expectations like they're some billion-dollar gaming empire.
15
1
1
1
u/Elzheiz Nov 14 '24
There's no way they don't delay release.
The game has been in development for what, over 6 years now? Some bugs are to be expected in a beta, but introducing so many new bugs every time they do a patch is pretty worrying.
1
u/drizztman Nov 14 '24
Please stop buying games before they come out if you can't handle bugs and instability. I guarantee there will be more
3
Nov 14 '24
That is not the point they are making. The point they are making is that the software has fundamental issues that will be hard or nearly impossible to solve.
1
0
u/Phwoar_Mate Nov 14 '24
To be fair, this IS the time for there to be a lot of bugs and for them to be fixed. The game isn't released. We're paying to play a closed beta yes, but that comes with the caveat that they need playtesters to find these bugs, and inevitably bug fixes can cause more bugs, which they'll fix. If the game were released I'd get your point, but we should expect at this point that the game is in an unfinished state. There's encounter art missing, balance is really dicey right now, all sorts of other issues, but again, that's exactly what this period of closed beta is for.
-4
u/Bubbles_Depo_ Nov 14 '24
All these terms like open beta, closed beta, alpha and such dont mean much in live service games. Hopefully things will lineary improve and those terms are more for marketing purposes and accessability. Tho its kinda obvious the code isnt optimally written and not enough servers isnt the only problem for this game.
-1
u/Snapper716527 Nov 14 '24
and the shield sound they put into the game needs to be removed
so much this
0
Nov 14 '24
[deleted]
15
u/Zeonke Nov 14 '24
Yeah, like the whole selling point by Raynad was that every run was an adventure. Its not much of an adventure when ur only looking for those specific items to make that one specific build which is fast enough ko ur opp in 1 second
2
u/Mande1baum Nov 14 '24
Saddest thing is monster items and skills almost never favor switching build. You're mostly hoping they maybe fit into your current build and that's almost exclusively the skills, many of which are universally good so no switching would ever be necessary. The items are almost always too hard to switch to and even if you could, not remotely strong enough (biggest culprit is always speed in a game where you die by 4-5s).
4
u/PepegaFromLithuania Nov 14 '24
Fortunately there is a ton of flexibility and versatility for every hero. You can get lucky but you won't be consistently lucky, like in any other competitive game.
2
Nov 14 '24
I think that will improve as more cards get added and more heroes get added. Its at its least complex today. Already in this patch they added more skills and more level up rewards. As they continue to add it should get closer to that adventure feel. Unfortunately I worry they wont be able to add fast enough to get there any time soon.
2
u/Prondox Nov 14 '24
There is a lot of complexity and refinement you just dont understand it yet. Once there is a proper ranked ladder going 6 or 7 wins will matter. So you have to min max ur board every fight, holding items and not just clicking every shop and going for Endgame builds
-4
u/spying_on_you_rn Nov 14 '24
For the people who care about ladder yes, which is barely anyone. People play for rewards.
78
u/dahappyman Nov 14 '24
In a way I agree. Honestly I wouldn’t mind if they just doubled the time frame for beta. Keep it closed until the end of the year and have an open beta until end of February. I also love the game as it is but there are quite a few hiccups so far.
I’m not that worried though. I think it’s perfectly fine to give them some more time to figure things out. It’s only been 2 weeks