Miscellaneous: An Appeal to Improve Contributed Research & Development: A User Story

[azxten]

The current research and development contribution stickies in this forum are 10 years old now. Much of the information is completely irrelevant now or doesn't conform to how things are really working. I've played on P99 for 10 years and basically did zero bug investigation or contributory work. This wasn't due to a lack of desire but rather an observed lack of a serious process for community contribution. After getting bored I decided to start investigating things that bothered me about the game in an effort to get closer to the classic EQ experience that I enjoy. I figured I would brave this known arbitrary black hole bug forum to try to figure out why things are the way they are and how they might be improved. In the process I also became part of the problem and will detail that below.

To date I have contributed 1 change that made it onto P99 which was reverse engineering the spells file to change spell effects which was built on by Telin to recreate classic effects and then included on P99. This was what initially motivated me to think that maybe I could make contributions that would be valued. Below is what in software development is called a user story and below that is a few suggestions I have on how we could improve the experience described in this user story. This post is intended for Nilbog primarily but the other staff as well. It's not intended for the random forum trolls or anyone else. It is lengthy because it describes years of playing here and months of intermittent research and code contribution effort. It, combined with my channeling bug fix, represents my last chance to try to affect change on P99 in a classic direction that would interest me in being part of this community again.

These are the stages I went through from the idea of contributing to making what I consider a valid bug report:

1. It's no secret I think Enchanter is OP. I've never played Enchanter so it was difficult for me to nail down exactly why this is. In an effort to figure this out I made an Enchanter and I made posts about things I didn't remember being classic about Enchanter. Some of these posts were wrong and mechanics I thought weren't classic actually were. Some of the posts exposed mechanics which probably aren't classic but there isn't sufficient evidence to prove it. Some of the posts exposed mechanics which aren't classic and provably so with what I consider strong evidence.

2. This was basically a shotgun style approach. I threw stuff at the board and found out what stuck. I ended up with some stuff not sticking, some stuff half sticking and rolling down the wall, and other things clearly sticking. However, that process meant clutter in the bug forum. It meant people taking their time to point out that this discussion had already happened in the past. Sometimes pointing out that a mechanic was correct, other times pointing out they'd already reported that issue long ago and it has gotten no response, and mostly a lot of arguing amongst P99 players.

3. Now I could reassess the things that stuck but this raised another problem. What I think is good evidence isn't necessarily considered good evidence to others. At this point, if I want to "fully contribute" to making P99 more classic, I'm going to have to either wait for the staff to say something on a bug report which more often than not seems to only be "fixed pending update" or I'm going to have to assume that the evidence is sufficient and I can contribute more by proposing a code change. In my case I decided to focus on fixing the channeling code because it has extremely strong evidence but I still had to contend with other people on the forum introducing misinformation in an attempt to avoid the change being made and with no confirmation or denial from the staff. There are other things I would propose changes for but I don't think they would get a response or may have insufficient evidence so I'll wait.

4. If I wait for the staff in the vast majority of cases I will be waiting forever. There seems to be no reasoning or logic behind which bug reports will get a staff response, which will get staff effort to fix, etc. Some bugs which seem to have overwhelming evidence languish forever. Other bugs which seem to have almost no evidence get a response "fixed pending update." It doesn't seem to relate to impact on game play, controversy, etc. Druid Tracking is probably the best example I saw of this. A clear bug, well supported by evidence, which is not controversial was bumped for months with no response through multiple patches until it seemed to become a louder and louder cry as each patch went by with the bug report being ignored and finally it got attention. This indicates that if you want a bug report response the best method is to get as many players to complain about it in other areas of the forum as you can. It wasn't a matter of needing evidence, code, or some kind of community consensus. Also it was presumably custom P99 code because I believe Druid tracking was working properly and was nerfed to fit classic without a corresponding rule to revert the change according to timeline meaning code contribution probably wasn't possible.

5. If I decide to try to fix the issue myself and contribute a code change I'm completely in the dark fighting with EQEmulators notoriously difficult build process. For me personally this involved setting up a Ubuntu 20 docker instance, using Dolalin's docker compose, trying Ubuntu 16 VM source compile, and then finally trying Ubuntu 18 VM source compile. The code was written in about an hour but the process to compile it into the server and test it took several hours for want of either a simple existing Docker image that has the code compiled with instructions to edit files in the X directory and then run the make command again and test or even more straightforward the simple advice that Ubuntu 18 seems the be the version that works with the "simple install" instructions on the EQEmu Wiki which mentions nothing about build environment and jumps right into running a script ignoring that you will likely have issues if you aren't using this unmentioned build environment. In addition to that I had to learn how to set things up if my server isn't running on the same system as my client which is the most common development environment anyone would setup today. I had to learn how to make myself a GM and how to enable logging in order to debug my code while in game. Also the guidance offered is to make two quotes, "Existing code" and "Changed code" but this is somewhat impossible if you're changing spells.cpp for example which has 6000+ lines and the changes occur throughout the file. Instead I opted to create a patch file which shows a clear diff between code versions and can also be applied via command to source. All of this is unnecessary duplicated effort by contributors and acts as a road block. Do we want contributions or people to prove they can build EQEmu server code and figure out the server setup and commands to do basic debugging?

6. Now I have a desire and ability to contribute, I've found an issue with strong evidentiary support, and I've made the necessary code changes including testing them in game and including considerations for edge cases and performance impact. However, ultimately, I'm still back at step 4 waiting for the staff to respond which is fine by itself but this may also involve having to bump threads for months, gaining community support that will complain over those months, etc. I'm left in the dark wondering. Is my bumping this thread doing anything? Are the complaints necessary? Maybe this has already been seen, maybe it hasn't been seen and I do have to bump it endlessly until a dev happens to look, maybe it's not going to be fixed for whatever reason, maybe it's being tested and no one said anything, who knows?

7. The end state is months of research from multiple people, arguments and misinformation from players derailing the report including claims about fictitious staff statements, a lingering possibility that the evidence isn't strong enough, hours spent setting up a dev environment that can compile the source due to lack of some simple information or a pre-built contribution image/instructions, the time spent actually making the code changes, and now the possibility of having to manually follow up on my report regularly with a high likelihood of no response for several more months as happened with Druid Tracking.

When Nilbog made the research contribution post 10 years ago he said that if people would spend time researching and presumably also creating code changes he could spend more time fixing or implementing things. This at least indicates that at some point in the past this kind of contribution was encouraged and valued. However, given this and other examples, are the contributors being valued? Does this demonstrate that we, meaning the staff and community collective, do want to improve P99 and leverage community contribution? My experience with actually trying to research and contribute has been a struggle. One where you're basically left arguing with trolls who admittedly have no interest in recreating classic code or experience. You're left recompiling previously submitted bug information that other people gave up on due to a lack of staff response. You're left wondering if the amount of evidence obtained is sufficient to meet an unknown or arbitrary bar leaving you questioning what you should do next. Should I keep searching for evidence? Should I try to propose a code change? This is compounded further by situations where you find bugs that are ignored and have comments that essentially amount to "the staff knows but has decided not to do this for some reason." Is that true? Is it someone trolling because they don't want the change? Does that apply to this other bug and I'm wasting my time on something that even if I prove it and provide a code change will never be implemented? Also you're left with a nightmare EQEmu build process with no guidance that may or may not even be necessary but in my case I wanted to ensure my code compiled and ran appropriately. What if I don't even know how to code but I want to make database changes I proved would be classic and test them in game? Quests? Mob spawns? Etc. That is locking out a lot of potential contributors who if given a helping hand could otherwise get past the scary code and build stuff and just focus on their particular bug they want to fix which may not even involve code.

So, reflecting on all that, I would like to respectfully propose the following:

1. The research and code contribution stickies should be rewritten. If the staff has no time or desire to do this they should indicate a desire to have this updated so the community can rewrite it. It is very telling that these posts haven't been updated in 10 years. My suggestion is that if we really want things to progress and P99 to get better engaging volunteers is critical and shouldn't be an afterthought with the staff mostly being a black box where you have to fight for a contribution to be acknowledged. Nilbog as project manager shouldn't researching or fixing he should be managing researching and fixing by giving out "good job but needs more evidence" or "this seems broken but we don't have time to make a fix please write a code suggestion" or "this is provably unclassic but we won't implement this mechanic because it's too unbalancing" kind of posts to build up volunteers rather than leaving them to be attacked by trolls or endlessly bumping their threads until they give up. Nilbog seems to acknowledge this himself in the research thread that he has a desire to not be doing that work but the framework to foster others doing it is absent probably because he is doing that work instead. There are people here, we'll do the stuff, this community is much larger now as well and maybe 10 years ago there weren't any contributors to rely on which should be revisited today.

2. The staff should try to rely more on the community. Coming back to Nilbog's statement. Right now as a contributor I'm left wondering if I put in the effort will it amount to anything? Coming back to the contribution stickies as an example, if I rewrote them would they be updated? Would I just end up in the same situation bumping my thread endlessly waiting for a response and having already invested my own time with no assurance it is desired? Nilbog posted on the research thread acknowledging it is outdated now and should be rewritten but why not then follow up asking for someone to rewrite it? This is what I mean by rely more on the community. Get those things out there that the staff knows should be done and actively ask someone to do it rather than the passive model we have now or holding all the issues in a log somewhere until the staff feels they have time.

3. Start acknowledging and building processes around what seems like 4 stages of a bug here.

The first stage is things that are already known but there is no way to know it is known. This means bug reports that are already known to be classic mechanics. Bug reports that seem like legitimate bugs but someone else already submitted it sometimes multiple times. This is all the clutter created by well meaning people who are just starting to consider contributing and are asking about something only to be met with an inability to find out what is already known about that thing in order to avoid duplicating discussions. Most likely, as I've suggested before, this means using a bug tracking system rather than a forum. Discussions happen in a bug report but using a forum means bug reports are always falling out of visibility. Bug tracking systems lets the staff maintain a list of actual bugs that contributors can see as well. What if I just want to fix some bugs? I have no way of knowing right now what bugs I should work on or if the staff could use help either researching them or providing a code change. Just an endless list of forum posts to look through with ambiguous states of research, fix, won't be fixed, etc that we're left guessing about.

The second stage is defining or acknowledging once a bug has enough evidence to be considered something we want to fix. Currently players generally just argue amongst themselves about if evidence is good enough or not. There is no rules about this we can apply ourselves and there is very rarely a staff response indicating something seems broken. Without this bugs are almost always left to die in this state where the contributor or anyone else viewing has no idea if the staff would consider a proposed fix and so people just talk endlessly trying to find more evidence and bumping the thread. Often part of the evidence is just proving it is broken on P99. Take the INT/WIS 200 cap or Enchanter pet HP bug reports. Do I have an Enchanter high enough to test this with the research data? Do I have the ability to go over 200 INT/WIS and test my mana pool? Contributors are blocked in some cases by having access to P99 in game situations to confirm a bug especially old bugs they may want to fix or bring up again. Many bugs are likely already provably in need of a fix but just languish being bumped and researched more often, as I've seen, with previously existing evidence being lost as links 404. This stage describes the state of most bugs on the forum excluding people mistaking classic mechanics for a bug. The evidence varies from weak to strong but without a bar to meet no one can know just how weak or strong the existing evidence is but often there is evidence of some kind. Having some kind of guidance about what is strong enough evidence or a process for staff to acknowledge the evidence is strong enough to implement a change would be very helpful.

The third stage, which almost never happens for contributors, is a bug being acknowledged as valid and worth fixing. Instead staff typically would only respond if they are already working on fixing it or have fixed it already. The channeling bug I would say entered this state only due to the strength of classic evidence but without any acknowledgement from staff. There is no point at which a code contributor is encouraged to try contributing code and this creates a big disconnect between researchers and coders. Again, as someone who can write code, what bugs should I work on? There is probably not a single bug in this entire forum even if I read every single post where I would be able to say, yes, this is a bug that the staff agrees should be fixed and my code contribution is worthwhile AND they're not already working on it or it doesn't involve some custom code i can't access. Being able to see a list of bugs the staff actually wants fixed and that could be fixed by a contributor would be very helpful. This comes back to the bug tracker idea.

The fourth stage is a bug has strong evidence, proposed changes to fix it, and is now awaiting staff response but will likely disappear down the forum pages if not actively bumped and discussed on the forums. After going through the effort of researching, discussing, coding, etc it would be nice if contributors could trust that they will get some kind of response without having to continue owning their bug report indefinitely and bumping it possibly without ever getting a response.

In conclusion, there are currently ~7,000 bug reports in this forum and ~50,000 posts that aren't in the resolved section. This likely represents hundreds or thousands of contributors efforts towards finding out if something is a bug, trying to research proving a bug and/or how it should work, and making recommended changes to fix bugs. I don't think the staff doesn't respond to these because they don't want to or they don't care but rather because it's an impossible volume of work for the size of the staff who holds all or most of the necessary authority to move bugs between the stages listed above. If that is true and the staff still wants people to contribute and feel valued and get these things fixed for P99 then maybe, coming back to my first suggestion, more staff effort should exist around leveraging contributors to help respond and filter that work.

There are clearly some contributors here who are well known and possibly they would volunteer if approached to take on more responsibility. For example, Daldaen and Dolalin and similar well known researchers could act as a filter for stages 1 & 2 above. They typically know if a bug is actually a bug or an intended mechanic or if it has already been submitted. That simple function and enabling them to delete, merge, etc threads in the bug forums might go a long way by itself. They are also likely somewhat trustworthy sources on if a bug probably has sufficient evidence or not that it would be considered worth proceeding with trying to fix it and could add some official weight by giving their approval that a bug has met the necessary evidentiary bar. This would get us to stage 3 because we could see which bugs they have approved for trying to fix and it could also allow the staff to get a short list of pre-reviewed bugs that they themselves might want to respond on either to confirm/refute that initial review or to at least respond after a code change is recommended so that the researchers and coders don't have to endlessly bump and complain about the bug to try to get a response. Presumably if the staff just had a list of bugs that were pre-reviewed for valid research and/or definitely included a suggested code change it would be a lot easier for them to respond and implement those fixes. I'm not saying Daldaen or Dolalin or anyone else would take on this role but some kind of process like this would be pretty nice to get things progressing on fixes and allowing contributors to feel like they're not wasting their time and unappreciated.

I have played P99 for a decade, I've made donations, I've contributed changes that went live, I've researched, and I've made what I consider to be a full complete bug report with strong evidence and suggesting code changes. The process of contributing has been extremely annoying for all the reasons listed above and I'm left with the impression that the staff doesn't care about contributors here but rather has their own list of bugs they care about and this forum purely exists for the staff to review if anything pops up they happen to care about like Velious custom helms breaking but they can ignore Druid tracking as less important from their perspective. I have no idea what the true state of things is that is just the impression I'm left with. I would not contribute anymore here until something changes and especially not until that channeling fix gets a response without me having to bump it for several months like in the other thread on that topic. My current feeling is that while I enjoy P99 and would like to play here again if some mechanics could be adjusted to make things more classic trying to get those changes done by contributing in this forum is a painful process with no definite value to P99.

Thanks to the staff for P99, it has been great, but for now this marks the end for me. The server has become too stale and formulaic due to what I think is a lack of progress on making things more classic. It's too easy, there are too many abused classes and mechanics, and there seems to be little effort towards improvement that doesn't originate from the black box of staff devs with seemingly no consideration for gameplay or consistency. Too many classic mechanics are nerfed and too many non-classic mechanics are ignored which seem to have strong evidence for how they should change with relatively minor code changes.

A lot of contributors here take a "staff will decide maybe someday if they care" approach and simply acknowledge their bug reports get no response and they seem okay with that. I'm making this post because I enjoy P99 and want to see it continue to grow and improve. I don't think ignoring the situation outlined above is helpful and while maybe some people think I "have no right" to complain, and that may be true, I'm still going to "speak truth to power" because sometimes that actually gets something to change in a positive way. I've been banned on P99 forums and in game, I've been unbanned, banned, unbanned, sideways banned, and I've even insulted Rogean and Nilbog in the past and they still unbanned me after it which I have huge respect to them for. I try to walk this line between ultimate respect and being an asshole because I believe that sometimes people are too passive and problems aren't just ignored because of it but rather aren't even seen in the first place until someone dares to risk their accounts to speak out and get yelled at by all the trolls and fall out of favor with the staff or community. Maybe the staff is fine with things the way they are now but if their past statements about desiring community input are true then they should consider what is stopping some people from wanting to contribute and how we could fix that.