Is it possible to download the wiki to have it locally? [Archive]

loramin

02-25-2026, 07:53 PM

There is no link, because if it existed it would constantly have to be rebuilt every time anyone updates their Magelo (or anything else).

As for scraping, there might be a tool out there that can do it, but personally I'd just use AI to do it with custom code.

However, if you do it, one detail is critically important: only scrape at off hours (eg 1am PST) and "rate limit" your scraper to only fetch a page a minute. If you don't do that you're effectively attacking the wiki and preventing anyone else from using it ... and I wouldn't be surprised if Rogean has code to blacklist IPs that do that sort of thing.

Mortdecai99

02-25-2026, 08:47 PM

If you don't do that you're effectively attacking the wiki and preventing anyone else from using it ... and I wouldn't be surprised if Rogean has code to blacklist IPs that do that sort of thing.

Isn't that exactly what's happening to the wiki now? Obviously there is no code to prevent it if so. https://i.imgur.com/sV56Gnl.png

loramin

02-25-2026, 09:40 PM

Swish

02-25-2026, 11:07 PM

Use the wayback machine if its being slow

http://web.archive.org

Hannibal

02-27-2026, 03:50 PM

Use the wayback machine if its being slow

http://web.archive.org

I have been here since 2012. I have used the wayback machine when asking for old forum post info etc. And yet, I never thought to use it for our own wiki when it takes a nosedive? Dammit... Thank you Swish!

Mattson

03-08-2026, 04:57 PM

Right, but whoever is spamming the wiki right now is probably not a player here; it's some loser hawking penis pills and other garbage.

Presumably you don't want to be like them because A) you don't want to ruin the wiki for others, and B) unlike the spammer, if Rogean does blacklist your IP you'll actually care (because you won't be able to play anymore).

To be clear, I have no idea what protective measures Rogean does or doesn't have in place ... but again, even if B) isn't a concern, I'd hope A) is.

Every time someone with pigparse considers a mob it sends a request for the relevant p99 page.

Some people just spam the consider button. Just remember if you have pig parse and you hit the C button you're making it worse for everyone.

Its worse on raiding days because all those raiders are hitting consider and 99% of the time they're not even using the mob info window so they don't care that they're killing the wiki.

loramin

03-08-2026, 05:52 PM

Every time someone with pigparse considers a mob it sends a request for the relevant p99 page.

Some people just spam the consider button. Just remember if you have pig parse and you hit the C button you're making it worse for everyone.

Its worse on raiding days because all those raiders are hitting consider and 99% of the time they're not even using the mob info window so they don't care that they're killing the wiki.

I would imagine/hope the coder behind it has heard of caching ... in which case, you're mistaken.

EDIT: Oh, it's a tunnel quest site. If it literally has a "update from wiki" button then yes, hitting it hits the wiki ... but why would anyone spam that button? Wiki prices don't even change daily on most items, let alone hourly or sooner?

Regardless, there's a world of difference between hitting one page, even a few times ... and hitting every page in a short period. How is that not obvious to you?

Jimjam

03-08-2026, 06:15 PM

The beauty of a wiki is it is a product of player contributions and edits, so you can simply create your own p99 wiki to browse when offline.

bcbrown

03-08-2026, 07:39 PM

I would imagine/hope the coder behind it has heard of caching ... in which case, you're mistaken.

EDIT: Oh, it's a tunnel quest site. If it literally has a "update from wiki" button then yes, hitting it hits the wiki ... but why would anyone spam that button? Wiki prices don't even change daily on most items, let alone hourly or sooner?

Regardless, there's a world of difference between hitting one page, even a few times ... and hitting every page in a short period. How is that not obvious to you?

Pretty sure it's you who's mistaken. I don't think you understand what PigParse is. It's not just a tunnelquest website, it's also a log-reading utility that does DPS parsing and mapping, and I guess loading all items the wiki lists as dropping from a mob every time you /con it.

Here's the ConHandler hitting the the WikiApi:
https://github.com/smasherprog/EqTool/blob/ecec31900d679904235eff849a0ef4637e118920/EQTool/Services/Handlers/ConHandler.cs#L46
Here's the WikiApi directly hitting the wiki:
https://github.com/smasherprog/EqTool/blob/ecec31900d679904235eff849a0ef4637e118920/EQTool/Services/WikiApi.cs#L107

Sure don't see any caching happening.

If it's hitting the p99 wiki every time any player using pigparse /cons a mob, that's a lot more than "one page, even a few times". And sure, I'm aware of the difference between that and an non-rate-limited multithreaded crawler, but unless you have some special insight into wiki traffic from being an administrator, neither of us knows whether the wiki slowness is caused by crawlers or something else. In contrast, the issue pointed out by Mattson is a real source of a nontrivial amount of unnecessary traffic. So why bother responding dissmissively if you're not even going to put in the effort to understand what the issue is?

It seems like the maintainer is aware that this is... not a best practice, and I see some (unfinished) work today to begin to implement caching:
https://github.com/smasherprog/EqTool/commit/b962a7a8d4d22dbe985d1c00f30ae286a691d333

Goregasmic

03-08-2026, 10:28 PM

When you want to lull from out of LOS you're going to spam consider a fair bit every time to see if faction goes from X to threatening when it lands, that's your cue to bail out if it does.

I don't use pigparse UI but if it pulls up the wiki page every time...

loramin

03-09-2026, 11:14 AM

Pretty sure it's you who's mistaken. I don't think you understand what PigParse is. It's not just a tunnelquest website, it's also a log-reading utility that does DPS parsing and mapping, and I guess loading all items the wiki lists as dropping from a mob every time you /con it.

Here's the ConHandler hitting the the WikiApi:
https://github.com/smasherprog/EqTool/blob/ecec31900d679904235eff849a0ef4637e118920/EQTool/Services/Handlers/ConHandler.cs#L46
Here's the WikiApi directly hitting the wiki:
https://github.com/smasherprog/EqTool/blob/ecec31900d679904235eff849a0ef4637e118920/EQTool/Services/WikiApi.cs#L107

Sure don't see any caching happening.

If it's hitting the p99 wiki every time any player using pigparse /cons a mob, that's a lot more than "one page, even a few times". And sure, I'm aware of the difference between that and an non-rate-limited multithreaded crawler, but unless you have some special insight into wiki traffic from being an administrator, neither of us knows whether the wiki slowness is caused by crawlers or something else. In contrast, the issue pointed out by Mattson is a real source of a nontrivial amount of unnecessary traffic. So why bother responding dissmissively if you're not even going to put in the effort to understand what the issue is?

It seems like the maintainer is aware that this is... not a best practice, and I see some (unfinished) work today to begin to implement caching:
https://github.com/smasherprog/EqTool/commit/b962a7a8d4d22dbe985d1c00f30ae286a691d333

So PigParse is DDOSing our wiki? Awesome! :mad:

Look, I googled and saw an auction site. I can't be responsible for every tool and knowing how they all work ... I just figured no dev would be so stupid (and cruel) as to effectively encourage users to DDOS the wiki!

But, to be fair, I'm still unclear how many users it has, how often they spam the "take down the wiki" button, etc. If it only results in a few requests per user per minute, that's the same as a human wiki user (and not a problem). But if it's making a whole bunch of requests per second, frequently ... the PigParse dev is a true jerk, and Rogean should block the tool from wiki access.

bcbrown

03-09-2026, 05:58 PM

So PigParse is DDOSing our wiki? Awesome! :mad:

Look, I googled and saw an auction site. I can't be responsible for every tool and knowing how they all work ... I just figured no dev would be so stupid (and cruel) as to effectively encourage users to DDOS the wiki!

But, to be fair, I'm still unclear how many users it has, how often they spam the "take down the wiki" button, etc. If it only results in a few requests per user per minute, that's the same as a human wiki user (and not a problem). But if it's making a whole bunch of requests per second, frequently ... the PigParse dev is a true jerk, and Rogean should block the tool from wiki access.

To be clear I'm not trying to start any beef with you. I like and respect you, and you do far more for this community than most other players.

I don't expect you to know every tool and how they work. But I do expect you to do better than dismissively deny the possibility that a tool you don't know about does <a thing>. Besides, I know you've seen the quality of code that other p99 enthusiasts have written - expecting amateurs to meet professional standards isn't very reasonable :)

And for sure, none of us have visibility into why the wiki's so slow, whether this tool is a significant source of traffic, etc. But here's a simple Fermi estimate: 100 average concurrent Pigparse users who /con every 10 seconds on average vs 2000 Wiki users who view 100 pages a day on average.

Also, Rogean can't easily block the tool from wiki access since it's client-side logic, not server-side.

loramin

03-09-2026, 08:36 PM

You've got me: I thought anyone coding a tool that hits the wiki would also understand how requests and servers work ... and with that understanding, wouldn't still write their code in a way that hurts the wiki.

So yeah, I'm a terrible person/poster for assuming other programmers aren't terrible people (and having to assume because I don't know every tool and how it's coded).

As for Rogean not being able to block the tool ... there are ways. Tools have patterns and make requests with those patterns. Again, I still have no idea if PigParse has even contributed, meaningfully, to the wiki issues ... but if it does, I'll personally volunteer to write code to block it from accessing the wiki ... if Rogean can't deal with it himself (I strongly suspect he can).

P.S. Also ... where did I ever rule out the idea that the tool could be responsible? Quote me, because all I see are words like "imagine" and "hope".

I truly don't understand why people on this forum need to attack others for what they imagine they've said. I've actually said plenty of things worth arguing over: pick one of them to start a fight over if you just want to argue with an Internet stranger.

DeathsSilkyMist

03-09-2026, 10:21 PM

I truly don't understand why people on this forum need to attack others for what they imagine they've said.

You do this often. It is indeed strange as to why posters like yourself do this.

For all of the disagreements myself and Bcbrown have had, I don't see him attacking you here. He is simply asking you to listen to others, instead of simply dismissing them out of hand. You did indeed do that to Bcbrown.

bcbrown

03-09-2026, 10:53 PM

DSM, showing up with nothing other than offtopic personal attacks ("You do this often.") fits within my definition of trolling. I appreciate the support but I know you can do better than this.

I don't think you're a terrible person or poster, Loramin. If you thought I was attacking you I apologize. I thought you made a mistake, and the thing about mistakes is we all make them. Let's get back on topic, shall we? :)

Mattson, I wrote up a little tool that you (or anyone else) might try out and see if it could be useful. It's a local webserver that proxies the p99 wiki. So you can browse the wiki just like normal, but every page you visit gets cached locally. That means that if the wiki is down or slow it will fall back to the local cache.

It's only half-finished and still quite rough around the edges but if it proves to be useful I can polish it up.

Instructions
1. Install Python3 on your computer (https://www.python.org/downloads/)
2. Download the attached main.txt file somewhere on your computer - the cached wiki files will be downloaded to this location
3. Rename from main.txt to main.py
4. Open a command prompt or terminal at that location
5. type "python3 -m pip install requests" to install the necessary 'requests' library
6. type "python3 main.py"
7. open a web browser and go to "localhost:8088/Main_Page"

Current Limitations
* Search does not work. You have to navigate to the specific page by URL or links
* The wiki uses redirects (like "Skyshrine_Armor" redirects to "Skyshrine_Armor_Quests" and there's probably some corner cases that won't work correctly
* The wiki is case-sensitive ("Skyshrine_armor" is different from "Skyshrine_Armor") and there's probably some corner cases that won't work correctly
* There's no cache updates right now, so the page you see will always be frozen at the first version cached. This would be the next thing I'd work on if there's interest in this project.
* CSS styling isn't working and I don't know why, so the formatting looks all wonky.

DeathsSilkyMist

03-09-2026, 11:09 PM

DSM, showing up with nothing other than offtopic personal attacks ("You do this often.") fits within my definition of trolling. I appreciate the support but I know you can do better than this.

Loramin brought it up directly (which was off topic). I wouldn't have responded otherwise. Loramin does indeed need to stick to the topic instead of trolling other posters like yourself. Loramin needs to be held to a higher standard since he is a wiki admin. He shouldn't be doing this.

bcbrown

03-09-2026, 11:10 PM

Loramin brought it up directly (which was off topic). I wouldn't have responded otherwise. Loramin does indeed need to stick to the topic instead of trolling other posters like yourself. Loramin needs to be held to a higher standard since he is a wiki admin. He shouldn't be doing this.

Loramin is not trolling anyone, but you are. You can do better.

DeathsSilkyMist

03-09-2026, 11:12 PM

Loramin is not trolling anyone, but you are. You can do better.

I disagree with your assessment, and Loramin's past behavior disproves the claim Loramin doesn't troll. Please stick to the topic, as there is no need to continue this back and forth. Hopefully he will respond to you with a proper post this time.

You've got me: I thought anyone coding a tool that hits the wiki would also understand how requests and servers work ... and with that understanding, wouldn't still write their code in a way that hurts the wiki.

I don't think you should assume this, nor is there a reason to be snarky about it. The principle behind PigParse is simple enough to where someone could vibe-code it with AI. Not everybody frequents the forum. They may not even know that the wiki has issues. Many amateur programmers are not focused on these kinds of secondary effects. They may just be happy that they got PigParse working in some form.

Use the wayback machine if its being slow

http://web.archive.org

This is a good idea!

loramin

03-10-2026, 11:52 AM

This message is hidden because DeathsSilkyMist is on your ignore list.

I've been here since 2013, and I've ignored exactly seven people in that entire time. Just seven: every other bad poster I simply read past. Six were annoying sock puppet accounts, which I got tired of trying to remember who their real poster was ... and one was DSM.

What does it say about you DSM that in over a decade of posting on this forum ... which has had a lot of bad participants ... you are the only one I've ever ignored for just being a terrible person?
I don't think you're a terrible person or poster, Loramin. If you thought I was attacking you I apologize. I thought you made a mistake, and the thing about mistakes is we all make them. Let's get back on topic, shall we? :)

Mattson, I wrote up a little tool that
If you don't have beef, I don't either: we're good :)

Cool proxy server idea, hope it helps people!

DeathsSilkyMist

03-10-2026, 11:59 AM

I've been here since 2013, and I've ignored exactly seven people in that entire time. Just seven: every other bad poster I simply read past. Six were annoying sock puppet accounts ... and one was DSM.

What does it say about you DSM that in over a decade of posting on this forum ... which has had a lot of bad participants ... you are the only one I've ever ignored for just being a terrible person?

You keep saying I am on your ignore list, but you often reply. You have already admitted to reading my messages post ignore list.

It doesn't say anything about me. Your idea that I am a "terrible person" is completely in your imagination. Every lie you tell about me has zero evidence. Every time I ask for evidence, you make an excuse.

I am sorry you are delusional. This is exaclty what you do:

I truly don't understand why people on this forum need to attack others for what they imagine they've said.

You imagine me saying things, and use those imaginary posts to make your conclusions about me. It is easy to show you misreading my posts over and over. The post history is still there.

I have been here since 2014. You are the only wiki admin I have seen actively abuse their power to support users trolling the wiki and other forum posters. You have purposely made the wiki a worse place, and you should be ashamed.

Mattson

03-16-2026, 07:27 PM

Wait the admins are facilitating the DDoS?

DeathsSilkyMist

03-16-2026, 11:53 PM

Wait the admins are facilitating the DDoS?

No. Loramin is not responsible for the DDoSing to my knowledge. Nor is any other admin. I am referring to a different incident.

Essentially a forum user posted a video of themselves soloing Fungi King. A few trolls attacked the poster of this video to diminish his accomplishment. Then they trolled the wiki by changing the solo challenge rankings to purposely give the poster of the video a lower score. This was obviously malicious.

When Loramin saw this thread, he somehow thought the people purposely trolling the wiki by lowering the rankings were editing the wiki in good faith. This is despite the mountain of evidence otherwise. Loramin then supsended anyone who edited that wiki page other than the trolls.

Loramin gave preferential treatment to clear trolls and wiki abusers, and then abused his wiki admin powers to suspend anyone that disagreed with him.

Be wary when dealing with Loramin. He will read your posts incorrectly, and then harass you across the forum. There are many instances of Loramin clearly misreading my posts, and then claiming I said something different. That is why he thinks I am a "terrible person". He has various fantasies about what he thinks I said.

Goregasmic

03-17-2026, 09:15 AM

Ok but can we get a working wiki? plzkthx.

Swish

03-17-2026, 11:18 AM

Use the wayback machine if its being slow

http://web.archive.org

mcoy

03-17-2026, 07:56 PM

We need a new "godwin's rule" for p99 that every thread eventually descends into DSM vs Loramin vs Bcb...

-Mcoy

DeathsSilkyMist

03-17-2026, 08:35 PM

We need a new "godwin's rule" for p99 that every thread eventually descends into DSM vs Loramin vs Bcb...

-Mcoy

The actual "godwin's" rule is every thread eventually gets spammed with nonsense by trolls like Loramin. You are doing your part to prove this rule via this post! The solution is simple, don't let trolls get away with this stuff. They are similar to bullies. When nobody supports them, they can't get what they want, and they stop.

loramin

03-17-2026, 08:47 PM

We need a new "godwin's rule" for p99 that every thread eventually descends into DSM vs Loramin vs Bcb...

-Mcoy

No idea what DSM is going on about in this thread, because I forum ignore him (and said nothing about him to trigger him) ... but maybe if we all put him on ignore, he'll stop bringing up his crush on me in every thread I post in?

https://media0.giphy.com/media/v1.Y2lkPTZjMDliOTUyNmlvdTF6a3M0eDA3dzMxdWp3YW9nY3h veHZ1d2NubHdzM2x3dWd5YyZlcD12MV9naWZzX3NlYXJjaCZjd D1n/KVZZnfAvZZlty258kq/200w.webp

:D Who am I kidding, we all know nothing will stop the mentally ill. But I still highly recommend ignoring him; it's improved my forum experience significantly.

DeathsSilkyMist

03-17-2026, 08:53 PM

No idea what DSM is going on about in this thread, because I forum ignore him (and said nothing about him to trigger him) ... but maybe if we all put him on ignore, he'll stop bringing up his crush on me in every thread I post in?

:D Who am I kidding, we all know nothing will stop the mentally ill. But I still highly recommend ignoring him; it's improved my forum experience significantly.

Loramin keeps claiming I am on ignore, while he specifically mentions me in the threads where I bring up his obvious and proven abuses of his wiki admin powers. How does he know I am talking about him here? He didn't post in the other threads where I don't bring him up. I don't know why he keeps pretending. If he is ignoring me, he simply shouldn't post. But considering he randomly attacked me across multiple threads, Loramins obsession is clear.