Mitigating the DDOS attacks. Possibilities inside.

[Krazed]

Found this today. Haven't thought too muh into it (still half asleep).

Could this possibly work for us? The only issues I see would be possible delays in response time to/from server and client.

Again, haven't thought too much into it. Just trying to help as I see how this works well from a website mitigation perspective.

http://arstechnica.com/security/2013/03/how-whitehats-stopped-the-ddos-attack-that-knocked-spamhaus-offline/

http://en.m.wikipedia.org/wiki/Anycast

[thryme]

Sounds expensive to employ any company to help with this.

[Swish]

DDoS'er will run out of funds at some point... funny how people will burn money on DDoS'ing services. I wonder if he's tweeting live from mom's basement?

[khanable]

Why does everyone assume the person behind the ddos is paying for it?

Bad assumption

[Pringles]

Why does everyone assume the person behind the ddos is paying for it?

Bad assumption

[thryme]

Why does everyone assume the person behind the ddos is paying for it?

Bad assumption

I agree - I was just thinking hiring a company to combat it for us would seem to be costly.

[Glorindale]

Do we know that p1999 is even the target? A DDOS attack could cause congestion on a network segment that just happens to be shared with the true target which effectively denies service to EVERYONE who shares that segment. If it is truely targeted at p1999 servers many times these attacks are sourced from a single (or maybe a few regions) such as China. You could mitigate the effectiveness of the attack by blocking traffic to IP ranges used in these regions. In fact if the server is hosted somewhere I'm sure the staff of the facility would love to help push these DOS traffic out and would love to help.

[Clark]

We know it's DDOS and duping because 8 AONs showed up for sale

[Glorindale]

Why does everyone assume the person behind the ddos is paying for it?

Bad assumption

I wouldn't assume that someone is paying for it. Nor would I assume these is some uber hacker who has compromised thousands of computers who would use them to DDOS a free EQ server responsible for the attack.

[Glorindale]

We know it's DDOS and duping because 8 AONs showed up for sale

Ah...so we know that the server has to actually be rebooted to restore service? Ok...that makes sense then.

If the majority of the attacking computer are in North America it becomes much harder since that is where the majority of players are. However, if it is outside of North America I saw screw them and just start blocking IP rangers.

[Volibear]

Do we know that p1999 is even the target? A DDOS attack could cause congestion on a network segment that just happens to be shared with the true target which effectively denies service to EVERYONE who shares that segment. If it is truely targeted at p1999 servers many times these attacks are sourced from a single (or maybe a few regions) such as China. You could mitigate the effectiveness of the attack by blocking traffic to IP ranges used in these regions. In fact if the server is hosted somewhere I'm sure the staff of the facility would love to help push these DOS traffic out and would love to help.

They are sending everquest packets

[Swish]

If the majority of the attacking computer are in North America it becomes much harder since that is where the majority of players are. However, if it is outside of North America I saw screw them and just start blocking IP rangers.

From the Euro/Asia/etc community...

http://i.imgur.com/FZKVxl5.gif

[EvaShogouki01]

Can anyone give a number range for the costs of reinforcing the server rather than just saying "expensive?"

[Agatha]

Anyone who runs a botnet makes sure they have all corners of the world covered, i know somone who used to run a 3mil+ botnet and he would watch his countries wake up. he'd be like oh look europe just woke up, oh look asia just woke up. was kinda neat. but yea, in the emu community its very niche, you arn't dealing with your normal script kiddy. These are adults who run for profit botnet companies most likely tied in with the undernet.

What im getting at is its not going to be possible to get away from this in the conventional means.

[Adolphus]

Can anyone give a number range for the costs of reinforcing the server rather than just saying "expensive?"

+1

I like how everyone assumes that of the thousands and thousands of people who play P1999, none of us will be a rich guy willing to help out.

Actual information on the exact equipment necessary including cost may lead to surprising results for the server.

[Turp_SmokinPurp]

Would OP idea work? We do not need to hire them forever, if can get a few months or few attacks worth of coverage (is this possible) it might kill the DDOS culprit.
This is how i pic the graph you sent, is this how it works?
Anycast methodologies may be exploited to distribute DDoS attacks and reduce their effectiveness: As traffic is routed to the closest node, a process over which the attacker has no control, the DDoS traffic flow will be distributed amongst the closest nodes. Thus, not all nodes might be affected and no lag occurs.

http://i1111.photobucket.com/albums/h468/Turp420/800px-Anycastsvg.png (http://s1111.photobucket.com/user/Turp420/media/800px-Anycastsvg.png.html)

http://arstechnica.com/security/2013/03/how-whitehats-stopped-the-ddos-attack-that-knocked-spamhaus-offline/

[Gaffin Deeppockets]

The more you morans talk about it the more its gonna happen.

[Glorindale]

Anyone who runs a botnet makes sure they have all corners of the world covered, i know somone who used to run a 3mil+ botnet and he would watch his countries wake up. he'd be like oh look europe just woke up, oh look asia just woke up. was kinda neat. but yea, in the emu community its very niche, you arn't dealing with your normal script kiddy. These are adults who run for profit botnet companies most likely tied in with the undernet.

What im getting at is its not going to be possible to get away from this in the conventional means.

That all depends on how sophisticated it is. I just find it hard to believe someone who has access to a large and widely dispersed botnet would bother using it to attack little ole project1999 for weeks on end.

[Glorindale]

The more you morans talk about it the more its gonna happen.

What the hell are morans and do they have anything to do with IP rangers? ;-)

So it will just go away if we stop talking about it? Wow..why didn't think of that?

[kingsBlend]

What I don't understand is.. Rogean, you are the man. You know your shit when it comes to servers and networking, you proved it to us. How do you not know just a little bit on Network Security?

[Splorf22]

Anycast will not work for P1999 because we cannot replicate our service across multiple data centers (that would result in 10 copies of the server). Turp the problem with your diagram is that somehow your router is magically detecting which traffic is 'bad' and sending it elsewhere, and unfortunately that is not possible.

I think Rogean could actually do a lot more against these attacks then he has, probably because he has a job and such. Some interesting things:

Why not keep a list of IPs that send each packet and crossreference against the list of IPs that are logged in? It would not surprise me at all if this is one guy, or one IP is sending a hugely different set of packets than anyone else. Probably they have only found one vulnerability and are just hammering on it, so you should see 1 IP with 95% "requesttrackinginfo" packets or something when no one else has more than 20%.
Detecting a DOS attack should be fairly easy (just have a ping process or monitor cpu load or whatnot) and at that point enable profiling to see which part of the code they are attacking (if they are overloading the CPU, not the network)
Search the logs for AON transactions and try to trace them back to their source to find which ones were duped and which characters acquired the duped ones.


The corollary to all of this is that I'm making the assumption they are sending Everquest packets because they have found some vulnerability in the server code. If they are just flooding the datacenter with DNS packets or whatnot, there is nothing Rogean can do other than pay for more bandwidth.

[Rogean]

How do you not know just a little bit on Network Security?

It's not a lack of knowledge. It's a lack of time and resources.

I have a full time job that gets increasingly busy in the summer. I have commitments all this weekend. I have a trip coming up that I leave for very soon that will put me away for a week. The timing of all of this shit happening is the worst it could possibly be.

Look up DNS Amplification attack, and you guys will see just how little there is that I can do about it myself. No amount of equipment I put on my side of our data center drop will help line saturation. It's up to our data center. I'm seeing what they are willing to do, as well as their upstream providers (Level3).

We used to have DDoS protection. It's one of the reasons that we moved to the data center we're at now. But then they decomissioned the device and decided to not replace it, so now we're stuck in the data center without mitigation. If there's nothing they can do to stop this then we're looking at literally a month or two for us to find and move to a data center that can.

[captincrust]

As far as I am know, this is the most common and effective response to combating DDoS attacks. Often your internet service provider will do this sort of stuff as well since it clogs up their network to a degree. I think this is the best option.

There may be something to be done with the login server - EQEmu has been getting pummeled simultaneously and I suspect there is some bug being exploited with the login server. Various eqemu cheat sites (ie: RedGuides) have alluded to this very recently.

[Glorindale]

It's not a lack of knowledge. It's a lack of time and resources.

I have a full time job that gets increasingly busy in the summer. I have commitments all this weekend. I have a trip coming up that I leave for very soon that will put me away for a week. The timing of all of this shit happening is the worst it could possibly be.

Look up DNS Amplification attack, and you guys will see just how little there is that I can do about it myself. No amount of equipment I put on my side of our data center drop will help line saturation. It's up to our data center. I'm seeing what they are willing to do, as well as their upstream providers (Level3).

We used to have DDoS protection. It's one of the reasons that we moved to the data center we're at now. But then they decomissioned the device and decided to not replace it, so now we're stuck in the data center without mitigation. If there's nothing they can do to stop this then we're looking at literally a month or two for us to find and move to a data center that can.

Ah. Now all of you conspiracy theorist can put their "duping" conspiracies to rest. This attack isn't exploiting the game. It is exploiting the TCP/IP stack.

Rogean, it sounds like this attack is affecting other hosted customers at your ISP? If so maybe they will actually do something about it.

[kingsBlend]

I'm sure there are plenty of network security specialists here, which given the right information, could easily put a stop to it.

[Rogean]

I'm sure there are plenty of network security specialists here, which given the right information, could easily put a stop to it.

Again, it's not a lack of information. We already know how to stop it. It's up to the data center if they want to help or not.

And no, other customers are not getting affected. The attack would need to be over 10 GBit for that to occur.

[Agatha]

I'm sure there are plenty of network security specialists here, which given the right information, could easily put a stop to it.

As a network security specialist myself. I can't recommend just offering anyone with their hands up saying they are a network security specialist and just offering their services a position. As i would tell my client that is not very secure thing to do.

On the other hand, I am a computer security specialist that has worked in many job sectors, defense and private alike. Get at me Rogain i can help you out.

Edit: take an ip address that is sending verified ddos attack, gain root, recover the bot from that computer,debug,see where it connects, join as zombie, see what login commands owner is using, use them to gain control to his net and add it to mine, i mean destroy it.

[Glorindale]

I'm sure there are plenty of network security specialists here, which given the right information, could easily put a stop to it.

If it is line saturation like Rogean says there isn't anything he can do about it since that is the ISP's equipment. It isn't just about knowledge. He can ask them to do something about it but it is ultimately up to them which course of action to take. If this is affecting their other customers as well they might be willing to do something. Or they might have to put pressure on their upstream provider. However, Rogean cannot force them to do anything and certainly has no influence with his ISP's provider.

My experience with problems like this is they are not really worried about it unless you are a large customer or if it affects many customers. However, it sounds like it is only affecting project1999's link to the ISPs network.

Even if Rogean installed a tricked out firewall with IPS capabilities it wouldn't matter cause the attacker is sending so much traffic it is saturating his pipe. If Rogean paid for a bigger pipe it would likely get saturated, it would just take that many more DNS responses to do it.

[Rogean]

It is exploiting the TCP/IP stack.

DNS is UDP Traffic, not TCP.

[SamwiseRed]

Rogean, any chance of throwing up a new temporary red for us to play on. It would be cool to see how well it does. Just a thought, no idea if it would be possible to setup a server in less than an hour or so but it would be pretty fun. Fresh pvp servers are the best.

[Nimblefork]

DDoS'er will run out of funds at some point... funny how people will burn money on DDoS'ing services. I wonder if he's tweeting live from mom's basement?

You honestly think you need to have "funds" in order to hack around 20 computers and send requests through them? No funds required dude.

[Splorf22]

Rogean, is this actually a bandwidth attack?

Also if you ninja patched in some reset code I will hug you.

[Glorindale]

DNS is UDP Traffic, not TCP.

UDP is part of the TCP/IP stack. Sorry to play gotcha but you started it.

[Agatha]

my method of fixing is probably the only way to go about this in a timely manner. the only other way is to call the FBI, and they take years if you arn't CIA.gov

[Champion_Standing]

my method of fixing is probably the only way to go about this in a timely manner. the only other way is to call the FBI, and they take years if you arn't CIA.gov

I'm on the phone with the Internet Police right now, they are on the case!

[Agatha]

I'm on the phone with the Internet Police right now, they are on the case!

You know there is actually internet police, but they dont handle small things currently. They are called cyber command and they are out of FT meade, i have worked with them on occasion and they are highly effective yet too late to be implemented and are catching up fairly quick in relation to china's internet security services.

Right now they are mostly focused on national security issues but they are trying to expand to things like this down the road.

[Glorindale]

Right now they are mostly focused on national security issues but they are trying to expand to things like this down the road.

That is sort of scary actually. It should give one pause anytime the government wants to "expand" anything.

[Akkadius]

UDP is part of the TCP/IP stack. Sorry to play gotcha but you started it.

It uses both, primarily UDP

[Nirgun]

There are 24 denial of service vulnerabilities in the version of PHP that rogean is using.

Update PHP, DDOS gone.

[Glorindale]

There are 24 denial of service vulnerabilities in the version of PHP that rogean is using.

Update PHP, DDOS gone.

You clearly don't know what you are talking about.

[Nirgun]

You clearly don't know what you are talking about.

Educate yourself dumbass.

http://dalnoth.rogean.com/test/php/test.php

http://www.cvedetails.com/vulnerability-list/vendor_id-74/product_id-128/version_id-90935/PHP-PHP-5.2.13.html

[Agatha]

Educate yourself dumbass.

http://dalnoth.rogean.com/test/php/test.php

http://www.cvedetails.com/vulnerability-list/vendor_id-74/product_id-128/version_id-90935/PHP-PHP-5.2.13.html

those are really just the public vulns that people have been using to gain admin on this forum.

[Nirgun]

All users of PHP 5.2 are encouraged to upgrade to PHP 5.3.

[Glorindale]

Educate yourself dumbass.

http://dalnoth.rogean.com/test/php/test.php

http://www.cvedetails.com/vulnerability-list/vendor_id-74/product_id-128/version_id-90935/PHP-PHP-5.2.13.html

Dumbass? Try reading the thread before claiming to have answers. The attack is DNS Amplification Attack jack ass. It has nothing to do with the language this forum happens to be written in.

[Nirgun]

Dumbass? Try reading the thread before claiming to have answers. The attack is DNS Amplification Attack jack ass. It has nothing to do with the language this forum happens to be written in.

HAHAHAHHAHAHAHAHAHAHHAHAHAHAHHA

[Agatha]

http://youtu.be/O2rGTXHvPCQ

the answers you guys are looking for.

[Tiggles]

http://youtu.be/O2rGTXHvPCQ

the answers you guys are looking for.

That hurt

[Joyelle]

"Luckily, I speak leet"

lol...

[Splorf22]

The attack is DNS Amplification Attack

Where did we decide this? Yes, that's what happened to Spamhaus. But I have yet to see Rogean claim that is what's happen to Project 1999. Maybe I just missed it.

[Glorindale]

It's not a lack of knowledge. It's a lack of time and resources.

I have a full time job that gets increasingly busy in the summer. I have commitments all this weekend. I have a trip coming up that I leave for very soon that will put me away for a week. The timing of all of this shit happening is the worst it could possibly be.

Look up DNS Amplification attack, and you guys will see just how little there is that I can do about it myself. No amount of equipment I put on my side of our data center drop will help line saturation. It's up to our data center. I'm seeing what they are willing to do, as well as their upstream providers (Level3).

We used to have DDoS protection. It's one of the reasons that we moved to the data center we're at now. But then they decomissioned the device and decided to not replace it, so now we're stuck in the data center without mitigation. If there's nothing they can do to stop this then we're looking at literally a month or two for us to find and move to a data center that can.

[Seary]

Cast is going HARD on the ddos attack this weekend, never should have doubted his persistence.

[quido]

Hardware-based whitelist!

[Nuggie]

Good read. Not sure which tech geek to believe.

That sucks they shut down whatever was protecting us before.

Keep up the good fight.

[Sadre Spinegnawer]

Hard to follow what you guys are saying, but luckily I speak leet

[Turp_SmokinPurp]

It's not a lack of knowledge. It's a lack of time and resources.

I have a full time job that gets increasingly busy in the summer. I have commitments all this weekend. I have a trip coming up that I leave for very soon that will put me away for a week. The timing of all of this shit happening is the worst it could possibly be.

Look up DNS Amplification attack, and you guys will see just how little there is that I can do about it myself. No amount of equipment I put on my side of our data center drop will help line saturation. It's up to our data center. I'm seeing what they are willing to do, as well as their upstream providers (Level3).

We used to have DDoS protection. It's one of the reasons that we moved to the data center we're at now. But then they decomissioned the device and decided to not replace it, so now we're stuck in the data center without mitigation. If there's nothing they can do to stop this then we're looking at literally a month or two for us to find and move to a data center that can.

Again, it's not a lack of information. We already know how to stop it. It's up to the data center if they want to help or not.

And no, other customers are not getting affected. The attack would need to be over 10 GBit for that to occur.

Thanks for the info.

So hopefully the data center helps or if not maybe we can get the equipment they failed to replace, if not move it!

Only 10g, hopefully can fix it. The DDOS culprit is not paying for that weak shit its probably a home setup. Track him down!

[Pringles]

It's not a lack of knowledge. It's a lack of time and resources.

I have a full time job that gets increasingly busy in the summer. I have commitments all this weekend. I have a trip coming up that I leave for very soon that will put me away for a week. The timing of all of this shit happening is the worst it could possibly be.

Look up DNS Amplification attack, and you guys will see just how little there is that I can do about it myself. No amount of equipment I put on my side of our data center drop will help line saturation. It's up to our data center. I'm seeing what they are willing to do, as well as their upstream providers (Level3).

We used to have DDoS protection. It's one of the reasons that we moved to the data center we're at now. But then they decomissioned the device and decided to not replace it, so now we're stuck in the data center without mitigation. If there's nothing they can do to stop this then we're looking at literally a month or two for us to find and move to a data center that can.



I am just speculating here since I dont know the scope of the attack, only what you noted about DNS amplification attack, but what about firewalling all DNS related traffic on the p99 boxen, and have us to use our own DNS resolution for the server (windows hosts file). Would that at all help? I wouldnt mind making host entries to resolve p99 DNS so that you can shut it off.

[Glorindale]

I am just speculating here since I dont know the scope of the attack, only what you noted about DNS amplification attack, but what about firewalling all DNS related traffic on the p99 boxen, and have us to use our own DNS resolution for the server (windows hosts file). Would that at all help? I wouldnt mind making host entries to resolve p99 DNS so that you can shut it off.

I think the only thing that would mitigate the problem is a devices that sits on the ISP's side of Rogean's drop (or somewhere in the path of their connection to the rest of the world). That device would need to be able to track DNS name resolution requests so that when the name resolution responses are returned it could then match them up with the requests and block any responses that don't have matching requests (thus blocking the responses to the spoofed requests). Unfortunately doing that on Rogean's side of the drop wouldn't prevent his drop from being saturated which is what he described was the problem.

I think if his ISP isn't willing to help he has no choice but to move to one that would be willing to help if this happens again.

Boiled down....Rogean really cannot do anything himself to prevent this.

[DoucLangur]

However, if it is outside of North America I saw screw them and just start blocking IP rangers.

True American dumbass speaking... You're a disgrace to the decent people in the USA.

[Glorindale]

True American dumbass speaking... You're a disgrace to the decent people in the USA.

Ok...I can live with that.