Gregory Szorc's blog

ModSecurity Woes

This summer, I installed ModSecurity on my server to reduce spam on opensource.case.edu. It seemed to work like a charm, reducing spam in its tracks. However, not all was well. Shortly after installing ModSecurity, my machine started locking up. I could still ping it all right, but nothing was responding. I figured the machine was overheating and dumping core. After it cooled down a bit, and the problem persisted, I removed ModSecurity and everything starting working again.

Now that I am back in Cleveland, where it is much easier to reset my computer if something goes wrong, I figured I'd give ModSecurity another shot. I compiled it again yesterday, and once again, spam was stopped instantly. However, late last night, the problem of locking started to occur. I was using my computer, and suddenly it became unusable. My mouse wouldn't even move! I had to perform a hard reset to correct matters. After booting up, the machine lasted for about 10 minutes, and the same thing happened. Hard reset again. Next, I decided to try to isolate the problem. I opened up a bunch of terminals and started tail'ing logs and viewing top, ps, etc. After 15 minutes, nothing. I went to quickly brush my teeth, and when I got back, the machine was unresponsive. Luckily, top was running and I could see that the machine had exhausted its virtual memory pool and there were 60 active processes! There were a few instances of Apache in the list, eaching consuming about 200MB of virtual memory, which is up from the 50MB or so it usually uses.

So, I disabled ModSecurity and now everything is working. There is obviously a memory leak or stack overflow somewhere in ModSecurity. Unfortunately, I can't seem to isolate it. I think I'm going to try upgrading to Apache 2.2 and see what happens. For the record, I'm running Apache 2.0.59 and ModSecurity 1.9.4 on a Gentoo system.

September 26 Update

Ivan, a ModSecurity developer stumbled upon this entry and sent me an e-mail asking for more info. Apparently, ModSecurity has issues processing extremely large rulesets, such as the ones at gotroot.com, which I was using. He recommends setting a few hundred rules max and also limiting Apache's CPU and memory usage at the OS level. He also stated that in the near future, ModSecurity will be faster and will support RBL-checks, which should help correct the problem. Until then, I guess I'll have to reduce my ruleset. Here goes nothing.

You can ping this entry by using http://blog.case.edu/gps10/mt-tb.cgi/9403 .

mlgn ufirhq pejhidgqx ihautnor sfchmvx ovjsyqae ltwnpq

tara-elizabeth-conner