| 04:26.07 | *** join/#fredlug plarsen (n=plarsen@72-255-9-126.client.stsn.net) |
| 05:05.55 | *** join/#fredlug nombyte (n=nomb@c-71-62-193-105.hsd1.va.comcast.net) |
| 05:37.57 | stickster | Wow, people are on late tonight! |
| 05:53.58 | nombyte | im always up late |
| 06:03.16 | stickster | Not me. |
| 06:03.20 | nombyte | ;d |
| 06:03.24 | stickster | Tomorrow is a day off though |
| 06:03.27 | nombyte | nice |
| 06:03.33 | stickster | But I'm about done with work for tonight |
| 06:03.45 | nombyte | its always nice to relax |
| 06:06.42 | stickster | G'night! |
| 06:30.57 | plarsen | night ;) |
| 12:35.43 | *** join/#fredlug nombyte (n=nomb@c-71-62-193-105.hsd1.va.comcast.net) [NETSPLIT VICTIM] |
| 14:09.45 | *** join/#fredlug plarsen (n=plarsen@208.176.91.226.ptr.us.xo.net) |
| 14:09.58 | plarsen | good morning! |
| 14:13.55 | *** join/#fredlug jsmith_ (i=jsmith@nat/digium/x-1c8f5b4c40d7c9f1) |
| 15:15.34 | stickster | woohoo! |
| 15:39.17 | *** join/#fredlug jsmith (i=jsmith@nat/digium/x-feac43641f867c03) |
| 15:56.09 | plarsen | stickster: well, your suggestions didn't work; sorry :) |
| 15:56.14 | plarsen | But nice try ;) |
| 15:56.56 | stickster | So I'm sure this has somethng to do with the OOM-killer in the 2.6 kernel |
| 15:58.12 | stickster | plarsen: But that PDF link I gave you probably has the answer buried in it somewhere. It sounds like your low memory is being exhausted. |
| 15:58.35 | stickster | plarsen: Do those machines have the "lm" flag in their /proc/cpuinfo? |
| 16:01.00 | plarsen | stickster: well, that's the issue. I don't see a memory exhaustion on the OOM dump. Do you? |
| 16:01.09 | stickster | Nope. |
| 16:01.34 | stickster | plarsen: How about that "lm" flag? |
| 16:01.35 | plarsen | yep - both (dual) cpu's have lom |
| 16:01.37 | plarsen | lm* |
| 16:01.45 | stickster | plarsen: So they could be running 64-bit then. |
| 16:02.04 | plarsen | those are 64 bit processors; however these VMs are set for 32. |
| 16:02.13 | plarsen | The kernel is NOT a 64 bit kernel |
| 16:02.32 | stickster | So these are guests having the problem? |
| 16:02.37 | plarsen | Besides this stuff WORKED for months without a hickup |
| 16:02.42 | plarsen | yes |
| 16:02.51 | plarsen | I think it's a VMware issue |
| 16:02.54 | plarsen | but I'm not sure |
| 16:02.56 | stickster | I'm pretty sure that's the case. |
| 16:03.11 | stickster | This turned up a LOT of googlejuice, so you're not alone. |
| 16:03.25 | stickster | What VMWare Server are you running? |
| 16:03.29 | plarsen | ESX 3 |
| 16:05.59 | stickster | So give the rundown: Host RAM, Guest RAM |
| 16:06.11 | stickster | Host OS, Guest OS |
| 16:06.34 | stickster | plarsen: I think I missed some of those details before. |
| 16:10.40 | plarsen | ESX is not reporting memory exhaustion. Host 13GB, Guest 4GB. |
| 16:10.45 | plarsen | Host OS = ESX |
| 16:10.51 | plarsen | Guest OS = RHES4 |
| 16:11.42 | stickster | I haven't ever run ESX -- it provides its own kernel and the rest of the OS stack on its own? |
| 16:14.07 | plarsen | yup |
| 16:14.11 | stickster | Interesting! |
| 16:14.13 | plarsen | That's the point of it. |
| 16:14.31 | stickster | http://losalamos.redhawk.org/index.php/content/view/65/9/ |
| 16:15.03 | stickster | I know that link is for 64-bit, but you should check it out anyway. |
| 16:16.18 | stickster | http://www.vmware.com/resources/techresources/531 |
| 16:16.57 | plarsen | Interesting ... well, the note itself doesn't mention 64bit; and if you look at the OOM DMA is the only one where the flag all_unreclaimable is set for yes |
| 16:17.32 | plarsen | Last one is way old ;) |
| 16:30.00 | plarsen | hmmmm - looks like it slows down the VMs when you enable the memory gab |
| 16:33.15 | stickster | The one other thing that I saw that might be applicable was resetting the kernel's overcommit policy to "2" or "don't do it when it would exceed the ratio in overcommit_ratio." |
| 16:33.28 | stickster | You might want to back out of any other changes and see how that affects performance. |
| 16:33.59 | plarsen | all the /proc/sys/vm changes have been "cleared" |
| 16:34.11 | stickster | Right-o then |
| 17:32.37 | plarsen | well, it's been holding up for 45 minutes now ;) |
| 17:53.45 | plarsen | Damn - I jinxed it!!! |
| 18:52.46 | *** join/#fredlug jsmith (i=jsmith@nat/digium/x-09adf725cf47c940) |
| 20:23.16 | *** join/#fredlug nombyte (n=nomb@c-71-62-193-105.hsd1.va.comcast.net) |
| 22:23.20 | *** join/#fredlug jsmith (i=jsmith@nat/digium/x-c6dc1ada2a38ae9e) |