| 02:28.32 | *** join/#fredlug GGD (~GGD@pool-70-109-55-219.clppva.fios.verizon.net) |
| 02:31.09 | GGD | hey stickster_afk |
| 09:06.30 | *** join/#fredlug nmcbride (~nmcbride@12.48.235.5) |
| 13:26.33 | *** join/#fredlug GGD (47ffe2b8@gateway/web/freenode/ip.71.255.226.184) |
| 13:26.43 | GGD | morning stickster |
| 13:27.00 | GGD | jsmith-away: |
| 14:07.42 | GGD | hi jsmith |
| 14:08.25 | jsmith | Morning GGD |
| 14:08.46 | GGD | things are not so fun fun over here at the GGD land |
| 14:08.58 | GGD | i was working with paul yesterday but work got in the way... |
| 14:09.10 | GGD | and yes i hate it when that happens |
| 14:11.38 | GGD | any ideas? |
| 14:20.10 | GGD | i can log in but after that the screen totally freezes |
| 14:21.53 | stickster | GGD: Have you tried creating a brand new user? 'useradd newguy' and then 'passwd newguy' |
| 14:22.08 | stickster | Then see what happens to that user. If he can log in OK, you know it's a local config problem in your account. |
| 14:22.11 | GGD | hold on and i will do that |
| 14:27.12 | GGD | no dice |
| 14:28.21 | GGD | "there is a problem with the configuration server (/usr/libexec/gconf-sanity-check-2 exited with status of 256) with a close button |
| 14:29.01 | GGD | and there was a gnome power manager has not been installed correctly please contact your computer admin |
| 14:29.08 | GGD | error that popped up as well |
| 14:29.35 | stickster | So that happens with the brand new user? |
| 14:29.43 | GGD | yes |
| 14:29.58 | stickster | That seems to imply you've got RPM problems on this system -- a bad schema in a shared location that otherwise should be fine |
| 14:31.09 | stickster | Can you run 'package-cleanup --problems | tee pcreport' and see what's reported? |
| 14:31.12 | GGD | which is odd as i didn't do anything crazy with it |
| 14:31.19 | GGD | ok hold on |
| 14:32.16 | GGD | could i pipe that to fpaste? |
| 14:32.21 | GGD | the outcome? |
| 14:32.42 | stickster | You can fpaste the 'pcreport' file that was just created |
| 14:32.55 | stickster | 'tee' means 'copy the output also to the following filename' |
| 14:33.03 | GGD | ahh |
| 14:33.10 | stickster | It's nifty! |
| 14:33.36 | GGD | "nothing found" |
| 14:33.42 | stickster | OK, that's good then. |
| 14:33.54 | GGD | correcion "no problems found" |
| 14:34.02 | stickster | Do 'rpm -V gnome-power-manager' for the heck of it |
| 14:34.17 | stickster | You should ideally get no output |
| 14:34.51 | GGD | done with the power manager |
| 14:35.30 | stickster | GGD: One other 'simple' suggestion for now |
| 14:35.33 | stickster | As root, run this: |
| 14:35.38 | stickster | touch /.autorelabel |
| 14:35.51 | stickster | Then reboot the system. It will relabel your SELinux contexts system-wide, which may take a while. |
| 14:36.10 | stickster | I'm assuming you're running in SELinux enabled mode. |
| 14:36.24 | stickster | (which is recommended) |
| 14:36.39 | GGD | yes as it throws up a whole bunch of those when i boot on my regular login |
| 14:37.09 | stickster | Hm, that may indeed be one of the problems then. |
| 14:37.25 | GGD | 16 of them to be exact |
| 14:37.29 | stickster | you can see if there are AVC errors by doing 'grep AVC /var/log/audit/audit.log | less' |
| 14:37.51 | stickster | Do the autorelabel shown above, reboot, and see if you're in any better shape then. |
| 14:38.11 | GGD | rebooting now |
| 14:38.41 | GGD | is it normal to get modprobe errors at first boot up? |
| 14:38.50 | stickster | Not usually |
| 14:38.58 | stickster | It depends on the module |
| 14:39.28 | GGD | i always seem to get them when i boot up a fedora machine.. no mater the hardware,. |
| 14:39.47 | GGD | ok we are doing the relabel now |
| 14:40.14 | GGD | and i want to very much appologize for yesterday |
| 14:40.20 | GGD | to your Paul |
| 14:40.37 | GGD | i was trying to get things done here and work got in the way... |
| 14:43.49 | GGD | ok relabel is done |
| 14:44.11 | GGD | still get that gnome power issue error again |
| 14:44.50 | stickster | Does the login work though? |
| 14:45.15 | GGD | the test one or my main log in or either? |
| 14:45.22 | stickster | The new one |
| 14:45.27 | stickster | The test one |
| 14:45.36 | GGD | no same error |
| 14:45.44 | GGD | the config server error |
| 14:45.53 | stickster | Can you click through it and get a session? |
| 14:46.38 | GGD | i get a selinux message box with 19 errors |
| 14:46.44 | GGD | or alerts rather |
| 14:46.47 | stickster | That's fine, dismiss it |
| 14:46.53 | stickster | Do you get a session? panel, menu? |
| 14:47.12 | GGD | get a crash in the setroubleshoot server |
| 14:47.15 | GGD | two of them |
| 14:47.51 | stickster | Again, dismiss all this stuff |
| 14:47.54 | stickster | Do you get a panel? |
| 14:48.02 | GGD | no freezes |
| 14:48.09 | stickster | It totally freezes? |
| 14:48.14 | GGD | yes |
| 14:48.22 | stickster | Hit Ctrl+Alt+F2 -- do you get a text screen login? |
| 14:48.24 | GGD | i have mouse movement and such but nothing else |
| 14:49.26 | GGD | yes i can get to that |
| 14:49.31 | stickster | OK, login as the test user there |
| 14:49.31 | GGD | the text |
| 14:49.45 | stickster | Then 'fpaste ~/.xsession-errors' |
| 14:49.49 | GGD | im in |
| 14:50.35 | GGD | "no text to send" |
| 14:50.49 | stickster | do this: |
| 14:50.54 | stickster | ls -l .x* |
| 14:50.58 | stickster | That's dot, x, star |
| 14:51.05 | stickster | oops |
| 14:51.08 | stickster | ls -ld .x* |
| 14:51.35 | GGD | ok i get two files |
| 14:52.01 | stickster | ls -ld .x* | fpaste |
| 14:52.02 | GGD | .xsession-errors and one that says .xsession-errors.old |
| 14:52.36 | GGD | http://fpaste.org/LJdT/ |
| 14:52.45 | stickster | Wow, interesting |
| 14:53.45 | GGD | is this on the same realm as the cobbler server issue the other day or is this a whole NEW realm? |
| 14:53.48 | GGD | lol |
| 14:53.53 | stickster | No, brand new |
| 14:54.20 | GGD | and btw that lappy got so hot that it shut itself down the other night.... |
| 14:54.52 | stickster | At the terminal run |
| 14:54.57 | stickster | /usr/libexec/gconf-sanity-check-2 |
| 14:55.50 | GGD | y or n on the continue? |
| 14:56.27 | stickster | Does it give you some output? |
| 14:56.44 | stickster | has no idea what this produces since it always completes without output (i.e. no problems found) on his system |
| 14:57.22 | GGD | "the files that contain your perference settings are currently in use |
| 14:57.41 | stickster | That's OK, you can continue then |
| 14:57.41 | GGD | you might be logged in to a session, |
| 14:57.50 | stickster | Oh wait |
| 14:57.51 | stickster | better yet |
| 14:58.24 | stickster | hit Ctrl+Alt+F1 to return to the GUI, logout there (or hit Ctrl+Alt+Bksp to kill the X server), then Ctrl+Alt+F2 to come back here |
| 14:59.10 | GGD | interesting... |
| 14:59.18 | GGD | i did that and re did the command |
| 14:59.46 | GGD | "please contact your sysadmin.,.. to resolve this following problem |
| 15:01.14 | GGD | "could not resolve the address "xml:readwrite:/home/test/ .gconf" in the configuration file "/etc/gconf/2/path": failed: could not make directory '/home/test/.gconf' : no space left on device |
| 15:01.26 | stickster | There's your problem :-) |
| 15:01.29 | stickster | Type 'df -h' |
| 15:01.32 | GGD | however i have a 250 gig hd and i know that its not full |
| 15:01.41 | stickster | Actually... |
| 15:01.43 | stickster | df -h | fpaste |
| 15:01.53 | stickster | keeps forgetting that useful pipe :-) |
| 15:01.57 | GGD | i have a 250 gig ext that it thinks is a system drive ( i think) |
| 15:02.01 | GGD | lol |
| 15:02.34 | GGD | http://fpaste.org/EU0Y/ |
| 15:03.00 | stickster | Yup, you've filled your whole system drive somehow |
| 15:03.15 | GGD | how?? |
| 15:03.23 | stickster | I'm betting the culprit is in /tmp but let's see |
| 15:03.33 | GGD | 300 gig???? |
| 15:03.36 | stickster | run this: |
| 15:03.37 | stickster | du -sm /* | fpaste |
| 15:03.55 | stickster | It will probably take a while. |
| 15:04.24 | GGD | i get a WHOLE bunch of permission denied's |
| 15:04.40 | GGD | http://fpaste.org/WKiU/ |
| 15:05.06 | stickster | GGD: My apologies |
| 15:05.08 | stickster | You need to run that as root |
| 15:05.13 | stickster | logout |
| 15:05.14 | GGD | no prob man |
| 15:05.18 | stickster | Then login as root this time :-) |
| 15:06.17 | GGD | http://fpaste.org/ywKV/ |
| 15:06.45 | stickster | OK, so /var seems to be quite full -- ~169G |
| 15:06.49 | GGD | i got a few could not access errors |
| 15:06.55 | stickster | Do you mirror a bunch of web content there in /var/www? |
| 15:07.10 | GGD | i don't mirror anything |
| 15:07.14 | stickster | OK, run this: |
| 15:07.19 | stickster | du -sm /var/* | fpaste |
| 15:07.49 | stickster | Interestingly, there's a cool tool called 'baobab' that does a graphical version of what we're doing now... if you could login that would be very useful :-D |
| 15:07.55 | GGD | http://fpaste.org/311p/ |
| 15:08.01 | *** part/#fredlug stickster (~pfrields@fedora/stickster) |
| 15:08.13 | *** join/#fredlug stickster (~pfrields@fedora/stickster) |
| 15:08.21 | stickster | Nuts, I hit a wrong key |
| 15:08.27 | stickster | Can you repaste that, GGD? |
| 15:08.29 | GGD | http://fpaste.org/3IIp/ |
| 15:08.43 | stickster | Wow, /var/lib? That's totally bogus |
| 15:08.49 | stickster | Next... |
| 15:08.51 | stickster | du -sm /var/lb/* | fpaste |
| 15:09.00 | stickster | Hopefully you're seeing my pattern here :-) |
| 15:09.34 | GGD | yes i am... :) |
| 15:10.06 | GGD | i take that back.. i think i do some web caching on the machine but not 169gig of it |
| 15:10.13 | stickster | You ran into one of the more interesting problems -- having a full filesystem can cause all sorts of nasty things that are more difficult to track down than the average error |
| 15:10.21 | GGD | http://fpaste.org/q6HQ/ |
| 15:10.34 | stickster | Ah, I see the problem |
| 15:10.36 | GGD | Paul.... |
| 15:10.41 | stickster | You are using a big bunch of disk in /var/lib/libvirt |
| 15:10.47 | stickster | Maybe you have a really large disk image there. |
| 15:11.15 | GGD | i always seem to have the "more interesting problems" LOL |
| 15:11.44 | GGD | yea i have a couple there i am working with... but these are like 10 -20 gig machines |
| 15:12.06 | stickster | GGD: Have you maybe removed some VMs without removing their disk files, even though you maybe meant to? |
| 15:12.19 | GGD | thats a very good possiblity |
| 15:12.55 | GGD | ive had problems with virt in general as well |
| 15:13.14 | stickster | cd /var/lib/libvirt/images |
| 15:13.19 | stickster | Then look at what you see there: |
| 15:13.20 | stickster | ls -l |
| 15:13.39 | stickster | You'll need to figure out which of these images is in use, and which you can get rid of |
| 15:14.21 | GGD | i could dump all of them and start over fresh |
| 15:14.30 | GGD | if i can do that |
| 15:14.48 | stickster | GGD: That's up to you -- but if you do that, keep in mind you'll have XML definitions for those machines in /var/lib/libvirt/qemu/ that will reference disk images that are no longer there |
| 15:15.13 | GGD | ok how do i do that the "clean way"? |
| 15:15.50 | GGD | with out having the xml diffinitions there |
| 15:16.03 | stickster | GGD: Sorry, the XML definitions are in /etc/libvirt/qemu/ |
| 15:16.13 | GGD | ok |
| 15:16.17 | stickster | Well, it should be 'clean' to do this: |
| 15:16.38 | stickster | rm -f /var/lib/libvirt/images/* /etc/libvirt/qemu/*xml |
| 15:16.43 | stickster | service libvirtd reload |
| 15:16.52 | stickster | That will wipe ALL images |
| 15:16.54 | stickster | AND the machines |
| 15:17.00 | stickster | So make sure that's what you want to do! |
| 15:17.07 | stickster | is not responsible for data loss :-) |
| 15:17.15 | GGD | those were test machines anyway... |
| 15:17.23 | stickster | If all you've been doing is playing around, should be OK then |
| 15:17.41 | GGD | yes |
| 15:18.05 | stickster | Once you're done, let me know |
| 15:18.07 | GGD | although... i am having issues with the virt currently as well |
| 15:18.08 | GGD | ok |
| 15:18.10 | GGD | hold on |
| 15:18.19 | stickster | The 'disk full' problem would affect *A LOT* of stuff on your machine |
| 15:18.31 | stickster | Because you're sharing one disk partition for everything (system, virt, user data, etc.) |
| 15:18.45 | stickster | If no process can write more data to the disk, lots of things will start failing |
| 15:19.29 | stickster | Once you're done, I would recommend rebooting so that your message bus and other stuff can restart normally. This is not normally needed in Linux but a disk full error could have affected other things at startup time, so rebooting would be a good thing in this particular case. |
| 15:20.25 | GGD | rebooting now |
| 15:20.41 | stickster | GGD: When it comes up, try logging in as your normal user |
| 15:22.25 | GGD | that was it THANK YOU PAUL!!! |
| 15:22.27 | GGD | :) |
| 15:22.31 | GGD | it came up |
| 15:23.23 | stickster | Bingo! |
| 15:23.56 | GGD | hmm i am looking at the disk usage analyser |
| 15:24.01 | GGD | and it still says 100% |
| 15:24.44 | stickster | That seems odd. Can you open a terminal, and run 'df -h | fpaste' again? |
| 15:26.39 | GGD | http://fpaste.org.org/2BOX/ |
| 15:26.47 | GGD | http://fpaste.org/2BOX/ |
| 15:27.28 | stickster | Nope, looks like you're in fine shape |
| 15:27.37 | GGD | odd |
| 15:27.49 | stickster | You may want to look closer at the disk usage analyzer -- it may be telling you what percentage of a certain area is being used and by what |
| 15:28.31 | stickster | It should say "Total filesystem capacity" at 100% but that's not really usage, just the total of... your total :-) |
| 15:28.36 | stickster | The real usage is in the gray bar above that IIRC |
| 15:28.53 | GGD | ok |
| 15:28.56 | stickster | You can do 'scan filesystem' in the toolbar to get a real readout |
| 15:29.12 | stickster | It will take a while |
| 15:29.31 | stickster | gets back to $DAYJOB now that crisis is over :-) |
| 15:29.48 | GGD | LOL |
| 15:29.58 | GGD | i will bring my lappy the next meeting |
| 15:30.06 | GGD | that will be the fun one.. |
| 15:30.08 | GGD | ;) |
| 15:30.17 | GGD | again thanks a bunch stickster |
| 15:30.27 | stickster | np, enjoy! |
| 15:30.32 | GGD | :) |
| 23:05.56 | *** join/#fredlug theunwiseman (~matthew@pool-71-171-87-156.clppva.fios.verizon.net) |