Don’t forget your log files
I was originally going to title this post “The Reaon I’m a Terrible Systems Administrator”, but on further reflection I changed it to “The Primary Reason… ” and finally to “One of the Many Reasons…” before I decided to give up and put the title more in context. For now I’ll just thank my lucky stars that my current gig has me doing heavy PL/SQL development instead.
I’ve wasted a lot of hours the last two weeks trying to troubleshoot issues with my network at home. In two specific cases, I could have avoided a lot of frustration and grief if I had only checked the log files immediately, instead of pounding google and attempting a hodge-podge of fixes that didn’t work
The first major issue I had was opening some DMZ pinholes for samba shares between blue (wireless) and green (local network) on my new IPCOP firewall installation. I googled up the proper ports, entered them into the IPCOP interface, and then went to my laptop and tried to mount the drive - no luck. I spent a couple of hours a day for 3 or 4 days (sometimes, that’s all the time you can spare, right) double-checking my subnet settings, looking at the connections screen in IPCOP, searching through the internet, undoing everything and doing it again. I tried two different PCs on the wireless and then was miffed when they both worked when plugged into the LAN. I just couldn’t figure out what was going on. Then, finally, I took a look at the log files and instantly knew what the problem was. When I originally set up Samba, I didn’t have a fancy firewall setup with multiple zones. I set it up to only allowed 192.168.0. and 127. Two minutes later I had added 192.68.1. to the smb.conf file and shares were mounting without issue.
And since I’d set up a new web/mail server for the DMZ a couple of weeks ago, I’d been frustrated because I wasn’t receiving the daily logwatch email. I had put a .qmail file in the vpopmail domain to forward all root mail to myself, but I wasn’t getting anything. This morning I had a couple of hours, so I started digging into it. Again, I spent a lot of time on google, looking for an answer, I changed, double-checked, changed, reverted, and check-again all my settings. I sent about 20 email from the command line to root, and I wasn’t getting anything no matter what I tried. Finally, I went and checked the log file, and the answer was so obvious and right in front of my face I cursed myself. Permissions issue on the .qmail-root file. Again, a simple and easy fix that I would have had instantly had I checked the logs first.
So I’m not really writing this for you… I’m writing it for me. Because when I see this stuff up here, I tend to remember it once and for all. In that vein, I’m going to leave myself a couple more hints. At the scent of trouble, open up another terminal and use tail -f on that log file so that you can see exactly what’s happening when it’s happening. It’s a lot faster and easier than issueing commands and looking at the log alternatively in the same window.