Skip to content

Gauging Jabber/XMPP usage in your social network.

Today as I was looking at my ‘All Mail’ folder in my gmail I got to wondering, ‘Gee, I have close to seven years of email here and probably thousands of different domain names. I wonder just how many of them use jabber?”

So I came up with a quick little script that runs through the ‘All Mail’ folder pulling out the domain names from the email address, and checking to see if it has any of the tell tale jabber SRV records.  After which report back to me the results.
Now this script makes a few assumptions…

  1. You’re using gmail (or GAFYD, as I am)
  2. You have IMAP enabled
  3. That the people have setup SRV records for the domain after the ‘@’ symbol pointing to their jabber server (I have found this to be the most common setup)
  4. You have the dnspython package installed

With all that out of the way you can just fire off this script and let it rip, it will take quite a while if you have many thousands of emails as we are just running one thread here although I have added some common sense optimizations such as only checking a domain and email address once.  Also I wrote this in the last 45min or so of my lunch break so it is kind of sloppy and doesn’t really handle exceptions.  It may need to be tweaked a bit to run on IMAP servers other than gmail (change the server address for one), but I didn’t try it on any others than gmail.

So what where the results of my test?  Out of 5,980 (1,034 domains) emails I found 119 people that have jabber setup.  There where a few big names that are using Jabber  such as AT&T + the baby bells, vonage.com and earthlink.net

What where your results?

imapjabber.py

P.S. I have no idea if this runs on Windows boxes.  I assume it does but I don’t own a windows machine to test it out.

Tagged , , , , , ,
2009 04 09 Ryan Parrish Internet
Python
XMPP Comments (1) Permalink

Repairing a motherboard

Here is this and my other websites server, it’s just an old Shuttle SV24 compact computer that I had laying around that with it’s P3 1GHz and 512MB RAM is more than adequate for light web hosting duties.  His name is half-pint.
half-pint 

About a month ago I started having problems with it locking up hard every 15min or so and requiring that I go into the server room to cold boot the machine to get it back up.  I would get no indication in the logs of the freeze, and nothing on the terminal that it was hooked up to, just a frozen server.  These kinds of hard lockups are usually indicative of  something physically wrong such as overheating, bad power supply, bad motherboard, bad CPU, (bad RAM would be a little more random as it hit bad pages so I did not suspect it to begin with) etc…

I ruled out overheating easily as I keep this server in an air conditioned server room and while it was running I could call up mbmon and see that it’s running at 37ºC, so my next guess was the power supply had gone flakey which is not all that uncommon with cheap power supplies.

I opened up the server and connected the ATX header connector to my Fluke 87 and let it log the power outputs so that I could see if it was spiking or having low voltage problems.  No problem there (although I didn’t really test it properly with a dummy load  resistor attached).

Now I had to do the real fun part of taking the mother board out to inspect it, as you can see with a small computer here that stuff is really packed in tight.
img_3699 

After getting everything out so I could remove the MB I was greeted with a nice surprise.
img_3685img_3687

Three popped capacitors (the orange crusty stuff on top of the cluster of three cylinder components).

Now at this point many people would just trash this MB for a replacement, or bite the bullet and rebuild the server on new hardware, but that’s such a waste.  For starters this MB is next to impossible to find for a reasonable price, second electronic waste is becoming a real problem for our planet - did I really want to unnecessarily contribute to it?  And finally, what about my time configuring this machine?  My time is not worthless and I didn’t want to spend another few evenings redoing everything on a fresh box (yes I have backups, but there is still plenty of work to be done) when I could be playing with my kids.

So I did the only logical thing. Replace the capacitors.
The markings on the side of the capacitor will tell you everything you need to know to find a new one, here it what was on mine.

  • 3900µF
  • 10V
  • ±20%
  • 105º
  • LESR

What that all means is - The capacitor is rated at 3900 micro farads, 10V DC (this is a polarized electrolytic capacitor, so no AC here), 20% tolerance which means it can experience a 20% sway (plus or minus) in its capacitance rating through its operating temperature range which maxes out at 105ºC.  Finally there is the LESR which stands for low equivalent series resistance.  I won’t go into what LESR really is and why it’s needed here, as it’s not something I can explain in one quick sentence.  I also took the measurements of the capacitors so I was sure to get ones that where the right fit, they measured 12.50mm x 30.00mm.

The rules for replacing a capacitor (even on a MB) are not as strict as you might think.  You can always replace the broken units with ones that have a higher voltage rating, and you can generally replace the unit with one that has a bit higher capacitance rating also.  But you should always try to get ones that match right up.

Next I head on over to Digi-Key and do a search for specs I laid out earlier and was able to get a set of four (I like to order an extra on just incase one breaks) for $3.92, just $6.37 with S/H - way cheaper than buying a new MB.

(fast forward one week to when the parts arrive)

Now with the new capacitors in had it was time to desolder the old ones.  Here is a nice tutorial on desoldering.
img_3689 

Put the new caps in.
img_3693

Solder and clip the excess leads.
img_3694

Put everything back together and power it all up.
img_3698 
It works! (notice the barely visible green light)

 

So there you have it.  45 minutes and $6.37 later I have a working server again.

Tagged , , ,
2009 01 07 Ryan Parrish How-To
server admin Comments (1) Permalink

PacketLife cheat sheets

Saw these networking related cheet sheets via the SANS blog today and thought I would throw up a post about them as these are just too good to pass up (there are a couple that will be going up in my server room, that’s for sure).

Anyone know of any other good cheat sheets? I have one for vim that a reference from time to time, and nice fold up sheet for bash that I got at a Linux world expo about ten years ago.

2008 11 17 Ryan Parrish Internet
server admin Comments (0) Permalink

Setting up a serial console with DRAC 4/I and Ubuntu 8.04 (Hardy)

Previously I had always setup console redirection on my Ubuntu servers using this (PDF link) guide, which worked great up until Ubuntu switched to Upstart and there was no more /etc/inittab file.  So for my own, and anyone else’s future reference, here is what you need to do…

First follow the linked guide on how to enable serial redirection in the BIOS and on the DRAC card, these have not changed.  
Next add the following anywhere above the ### BEGIN AUTOMAGIC KERNELS LIST line in /boot/grub/menu.lst


#DRAC serial access
serial --unit=0 --speed=57600
terminal --timeout=10 serial

Now find the line that starts with # defoptions=, in my vanilla install it is on line 93 and by default says # defoptions=quiet splash.  Append to the end of the line console=ttyS0,57600 so it now looks like…

# defoptions=quiet splash console=ttyS0,57600

Make absolutely sure you leave the line commented out, that is how it is supposed to be.  The reason we do it this way rather than appending directly to the kernel options is so we don’t have to add this line onto every new kernel we install, grub will do it automatically.

No onto the big difference with Upstart.  Create a file /etc/event.d/ttyS0 with the following…

# ttyS0 - getty
#
# This service maintains a getty on ttyS0 from the point the system is
# started until it is shut down again.

start on runlevel 2
start on runlevel 3
start on runlevel 4
start on runlevel 5

stop on runlevel 0
stop on runlevel 1
stop on runlevel 6

respawn

exec /sbin/getty -h -L 57600 ttyS0 vt100

Now double check that “ttyS0″ is listed in the file /etc/securetty, it was already there on my install towards the top.

Now reboot your server and SSH into your DRAC’s IP and issue the command connect -h com2 and you should see the login prompt waiting for you just as it will at 3AM when your pager goes off and you cannot connect to the server by normal means. I assume this will be the same for DRAC 5 cards also, but I don’t have one to test so YMMV.

Tagged , , , , ,
2008 10 23 Ryan Parrish Ubuntu
server admin Comments (0) Permalink

kerberos permission denied in OS 10.5

Experienced a weird problem with a fairly new OS 10.5 machine today.  My boss had spent the past couple weeks configuring an install of 10.5 on an external FW drive (keeping the 10.4 install on the local drive so as not to disrupt work for days on end while configuring / installing apps) and the time came for me to finaly make it live on his local drive.  So I started up his machine in target disk mode and plugged it into my computer, along with the external FW drive, fired up Carbon Copy Cloner and proceeded to clone the FW drive over his local drive.  It should be noted at this point that everything in 10.5 was working just fine while booted on the external drive up to this point.

Upon rebooting the laptop with its new fresh clone things appeared to be working but I soon discovered (upon starting Mail.app) that the machine was failing to get a kerberos ticket which was normally issued upon login.  I fired up the CLI and tried getting a ticket manually with kinit and got a strange permission denied, so I then tried the GUI Kerberos.app and received a similar permission denied message.  As I tinkered with it some more trying to get a ticket I eventually got one, at least I thought I did, I could see a ticket with klist and it showed up in Kerberos.app but the ticket didn’t work for any services.  Next I tried to destroy the ticket with kdestroy to see if I could get a ticket again, but that too failed with a ‘Permission denied’ error - now things are getting really weird.

Next up came some heavy googling, reading about Leopards new local KDC, and trying kinit in other accounts on the computer to verify the clone was good (they all worked).  

Out of options I figured I would try doing an strace on the failing programs only to learn that OS X doesn’t have strace, so some more googling I learn that OS X has ktrace, pulling my hair out trying to figure out why I can’t find ktrace I learn that ktrace got replaced with dtrace and dtruss (which is simlar to (s|k)trace) in leopard.  So now on the right path with dtruss I did a trace on (or is that a ‘truss’ ??) on kinit and found that the last files and directories it was stat’ing was in ‘/var/folders’, a directory that I have never heard of.  Doing some more searching around it appears this is some kind of cache folder for OS X and the permissions differed from what was on my leopard machine.  My computer had /var/folders with the directories underneath it being root:root, 755 - however my boss’s computer had some other groups.

So I just went for it and rm -rf’ed the directory that dtruss revealed while logged in as another local user, after all I did have a backup of the 10.4 install and the 10.5 on the FW drive.

Low and behold, upon logging in with my boss’s account after the cleanup the computer was able to get it’s tickets properly and authorize to all the services that I couldn’t before.

What and why this happened, I just don’t know - that just may be a little more than I know about OS X’s BSD sub-system.  I can only hope this post may find someone out there that was in the same situation as I was and help them to a speedy recovery.

Tagged , , , , , , , , ,
2008 08 12 Ryan Parrish OS X
server admin Comments (0) Permalink

Version 0.3 of servermonitor.py, now with 100% more web 2.0′ness!

Here is version 0.3 of servermonitor.py

Added in this version is if a service has a history of going up and down there will be a toggle link to display the history of the services states.  All the HTML that is loaded is pre-rendered and the js lib only does a GET and append to the DOM (there is no JSON or real AJAX going on here), so there is still no additional requirement of any kind of dynamic web programing language such as PHP or mod_perl to query a DB and generate the page dynamically.  Also, to keep with the one file philosophy I have used the Google javascript CDN to load jquery rather than distributing the library in a separate file.

 

Here is a sample of the HTML output http://black.kracknet.net/~stickystyle/servermonitor/

Tagged , , , , ,
2008 06 30 Ryan Parrish Python
server admin
servermonitor.py Comments (0) Permalink

XMPP Z-Machine

Since prakbot that I mentioned a while back may be dead (I can’t get to it anymore) I have found another Z-Machine gateway here.  He has even put in a few features to make playing on a blackberry easier, such as being able to set the screen width to a custom value (e.g. 68 to fit JiveTalk).  So check it out if your looking for a time waster on your BB.

Tagged , , , ,
2008 06 30 Ryan Parrish Links
XMPP Comments (0) Permalink

Version 0.2 of servermonitor.py

Well that didn’t take long for me up get around and update the script to include a method to update the hash of a ConnectAndVerify service :-)

Next step will be to look at the code I wrote and possibly refractor a few bits just to make sure I’m not duplicating code anywhere or doing things in a manner that is not as efficient as it could be, since really most of this script was written on a saturday afternoon while I was board.

see http://www.subjectivereality.org/category/servermonitorpy for the most recent version

Tagged , ,
2008 06 21 Ryan Parrish Python
server admin
servermonitor.py Comments (0) Permalink

Simple server monitoring with servermonitor.py

The problem with most server monitoring systems is they run from inside of your network, as such if there is a major outage past office hours where their monitor server is not capable of sending an alert email out (be it the mail server is down, or internet) you may not get the notification at the time of the outage.  

There are ideal solutions to this such as signing up for a paging service so your servers have some kind of Out-Of-Band alerting or paid external monitoring but for very small shops that may not have the extra equipment or funds to install such hardware or pay for a service and the situation of having an externaly facing server go down is still a big deal I offer this script.

It’s purpose is to use one of the many free online shell accounts to provide you with additonal external monitoring to you already existing internal system for no cost.  It would also work great for all the home server folks out there that may have one box hosting there domain and want a simple monitor to know if there site, internet, or mail service has stopped working.

Here is an example of the status page the script generates script running at on of my free shells http://black.kracknet.net/~stickystyle/servermonitor

So here is v 0.1 of my script whose goal is to rely on nothing standard built-in python modules, and keep everything in one file.  It has an MIT license so everyone should be able to use it without worry of “GPL contamination”, also if you do use it, drop me a comment let me know how it works for you, that may inspire me to put some more work into it :-)  (patches are ALWAYS welcome)

See http://www.subjectivereality.org/category/servermonitorpy/ for the most recent version

TODO: Need to add a function to update the hash on ConnectAndVerify, right now you need to delete the service and re-add it if your banner or web page changes.

Tagged , , ,
2008 06 08 Ryan Parrish Python
XMPP
server admin
servermonitor.py Comments (0) Permalink

Free shell accounts.

Been hanging with the guys at kracknet.net recently and they are a pretty good group of guys, so good in fact that they are offering free shell accounts with very liberal permissions on what you can do (basically no IRC bouncers / psybnc).

To get a free account ssh on over, with the username and password of new/new…
$ssh -l new shell.kracknet.net

kracknet.net

Update: kracknet has been dead since around Aug 2008, ignore this post.

2008 05 30 Ryan Parrish Uncategorized Comments (0) Permalink