January 04, 2008

Case IM Roster Tool

I've written a little tool to help migrate your Case IM roster to the XMPP server in Case Google Apps.

Case IM Roster Tool.

It takes your Case IM roster (or "buddylist", if you prefer) and displays it as a comma separated list of Jabber IDs (with the "@im.case.edu"s replaced by "@case.edu" for Google Apps. You can then copy-and-paste that list into the "Add Contacts" window in Google Apps Gmail.

Unfortunately, there's no super-smooth way to get at the roster data short of reverse-engineering the database. So for now, it's based on a dump of all the roster data I made on November 28th. I'll probably update that next week (it's a time consuming process, because the export actually checks against every user in LDAP).

Also note that Case IM and Google Apps can interoperate with each other - Google Apps has the necessary SRV records to do federated XMPP. It could get confusing to figure out who's using which, though.

Posted by sdh7 at 02:04 PM | Comments (0) | TrackBack

September 10, 2007

Case IM now supports Google Talk

I upgraded the Openfire IM Gateway plugin today. The biggest new feature in the upgrade is support for using Google Talk via the Gateway. We've supported connections to Google Talk via XMPP federation for a while, but now if you want to chat with your Google Talk buddies using your Google account, but within Spark, you can now!

You'll need to download Spark 2.5.6 to support this - it's not on the Software Center yet, but you can download the Case-skinned version here.

There are also some other new gateway features (support for SIP/SIMPLE, other XMPP servers, hooking into an IRC server for MUC channels), but we don't have any plans to use them yet.

Posted by sdh7 at 05:47 PM | Comments (0) | TrackBack

August 20, 2007

Case IM is now Twitter compatible

When I first tried out Twitter, I was eager to try out the IM integration it offered with our IM server. But then I found out that it didn't work. You could add twitter@twitter.com to your roster, but it would stay in a sort of "pending" state, so you couldn't send it the claim code. My guess was that their XMPP server seemed to be requiring SRV records of us, which is only really required if your XMPP domain name doesn't match the server's DNS. Since im.case.edu is both our XMPP domain and our DNS name, it should have worked.

I sent a bug report to Twitter about it, quoting the pertinent bits of RFC 3920.

Today, I was notified that this was fixed in their updates last week, so you can all update your Twitter status via your Case IM account, if you're into that sort of thing.

Posted by sdh7 at 06:31 PM | Comments (0) | TrackBack

August 17, 2007

prettier presence embedding

Being able to embed your IM presence in a web page has been available for a while, and it works just fine as it is. However, embedding the text with the <OBJECT> tag is kind of clumsy and never really looked nice in the sidebar of my blog's index page.

So I improved upon this with some JavaScript magic (and a little back-end Perl):


<script language="JavaScript" type="text/javascript" src="http://im.case.edu/api/presence.cgi?jid=NetworkID"></script>

Replace NetworkID with your Network ID. You'll still have to add yourself to your Case IM buddy list for this to work.

Posted by sdh7 at 11:31 AM | Comments (0) | TrackBack

August 06, 2007

Case IM Avatars!

Ever try to add an avatar to your Case IM profile, but got an error that you didn't have permission to do so? That's fixed now.

I installed Hannes Wüthrich's LDAP VCard Avatar plug-in for Openfire. What this does is allow the avatar to be stored in the Openfire database, but all the rest of the user's information is still pulled from the LDAP server.

So, you can have avatars now. Go to it!

Posted by sdh7 at 05:40 PM | Comments (3) | TrackBack

July 27, 2007

Potential Windows Spark Issue

Some Spark users (mostly on Windows Vista, but some on Windows XP as well) have reported seeing incorrect timestamps on their IMs. This appears to be an issue in the Java Runtime Environment that Spark ships with. We've come up with a workaround that seems to help:

The Spark.vmoptions file hard-codes the time zone used by the JRE to EST. If you're in another time zone, you'll need to edit the Spark.vmoptions file and set it to the appropriate time zone.

The Mac OS X version of Spark is unaffected by this issue, since it uses the OS's JRE to run. It's unclear if the Linux version is affected.

Posted by sdh7 at 03:24 PM | Comments (0) | TrackBack

July 26, 2007

SparkWeb is now available

As I mentioned before, a web-based IM client is part of the newer versions of Wildfire/Openfire Enterprise. It's now deployed and available at im.case.edu.

Posted by sdh7 at 02:58 PM | Comments (2) | TrackBack

IM upgrades relatively complete

The upgrades I mentioned in my previous post are installed. There may be some post-installation tasks left to do. If you notice anything amiss or any problems, let me know.

It took less than the full three hours I alloted myself, too.

Posted by sdh7 at 05:05 AM | Comments (0) | TrackBack

July 20, 2007

IM updates are coming!

OK, the long awaited upgrade to the Case IM server is coming! It should happen some time next week - probably Thursday morning.

Also, the latest version of Spark is going to be in the Software Center soon.

There may be some more "surprise" features as well, but they'll require some post-upgrade testing and configuration.

Posted by sdh7 at 10:06 AM | Comments (2) | TrackBack

February 19, 2007

I guess im.case.edu is in production...

...since it made today's Case Daily. We also topped out at 116 simultaneous users today.

We'll probably start looking into upgrading to the latest version of Wildfire soon - that way we'll have a web client, and be in position for the upcoming Jingle-enabled version of Spark coming soon.

I'm also starting to play with the Fastpath feature. On my index page, there's now a click-to-chat button, which uses Fastpath. We won't be able to do this for individuals (too much manual setup), but groups interested in having this kind of functionality can contact im-admin@case.edu for more information.

Posted by sdh7 at 03:40 PM | Comments (0) | TrackBack

February 02, 2007

IM update

The IM service is now very nearly production ready.

Unfortunately, we were unable to set the JIDs to "userID@case.edu". The problem is SSL related (it seems that Entrust wouldn't let us issue a cert for "case.edu".) Instead, all of the JIDs are of the form "userID@im.case.edu".

We also now have Case-branded versions of Jive's Spark client - you can get them right now at the rather sparse im.case.edu, and they'll be available in the Software Center soon.

As a result of this, the "messenger.case.edu" moniker is now deprecated in favor of "im.case.edu". You can still connect to the server using the hostname messenger.case.edu, but all the JIDs are @im.case.edu, and there's no way to do JID aliasing that I know of, though that is a good idea...) This shouldn't affect most people unless they've been doing server-to-server connections already.

Posted by sdh7 at 05:14 PM | Comments (0) | TrackBack

January 30, 2007

The Latest on the Case IM System

1. Oracle Messenger has been officially retired. This actually happened a few weeks ago, but I didn't get around to announcing it until now due to vacation, etc.

2. Because of (1), we've now got the Wildfire server listening for SSL connections on the correct SSL XMPP port (i.e. 5223). If you're using a client that needs to use the old SSL port, adjust accordingly. We're also working on getting a real SSL certificate installed, which should stop iChat from giving errors upon making SSL connections.

3. The "change-the-JIDs" move I keep talking about is going to happen. This may also be combined with some other changes that will improve reliability. Stay tuned for details.

4. We have now purchased the Enterprise version of Wildfire. Because of this, we will be soon distributing a customized version of Spark.

5. I'm also starting to look at the FastPath functionality in the Enterprise Server. This will allow specially-defined workgroups to do "click-to-chat". I've already got one group interested in this, which partially came out of a conversation I had during my vacation(!).

Posted by sdh7 at 01:18 PM | Comments (0) | TrackBack

December 18, 2006

Case IM Beta update

We're making good progress with the IM beta so far. Here are some notes on the current status of the service:

1. SRV records - Hopefully, we'll soon have SRV records in DNS. This will improve server-to-server traffic, as there are some Jabber servers(supposedly this includes GTalk(?)) that are known to not try s2s to servers that don't publish XMPP SRV records.

2. Domain name - Right now, the JIDs issued to users are of the form "NetworkID@messenger.case.edu" - I'm wondering if changing it to "NetworkID@case.edu" would be better? This would require a little bit of work - as then the SRV record would move from being a nice addition to being required, and any instances of "messenger.case.edu" in the database would have to be changed.

3. Web Services - At some point, we'll hopefully be making a web-based IM "presence" service available. This would allow users to embed an image or text on a web page (such as a blog indicating their current IM status.

4. Web-based IM - I'm still thinking about deploying some kind of web client for the service. The most likely choice is currently Claros Chat. I'd love to know if there are any good alternatives out there.

One other thing that I haven't mentioned before - the IM server is available to Alumni and anyone else with a valid Network ID, so check it out.

Posted by sdh7 at 01:34 PM | Comments (2) | TrackBack

December 11, 2006

Wildfire IM public beta

In my last post I mentioned that we're looking for users to try out our test deployment of the Wildfire IM server. Well, now I'm going to make the details public.

You can point your Jabber client to messenger.case.edu, port 5222 (yeah, it's a nonstandard XMPP port... When this goes to production, it'll be the normal 5222). If you don't have a Jabbber client, you can try Jive's open-source Spark. If for some reason your client can't do TLS, you can do the "old style" SSL encryption on port 5223.

Server-to-Server XMPP is enabled, so you should be able to IM Google Talk and other Jabber users. We've also got the IM gateway plugin enabled, which will allow you to connect to AIM, Yahoo, MSN, ICQ, and IRC to some extent from your Jabber client.

update: All the connection details in this post are now correct, since the port move happened on Wednesday.

Posted by sdh7 at 03:28 PM | Comments (0) | TrackBack

February 17, 2006

anatomy of a mail server crash

ok, so the mail server crashed twice today.

Want to know why?

We knew that the initial cause was the IMAP server dumping core. We may have even had an inkling as to what may have caused it. Here's a little bit of the output of a pstack on the core file:

----------------- lwp# 12 / thread# 16 --------------------
ff129394 txn_abort (0, f30a14d8, f30a14c0, ff12e164, 0, 0) + 4
00085fbc mboxlist_trans_finish (0, 16, f30a14d8, f30a14c0, 0, d) + 24
00091f94 mboxlist_maybeRecoverSubsFromFile (1e25994, 2f, 0, 0, 0, 2f) + 68c
0008ebdc mboxlist_findsub (1799550, 1, 0, 1e25994, 1c7b538, 625b0) + 6c
00053bb0 cmd_list (1e25920, 1e2f0b8, 1, 11995d8, 1799550, 0) + 288
00047d90 cmdloop (1e25920, 708, 2000, 2710, 0, 0) + 3058
000434d8 sock_readable (1e25920, 0, 7c2b48, 0, 0, 0) + 580
ff1860bc GDispCx_Dispatch (221204, faa47c, 7bed4c, fffffff8, 0, 6d47c1) + ec
ff1864b8 GDispCx_InternalWork (7bed4c, ff186188, fee6c000, c, 7c2b48, 0) + 330
ff0b2ad0 _pt_root (7c2b48, ff0cc24c, 0, fee78d04, 0, 2) + a4
fee5b03c _thread_start (7c2b48, 0, 0, 0, 0, 0) + 40

So something weird was going on and causing the IMAP server to crash, which then left something locked in the store daemon (the process that actually touches the mail, whether via POP, IMAP, webmail, or SMTP), which then caused further problems.

After the second crash under similar unusual circumstances, I went digging into the IMAP log files, hoping that something would turn up. Something did:

[17/Feb/2006:16:08:16 -0500] northuist imapd[33856]: Account Information: login [x] uid proxyauth
[17/Feb/2006:16:08:16 -0500] northuist imapd[33856]: Store Critical: Mailbox database error: missing or empty key value specified

Well, that's interesting. Let's see if that error turned up in the earlier crash:

17/Feb/2006:14:05:59 -0500] northuist imapd[31550]: Account Information: login [x] uid proxyauth
[17/Feb/2006:14:05:59 -0500] northuist imapd[31550]: Protocol Information: uid created mbox UM-Messages
[17/Feb/2006:14:05:59 -0500] northuist imapd[31550]: Store Critical: Mailbox database error: missing or empty key value specified

Note that I've cleverly disguised the user's network ID as "uid" here. But interestingly, both crashes involved this same person's mailbox. It looks like they were being provisioned for Unified Messaging on the earlier crash, and attempting to access voicemail on the second.

Time to look at their mailbox!

northuist%ls
=+U+M-+Messages/ store.idx store.sub store.usr

Nothing terribly unusual from the initial peek. There's no actual mail in the INBOX (but it turns out they forward their mail, so that makes sense)

Let's look at the UM-Messages folder:

northuist%ls -l
total 16
-rw------- 1 mailsrv nobody 7856 Feb 17 16:41 store.idx

Very normal looking too. Let's go back and take a closer look:

northuist%ls -l
total 526
drwx------ 2 mailsrv nobody 96 Feb 17 16:41 =+U+M-+Messages/
-rw------- 1 mailsrv nobody 7856 Feb 17 16:41 store.idx
-rw------- 1 mailsrv nobody 259656 Sep 3 2003 store.sub
-rw------- 1 mailsrv nobody 37 Feb 17 15:48 store.usr

hmmmmm... That store.sub file looks a little big. It's just supposed to be a list of all the folders you're subscribed to (i.e. your own folders) This user only has one folder, so why is it so big? Let's take a peek:

northuist%more store.sub
/hBOcdctpYVDO46Y
W16kL91GMdfKlokIlr+9qDX31vApE+z7ENLl+8pyz4TuZe3YCN2Bbnml9DKtxzImBtBjCiM8SSiF
jbJ2c4rGfW+KTC+VtSq+x6vK/qXINxqtfl+58EqPHrEj9SQ9bN7TWh4TiZ7f3pm6C4Vj9lIcG2+q
9z9ErtDxB2F56fBzHb+fsOXhtnNQ00M4ht8Z+PbW7tpiBq3ygXCm0Yg2/CHY/xAvWL9ro247Yu6l
jUKZhFeCFl8mZLln2gOdx9bnfirC6zCkbUPnzW82to03ij1Q31EZWjj+P0bQjX5Xy2fo2i1LWAXK
CwnxMmqkbMWSG5D3vbrgezE3I7dR7nhB0bo9p6X0h/LodW42MwPsfX3bt1Es93xc9xp1b3jnHqVT

That doesn't look good. For comparison, here's an excerpt from my own store.sub file:

user/sdh7/Sent
user/sdh7/Sent Items
user/sdh7/Trash
user/sdh7/UM-Messages
user/sdh7/JunkMail

A-ha! A little magic later (i.e. rm'ing the file and re-massaging the mailbox database) and I think everything should be happier now. We'll see what happens.


Update: We've finished running a database reconstruct and all still seems well. I did just happen to notice that that bogus file dated from 2003, so these crashes appear to be a holdover from an old crash, rather than a recent corruption. Given that more people are going to be set up for UM soon, we should probably run a check for things like this.

Posted by sdh7 at 04:51 PM | Comments (0) | TrackBack

February 13, 2006

Unified Messaging and Mail forwarding

So, many users have been converted over to Unified Messaging from the old DirecTalk voicemail system. If you're one of them, you may have noticed some peculiar things about how the system works:

1. You actually get two copies of each voicemail message - one in your INBOX and one in a separate folder called "UM-Messages". This is in order to be useful to the two different camps of users we have on campus (POP users and IMAP/Webmail users), and the two different methods of message access (via e-mail and via telephone, or "TUI"). Since we try to be everything to everyone, we set up what's known as a "System-level Sieve filter" that checks each incoming message and does the special delivery if it's a voicemail message.

2. Mail forwarding causes some strange behavior, or does things you don't want it to. When mail is forwarded, the UM-Messages copy doesn't occur, so you don't get to check your mail via the TUI, only through your forwarded mail.

We're looking into some more automatic workarounds for #1 for power users (i.e. those who only want one copy of the message or the other, and not both). Stay tuned...

To solve #2, Joel Kraft actually went to the trouble figuring out enough to write some of his own Sieve templates to add into Delegated Administrator. They did require a couple of minor edits before they were working 100% correctly, but they're in there now.

Here's how to use them:
1. Go to Delegated Administrator and log in
2. If you are currently forwarding mail, turn that off, and turn mailbox delivery back on- it sounds counter-intuitive, but trust me on this one.
3. Now go to the "Set Mail Filters" view. (Unfortunately, this tends to render weird in Firefox, but looks fine in IE or Safari). From the menu, select "Forward Everything (Copy Voicemail). Make up a name for the rule, and enter the e-mail address you want to forward to.
4. Click OK, and then save your rules. You're done.

Posted by sdh7 at 04:24 PM | Comments (1) | TrackBack

January 07, 2005

some more alias migration fun

For a couple of reasons(speeding up migration time, testing out some enhanced spam control facilities for mailing lists), we migrated another alias from the master alias file to Sympa. The process was similar to what was done the other day, though this was a "simple" alias, rather than an Admin Alias. The procedure is now down to about six minutes (a couple of which were spent tracking down the appropriate crontab entries to run by hand).

For some extra background, here are some of the different kinds of things we have lurking in our master alias file:


Note that Personal Aliases aren't part of this. When they were initially deployed, they were stored in the Master Alias file, but those have now been migrated to LDAP.

We eventually plan to migrate most of the aliases listed above to Sympa, though that's probably going to be a somewhat long and drawn out process. I suspect that admin aliases may actually be the hardest, since ownership of the aliases isn't as strongly defined as majordomo lists (most admin aliases are actually "owned" by our Network ID people).

Posted by sdh7 at 01:57 PM | Comments (1) | TrackBack