brian's blog

Entries for March 2007

Credit card systems are super annoying

So I have a family credit card I sometimes use for small things and emergencies. A few days ago my wonderful grandparents told me I could use it to buy a new laptop as a present for my eventual graduation.

It was a pretty easy decision to buy a MacBook. I looked at one other company's website and was disgusted by the typical electronics shopping experience—crossed out price values highlighting urgent deals and savings, and other insulting bullshit.

So I went to the Apple Store with Steve and Sara. I told them what I wanted at the counter; for some annoying reason they think this means you want to chat with someone instead of spending money and getting out of there, so I waited while they found someone for me to talk to.

Eventually they brought out the box and I gave them my credit card. It was declined with no reason given. Great. After trying to contact my family, I called Visa and spoke with someone who said that the charge attempt never showed up—Apple's system just failed to contact theirs.

So I went back and tried again. It made them authorize by phone this time, and after a wait, declined claiming to have insufficient funds. The Apple store employee quietly related this information to me, to save me some embarrassment, I presume.

Much later my family got back to me, and was puzzled to hear what happened. They contacted Visa and there was plenty in the bank and no applicable limit—Visa claimed it was just a security feature and that the next attempt would be successful. This was ten minutes before the Apple store closed, and we had driven to another location by this time, so we hurried back and I made it in at the last minute, as evidenced by the lack of music on the store speakers. By now of course I looked like a total shady character trying to use a stolen or maxed out credit card.

The salesperson in training fetched the laptop a third time and took my Visa and identification. Just as he was ready to send me home with a new laptop, the other employees informed us how Apple's credit card system would just decline me again, since their system only allows a card to be used once in 24 hours. The previous attempts had ruined my chances for the rest of the day. I left the store defeated a third time.

So thanks to Visa and Apple for costing me three hours and some dignity and leaving me empty-handed thanks to some stupid policies. Clearly you don't want to help me legitimately buy something.

Inkscape plugin idea: Interface mockup builder

Usually for HTML/CSS interfaces I jump right into the coding and tweak until it looks right. But for Pagoda, we often each have some ideas in mind about how the interface for a feature should be presented, so spending time on easily changeable mockups instead of a less flexible prototype (pixel pushing with HTML/CSS is often time consuming and creatively limiting) is a good way to see whose ideas work out the best. I'm our mockup guy, and as in my previous entry, I use Inkscape for this. Some features actually lend themselves very well to this purpose; some that come to mind are Paste Style, the Align & Distribute dialog, and Ctrl-dragging to limit the direction of movement.

Today after drawing up some mockups, I had an idea: an interface builder plugin for Inkscape. It would have a library of common widgets found in web interfaces and desktop toolkits. You could drag these pre-built graphics onto the canvas and be able to change their state easily. For example, drag on a checkbox and change it to be unchecked, or drag on a tab and change it to be the selected tab. Tree widgets are especially time consuming to draw, don't get me started about changing the items later...

This would be nice because unlike various toolkit-specific interface builders, you still have all the power of Inkscape to think outside the realm of just widget layout, which is a big factor of web interfaces. Being free-form is a big plus.

So, some of the features in list form:

• Widget library


• Transformations to change widget state (selected, disabled, etc.)


• Some additional align & distribute options that are useful for interface building, and resize options to make resizing grouped shapes easier (for many widgets you only want some shapes to resize in certain dimensions)


• Something with layers to show different stages of an interface

Any other ideas? Is there something out there like this?

With help from Inkscape's blur filter...

I often enjoy putting my coding abilities on hold for projects that need graphics. Here's some original artwork for a top secret project I'm working on with a friend. Made with Inkscape, my weapon of choice.

How did I survive before the Gaussian blur filter? It can really light up an image, especially when it's really supposed to shine or glow.

All those details really give Inkscape a rendering workout, especially on my old computer. Exploring all the dark corners of Inkscape also reveals some annoying bugs I'm sure the wonderful developers are hard at work on. Can't wait for the next release...

Sara on Linux

A few days ago Sara agreed to try out Ubuntu Linux on her computer. Windows had been misbehaving in some very annoying ways... not that I can stomach its normal behavior anyway. The final push came when we were at a party and noticed that her friend's boyfriend had done the same with her computer, and it seemed to work out well.

Everything went pretty smoothly despite some minor annoyances, and the results have been quite enjoyable. Some of our experiences, good and bad, follow...

• "What is this, Lion King Catharsis? Who makes this stuff?" - Sara on the overly dramatic Ubuntu login sound. (Does anyone not immediately disable all sounds? Shhh!)
• "That's all I have to do to install stuff? This is awesome." - Sara on package management.
• "Huh, is this better than the other photo manager?" - Sara on F-Spot. It took me a second to figure out she meant Nautilus, which displays image thumbnails. I really think images should have normal mime-type icons in addition to a small thumbnail overlay in a file manager. Some thumbnails just make for really annoying icons.
• "Wait guys... let me just try a couple more packages." - me, several times, while trying to watch a DVD with people. We gave up and used a Mac.
• I hate packages that think the world revolves around them. Do I really need all of Sun Java 5.0 Plugin Control Panel, Sun Java 5.0 Policy Tool, Sun Java 6 Plugin Control Panel, Sun Java 6 Policy Tool, and Sun Java 6 Web Start cluttering up my menus? This should be one or zero menu items. Who uses that bullshit? Die, Java. (With thanks to Menu Layout—checkboxes instead of deleting stuff is brilliant.)
• Even though I prefer KDE, I'm convinced GNOME was a good choice for vanilla Ubuntu, and I can't wait till 2.18 gets pushed into the repositories. Many of the reasons I prefer KDE are because I'm a developer and couldn't live without things like IO-slaves. That said, I ended up having to install KDE libraries anyway because Amarok and Akregator blow away GNOME's offerings in my opinion.
• I could only get Cisco VPN support to work from the console. Save us, NetworkManager!

That's all for now. I expect Sara to start contributing to the Linux kernel any day now...

Thinking about taking over the django-sqlalchemy branch

For the past couple weeks I've been telling myself that I'm the guy who could make the sqlalchemy branch a viable option. I think I'm familiar enough with both django.db (after my shot at Merquery and hunting down bugs from Trac) and SQLAlchemy (after modelling and remodelling inheritable, multilingual, revisionable tables a few times — an excellent way to far surpass any beginner's material in a matter of days) to be able to pull it off.

I talked to Robin Munn (the original maintainer) at PyCon and he is ready to hand it off to someone. I also spent time at PyCon dissecting the metaclasses used by Django, Elixir, and ActiveMapper to find out how they're different and why. Still not clear on the 'why' part, but I think I know enough to give it a go.

So, knowing very little about what Robin's already accomplished, my approach would look like:

• Start a django.contrib.sqlalchemy.models.Model that would serve as a drop-in replacement for django.db.models.Model, as Adrian first suggested.
• Almost all methods and classes used by this Model would be completely from scratch. There will be some opportunities for reuse, but I would much rather have something that does everything The SQLAlchemy Way. Don't even look at the django.db code unless it's clear that it solves a problem that SQLAlchemy doesn't, or resolves an undocumented feature.
• Reimplement all of Django's fancy fields as custom SQLAlchemy types. This is theoretically a chance for reuse, but this part is trivially easy compared to making a model with the same QuerySet semantics. These types wouldn't support any of the oldforms stuff that the current fields do, since oldforms would likely be dead by the time this is finished.
• Use SelectResults as a base for QuerySet, but use django__joins and 'strings' instead of Quoted.c.column objects where appropraite.
• Totally ignore the undocumented API (like _meta) until the documented API works.

I was bored on the flight back from PyCon and started writing some notes on a napkin mapping Django's Model features to SQLAlchemy's syntax. I don't think it's a problem that requires solving any super hard problems, just lots of careful coding (to keep it clean).

So, will I be the one to do it? I'm not sure yet. I'll be spending a lot of time working on Pagoda, and already have a hard time maintaining geopy and other projects, which people often email me about. Maybe if I spend a weekend totally focused on integrating Django and SQLAlchemy, I'll come up with something that other people can contribute pieces to without having to completely understand the mapper magic.

Better Python Editing in Kate

Perhaps this will damage my hacker credibility, but I use KDE's Kate Editor as my development environment. For a while I used KDevelop, but then I realized the only features I was using were Kate's (note to KDevelop developers: having to decide on a location and filename immediately when I hit New File is extremely annoying. Give me an Untitled!)

The included python.xml (the file that tells Kate how to categorize tokens for highlighting) is a bit insufficient for Python programmers: no differentiation of class definitions, function definitions, or decorators. I've added these and some other minor improvements to my own python.xml and packaged it up with a custom color scheme in kate_colors.tar.gz. If you extract this to your home directory, it will replace a few Kate-specific files. Your configuration options will be kept, but I think any custom colors you've set will be replaced (Kate lacks a way to split these up to make sharing easy). If you select "exogen - Dark" as the default schema in Fonts & Colors, your HTML, CSS, JavaScript, and Python files will look pretty:

Of course, you don't have to use the included schema. The enhanced syntax highlighter is the biggest improvement, giving you several more contexts under Text Highlighting Styles.

I shared these changes with Adrian Holovaty at PyCon, who I noticed also uses Kate. I guess he couldn't get used to the high contrast color scheme, but here's hoping the new syntax file is serving him well. ;)

(I also recommend the Tab Bar Extension, the Word Completion Plugin, and learning to use those double arrows in the Filesystem Browser panel.)

Pagoda CMS Notes

If you were at PyCon 2007 or read Gary's blog or read the TurboGears mailing list, you may have seen Pagoda CMS mentioned. Pagoda is an open source content management system I've been working on with Chris and Ian. It's built on TurboGears and is focused on being simple yet extensible. We put up an introductory screencast at pagodacms.org that we hurriedly made the night before PyCon.

We've tried a bunch of content management systems, both open source and commercial, and developed for small shops, big corporations, government organizations, and of course Case itself. There are features that are consistently implemented poorly, hard to understand, or simply missing. Pagoda is a result of the observations we've made of how content management systems are really used in a production setting.

These are just a few of the notes and design goals we've been using along the way.

Don't overengineer it

Somewhere along the line someone decided that if you're going to make a content management system, you have to build everything on top of a dozen layers of abstraction. Some pretend that there's no difference between static page content (like a blog entry) and dynamically generated content (like a news feed). Some pretend that building complex workflows that are exactly suited to the way your organization is structured is a common need (we've found that people already have real-life workflows and rarely do they want this duplicated in a CMS).

Experience has shown us that such complexity is rarely needed. We don't try to fit every feature into a "plugin" structure or an "actions" framework. We've streamlined the features based on our experience, and hopefully kept it fun to hack on (when you do need something extra) by avoiding meaningless abstractions.

Do one thing really, really well

A lot of content management systems try to do everything involved in running a web site. Database management, email management, form design, you name it. We don't want a content management system that takes over every aspect of making a web site. We've made conscious decisions to leave a lot of features out. In addition to the above (which can all be found in Zope + Plone, for example), we've spent a lot of time deciding how far certain features should reach and what should be left up to the webmaster.

One example is theme switching. When you're first building a web site, being able to download prepackaged themes might be nice. But for production sites, this simply does not happen. Imagine the Cleveland Museum of Natural History or a university department downloading new themes and swapping them out. Not gonna happen. Instead this is limiting, because prepackaged themes require predetermined markup. As a result, most Plone sites look the same and are structured the same way. They have the little tree on the left and those tiny tabs and a logo above that. And then you're scared to modify too much CSS because there's a bunch already dedicated to making those tabs pixel-perfect. We don't have a default theme or even default markup. Markup and design are meant for programmers and web designers, let's not pretend otherwise.

Use simple terminology

As Jeffrey Veen mentioned in Making A Better CMS, stop it with the jargon already! "Mambots", "archetypes", "portlets", and I'll admit it, I'm not even a fan of the term "widgets". We've tried to use understandable terminology throughout Pagoda and not extend failed analogies.

One example where we created a feature and spent some effort on choosing a name is Placeholders. This is a feature that we've actually needed on production sites but haven't found in other content management systems. The idea is that there is text that appears on multiple pages within the content, and it would be nice to only have to change in one place so we don't have to hunt down every page in the future. Phone numbers, store hours, admission prices, and press contact information are some examples. These aren't template variables because they have nothing to do with templates (to the user) and aren't arbitrary Python objects, and they're not code snippets because they have nothing to do with code. They're simply content placeholders. Here's the mockup we used while implementing this feature:

Borrow features that work

We've had a lot of inspiration along the way and used it to solve real problems. For example, if you need to have a downloadable file on your web site, a lot of content management systems will force you to ask "where do I put this?" and once you've decided on a place, require you to find your way there in the filesystem. We decided on pages having Attachments. Most downloads are associated with a particular page, so just upload them to that page and that will determine their location. We used 37signal's Campfire for inspiration, where people can upload files to the room they're in and they appear as attachments.

Reduce the number of clicks

We're lucky enough to have started developing after AJAX became popular. The "Web 2.0" buzzword might be annoying, but this is really something we can use to make content management quicker and easier. Navigation and messing around with page options won't require dozens of clicks and page reloads anymore. Instead of having to retrofit our software to take advantage of AJAX, we can design with it in mind.

Built a content management system, not a new framework

Similar to doing one thing really well, we're not building a web framework. That's what TurboGears is for. People can still use their existing TurboGears controllers, models, and templates. We're using SQLAlchemy for Pagoda's models and Genshi for the templates. To install Pagoda for your existing TurboGears project, you'll just have to subclass from PagodaController instead of the default RootController, so Pagoda can dispatch requests to the appropriate page.

So hopefully it sounds like an interesting project. We're still hacking on the core and hope to release a demo before the end of March, when we'll also invite people to help out and find weak spots. We have some mailing lists on Google Groups for discussion: pagoda-talk (general discussion), pagoda-coders (core development), and pagoda-announcements (for releases and other notices). For the first few releases we'll also make announcements on the TurboGears list.

Japan Loves Sucks/Rocks

Last night, Gary and I were in Jennings making some improvements to sucks/rocks. We switched to Paste's HTTP server so that queries to the search API wouldn't block the whole server (which was single-threaded). This was a big problem because after running out of both Yahoo and Google queries, we had to fall back to Microsoft Live's, which is the slowest to respond by far, and has an annoyingly small index.

While doing that and other minor changes, Gary noticed the server log start going crazy. "Holy crap! Someone's sending tons of traffic our way!" I asked him to check the referrer. "I just hope it's not — shit, it's Slashdot! Japanese Slashdot!" We checked slashdot.jp and sure enough, it was the latest story. We went into panic mode as our CPU usage on WebFaction grew to 50% as we served multiple requests per second. We were able to make a few changes and get it back down around 6%, and Paste's HTTP server kept us online the whole time! We also noticed a bunch of other Japanese sites in our logs, including headlines.yahoo.co.jp , 100shiki.com, and japan.internet.com. Japanese people love sucks/rocks! Our Google Analytics graph for this morning is below.

While Slashdot Japan was busy hammering us, we added a Latest searches box where you can watch incoming searches scroll by. We had been doing this before by watching the server log, and figured others would enjoy it too. This was flying by with all the traffic last night. (You can see below how crappy Microsoft's index is — they don't even have any results for xbox360, with or without the space!)

I'm not sure how many more features such a silly little project can handle, but Gary seems to have started working on a new (unrelated) project.

A minister, a professor, and a knight walk into an airport...

Last Monday night we were rushing to the Dallas/Ft. Worth airport in our rental car for our flight back to Cleveland, talking about all the awesome people we met at PyCon. We returned the car to the appropriate lot, located at the south end of the five-mile-long airport, and took a shuttle to our departure gate. Our flight was boarding in five minutes.

Ian printed our boarding passes and distributed them. Luckily, there was nobody in line at the security checkpoint, and we were prepared to be sitting on the plane in no time. An older woman took our boarding passes and IDs to wave us through. After a moment, no such waving was granted. "Where's the expiration date?" she asked me, looking over my Rhode Island identification. I told her that it was just a state ID, not a driver's license (I don't drive), and there is no expiration date. I'm accustomed to saying this at every airport, and the reply I usually get is "You're absolutely right sir, go right ahead."

Not with this woman. After trying to persuade her in a friendly manner it became clear that she wasn't having any of my "questionable" identification. "This says you're under 21, which is not true, and the picture doesn't even look like you." I offered to remove my glasses, but that didn't cut it. "Do you have any other forms of 'current' identification?" Ah ha! My Case RTA pass from last semester — it has an expiration date and a more recent photo. She looked it over and apparently it was not good enough since it expired on January 17th.

This is when Ian and Chris joined in. An ID with my name and picture on it that expired one month ago is not current enough? Surely it's not much different from the same form of ID that expires, say, tomorrow? Chris argued with her over her strict definition of "current" while I pulled every card I had out of my wallet. "How about a credit card?" I offered. "It has my name and an expiration date." She looked it over and refused it because it wasn't signed — I had instead printed "SEE MY AWESOME RHODE ISLAND IDENTIFICATION" on the back. "I'll sign it for you right now, let's do it." Not good enough. Surely the combination of all three cards — my signed Rhode Island ID, a bus pass with a recent photo, and an unexpired credit card in my name — should be sufficient? I even let her know that I've been to several airports and never had a problem, to which she said "Well this is DFW!" Oh, my mistake. We continued to argue over how silly it is that a Sam's Club card would be sufficient (her suggestion), but none of what I had.

Chris asked to talk to her supervisor, since we clearly weren't getting any closer to the plane. We all walked over to him, while the woman explained the situation to him in a way that was clearly trying to convince him that I was bad news. He took my boarding pass. "Professor Brian Beck?" he asked, looking it over. "Yes, that's me," I replied sincerely, stepping forward. Since I booked our seats, we were flying as Professor Brian Beck, Reverend Ian Charnas, and Sir Christopher Hesse, naturally. He took my other forms of identification and asked if I had anything else. As a last resort, I selected a family health insurance card from my wallet, positive that it would serve no purpose, having no photo, no expiration date, and several other people with different last names listed.

He held up the insurance card. "What are the last four digits of your social security number?" I quickly recited them. He wrote something on my boarding pass. "Anyone else have any Awesome Rhode Island Identification?" he asked, clearly amused. He returned my boarding pass and we were allowed to continue to the security checkpoint. I was taken aside and patted down while my bags were searched, thanks to a special mark the supervisor had scribbled on my boarding pass. "How paranoid do you have to be..." I thought. I gathered my things after being touched while Chris and Ian waited, then we headed for our gate.

I read my health insurance card closely. There was no trace of a social security number printed anywhere.

Once on the plane, we met up with the remaining members of our criminal mastermind organization. "The old driver's license trick?" they asked, and we all had a laugh. "Fooled them again!!" I thought, arming my wristwatch.