Pocket Revolutionary

For those who care to listen, I try to give advice from things I’ve seen done right. Occasionally, I’ve done something right, but normally I like to look at other people doing things I should do better. This is not about doing things right, though. This is my story of what I did completely backwards and wrong in about a week, and how I had to fix it in one day. Maybe you can learn from my mistakes. Or you can laugh at me. Either way, I hope you can get some amusement or education from my five not-so-easy steps to really screwing up software design:

1. Design a Perfect Solution. We had a simple problem. We were going to have our first “virtual conference” where real registrations would be paid for by real credit card transactions for access to a “virtual” online conference with streaming video, forums, and chat. Our problem was that our current online registration system was designed for “real world” conferences. I had hacked it into our shopping cart a long time ago, and it worked great. It could provide us a list of attendees before the conference, and emailed them their conference information. It didn’t however, talk to our website’s backend. Our TYPO3 website already has all of the user management tools, and would take care of granting “attendees” access to the forums, chats, and video features based on group permissions. Our shopping cart just didn’t know how to add or edit users in the database. Now, some of you are already shaking your heads. What was I doing running a website where the content management system was not talking to the shopping cart? Have I not learned anything about “seamless integration” and “smooth user experience”? Well, now I need you to get off your soap box and shut up. I would have loved to integrate these two complex, perfectly independent, working systems before. It wasn’t a priority because no front-end users could tell the databases weren’t talking. It was a larger-than-life project. Seriously, you just need to drop it. Anyway, I immediately set forth working on the perfect solution. I designed the entire thing on a whiteboard. I mapped it’s user interface, functionality, and even the variables I would use. It was perfect. Using only 80% of my development time, I had designed the monster. This left me with –crap– a couple of days to actually build the whole thing. I mean, design is so important, but maybe that’s just too much time out of the total timeline. That’s what I should have thought. I didn’t think that. I thought the design was too important to reduce it down to just a day, and so I had over-designed like there was no tomorrow. Literally.
2. Build for Scalability. What did I design this little bridge for? For the 500 estimated registrants? For this one conference, and maybe the one after? No way. I decided to design a system that would work for the most amount of people and would never have to be changed. I could’ve realistically handled 10,000 registrations, and all we would have had to do for any future conference is create a new instance with a click of the mouse. Obviously, this kind of scale limited my options a little bit. I couldn’t simply modify our current registration system. If I did (and I worked this out), there was no way for the users to be automatically added to the correct group in TYPO3. The problem with designing for 10,000 people was that it all had to be automatic. We couldn’t keep up with manually approving anything, even if it only took one second per user. I also couldn’t make a system that worked just for this conference as a test. No, the entire thing had to customizable to the nth degree considering every conference we host has different demands, and it had to handle all of them (even the ones I can’t think of). The problem with this? There are probably only going to be 500 users at most this time around. If I have to manually touch each of their records for just one second, that’s only 500 seconds in the next month. That’s right, I spent days designing a system that would save me eight minutes and twenty seconds. And future conferences? Yeah, I’ll probably have to tweak the code a little bit no matter what I plan now. In my “perfect solution”, though, that code was an 800 pound gorilla of an extension instead of the 2-3 PHP files that can be edited without worry.
3. Use the Best Tool for the Job. That’s actually a bit dishonest. I didn’t, technically, try to use the best tool for the job. I tried to use the absolute perfect tool for the insane job I had over-designed. What was my tool? To do basic PHP editing on the shopping cart registration forms? No, that’s not scalable. That’s not elegant. No, my tool was going to be a fantastic TYPO3 extension that would bring the whole world together. Now, I ignored some basic facts like (1) I had never written a simple TYPO3 extension and (2) I had successfully written a hundred modifications to the shopping cart. I would not be deterred, though. Like the thousands of PHP developers who dive into a new learning curve because they know that Ruby on Rails is the best way to build a one-off blog in the afternoon, I cracked open the books. I didn’t ask myself if I was going to need this education again in the near future (probably not) or if I truly had the time to add training to the timeline (definitely not). I just started working on the perfect solution that would take me about two days just to train myself on. Don’t get me wrong, though, because I love learning new solutions. I love learning new languages, frameworks, and techniques. Maybe, though, this was a bit too much training for what could have been a half-day project. Once again, that would have been the perfect thought to have a week ago.
4. Commit to Your Decisions. Now that I had only done one complex design and started training on a solution I had never used before, I had to commit. Even when I started to fall behind, I reminded myself (and my boss) that I had to launch all or nothing. Why? Because I had conveniently left myself with no other options. I had nothing right now, and the only way to fix that was with this too-big solution. I couldn’t launch just part of it, because it was a monolith. I couldn’t go back to the easier solution because, well, it wouldn’t have all the same features and it wouldn’t be infinitely scalable and I had already read all the books and… and… I was committed.
5. Fight Through the Pain. Now that I had committed to an over-designed behemoth solution that could scale to twenty times more users than necessary using a system I didn’t know, I had to buckle down. I couldn’t stop for food or sleep. Even when I came back from SXSW exhausted, I started pulling all-nighters. Then, I got sick. So, I muscled through that. Until I got sicker. I muscled through some more. Until my boss made me rest for one day. One whole day. He made me come to our weekly prayer meeting, admonished me for making myself sick instead of asking for a few more days, had everyone pray for me, and then dismissed me to my house to rest. I was not allowed to work on registration. It was horrible. It was the worst thing that he could do. That is, until I rested for a while. I realized that I had designed the wrong solution. What was too hard to realize earlier in the week when I was hungry and sleep-deprived became self-evident. I was being a newbie project manager all over again. So I rested and sub-consciously thought of a whole new solution. I didn’t work on it that day, though. I just napped, watched movies and let it stay in the back of my head for an entire afternoon and evening.

Okay, so that’s everything I did spectacularly wrong. How did I manage to fix it? I gave up on everything I had done. I copied our registration form and modified it slightly for this event. I added in a few quick hooks that added new users to the TYPO3 database by good ol’ MySQL insert. It took me about three hours. What doesn’t it do? Well, it still doesn’t automatically add them to the correct group for approval. But, it marks them using a magical, proprietary system in the user list to show that we need to approve them. Okay, it prepends “1_” to the username. Shut up. It’s great. Once every few hours, we look at the user list in the TYPO3 backend and add everybody with a “1_” to the event group, then delete the “1_”. It takes a little less than a second per person. And now I sleep properly. The wrong design took me about a week of hard work. My next-best design took me a morning. Hopefully I learned my lesson. Maybe I can tattoo my easy steps for designing bad software on my arm for me too look at every day. If you have any more tips that nobody should be taking, you can put them in the comments or email me. Maybe I can tattoo those on my other arm…

Michael Lopp (a senior engineering manager at Apple) gave a great presentation this past weekend at SXSW with John Gruber about the blood, sweat and fear in great design. He talked about the long, hard process they go through at Apple, and it was probably one of my favorite panels. I was so excited by some of the ideas that I recited everything I could remember (without my notes) to our graphic designer at Generals this morning before coffee. I was so excited about it that I was going to post all of my impressions here… until I saw that Helen Walters had already written a better report on it for Business Week here. So, I highly recommend reading the article, and I’ll go back to trying to write my original thoughts about other people’s words later.

[via TUAW]

So far, South by Southwest has been amazing. Honestly, it’s a reenergizing and inspiring experience just being around this many great and upcoming designers and developers. I tweaked my CSS tonight just because I was getting restless with my original design that had been “good enough”. I got to see some great blogger heroes today in panels with John Gruber, Michael Lopp (Rands), and Merlin Mann. Now, hopefully I’ll get the nerve to introduce myself to some of these awesome bloggers and internet celebs… anyway, I’m hoping to start hacking away on the new group blog this weekend, and maybe I can write more about SXSW after it’s all over.

This morning turned unexpectedly exciting for me when I was glancing through my newsfeeds and saw the announcement that FLOW3, the new PHP framework from the TYPO3 5.0 development team, is now “public.” Now why is this that exciting? Do I always get this excited about seemingly obscure and nerdy development tools? Actually, yes, I do… I explain the intricacies of Textmate, MVC, and CakePHP to anyone who mistakenly asks me what I do for a living. That’s besides the point, though. I am even more excited about this announcement than normal.

What makes so me excited about FLOW3? First, I believe in the TYPO3 developers working on it and it really does look like a first-rate project (no hands-on with it, yet, but I have faith). I also just really love any elegant framework solution because I don’t have time or patience to invent a new wheel every three months and I really don’t need developers I’m working with to make their own octagonal, undocumented wheels in the middle of a project. Most importantly, though, I just like the idea of a common framework between my CMS (TYPO3), my CMS extensions (hopefully), and my external web applications (sometimes you can’t shoehorn an entire webapp into a TYPO3 extension, after all).

If my dreams are true, they should all be talking the same language with the same APIs and have the same basic architecture in the future. Even better than that, they will have the same “flow.” Every developer has had to do the “mash-up” before. Heck, that’s a big part of the job: make this piece of software share a database with that CMS and then dump the data into the charts for the marketing department. The biggest problem I run into isn’t the language issues (if the API doesn’t suck), but the fact that coding in one framework asks you to think one way and coding for the extension engine wants you to think a completely different way. You have to “change paradigms” at every step in the development, and it’s mostly needless. One of the goals of FLOW3 is to get developers learning the “TYPO3 5.0 way,” and that’s what I care about the most: a consistent “way” for my CMS, it’s extensions, and my external webapps.

Editor’s note: I got linked to by Robert Lemke on the TYPO3 news page here. I’m positively blushing.

We have a job opening at Generals International for some more help in the IT/Web Department. Check out the info below, the website (if you don’t already know about the ministry), and email me if you’re interested! Friends and family are welcome to apply.

Job Description: We need somebody to handle website updates, store updates, and the administration needed for those updates. Our normal duties for the website involve communicating with other departments about future updates, working with the graphic designer, editing raw text for pages, creating lead-ins or “blurbs”? for longer articles, graphically laying out newsletters and pages based on text from other departments, and creating new pages based on other department’s ideas and our own skill/experience. Store updates involve coordinating with the graphic designer and accounting and getting text or writing our own (based on current workload).

Job Requirements: Good communication skills (and grammar) are necessary for editing web-based text, creating lead-ins or blurbs, and especially for coordinating all updates with other departments. Professional aesthetic taste is just as important as being a good communicator. We don’t necessarily need somebody who could design a whole website from scratch, but we desperately need somebody who knows what works and what doesn’t and is willing to continue learning as the web changes. We don’t need a marketing person, either; we just need somebody who knows that blood red is not an acceptable background color and understands why too many logos on one page will overwhelm readers. Most importantly, we need people with the ability to learn. Web technologies keep changing all the time, but anybody willing to learn can pick up Typo3, FishCart, blogs, wikis, and whatever we decide to implement in the future. Past web experience is good, but we don’t need another programmer; anybody with the desire and basic ability to run their own blog or homepage and continue learning could have the technical skills that we need.

We Use: Mac Laptops, Typo3, wikis, etc.

Please Contact:Jeremy Greenawalt (IT Manager)jeremy@generals.org(972) 576-8887 ext. 205

Printable PDF