The other day I got a notification from the Plesk control panel for Carticipate telling me that the user account was about to expire. I logged into Plesk only to find out that the disk on the VPS was full.

Now this was really confusing, since the entire Carticipate system consists of a web2py install, on Linux with a fairly small MySQL database, and the VPS had 20 gigabytes of disk space.

I looked around and found that there were three full backups each about 14Gb, with several incremental backups of a gig or so. Thinking this was the issue, I deleted all but the latest backup.

Well apparently, those backups are stored somewhere not on the server, so that made no difference to the space problem. So I did the next natural thing which was to look for big log files, clear out /tmp, and anything that might be causing the problem. I only managed to clear out a few megabytes of space.

I dug around some more and found a lot of files under the /root directory that seemed to be related to updating Plesk. Since that’s provided as part of my 1and1 VPS service, I called the support number to see whether they could help me figure out where my space had gone. Unfortunately, for the first time, they were not very helpful, suggesting that I needed to research and find out where the big files were on my system.

After some frustrating arguing with the support guy, where I pointed out that I couldn’t get support from Plesk, only they could, I finally gave up and went to my old stand by of Google.

First a quick search to find where the mysterious gigabytes of space were being consumed. First I did a quick “find” command with:

Going to a few of these folders and running “du -hs” gave me the folders that seemed to have a lot of files in them and were eating up a lot of space.

A few more searches on the Parallels site, and found a couple more references telling me that these were both OK to clean out.

The /root/psa folder is where Plesk was configured to download updates, and apparently it doesn’t clean out those folders after the update is successful.

The other folder is where all the dumps from the local backups get placed, and that was the primary problem area. The /var/lib/psa/dumps folder was over 14Gb of the 20Gb, so cleaning that out got me started again.

Looking at the dump directory, it appears to have daily dumps of all of the Plesk stuff going back forever, so this may happen again, but for now, my VPS backup is down to a much more reasonable 2Gb.

Today I spent an hour or so trying to clean up some iPhone code that I’ve been working on for Carticipate. I found that I was having trouble with the build that I checked out from our Subversion repository for a bunch of different reasons.

The very first one was that the code was built on an older version of Xcode, as well as an older version of the iPhone SDK, and on top of that the build machine was running Leopard (10.5) while I’ve been running Snow Leopard (10.6) almost since the day it was released.

I tend to try and stay as up to date with software as I can, and most of the time it serves me well. In this case, I ran into several very interesting issues. First thing I did was to check out the code using the command line tools for Subversion. After a few false starts, I got the full directory from the server downloaded to my Projects folder.

I opened up Finder, double-clicked on the project file, and got a warning about the file being for an older version of the SDK. Since I had loaded the latest version of the iPhone SDK, it wanted to upgrade the project build files to the new format. I figured that would probably be OK as long as I didn’t forget and check things back in, so I went ahead.

Then hit “build and run” only to get compile errors. Seems a few of the calls on the code that is currently being used in the wild aren’t allowed for the latest SDK. A few edits and no more compile errors, but there’s some other error. By default XCode just shows this little icon at the bottom of the project that shows the number of errors, but you don’t really see the build log. Clicking on the little error icon, takes you to the log, and of course there’s more greek there.

Something about not being able to copy a file from a user directory (one that naturally doesn’t exist on my machine). More web searches and digging around to try and figure out where copies are specified. If you’re OS X centric, it’s really pretty intuitive, it’s hidden somewhere and you have to find the right properties page to update. Finally I find the setting in the build targets, change it to point to an existing file, and everything runs like magic.

Now I notice that on the build machine, there is a Carticipate settings page under the iPhones Settings app, and it looks different than the one that you see if you have a production version of Carticipate from the App store. The one on my machine looks like the production build, so I figure it must be because I’m building for the release target, but switching to the debug target doesn’t seem to change anything.

I stumble around a bit more and find a DebugFiles folder that has the Settings.bundle file with the extra settings on it. Seems like it is set up to copy the file from that sub-folder if I’m building for debug, but no matter what I do, I can’t seem to get it to change. I clean, I build, I switch, I clean, I build, nothing …

Finally just to make sure the files are coming from where I think they are, I edit the root.plist in both Settings.bundles to be different, and … Magic, the debug settings show up. I switch to the release target, and the settings pane changes to the production one. So apparently something in XCode was being lazy, and not copying the file from the DebugFiles folder until I touched it (must use Make under there somewhere).

In the meantime, I see a checkin email from another developer that has the entire build tree being checked in. I dig around on the web some more only to find that the way Subversion is configured out of the box on OS X, there are no default ignores, and apparently XCode doesn’t have any intelligence of it’s own around what it checks in. Turns out there is a  ~/.subversion/config file set up, but the global ignores is commented out so nothing is excluded by default.

By default the global-ignores is commented out, even though it seems to include most everything we would want excluded. I chose to copy the list from somebody else for now, making sure I added one for “build” to exclude that for sure:

[miscellany]

### Set global-ignores to a set of whitespace-delimited globs
### which Subversion will ignore in its 'status' output, and
### while importing or adding files and directories.
# global-ignores = *.o *.lo *.la #*# .*.rej *.rej .*~ *~ .#* .DS_Store
global-ignores = build *.o *.lo *.la .*~ *~ ._* .DS_Store .Trash* *.pbxuser *.mode*

So all I had to do was remove that build folder from the repository, and give this same change to the other developers … Well, not quite …

After I deleted the folder from the repository, I had the other developer try and check it out. He got these really weird subversion messages that said something was locked. I’ve seen those before, and usually it just means you have to run an “svn cleanup” command to get things back in synch. But in this case that didn’t work, and eventually I found that I had to physically delete the folder on his machine before running the “svn update”.

Oh, and did I forget to mention, the other developer is using an older version of the client, so when trying to update he got this lovely message about his client being too old. We updated to the latest client from the Tigris.org site using the Mac OS X download, thinking that would work. Turns out that this update doesn’t replace the version that comes with OS X (in the /usr/bin folder), so you have to remove or rename that one, and add a symlink to the /opt/subversion/bin/svn executable in order to get the newly installed version to be run at the command prompt.

So with the newly updated checkout, the build worked on all machines, and the repository is fairly clean. Then came the next hurdle, the older machines (running the older version of the iPhone developer kit and therefore XCode) would no longer update the code using the built in subversion client. Found a small message about the version being too old displayed in the status line of XCode, so did some more searching.

Apparently on the older version of XCode, you could tell it which Subversion to use, but at some point, the developer tools started including a subversion library. Still haven’t figure that one out yet, but it looks like it might require some more moving files and setting up symlinks to get it to use the newly installed latest version.

Seems like one thing that I am continuing to learn is that it doesn’t pay to keep your software up to date on a Mac (unless you’re really brave) …

GTUG CampoutI recently attended the Google Technology User Group Campout at the Googleplex in Mountain View. This was a three day sprint to build something interesting with the latest Google product: Google Wave.

Google Wave, as it turns out is a very interesting experiment in social interaction. Google is trying to reinvent collaborative communication with a piece of software that is one part chat, one part Wiki, and one part WebEx.

I’d seen this product at the Google I/O conference a few months back and was impressed with the demos. Basically you get these shared documents (called Waves) that all of the collaborators can update at the same time. You can watch the hour and a half demo at http://www.youtube.com/watch?v=v_UyVmITiYQ

The demo included things like interaction with blogs, Twitter and other web technologies, as well as interesting programming doing things like on the fly grammar checking. I signed up for a sandbox account the day of the presentation (using my iPhone of course), and got set up a week or so after that.

Wave was written by the brothers Lars and Jans Rasmussen, who are the architects of the Google Maps API. In some sense, this is an experiment in building software caused by the lessons they learned with the immensely popular Maps API. By giving the developers access early in the build process, they hope to build a more solid platform that will serve the developers needs.

So Friday came and I drove over to Google with Bennett Fonacier (a friend I met through Job Connections some time back). After the 50+ people got through with their 5 minute pitches, we networked for another hour forming teams. There were many ideas that were very similar, and for the most part these groups joined up into a combined team. Bennett and Steffen Frost (CEO of Carticipate) both came up with the idea of matching people for ride shares using the Wave.

I’d originally thought I would join a team doing something health related, but since my goal was to get a working piece of code, and I was sitting with the car pool team, I joined that effort. We became one of the roughly 50 projects teams, and quickly talked through what we’d be building over the next 48 hours or so.

The other members of the Wave Rides team were:

  1. Steffen Frost was a great concept guy, and had an existing product we were going to try and emulate.
  2. Bennett Fonacier has some development background, but he was short a computer, and would be doing QA.
  3. Andreas Koll who had some experience with the Google Maps API volunteered to build the Gadget for our interface.
  4. Hannie Fan offered to provide some design expertise and CSS coding.
  5. Robert Herriott was a quiet supporter, offering constructive criticism

I took on the task of writing the Robot, which is the part of the Wave that would take the input from the Gadget and match the participants with ride partners. Andreas had a working Gadget in short order, and was able to embed it in a Wave.

While he was doing that, I was working on getting a Robot built using the guidelines in developing Wave extensions slides. I got a working “Hello World”, built the extension.xml file, and with help from the Google crew, got it so we could create a new Wave with my Robot added.
Carticipate Logo

I got the icon from the Carticipate site, added a bit of code, and the Robot was adding the Gadget to the Wave. So far I had gotten a working Robot, and Andreas had a working Gadget. Now all we needed to do was clean them up a bit and get them talking.

This turned out to be a bit tougher than expected. The current state of the world is that the Robot can add a Gadget, and send data to it when it is added to the Wave, but can only read the state from the Gadget, and not actually set anything after the Gadget is running.

Anyway, I eventually got some debug code going in my Robot that would dump out the properties of the Gadget, which helped Andreas to debug some issues he was seeing with the state of the users accessing the Gadget.

A Gadget is basically a snippet of HTML and JavaScript that gets embedded in an XML file for inclusion in the Wave. Because the working code is inside an XML document, it gets wrapped in a CDATA element, which makes editing and debugging the Gadget a bit challenging. Andreas approach was to cut the HTML code out of the Gadget and edit it as an HTML document, then paste it back into the Gadget. Not ideal, but it works.

Our original plans for the WaveRides robot was that it would behave roughly the same way as the Carticipate application does: ask the user a few questions about where they are going, if they are driving, and then show a list of everybody who is travelling in the same area and time. So as we worked, I kept prototyping closer and closer to that goal.

By the late Saturday night, we had a working prototype that launched the map gadget, and displayed back the data from the users interacting with the gadget. The gadget was displaying the location of all of the users on the map, and we were feeling pretty good about the progress (especially considering none of us had ever built anything with the Wave API before). Bennett and I headed home, expecting to finish up the next morning, leaving Andreas coding away on his gadget.

The next day we arrived at the Googleplex and found that Andreas had solved some of his remaining problems, and the gadget was looking good. I went to work on the Robot, trying to get it to match up the user data. Of course, since there was little time left, the Wave kept misbehaving (probably due to all of us pounding on the sandbox with untested code), and we kept running into walls.

My original design had been to add a blip with the map gadget and gather my data from there. I soon realized that it was difficult to keep track of the gadget that way, so I changed my code to add the gadget to the root blip, and started removing debug code. At some point, we decided to put the code up on code.google.com for safekeeping, so I spent a few minutes figuring out how to do that (you can see the code at http://waverider.googlecode.com).

It was still fairly early on Sunday morning, and Andreas had been up until the wee hours of the night, so he wasn’t around for us to ask him to make changes to his gadget. We had separated the development of the gadget and the robot, so they were actually being served by two separate app server applications. The gadget only provided input for one point, and to complete the robot to the point we could demo something interesting, we needed it to have a “from” and “to” for each participant.

So rather than reinventing what Andreas had done, I decided to change the robot to create a “from” and a “to” gadget in the Wave, and use that. Interestingly this turned out to be fairly painless. I was able to add the second instance of the gadget, and give them each a name. The Wave kept track of them separately, so I got the data from both separately.

I spent the last few moments before we were supposed to present, trying to get a simple match working. The nice thing about this was that I could version the app on the Google App engine, and keep a known working version deployed while continuing to test. As other teams presented, it became obvious that this had been a good decision, and I eventually dropped back to one of the earlier working versions for the demo.

We got to demo the concept, and explain what we would have liked to have done. I accomplished my goal of learning how to code a basic robot, and learned a lot about the API. We were by no means the slickest or coolest app there, but we had fun building it.

We’ve got the start to an open source project that could eventually be used to match people locationally, and used for all sorts of purposes, and we got to see some of the challenges in building apps for a piece of software as new as Google Wave.

Team picture

From left to right above: Steffan Frost, Fannie Han, Rob Weaver, Andreas Koll, Bennett Fonacier.

Steffen created a really cool video over that weekend as well that you can watch at http://www.youtube.com/watch?v=DkmuBmBZkBo