The WGBH.org Development Blog

As I write this, my buddy and co-developer Pete Bull is gallivanting around Hawaii, enjoying a well deserved vacation in paradise. So, until our own Don Ho gets back next week - no doubt tan, refreshed and mellowed out - I’m on my own coding up the new version of WGBH.org.

We’re making good progress with the new TV programs and schedules module. As mentioned earlier, the Programs A-Z list, schedule grid and full day by channel schedules are pretty much good to go. I’m currently up to my neck coding up the episode pages.

The big news as of late is that we’ve decided to forgo program pages (or, rather, series pages) and go only with episode pages. I’ll have more on that in an upcoming post. That, folks, is what they call a cliffhanger.

I’d actually like to take today’s post to introduce you to the newest member of the WGBH Online development squad (I prefer to think of us a “squad” more than a “team”; please note there is still no I in “squad”).

This member joined the team just about a month or so ago and has already proven to be an impact player. This member has also proven to be an invaluable cog in our machinery. I’m already having a hard time remembering what life was like before this member came on board.

So, without further adieu…

allow me to present…

the newest WGBH Online technical squad member…

my new MacBook Pro!

After years - actually, technically, decades - of working primarily on a PC (mainly out of convenience, not any religious thing), this little cupcake worked her charms on me and has caused me to switch sides!

It’s not that the MacBook is making me any more efficient; the tools aren’t all that different, in many ways. My main development tools went from Macromedia HomeSite, Firefox, and PuTTY to ActiveState Komodo, Firefox and the Mac terminal.

What it comes down to, I suppose, is that the thing is just so dang … sexy. I just want to use it! Silvery smooth, light weight, a keyboard that’s a delight to tap, it’s got all sorts of sensory goodness going on. You gotta give Steve Jobs and his squad mad props (whatever that means): they sure know about industrial design.

Anyhow, the MacBook has become my new official workstation of choice. I really like it. In fact, truth be told, I’ve become quite attached to it in a very short amount of time. Actually, I think I’m even developing feelings for it.

Ahh, heck, who are we kidding? I’m in love!

OK, time for a meeting, which means I must leave my MacBook (ever so briefly) … I’ll miss you sweetie! XOXOXOXOXO!

As you regular readers know, we’ve been struggling with a little memory problem here as of late. Basically, under several different circumstances, PHP would quit and tell us it had used up all of its allocated memory, which we had jacked all the way up to 512MB.

At first, I suspected it was due to some server configuration issues, since we had just built an entirely new environment from the ground up. After a while we figured out it was definitely a code issue, and possibly more than one. After examining our own code, poking around contributed module code and some trial and error, I hadn’t been able to pinpoint the problem. Things were starting to look … worrisome.

But, as they say, it’s always darkest before the dawn for the other day what to my wondering eyes should appear but a comment on this blog from Moshe Weitzman. Moshe is one of the original Drupal developers and remains one the key core and contributed module maintainers to this day. Apparently, he is also a local resident and WGBH fan.

He’s also a very nice guy.

Moshe had discovered this blog and offered up his considerable help. Right away he fixed a memory leak in the Devel module and released a new version, which solved our problem of sometimes running out of memory when invoking the theme editor.

But he wasn’t done helping us with just that tidbit. Oh me oh my no.

I gave him a rundown on our problem of running out of memory when doing our nightly TV schedule data import and he was able top quickly suggest some possible culprits. Sure enough, after some tinkering around, I found that the problem went away when I disabled the Pathauto and Token modules. I tinkered with the FeedAPI cron function to disable these modules at the start and reenable them at the end of the process. As a result, memory usage during the import of our schedule data dropped by ten-fold.

Whew!

This fix, however, did introduce one new wrinkle: we used the Pathauto module to set URL aliases for new TV program and episode nodes that are created during the nightly import. By disabling Pathauto, I then had to write my own bit of code to set these aliases during import. Not a huge deal and, really, quite a small price to pay.

The bottom line of all this is that I am now sleeping just a little bit better each night and we’ve been able to ratchet down the maximum amount of memory assigned to PHP from 512MB to 128MB.

Thanks again, Moshe!

This week brought good news and bad news. Actually, more like very good news and pretty annoying news. First, the very good news:

We have officially posted some pages for internal testing! Behold…

TV Programs A-Z (click to enlarge)

Nothing too fancy here. On our current site the A-Z list is a popup. But now it’s a full grown page! One note: the Search TV Programs form does not yet work, on this page or any other.

TV Schedule Grid (click to enlarge)

Now we’re talking! The grid is the big magilla of this project. It finally allows us to display schedule information for all of our channels at once. Basically, we’re finally catching up to the rest of the world.

As you can see, the grid displays schedule information in three hour blocks. The user can navigate forward or backward or jump to a specific block of time using the Pick a Time form at the top. There is also the calendar selector, which lets users view the schedules for a given day. Note how the calendar highlights the current day (or the day of the schedule that you’re looking at), as well as the schedule data window, the period of time for which we display schedule data which, as of now, is one week week back, two weeks forward (almost).

We still need to play around with limiting the number of characters in the program or episode title that we display on the grid. There’s always something…

Full Day TV Schedules by Channel (click to enlarge)

As you can see, the full day schedule shares the calendar selector with the grid, and replaces the Pick a Time selector with a Pick a Channel form. Nice!

Now we can proceed with some preliminary testing, while Pete and I get to work on the program/series, episode, search and other pages.

Ok, on to the pretty annoying news. This… (click to enlarge)…

…is still happening.

Under a couple of different scenarios, the underlying PHP process uses up its allotted memory and then - like one of my kids - holds its breath and refuses to continue until is gets what it wants (more memory!). The above error was generated simply by trying to enable the theme developer. It can also happen during our nightly schedule data ingest, though our current allocation of 512MB is enough to prevent this, thank goodness.

So far all I’ve been able to confirm is that it’s not a server configuration issue. It seems to be a code leak and there may be more than one culprit out there. It’s starting to give me real headaches and needs to be resolved in the not too distant future.

However, for now, I refuse to let it ruin my weekend!

Been a low-key week here at WGBH Online World Headquarters, or as we call it WOW HQ.

The big event of the week: I got a haircut. All that goofy hair on my head was making it hard to think properly, so I had quite a bit of it lopped off. Things should really start picking up now.

Pete and I have been finishing up our first pass at the TV schedule pages (a multi-channel grid, full day schedules by channel, and a Programs A-Z list). Doesn’t sound like much, just three pages (basically), but there’s quite a bit involved, as you might imagine.

The plan is to get these pages in shape and then post to our test suite for a preliminary round of testing. This testing will be done by people other than Pete and I and the purpose is really just to evaluate how well the data is being imported from PBS. We haven’t worked up a formal test plan yet, but it will most likely involve comparing the WGBH schedules on PBS.org (which use the exact same TV Guide data) to the schedules we’re producing with the new code. We won’t yet be testing the ability to modify this information locally.

Needless to say, testing is just an annoying formality. The odds of there being any bugs in our code are minimal.

I’ve also been working with our new project manager Louise (Hi, Louise! I know you’re reading this…) to help fully flesh out the project tasks and schedule. That’s been a long process but I think we finally have all of the pieces laid out and scheduled.

In addition to the schedule page testing, we’ve also blocked out a longer period of time for full functional testing after we complete all of the development for this module (i.e. schedule pages and program and episode pages), as you would expect. Again, that’s testing outside to be done by others distinct from the unit testing that Pete and I do during coding. We thought about trying to break out functional testing into a more discreet chunks, but given the relatively small size of this project, it didn’t seem to make sense.

OK, I gotta go wrap up these schedule pages. Don’t forget to call mom on Sunday! I mean your mom, not mine. I’ll handle her.

Happy May!

Hard to believe we’re already into the shortest (letter-wise) month of the year, isn’t it? After a fine week off last week to spend quality time with the kids (is there any other kind?), I’ve been back in the saddle working away on this little Drupal project we’ve got going on (think of it as the Big Dig of coding).

While I was away my friend Pete got himself fully immersed in the fun and made some great progress on implementing our theme. He’s spent this week working on the TV schedule grid. So far he hasn’t run away screaming. i’ll get him to write about that in a future post.

Once the post-vacation glow and feeling of calm and relaxation faded away (i.e. by 9:01am Monday), I was right back it. First task was to figure out why the Update Status module (which is now part of core) on my install wasn’t telling me anything about the contributed modules I had installed, like it’s supposed to. Well, turns out that if you are downloading this code via CVS, rather than a tarball - as we are - then, in order for update status to properly work you need the CVS Deploy module.

Who knew? Not me.

Anyhow, i deployed CVS Deploy and now Update Status works as planned. Using that, I went and upgraded a bunch of modules that had been updated whilst I was away. Most notably, the Views 2 module went from an alpha to a beta release, so I was anxious to see what was going on with that. In particular with the argument handling issue that I mentioned previously.

The Views 2 Alpha was completely devoid of the the Views 1 argument handling logic. So, to make it work, I jimmied and jiggered things around and did the following:

Set the view_php view attribute by hand via some code like the following

$view_name = 'tv_episode_airings_viewable';
$view = views_get_view($view_name);
$view->view_php = '‘;
$view->save();

Then, in order to have that code properly executed by Views, I added some old Views 1 code to views/includes/view.inc

// Execute the initial PHP code that a view can have.
if (!empty($this->view_php)) {
ob_start();
$result = eval($this->view_php);
if (is_array($result)) {
$this->args = $result;
}
ob_end_clean();
}

Bingo bango - that worked!

Now, in the Views 2 Beta they’ve added the ability to add some PHP code on a per-argument basis, which seems to be the way of the future (rather than one single block of code to handle all the arguments). However, currently, it provides functionality to set a default value for an argument or to validate an argument. I see no way yet to manipulate an incoming argument.

So, for now, we’re keeping things as is. Once Views 2 is officially released we’ll probably have to reengineer things a bit to handle arguments properly.

There’s one other big problem we’ve been wrestling with this week - memory, or lack thereof. Don’t recall if I’ve mentioned this before but during certain activities on the system PHP throws a fatal error, saying it has used up its alloted memory. It first showed up when I tried to run the nightly cron tasks, which mainly consists of importing the PBS TV Guide schedule data. Each night in the middle of this run that error is thrown. It also shows up when I try to use the Drupal 6 Theme Editor. Pete also ran into when trying to install the first version of our custom WGBH TV Schedules module.

At the moment we’re not really sure what’s up. In php.ini I’ve got memory_limit = 256M, which should be plenty of memory for what we’re doing. I’ve even boosted the allotted memory to 512M and we still hit the error. Stay tuned…

Last but definitely not least we’ve fully staffed up and now have an official project manager and designer for this whole project. Please join me in a around of applause for these new team members Louise and Tyler! Welcome to the party!