Welcome to the new Gallery website!

Gallery has a new website! It's been many months in the making and has taken a great deal of time from many people. We chose Drupal as our content management system and have transitioned all the data from our old website over to a shiny, new extremely fast server. The road to this new site has been long and rocky at times. Read more to hear the story of what we did and how we did it.


In the beginning...

The Gallery website has been around in one incarnation or another since about 2000. In the early days, it was just couple of HTML pages on an old server in a closet behind a DSL line. As the project started to grow and more people got involved, it was a drag to keep updating the HTML by hand so in December of 2001 we cast around and found PostNuke. A full-on content management system was far superior to simple HTML pages in that it enabled us to have many site administrators with varying degress of permissions. This lightened the load on our main developers and freed them up to spend more time writing Gallery code.

As the project grew, the website became more and more popular. Eventually we had to move it off the DSL line and onto a colocated server better suited to handle the site's resource requirements. As we made improvements we found that every time we reduced the latency on the website we received significantly more traffic, so optimizing the site became a major goal for us. By August of 2005 the site handled roughly 2.7M page views from 515K unique visitors. This is not an exceptionally large amount of traffic, but for a site where the majority of the traffic is interactive forums it meant that we had to continually tune the site to keep the latency down. In 2001 Postnuke was the best CMS available but as we started having to tweak aspects of it to do get what we needed we essentially created a fork of Postnuke that was very challenging to maintain.

One of the main problems with Postnuke was that much of our data was stored in the embedded best-of-breed phpBB forum software. When we originally started using Postnuke we were using phpBB1.4, which came with the app by default. But we rapidly hit limits in the early version (and had to regularly apply security patches) so we took the hit, installed phpBB2 using the PNphpBB2 module and ported all the forum data forward using one-off custom migration scripts. But even though phpBB2 was a significant improvement over phpBB1.4, we still had the problem that the forums data was essentially in a separate application that only looked like it was part of the same site. There was no way to have a forum post appear on the front page of the site, which meant that any comments on news stories added would wind up in a separate system from forum posts. Search integration was weak. Transient session ids would appear and disappear in the urls as you moved from Postnuke to phpBB2. The mods we applied to phpBB2 were very difficult to maintain through an upgrade.


Moving forward

General inertia kept us using Postnuke for several years during which time it had a harder and harder time coping with the load. We forked off a separate database solely for forum searches to keep them from locking up the main database. We routinely applied security patches and fixed little issues as they cropped up. Eventually we decided that with the next generation of Gallery we should put out a whole new website so we started looking around for alternatives.

When it came time to choose a new CMS, we wanted to make sure that we chose one that scale to handle our needs, was under active development, had a large and active community and was easy to customize. We wanted to make sure that if we did have to make changes, they could be mostly localized into the theme code so that we could avoid tweaking the framework code and winding up with another forked CMS. It had to be flexible enough that we could add in our own apps like our paid support system, many of which are only visible to a subset of our users or the development team. We gathered input from as many sources as we can and eventually narrowed our choices down to two four solutions: Xaraya, Postnuke, Mambo and Drupal.

Xaraya looked promising, but they had not hit their 1.0 release yet and we didn't want to be stuck in a bind if they were not ready when we needed it. The latest version of Postnuke promised the easiest upgrade path but among other reasons, we wanted to get away from having embedded applications like phpBB2 for our main content. We wanted very tight integration.

This left Mambo and Drupal and it was tough to decide between them. They both seemed to offer similar capabilities and features. Both had active development communities. Both had good scalability stories. Neither had an embedded Gallery2 story at the time, but we were reasonably confident that one would emerge over the months that it would take us to port the site. Both had achieved a level of stability and had a reasonable security story. As with all modern CMS, both were modular and had flexible theming systems. Mambo seemed to have a better forums system, but Drupal's seemed capable of handling our needs and Drupal's node-based forums gave us greater flexibility. It was Drupal's adoption by a few high profile sites that demonstrated that it really could fulfill our needs. That coupled with the fact that we personally knew several Drupal developers who could help us with the migration made us choose Drupal in the end.


The challenges

By far, the biggest challenge in porting was to move over all of our data. Because the Gallery website contains a great deal of information in the forums and has been around for a long time, there are many internal pages and external sites that link deep into the site. We wanted to be sure that we not only preserved all that data but to save all the links also. It would have been very easy to sacrifice these internal and external links or even start with a blank slate, but it would have meant that we'd be severing ties with the history of the project. There are still useful links out in the wild that point to our old phpBB1_4 installation so we wanted to preserve our history in one place, so we aimed to keep as much as we could.

We didn't have time to do this ourselves so we hired Kitt Hodsden on a contract basis to do the data migration for us. Her goal was to figure out how to move all of our data over to the new site in a way that would sacrifice as little as possible. At first glance this did not seem like such a difficult task but the details of the migration were extremely tricky and demanding. In order for us to have a useful beta cycle, we required that she automate the entire process so that at any given time we could run the script and it would blast the beta site and recreate a brand new one by importing all of the data from Postnuke again. Our data is diverse, our links are many, Postnuke and Drupal are dissimilar -- this was a complex task and it took Kitt many months of effort to get it to a point where it was 80% migrated. This also included writing several custom modules for us and migrating over the data for them as well. A secondary goal of this project was to fund a way for Kitt to create a migration path from Postnuke to Drupal that she could then leverage for her consulting firm. There were many, many niggling details to resolve but in the end we accomplished a push-button migration script that let us get a beta up to find and resolve all the remaining issues before going live.


We hit some snags

Partway through the process of porting to Drupal we started reorganizing our documentation. In order to make sure that we were using the right tools, we started experimenting with MediaWiki as our repository on a second server. This turned out to be a far better tool than the wiki implementation in Postnuke or Drupal and it did not take us long to collectively decide to migrate our data over there. This is when we discovered one of the downsides to Drupal -- it does not have a good MediaWiki integration story. By that point Kitt was halfway through porting to Drupal and we were committed. But this left us in the unfortunate bind of having two loosely linked websites without a clean integration.

During this process, James Walker got the Drupal/Gallery2 integration module far enough that we could use it almost out of the box on the new site. With a little work we added the features we needed (like having the random image block in the sidebar) and sent those changes back to James for inclusion in Drupal's CVS repository.

The Drupal forums module was not quite as good as we had hoped. Drupal follows a post-and-comments model which makes some of the nice forum moderation options like splitting certain posts off of a forum topic into a new topic very difficult. The default threaded comment presentation style did not mesh very nicely with our forum ideal. For a site that is heavily forum based, this was not promising. However, some investigation of the contributed Drupal modules turned up code that could be massaged into doing what we needed. It was not ideal, but provided enough functionality for us to get by.


Making it look good!

Mike Harding's Logo

As Kitt started getting close to a point where she had a working data migration, we started focusing on creating a new theme for the site. We wanted a light and welcoming theme that would provide our users with intuitive site navigation. To make this happen, we hired Trae McCombs to design mockups and create a prototype of a Drupal theme for us. He created the overall look of the site around Mike Harding's winning submission in the 2003 Gallery Logo contest. Trae then translated that into a PHPTemplate based Drupal theme based on Lincoln's Revenge. With some design help from Chad Kieffer, we put the finishing touches on it to create a theme that gives us a warm fuzzy feeling when we look at it.

You may notice that the orange feature box on the front page looks very similar to the one found on the Drupal.org theme designed by Steven Wittens. That's because it's actually a placeholder for a new feature box. During the theme design process, Bharat copied the Drupal.org feature box as a proof of concept to demonstrate the kind of design that he'd like to see. It was never the plan to actually go live with it like this, but with the time pressure and the last minute confusion of releasing the new website and Gallery 2 on subsequent days, this item fell off the todo list and we went live with it the way it was.

The feature box is a very important part of the website and clearly we did not want to lose it. But since we very clearly violated Steven's copyright we needed to replace it with a more permanent solution. Luckily for us, he very generously offered to perform contract work for us to replace the current feature box with something new and unique for Gallery.

We achieved our goal.. Now what?

Our goal was to launch the new site before we release Gallery 2.0 so that we kick off the new product with a whole new brand. It took a tremendous amount of effort from several people but we're very proud that we've made it happen. There are still some outstanding issues we need to resolve, like figuring out how to integrate our MediaWiki data into Drupal in a sensible fashion and exploring other ways to improve our forums without forking Drupal.

Our journey with Drupal is just starting. Now that we've got a shiny new website we're going to take good care of it so that it will last us for many years to come! We hope that you're as happy with it as we are!

A very nice, clean and modern site. Great job guys and girls. I'm very interested in seeing whether the details on migrating from your current setup to drupal will be published in the near future. :)

frederik.kunz's picture

Nice. You might want to re-check the background color of the three aligned screenshots, it's not exactly the same as the box surrounding it.

WOW! Very nice. Very clean.<br />
<br />
<em>AND IT WORKS!</em><br />
<br />
Ok.. are you going to share? I've been burning the midnight oil trying to get Drupal and G2 to talk to each other. Got them working up to a point.. it just doesn't show any images. :(<br />
<br />
So.. What version of G2/Drupal/gallery.module are you using.. did you have to do some custom coding?<br />
<br />
At least I KNOW they will work well together at some point.<br />
<br />
-Matthew

see http://gallery.menalto.com/index.php?name=PNphpBB2&file=viewtopic&t=23957<br />
for the existing integration.<br />
<br />
to my knowledge, bharat had to change a few things in the existing integration. i guess the changes will be merged into the downloadable integration at some point. don't know when yet. i guess, as soon as someone (bharat, walkah, ..) has time to do it.

This might give me a boost to convert all my phpnuke website to my new (beta) drupal website too! :)

Spockman's picture

I was hoping this would happen soon! The old is too old and was feeling rather dusty and arthritic.

dkgibson's picture

Orange on white text is hard to read, especially on large (1600x1200) screens typical of heavy graphics users, where the fonts already appear smaller than on a 1024x768 screen.<br />
<br />
Go for high readability!<br />

Not only is the new site lovely, but Gallery 2's functionality is stunning. I've used Gallery for years now, and I love Gallery 2 more and more every day. It's so easy to customize it makes WordPress look difficult. I got it set up in 10 minutes flat, and that included porting over 4000 photos with comments and descriptions. It took me 20 to get Wordpress installed! I can't say enough about Gallery 2!
Heather
www.crickie.com

I am more curious about how you transferred over all the phpBB accounts and forum data into Drupal. Can you share please?

thumb's picture

ewerx, take a look at this migration script on Drupal's site. Not sure if this is what Kitt used, but it looks like it'll do the job.

The search feature is broken :s

It looks like the search tool for the forum is broken in the new version... When I am searching "RSS" it tells that it yielded no topic... This worked previously !

very impressed - much more user friendly, brighter and cleaner :)

edit: When using IE6 to post comments/edit them the reply box is stretching to span the entire width of the screen. Using firefox this isn't happening...

http://www.gonewalkabout.info

stephenju's picture

Nice. Very nice. And it's not made of plastic. :)

Simply awesome. It is true: a warm feeling it gives you :-)

Ladies and Gentlemen

You have any reason to be proud of your work !

The new Gallery 2 web site just looks great and Gallery 2 itself is a real success story !
Don't know where you find all the time to work on these projects :-)

A BIG thank you to all the people envolved. Developers, new Web-Site team, data migration, just to all of them :-)

Regards,
Oliver

Arlo's picture

The new site looks great, but it needs a minimum width, say about 800px. Making the browser super narrow makes the site look pretty lousy :)

stephenju's picture

I can't find any link to report bugs on this site itself, so I'll just post it here.

The link to Localization Report is now 404:
http://gallery.menalto.com/modules/GalleryDocs/g2/localization.php

foremind's picture

On Drupal forum:

> However, some investigation of the contributed Drupal modules turned up code that could be massaged into doing what we needed.

I am thinking of converting some sites to Drupal but the forum is a show-stopper. The new Gallery forum seems to be okay though. Where can I find out what was done exactly?

I'm intending to start a new site using Drupal and Gallery2. Would the current gallery module, released on 15/04/2005 http://drupal.org/project/gallery work? Or is there anything extra I have to do to ensure gallery2 integrates well with Drupal?

Also, one other thing, how can I tweak the native Drupal forum to make it look like the one on this site?

Grab the Drupal Gallery2 module from cvs. Hints, as seen in other posts:

Install Drupal
Install Gallery2 & Configure as stand alone
Install Drupal Gallery2 Module

Forum: They have modified the "blocks" under adminstration->blocks->"Page specific visibility settings". For each block, they told it not to show for "forum*" which hides that block from those pages. The "look" itself is the the theme based, see ($DRUPAL_ROOT}/themes/{$YOURTHEME}/* style.css and xtemplate.xtmpl are key files.

I used a PHPtemplate that came from a drupal forum.

The genius is someone named farsheed and here is his instructions
http://www.drupalart.org/phpbb

I used it on my site and it has "buttery goodness!"

And can be viewed here http://www.gecko13.com/forum

Enjoy and I hope that helped.

What theme is the new Gallery site using? Think it looks awesome!

jamgold's picture

Drupal is a very good choice, IMHO, and I really like your new design as well. Much better and cleaner than the old site. Kudos.

Jan

Lunette's picture

I like this, much faster load and easier navigation.

You knocked it out of the park on this one. I just keep looking at the theme and my jaw is still on the desk. Great job.

I've converted one of my sites over to Drupal already and really love the flexibility. I'm considering changing another site over that uses dozens of Gallery installs and B2 blogs.

You've inspired me.

Mark