Gallery as a "platform" for web app

I need to build an app where images represent items that are traded between users. There will be mobile app (iPhone and Android) pieces that allow users to upload "items" (images, descriptions, and tags). Any Facebook user would be able to use these apps. Since a lot of the app is centered around images and image management, I am thinking of using a photo gallery app as the foundation. I assume I would need to write a lot of plugins to manage the tags and associations of items. I would need to be able to programmatically manipulate images and tags and associations with various albums.

Would Gallery 2 (or Gallery 3) be a suitable starting point for such an application or is that too far from the intent of Gallery to twist it into what I might need? I can give more detail about my requirements, but I wanted to start the conversation to get others' opinions.

If you think using Gallery as a starting point might work, should I look at Gallery 2 or go ahead with Gallery 3?

My earliest requirement is to be able to upload an image (and its meta-data) to an image repository, associated with a Facebook signin "token" and retrieve a URL link that can be embedded in a web page. I will need hooks so that I can plug in code to do things with the images after they are uploaded.

Later I will need to allow people to view each others images and tag, search, or "favorite" them in various ways.

What do you all think of this?

Thank you for your input,
Bill...