Zilla Smash!

Uploading lots of images with my new image uploader is fun, so I’m throwing up all of the backgrounds I’ve collected for my Zune. They’re mostly cropped versions of my wallpaper collection, with a few I’ve found around the Internet scattered here and there.

Image Uploading using YUI Uploader Widget

A few days ago the YUI team released the latest and greatest version of the YUI Library, YUI 2.5.0. Along with the usual round of bugfixes and speed improvements this release came with several new components. I personally was the most excited about the Uploader component, as the technology that powers it also underpins the upload process on Yahoo! Video. I didn’t write the upload pages but I’m expected to be able to provide bugfixes and enhancements to those pages, so what better way to learn how they work than from the ground up with the Uploader?

Yes, I know there’s quite a few better ways. This was the most fun sounding way. What can I say, I like 4 day projects in my spare time instead of just reading the existing code. Besides, I wanted a better understanding of it from a lower level than just how the upload code on VYC works.

As mentioned in the YUI Blog posting about 2.5.0, the Uploader is what not only powers VYC uploads but Flickr uploads as well. Flickr exposes more of it due to their allowing of multiple files uploaded at once. That’s not really very easy to do on VYC due to the much larger file sizes being used. Still, it’s there. I promise.

I’ve had a free-for-all, open to anyone image uploader chilling out for quite some time at http://tivac.com/upload/. I wrote it to solve a problem, namely the other image uploading services available at the time sucked. It was just a quick little thing, with some basic JS that would hide the form element and create a new one so you could upload more than one file at a time. When you were done it would barf out thumbs of the images along with some common types of link code for forums and the like. Nothing fancy at all. Well, the new YUI Uploader seemed pretty much tailor-made to work instead of creating new form elements.

Here’s a good representation of my thought process while I contemplated redoing the image uploader.

I think that paints a pretty compelling picture of why I’d go with the YUI lib over the original solution. Aside from requiring Flash 9 and not working on the latest OSX because Adobe and Apple are feuding, there’s really no downside. It’s provably better in every way. Since this is just a junky little personal project and not something important, I don’t even provide a fallback HTML form any more. That’s just how I roll.

Actual implementation was pretty basic. The YUI documentation is excellent as always. It more or less started life out as a copy of their Simple Upload Example and then I hacked the crap out of it. The flow ended up being something along the lines of the following.

1. Browse for image files.
   
2. If you selected one you don’t actually want to upload, clicking it’s filename will remove it.
   
3. Upload the files, watch all the fun progress bars whiz around.
   
4. As each file completes, its filename is replaced with a thumbnail and various different pre-filled link codes for HTML and forums.
   
5. You can also hit the Export button to just get a list of the URLs of the uploaded images.
   

It sounds complicated because I’m an engineer and don’t explain things well. The cool nerdy thing about this is how very event-driven it is. For example, take a look at this code chunk.

//certain things can only happen once the SWF is ready to rock

this.uploader.addListener('contentReady',   this.swfReady);

//event handlers

this.uploader.addListener('fileSelect',         this.onFileSelect);

this.uploader.addListener('uploadStart',        this.onUploadStart);

this.uploader.addListener('uploadError',        this.onUploadError);

this.uploader.addListener('uploadProgress',     this.onUploadProgress);

this.uploader.addListener('uploadCancel',       this.onUploadCancel);

this.uploader.addListener('uploadComplete',     this.onUploadComplete);

this.uploader.addListener('uploadCompleteData', this.onUploadCompleteData);

There’s an event fired by the flash object for pretty much everything you could want. The only event I found myself wishing it supported was a “allUploadsComplete” method. As it was I had to create a cache of all the file ids being uploaded, and in my handler for uploadComplete I would remove each id from the cache. If there’s no more files left to upload, hey presto we’re done! A little roundabout, especially when having to watch out for files that were selected and then removed.

The other fun thing I decided upon was that instead of using innerHTML and getting back big chunks of HTML from the server (wrapped in JSON, of course) I’d instead use skeleton structures hidden in the DOM. A quick

YUD.get('upload_skeleton').cloneNode(true);

and you’ve got yourself a nice chunk of HTML just waiting for some delicious data to be inserted. The actual insertion is pretty boring DOM traversals, but by using a skeleton as the base it avoids heavy node creation/appending or a massive innerHTML dump. The page’s DOM is a little heavier because it contains these stubs, but there’s only two of them and page weight isn’t a big concern for this project.

The progress meters are basically a total ripoff of Flickr, I feel a little bad about it but I really liked their approach of using a background image and just changing the x offset each time uploadProgress was called. It’s so simple and to the point that I didn’t see any reason to make it more complicated.

So that all works fine, but I was worried that if I didn’t copy the URLs correctly right away the images would be lost forever. That meant that I had to write a little PHP script that would output linked thumbs of all the images that had been uploaded. Since this is a totally unrestricted image uploader there’s been a few instances of people uploading porn and the like. I’d rather not deal with that, so there’s a little delete script as well. It’s protected by a .htaccess/.htpasswd basic auth setup. Nothing fancy, but it keeps other people from being able to delete my images.

The images are laid out in a big floating grid using Hedger’s very awesome work on getting display: inline-block to work across browsers. Item List Grid : Practice with display:inline block across browsers. It’s a technique we also used extensively on VYC so it was the first thing that came to mind when just floating a bunch of images that weren’t the same height went all crazy.

Oh, right. All the icons are the insanely awesome work of Mark James, specifically his Silk set of icons. I’m sure you’ve seen them used pretty much everywhere. There’s a good reason for that, they rock.

I’ve zipped up the source for anyone who wants it, it’s all pretty well-commented. You will have to handle the file permissions yourself though. UPDATE: Sitzmar wanted me to point out that all the paths are hard-coded for my uploader.  He says I should be ashamed, I say he should shut up.  Also I don’t think I included the .htpasswd (obviously) or the .htaccess.  Those are easily set up, a quick search on the internet will answer all questions.

upload.zip

As some of you may have noticed I was really excited when Yahoo! released their BBAuth tool right before their Open HackDay (My post about BBAuth). It seemed like a great idea, as tons of people already have Y! accounts. Little did I know that BBAuth wasn’t really a good solution as a single sign-on utility. It sounds silly for me to say that especially when the Official BBAuth Site says “BBAuth also offers a Single Sign-On (SSO) facility so that existing Yahoo! users can use your services without having to complete yet another registration process.” In the real world when I implemented Y! BBAuth as a SSO provider I ran into some issues.

First though, a quick overview of how BBAuth works. You set up an entry in the BBAuth DB for whatever site/webapp you’re going to use it with, providing some general info like contact details and the endpoint for when a user has successfully logged-in. Once that’s done you get a shared secret and an application ID. These are used in all future calls to hash info so that it’s reasonably secure and can’t be easily snooped along the wire. A special login URL is crafted that points the user to Y!’s login page, and once they enter their Y! credentials they are asked to give permission to whatever site for the info that has been requested. Once they agree they are redirected back to the site along with some encrypted info depending on what the site had asked for.

BBAuth works just fine, but even if you don’t ask for any Y! info your users still have to go through and agree to allow the site to use their info. My problem is then that I don’t want the user’s info. When I set up my site to use BBAuth I even told Y! that I didn’t want to use any info. All I want is Yahoo!’s promise that the user is a real Y! user and some sort of unique hash that my system can use. I’ll get the info I need for whatever app I’m running at the time, but none of what I do would be asking for Y! Photos info. I’m not saving any login credentials via cookies or anything, I just want a simple login mechanism that I don’t have to maintain. Clicking through one page doesn’t sound so bad until you realize that every time the user logs in they have to go through it. That’s pretty annoying and makes the jump to a Y! login page even more jarring.

There’s a post on the ydn-auth group by Jeremy Zawodny (Jeremy’s Post) that addresses this issue and says that this functionality is apparently a special case for BBAuth. Seems a little odd but I suppose that they were more concentrated on the usage cases for sharing a user’s Y! info with a 3rd party site. So they know about it and are looking into it, but until then I can’t really say I’m pleased with the actual implementation of BBAuth as a SSO provider.

After implementing BBAuth and being disappointed with the SSO performance, I began looking into other solutions. I really didn’t want to require users to sign up for yet another account because everyone hates that. It seems like there’s been a lot of buzz lately around OpenID (Technorati stats for “OpenID”). I figured I should at least give it a shot considering the large surge of interest behind it. First I had to figure out how OpenID actually worked.

Short Version: instead of a login that is some clever play on your name (Tivac lol) or something you identify yourself using a URI that links back to an identity provider. The identity provider stores your details and provides password authentication, so that when you visit an OpenID-enabled site you enter in your identifying URI and get bumped to your identity provider. You log in there, specify what details the site should be able to see from your profile, and then say “Ok” and get sent back. Note that this is what it’s like using MyOpenID.com because I was too lazy to set up my own half-assed identity provider.

So, how does OpenID work as a SSO provider? From a user’s point of view it’s pretty dead-simple. Looking at it as a developer adding it to an existing property proved to be considerably more work than I expected. I used the JanRain OpenID Library for PHP when working to get OpenID logins enabled on my testing site. Now I know the title of the article doesn’t say I’m comparing the libraries I used for each, but it’s still interesting to look at the differing amounts of code required for BBAuth and OpenID implementation in a fairly typical hosting environment.

The first thing that struck me as kind of odd was that I needed to move two folders totaling 33 files onto my shared path. I had forgotten that OpenID uses Yadis (Wikipedia page linked because the homepage at yadis.org has been vandalized) as part of the protocol. I didn’t realize that I’d need to be installing classes to support Yadis as well as OpenID. Once I had gotten all those files copied into the proper places and confirmed they were on my shared path it was time to look at the examples provided.

OpenID provides a decent example setup for both consumer and server roles, where consumer is providing the SSO login and server is actually running your own identity server. I only want to run a consumer right now, so I used just those files. After getting the form setup and trying it out I discovered that my PHP install has no big integer math library installed. This necessitates running in “Dumb Mode” which according to the documentation means my login form is more susceptible to relay attacks. Great.

After defining a new constant telling OpenID to run in dumb mode I was able to be successfully bounced out to my Identity Provider, sign in, and be bounced back to my site.

Avoiding any in-depth analysis of the security between the two as I’m not qualified to comment, which did I prefer implementing and using? That’s not actually an easy question to answer. BBAuth was certainly much smaller in terms of code required. It also has the advantage of using Yahoo!’s login system, it’s not exactly scientific but I don’t know anyone who doesn’t have a Y! account. So BBAuth has a huge built-in userbase right there. Both services require bumping the user out to a 3rd party login page so that’s a draw. I knew it was part of how both worked going in and it isn’t an issue for me in this case. Implementing OpenID had some bumps but thanks to the distributed nature of the project I don’t need to go sign my web application up anywhere. If I want to shift around page names I don’t have to remember to go update any entries in a Y! database anywhere. Any OpenID Identity Provider worth their salt will also provide a mechanism to say “Always Allow this site” which is something that BBAuth currently lacks.

In the end I wasn’t able to come up with a conclusive win on either one. BBAuth has that annoying extra page every visit but tons of people have Y! accounts. Implementation was also pretty quick and easy. OpenID took more work but with a decent Identity Provider will only require one page not on my site. It’s also got the whole distributed thing going for it. As it stands now I’ve implemented both and users can just choose their favorite. I’d prefer to just offer one but I don’t think there is one that offers a compelling enough featureset over the other to go with a single one yet. Going with both gives me flexibility and an excuse to keep playing with both of them.

I’ve been putting in a bit of time with the Yahoo! BBAuth stuff lately and while I don’t have it working quite like I want yet it’s pretty handy. I was previously using a homebrew user authentication setup for some of my tivac.com projects and while it wasn’t bad there was definitely some polish lacking. We’ll see if BBAuth solves my problem the way I want or if I’ll just have to put in some more work on my own system.

ResTek work is picking up some again finally, hooray for that!

Bitflux Blog :: Upload Progress Meter extension for PHP 5.2

This is pretty awesome, I’m all for not having to use the hacks that are out there currently to get a upload bar for users. Not sure where I’d ever use it at Restek but it’ll be just perfect at my Image Hosting.

I am still up because I just finished a homework assignment. Ugh.

I’ve been spending a lot of my time lately working with the CS department to try and hurry the hell up and graduate. I’ve been too lax in the past about not persuing the classes I needed and it has really started to bite me in the ass. I’m already taking one independent study this fall since the class is only offered next spring otherwise. This fall is going to be pretty busy, as I’ll be taking a full credit load while trying to get caught up on all the mayhem the other ResTek WADs have been causing in my absence. Judging by the CVS checkins I’ll be totally lost, should be a good time.

I haven’t had any brainstorms yet as to what I’m going to do this fall, I’ll probably substantially reduce my time spent on ResTek stuff to allow my replacement more work. Maybe I’ll take what I’ve learned at Y! and try and give Surveys an extra polishing. It could certainly use it based on what I remember of the code. Getting the YUI libraries integrated into all of our JS is also going to be a good challenge. While most of what we had was cross-browser, it was more by happenstance than design. Using YUI really helps with that since the libraries take care of all the nasy gotchas without needing any extra code.

Also holy crap there’s two more users now than I had before. Too bad one doesn’t even have any filters set up, oh well.

I end up writing a lot of code to avoid having to do in-depth searching for an already-built solution. I just wrote a basic cookied user authentication system for FeedFilter (and the rest of Tivac.com, I guess) because I didn’t want to go trawling through the MOUNTAINS of PHP code online that do just that.

I think in the end I made the right decision, as it was a pretty small problem and solving it myself means that I know the code inside and out and it does exactly what I need it to. There’s no extraneous code involved, although I will fess up to commenting more than usual. The reason for adding a user authentication system to FeedFilter is pretty obvious. When I first psoted it onto my blog someone went in and deleted my two filters to add some of their own. While I’m all for sharing information and letting other people see what I’m filtering I was a little put out by that. I was using those filters!

So all filters are now tied to a user, and the feeds they output are also uniquely indentified per user. So two users could name their filters the same thing and it wouldn’t be a problem. I’m still not entirely happy about the output arrangement I have. Currently all filter rss feeds are output to one folder that allows browsing. While I’m not particularly paranoid about this, someone else who may use this service (Unlikely though that may be) might not like that so much. Maybe I should do some sort of Google Calendar-esque solution. Maybe later. For right now there’s all of two people using this so unless it bothers one of us it isn’t likely to be changed.

Yes, shenanigans is overused. That doesn’t decrease my love of the word by any noticeable amount.

I’ve got my RSSDB (omg stolen name) updating and cleaning up after itself properly. This is important, as I was getting lots of duplicate posts and stale stuff previously. It’s now removing all feed items older than 1 month before updating. Obviously this might run into space/time issues if it became popular or something but that seems like an awful silly concern to me. I have no misconceptions about how popular any of the myriad number of random utilities I’ve created are.

In the past couple of days I’ve gotten my RSS filtering up and running properly. It’s doing all the things a good RSS filtering tool should do:

• - Adding a brand-new RSS feed to a filter will add it to the RSSDB and update it.
• - You can have multiple filters outputting to one filtered feed.
• - It has spiffy JS interactions using stuff that people much smarter than I have recommended, making the JS pretty quick.
• - Public & Private JS! (Nobody should care about this but it was a fun experiment.)
• - Silk Icons! Ok, this list is getting silly.
• - I’ve harnessed the power of Add To Any to allow easy adding of feeds to readers. That site provides one hell of a cool service. I approve.
• - XML files of feeds that have been deleted are now cleaned up nightly. I didn’t want a public-facing script doing that cleanup because people are jerks. This app is full of holes as-is, might as well not give people a hook into unlink() on my box.

All in all it is a start. I’m having fun writing it and am finding this incredibly useful. Now that I’ve got the RSSDB no longer deleting everything each time it updates (hey the difference between “< " and ">” is hard) things should be much smoother. Next on my plate is coming up with a quick and dirty global user system, I keep writing tools that require some sort of user auth and I’ve always either ignored it or come up with some sort of really awful solution. No more! Now I will have a really awful solution I can use globally!

Oh right, a link to the tool. Please don’t delete my feeds I like them and would hate to lose them. FeedFilter (yes it’s ugly shut up, form styling is a bitch.)

My internship this summer is a paid one, and while I’m making sure to save most of it having money brings along a certain temptation to spend it. To facilitate that I’ve been collecting RSS feeds of sites like fatwallet, techbargains, and the SA Coupons forum so that I can monitor them for good deals on the stuff I want. What I didn’t think about was just how much movement there is on these feeds. I set up a folder in Bloglines for all my “shopping” feeds and it quickly galloped to 500 unread items. I love reading feeds in Bloglines, but I don’t have time to even scan through that much just to find the one or two items I want!

So I’m going to do something about it. There’s a high probability something like this exists already, but it’s a fun side project to keep me from getting too rusty. I don’t get much DB interaction at the internship so doing some design and implementation work to stay sharp is a good thing. I’m working on a flexible RSS filtering system. You tell it the feeds you want it to watch, what you want it to watch for, and what you want to call it. Using a shared RSS DB/updater (I have a few other projects that deal with RSS I’m kicking around) this system will scan the feeds you told it to and build a feed of items that match the filter(s) you’ve set up for them. This output can them be consumed in your feed reader of choice, without turning a firehose of coupons and deals onto your face. Thinking about it, I actually already know something like this exists. Since I have a few issues with the implementation, I’m going to create my own less flexible version! The danger of stubborn people that can bang on a keyboard and have it do things…

If anyone’s interested in this I’ll let them in on the URL once I have something working. Right now it’s a DB schema and some forms that aren’t hooked up to the backend in any meaningful way. I had to get the central RSS DB up and running first though, and now that it’s running and updating all it’s feeds via Magpie I can continue with the filtering bit. Centralizing the RSS updating/storage is a definite Good Thing, as it means I only need one feed parser setup and won’t be grabbing duplicated feeds. Definitely gonna toss some YUI libraries into the mix for some oh-so-delicious JS, and I’ll probably try working on this using Aptana because it is shiny. SFTPDrive rules for this kind of thing by the way.