I have a suggestion for you

I’m a little bit behind the times. I was supposed to post something up here last week, but was so wrapped up in actually working on what I was going to write about that I completely forgot.

Gramene is going to be rolling out an autocomplete feature to offer suggestions to users sometime in the near future. You can sample the wonderfully suggestive goodness here until we go live.

The state of javascript in the world today continuously amazes me. I started writing javascript in the fairly early days. I go back to Netscape 2 with it and got seriously cranking developing things around 1997. I largely abandoned it in early 1999 when I got rather irate at the current implementations as I tried to write a clone of Pac-Man in it. I reached the point where Pac-Man would run around the maze and eat dots, but as I started to implement the ghosts I ran into a stumbling block – when the timers that controlled the ghosts were global objects, it ran fine. When they were attributes of the ghost object (so the ghost could move itself), then performance ground to a crawl. I got so annoyed at the arbitrariness of it all that I shelved javascript and didn’t look at it for a few years.

When I finally picked it up again, I was amazed to see that the current state of the world was actually all of the magical things that I’d been promised many years before. A consistent, easy to understand Document Object Model. The ability to independently update parts of the page. Dynamic pages that actually work. I was hooked again.

So. To Gramene. I had this bug kicking around for months but never took the time to delve into it. Conceptually it’s very easy – monitor the text boxes the user is typing in. Whenever they let go of the keyboard key, fire off an event to a handler. That’ll look at what’s typed in, and bundle it off to ship off to the server. The server will take that information and look it up…somehow…on the server side to get a list of suggestions. Those suggestions go back to the client and pop up in a little box below the text. Simple!

Mostly. At first I wanted to use Ken’s quick search routines and database, but that seemed like it might be slow for our purposes. And it didn’t have quite the information I wanted to see either. So him and I brainstormed on it and came up with a vocabulary list concept. We set up lists of vocabulary words for individual parts of the site – after all, if you’re typing in a text box on a Gene search page, you probably don’t want to see QTL information popping up. Ken was then nice enough to go populate these vocabulary lists for me. For those keeping score at home, we currently have 3,649,306 distinct vocabulary terms spread across 54 different vocabulary lists.

A few wrapper scripts and that part was done. The AJAX to hand off to the server side was straightforward. I’d written some AJAX routines for myself a few years back, and we integrated those into Gramene as part of the Help system. I needed to tweak a few options to make it operate properly in this case (for example – we only want to fire off one AJAX request at a time so as not to get suggestions back from the server out of sync), but it was basically done. The pop up box with the suggestions was just a DIV tag that was absolutely positioned slightly below the text box in question. We had a javascript function for that too. As a side note, finding the absolute position of a relative element on the page is still quite the nuisance, but I guess I can’t have everything I want yet.

I added in options for the user to disable suggestions, and then we were done! Time to blog it! This is when Ken fouled up the whole works by saying, “I’m having trouble autocompleting with more than one box in a form.” My response? “Crap.”

I had ever so carefully engineered and designed the thing and never really taken that into account. Because I wanted to keep the javascript simple to use, I set it up so that additional options for the script were set as hidden form elements. Which vocabulary list to use, for example. This worked fine when there was only one autocomplete box in the form. But once you had more than one, it blew up. After all, you might (as Ken did) want to define two separate search boxes which each use their own distinct vocabulary list. How would the server side script know which vocabulary item to use?

Further, how would it know which input box to look at?

I came up with 3 ways to solve the problem – the bad way, the good enough way (with some caveats), and the flexible way.

I skipped over the bad way, since it was, well, bad. I could tie the client side javascript and forms to the server side implementation. So that the functions the programmer calls in the browser do some particular parsing and analysis to submit the proper information to the server for the script to function. This is fairly easy to do, but introduced a tight coupling to the code.

Up until this point, the javascript and the server side were completely independent. It didn’t really care what the script was on the server side, it just handed off its parameters and got something back, which it stuck in a box. It could use the default implementation, or any other one. I felt that this flexibility was quite valuable – it allows us to easily deploy the code to our other probjects and it would allow us to easily open source it for anyone to use it, regardless of integration with Gramene. Maybe you have your own fast word lookup on your server. You shouldn’t need to change your code much (other than formatting the output) to use the results.

The good enough way was to extend the javascript to accept a lot of additional parameters. This way, Ken could hardwire in a bunch of values to allow the client side code to remain independent of the server. I set up a bunch of default values in the javascript that handle the usual case. You can even still pass in your values with hidden form elements (and are encouraged to!). But you can also send in all the options you need to your javascript. Voila! We have multiple forms!

I only consider that approach to be “good enough” because it has an irritating nuisance – what if the additional parameters we want to pass in are user configurable? Right now, it’s hardwired that one box searches the genes vocabulary list, and the other searches the QTL vocabulary list, for example. But it’s conceivable that we may want the user to choose which part of the site they’re searching, and then use that to populate the list. So no hardwiring values.

At that point, we need to analyze particular form elements and update arguments and hand around values. Or build new forms on the fly with carefully crafted definitions of what should be in them. Or any number of other nonsense like that. Or maybe something even more clever that I didn’t think of. Regardless, I opted to defer for another day until we actually needed the functionality.

As a result, after all that rambling about coupling and options and parsing, for most cases, it’s terribly simple to use.

<input
   type = "text"
   name = "search_for"
   id = "search_for"
   onkeyup = 'suggest(this.id)'
   onblur = 'hide_suggestions()'
   onfocus = 'show_suggestions(this.id)' />

And you’re off to the races. I always like it when software can be powerfully complicated on the backend, and completely hidden from the end developer so that most of the time, it’s trivially simple to use.