Obscene Art

As I mentioned in a previous post, I’ve been hanging out with a lot of mobile application developers lately. Now, I’ve never made a secret of my disdain for telephones. But seeing the things they’re doing with mobile technology got me to thinking. What’s a cool use for mobile technology that applies to what I’m doing every day with Webmin and Virtualmin? Well, it turns out it’s the same thing that gets me to spend $72/month on a Sidekick. Being able to administer my server, fix customer problems, and deal with email, while traveling…even if there’s no laptop in sight or WiFi in range. So, on our way home from dinner last Tuesday night, Jamie and I chatted about what keeps Webmin and Virtualmin from being useful on our phones. Technically we can login to Webmin via our phones (Jamie has a Treo and I have a Sidekick), the problem is that it’s so slow to load all of the graphics and deal with the frames and menus (not to mention clicking “OK” when any site with more than a few lines of JavaScript eats up all of the memory on my Sidekick and it can’t keep running them), that it’s faster to drop in via SSH (both of our phones also have SSH clients) and do it all on the command line. Despite the horrid little keyboard, this is faster than using the web interface.

So, we came to the conclusion that we needed a new theme just for mobile phones. A theme targeted to the most common operations, and stripped of everything unnecessary, so it would be fast despite the client being slow and limited. But, we’d also have to address the problem of showing that theme only when a mobile client connected, since we can’t reasonably make an AJAX-filled fully graphical theme degrade all the way down to tiny. So, Jamie added some code to detect mobile browsers:

# is_mobile_useragent(agent)
# Returns 1 if some user agent looks like a cellphone or other mobile device,
# such as a treo.
sub is_mobile_useragent
{
local ($agent) = @_;
local @prefixes = (
    "UP.Link",    # Openwave
    "Nokia",      # All Nokias start with Nokia
    "MOT-",       # All Motorola phones start with MOT-
    "SAMSUNG",    # Samsung browsers
    "Samsung",    # Samsung browsers
    "SEC-",       # Samsung browsers
    "AU-MIC",     # Samsung browsers
    "AUDIOVOX",   # Audiovox
    "BlackBerry", # BlackBerry
    "hiptop",     # Danger hiptop Sidekick
    "SonyEricsson", # Sony Ericsson
    "Ericsson",     # Old Ericsson browsers , mostly WAP
    "Mitsu/1.1.A",  # Mitsubishi phones
    "Panasonic WAP", # Panasonic old WAP phones
    "DoCoMo",     # DoCoMo phones
    );
local @substrings = (
    "UP.Browser",         # Openwave
    "MobilePhone",        # NetFront
    "AU-MIC-A700",        # Samsung A700 Obigo browsers
    "Danger hiptop",      # Danger Sidekick hiptop
    "Windows CE",         # Windows CE Pocket PC
    "Blazer",             # Palm Treo Blazer
    "BlackBerry",         # BlackBerries can emulate other browsers, but
                          # they still keep this string in the UserAgent
    "SymbianOS",          # New Series60 browser has safari in it and
                          # SymbianOS is the only distinguishing string
    );
foreach my $p (@prefixes) {
        return 1 if ($agent =~ /^\Q$p\E/);
        }
foreach my $s (@substrings, @mobile_agents) {
        return 1 if ($agent =~ /\Q$s\E/);
        }
return 0;
}

So, there’s a quick and dirty perl function (BSD-licensed, like all of Webmin core) that makes detecting a mobile client easy. Of course, there’s more granularity out there than “mobile” or “big” client. That’s where something like WURFL will come in (that’s not gonna happen in Webmin in the short term, but it’ll come later, as our themes get smarter and more dynamic). Anyway, so now that Webmin can detect what kind of client it’s talking to, it can choose a theme to suit the target. We’ve made a new dead simple, zero-graphics, menu-based theme for Virtualmin, that allows really quick administration on even the most limited web client.

Now, as of version 1.328 of Webmin and next weeks version of our themes, you’ll get a different interface based on how advanced your client is. Small clients will see something like this:

While big clients will see one of the all-singing, all-dancing, maybe AJAX-filled, themes with images and such. Everybody wins, and now I don’t really need an SSH client on my next phone–this interface is faster, and if I really, really, need a shell, I can wing it with the non-interactive CGI-based Command Shell module in Webmin. The neat thing is that this will also work in Usermin, so we’ll have a kick ass webmail client that is fast and clean in our mobile devices. I’m glad I’ve been thrown in with a bunch of mobile guys lately. It’s an eye-opening experience.

As I mentioned in yesterday’s post Webmin gets even cooler when you have more than one system running it. The BIND module can automatically create slave zones. Virtualmin can automatically setup a hold and relay backup mail server. There are a dozen or so cluster category modules that allow management of users, groups, packages, etc. across any number of servers. And, to continue our theme from yesterday, you can have your servers keep an eye on each other to make sure things are running smoothly.

Because one of the things that could cause your web server to go off-line is an actual system crash, hardware failure, or network outage, it makes sense to run status checks of important services remotely.

So, let’s setup a couple of servers in Webmin and create a couple of monitors.

First up, install Webmin on both of the systems you want monitor (and perform monitoring on). Then login to Webmin and browse to the Webmin category, and click on “Webmin Servers Index”. Click “Register a new server”, and fill in the form with the hostname, port, and OS type. Then select “Login via Webmin with username” for the link type and fill in the username and password for a Webmin user on the destination server (root will work, but you can also make use of Webmin’s users, groups and access control list capabilities to create a special user just for service monitor purposes). Save it.

Now, in the System and Server Status module, you can add any kind of monitor that you can add locally for execution on the remote machine, and you can make use of some of the remote service check types (which doesn’t actually require you to have the system to be tested setup with Webmin). Webmin offers a simple “Alive System” check, that confirms that the system (and Webmin) are running, so it’s a nice one to have setup for each of your servers.

I’ve been hanging around with a lot of mobile application developers lately, and so I’ve been thinking about the value folks get from services being available all the time. And, over the years, I’ve been asked many times how to get a page or an email sent to a cell phone when something happens on a server (where “something” can be “the web server crashed”, “disk space is running low”, “I got an email from my boss”, or “we just started getting referals from reddit.com in the access_log”).

The great thing about this question, unlike some questions (like “how do I migrate my application to a new server with zero downtime”), is that I already have a good answer and it’s pretty simple to explain (the “how do I migrate” question also has a good answer, but it’s not simple to explain, even if you’re using Virtualmin and everything can be bundled into a Virtualmin virtual server backup). The simple answer, in it’s shortest form, is “Use Webmin’s System and Server Status module.”

But, people never seem to believe that Webmin has a lock on best of breed solutions to so many problems…so they’ve been trying to use a bunch of standalone status and monitoring tools and gotten frustrated by how difficult they are to install, configure, or customize. While there are standalone products out there, and some of them are even pretty good, the Webmin option is dead simple to install, crazy simple to configure, and does probably all you’ll ever need.

So, install Webmin, if you don’t already have it on all of your servers. Go ahead and do that now. I’ll wait. If you have Virtualmin Professional, you can skip this step. It includes all of Webmin (hiding under the “Webmin” tab in the left menu).

Ok, so you’ve got it on all of your servers now, right? Wrong? Seriously, go ahead, install it on all of them. You don’t need to “try it” on one. You’re gonna love it and it’s free, and it gets more powerful the more servers you run it on. We’ll get to the “more servers, more power” thing in another post.

Now, login to Webmin (port 10000 in your web browser…something like http://mydomain.tld:10000, or https://mydomain.tld:10000, if you have the Perl Net:SSLeay module) and head on over the Others category. Click on “System and Server Status module.

There’s a bunch of monitors already setup, by default:

Yours will look different depending on the theme you’re using. All of the existing monitors are popular services type status monitors–the green check means the test resulted in a “true” result (which, depending on the test could be all kinds of things..in this screenshot, they’re mostly “is this service running?”). The red X means the test returned “false”, and the red box with an arrow pointing up means the test is “not available” (which could occur if the service is not even installed, or Webmin doesn’t know where to find the configuration and PID for the service).

Creating service status monitors is so easy, I think we’ll skip right ahead to something interesting. reddit is the coolest user-generated news site around, so if you starting finding reddit in the referer string on your server, your business might be picking up (and who knows, maybe the media will start calling to find out all about your new Web 2.0 AJAX-enabled frobnoxilator). So let’s setup a monitor to let us know about it the moment it happens.

So select “Execute Command” from the dropdown list. Enter a description (this will be in the title of all status reports via email). I called this monitor “Reddited!” to convey the necessary excitement. In the check on schedule field, select “Yes, and report on status changes”, so it’ll tell us when we start getting links from Reddit and after that we’ll only hear about it when we go a whole day without getting linked from Reddit. Finally, in the “Command to check exit status of” field, enter something like:

grep reddit /var/log/httpd/access_log

Where the log path should be replaced with the correct path to the relevant Apache access_log.

Save it. Now, unless you’re already getting hits from Reddit, you should see a little red X beside the Reddited entry. Now, get back to work to do something or write something cool enough to get your site on Reddit, so you can turn it into a green check!

Ok, now that you’ve done something cool to test your Reddited! status check, let’s set it up to send you a message. Click on the “Scheduled Monitoring” button. Set “Scheduled checking enabled?” to yes. Decide whether the default “Check every” periodicity is frequent enough for you (five minutes is the default, and that makes me happy, but maybe you wanna know within two minutes). Set the “Email status report to” to your email address, preferably the one on your phone so you’ll always be on top of things, even when skiing in the Swiss Alps with a couple of beautiful blond Scandinavians. Save it.

It can also use a pager to notify you of changes, if that’s the sort of thing you enjoy…but everybody these days has email-enabled phones, so let’s stick with the high tech route.

All done! Now have a look through the rest of the types of monitors that can be setup–there’s dozens of them. And if there isn’t already a perfect match, you can do what we’ve done here, and write your own command to do the job. Just write it such that it returns true when things are going right, so you get a green check when good things happen. This could also be used to call an arbitrarily complex shell script, perl/python/php/ruby script, or any other command on the system. (Don’t forget that grep can return false except when something isn’t found by adding “-v” to the command line.)

For more on this module, see the brief coverage in chapter 15 of the Book of Webmin (my book), and the much more comprehensive chapter 28 of Managing Linux Systems with Webmin (Jamie’s book). The latter is available in PDF format only at this time.

Edit (March 13): Webmin 1.332 and above has added an “email address” converter for folks who don’t know their mobile SMS email address. It knows about most of the major providers, so you just punch in your phone number and select the provider, and Webmin will be able to send you instant messages.

For the past couple of weeks, I’ve been focusing on Webmin/Virtualmin, and how much nicer they could be with an AJAX face-lift. Well, we gave a demo of the product before a bunch of really smart folks last night (many of whom had already seen the ongoing UI work on Virtualmin), so I wanted to have something new to show. So Sunday night I hunkered down and began the port of the first AJAX-enabled Webmin theme to Usermin. By the time of the demo the next evening it was ready to roll (or, at least, ready for a demo in which I could carefully navigate around all of the new bugs introduced by all of the new JavaScript code). The funny thing is that I believe the theme in Usermin is somehow more immediately “cool!” than the much harder project of getting it working for all of Webmin and Virtualmin (which are still under way, and won’t wrap up for another week or two).

Webmail is one of those things that has become a fact of life for nearly everybody. Yahoo and Google raised the bar with their next generation AJAX webmail products, and we’re not quite there with Usermin. But the new theme makes it more obvious how great the core mail reader in Usermin really is. With merely a pretty and AJAX-ified face, it has suddenly become the best Open Source webmail available (possibly excepting RoundCube, which is admittedly awesome–but give me another week or two…I only have to write the front-end to an already robust and complete mail client, while they’re writing everything from scratch). Sometimes, even I’m surprised by how much amazingly useful code is in Webmin and Usermin. Jamie’s like a damned machine, just churning out thousands of lines of great software every year.

And, sure enough, at our demo, the mail client drew quite a bit of the attention. Everyone understands mail clients. It’s like the mythical bike shed. Maybe when all of this is done, we’ll roll up a special version of Usermin that is focused on webmail. I think that’s probably why it isn’t used more. Nobody knows to look at Usermin for webmail…so they end up with SquirrelMail, which is fine, but there are several better products out their to satisfy the need. (Horde also has an image problem…it has a great mail client, too, but it’s far less popular than Squirrel.)

The cool bit about AJAX is, of course, the A for “asynchronous” thing (what, you didn’t really think it was the X for “XML”, did you?). Sure, everybody and their brother is saying they’ve got an AJAX app because they’ve added a highlight effect or something wobbly on one of their elements. But until you’re whipping things around in the DOM to make the page content change based on the users behavior, and without reloading that page, you’re not doing AJAX. Me being an expert and all, having worked with JavaScript for all of a week, I thought I’d share with you our first real AJAX element in Virtualmin: hooking up a dropdown menu to the contents of a tree menu just below it. When a new domain is chosen in our domain selector, the menus below it update to link to the administration pages for that domain.

Being a perl coder, I’m lazy, so rather than wrap up my menu as a JSON object and fire it off for rebuild, I’m just grabbing an HTML menu and re-running the treeMenu function I wrote a couple of weeks ago in the beginning of this adventure on that div that is being replaced. This stuff is easy! I can’t figure out why I haven’t been doing it all along.

So, here’s what we have in the markup to specify what will be replaced:

<div class='menu' id='domain'>
...blahblahblah...
</div>

Hmm…that doesn’t look like a lot of markup. But it’s all I added for this trick, so it’s all we care about–I could wrap anything in that sucker and replace it with a simple JavaScript function.

So, now in the dropdown, we have an onChange function call:

<select name="dom" size=1 onChange="rebuildMenu('domain',this.form.dom)">
<option value="116468149219819" selected>domain.tld
<option value="116899265022946">fcgidtest.com
<option value="116897866018104">suphptest.com
</select>

It calls a function called rebuildMenu, which is this:

rebuildMenu = function(menu, id){
  // Load a new menu for id and insert it in div menu
  var div = document.getElementById(menu);
  var dom = id.options[id.selectedIndex].value;
  var sUrl = "menu.cgi?dom=" + dom + "&menu=" + menu;
 
  var handleSuccess = function(o){
    if(o.responseText !== undefined) {
      div.innerHTML = o.responseText;
      var elements = YAHOO.util.Dom.getElementsByClassName('treemenu', 'div', menu);
      YAHOO.util.Dom.batch ( elements, treeMenu );
    }
  }
  var handleFailure = function(o){
    div.innerHTML = "Loading new menu failed.<p>"
  }
 
  var callback =
  {
    success:handleSuccess, 
    failure: handleFailure 
  }; 
 
  var request = YAHOO.util.Connect.asyncRequest('GET', sUrl, callback);
}

And that is where the magic happens. Whenever the selected item in the dropdown changes, it fires up this function with two arguments, the menu to load (which doubles as the id in the dom to load it into), and the domain id for the menu. I’m using the YUI util library to make the request for me with the URL built based on the information sent from the onChange call. All very straight-ahead, though because I’ve got a YUI-Ext tree built from markup, I have to call that treeMenu function again on everything that got reloaded (or I’d have a plain ol’ UL there after the update). Surprisingly, it all works.

Next up, making the content pane update based on the selected item in the dropdown. This sucker’s gonna look like a desktop application before too much longer. Rich UI, here we come!