Actually, a better title would be “What does the future hold for the real Webmonkeys?”

Tim Bray wrote up a piece about how his brother – some sort of tech manager – was stunned by the quality and quantity of resumes he got for a job posting for a relatively mediocre job: Just shows how bad the market currently is.

Don Box – a Microserf – apparently picked up on this thread and added his thoughts about this situation, which both Tim and I find interesting:

That’s an interesting and provocative take on it all, and I think it sheds a little light on what’s going on now, namely the following:

• The whole dot-boom was ignited by the propeller-heads that were doing this out of love of this sport. They had to do it. While the fuel that kept the boom rolling was $$ (VCs, dreaming of wealth, not bitwise operators), this also helped make it go “boom!” (note: it would have anyway; nothing is as good as the Net was hyped to be).
• A lot of people laid off from tech jobs are the very people who made it all happen: techies, desk support etc. The managers go last, and they have no clue outside a Visio diagram about how the company actually works. And a lot of the other people left – tech or not – came into this game for the $$ of it, not the fun of it. So even the tech people that are left are the passionless ones.
• A lot of people have gone into CS – as they went into Law or Medicine before – because it was a solid career move; you’d get out and be able to get a nice, well-paying job unless you were a real tool. How that has changed!
• And let’s be honest – unless you have a manager that is tech savvy and understands what makes a tech team work well (hint: not $$; yes, it helps, but…), who is a manager going to hire: The sharp-dressed, business savvy college graduate who sprinkles his sentences with XML, outside the box, pro-forma terms — or — the passionate nerd who grew up on MUD, builds his own whitebox computers and gets excited about cutting a tenth of a second off page load time? Passion loses; we all lose.

Just for fun – because I’m not – I recently rebuilt an ancient box I had Linux running on.

I wiped the hard drive and started from scratch.

I’ve installed Linux a fair number of time now (always from CD; still not a full-blown propeller-head), and so it was pretty much of a no-brainer.

The interesting part – to me – was getting the two databases I installed up and running: mySQL and Postgres.

Again, I’ve installed each of these at least three times in the past (mySQL on Windows, as well). But I never brought them up at the same time, attempted to get a “production type” system in place with each of them at the same time.

So the contrast was interesting.

Some observations:

• Ease of installation: This one is not even close: mySQL wins hands down. I’ve never had a painless Postgres installation – why do I have to build all these directories etc to start the database? Sure, if I want the database files somewhere else, I should be able to in a config file…why doesn’t this exist? mySQL, while less flexible, it more of a Windows-like install (you just install and it’s there, like where it is or not)
• Migrating data from existing databases: mySQL wins again. While there is the file import/export for Postgres – as well as pgdump – the tools out there for mySQL (I’m running SQLyog on Win2000) make migration a breeze. The tools that are out there for Postgres either 1) Suck, or 2) Aren’t up to the level of the mySQL tools (like SQLyog). That’s a problem.
• Security (locking down for Internet exposure): Postgres wins. The security model for mySQL – if you can even call it that is weak (GRANT….). I’m probably missing something, because I’m more of a SQL Server user than OSS database user overall, but … wow… weak. Postgres’ security is pretty damn good, which is to be expected from a database that is touted as an almost-Oracle RDBMS. Note: One of the reasons that Postgres is more cumbersome to set up is because of the security: You have to have a non-ROOT user running the server, recommended to add another user, set permissions on the database directories to reflect this etc. More work. More security. Depends what you need the database for.
• Ease of use (once installed and configured; data set): Let’s see. Postgres is ANSI 92 compliant, an ACID database, supports stored procs etc……mySQL lacks subselects (that’s my biggest beef with mySQL) and has weird – proprietary SQL (can you say CONCAT? sure, I knew you could…). Yes, Postgres wins hands down.

But – as mentioned above – it’s what you need the database for. Like Cold Fusion, mySQL is “not up to playing with the Big Boys…”, but how often do you play with the Big Boys?

Yeah, mostly it’s whacking together a quick solution to a problem. mySQL is much faster to get up and going for simple tasks.

It’s all about the best tool for a given job. My prejudices and preferences fall in the Postgres camp, but there are times when I need something simple and need it fast.

mySQL is there for me.

And yes, it’s all about choice, as well. And with these two out there – goosing each other to some degree – we all win.

OK, we all spend a lot of (..ahem…too much) time in front of a screen or in back of a rack of servers on a daily basis, and we begin to take a lot of things computer-wise (especially code wise) for granted. We just expect stuff to work, and when it doesn’t…well, of course it’s not user error.

I was just fiddling with this and that today, and it occured to me that I take this and that for granted.

What follows is just a short, incomplete, unordered list of things that just work – yet we never stop and thank them….

I’m a pretty patient guy, but having worked extensively on the three major platforms (Windoze, Apple, *nix) with a number of authoring languages (ASP, JSP, PHP…) and types of Web authoring tools (Perl, XML/RSS…), I can safely say that it’s a wonder I have any hair left.

Perhaps it’s good in the “job security” way that things are still not as transparent as they should be, but it’s tough to launch anything out there.

As my last entry noted, I finally whacked together an RSS feed. (When am I going to switch to either MoveableType or roll my own blogging tool [I have on in PHP running locally]?!?)

Fine. It works. Haven’t set the cron for it yet, but whatever. (Update: Set the cron.)

But it just is so frustrating to roll RSS out.

Nomenclature: Some call their files “[whatever].xml”, some “[whatever].rss”, some “[whatever].rdf”.

How can aggregators easily find these puppies? Must make them tear their hair out. And is there going to be a standard in the near future? I.e. all RSS feeds must be “[whatever].rss”?? Let’s decide now so there is no need for backward compatability in the tools that are currently coming out, and so authors (me) can avoid renaming/recoding. Makes sense to me…

And the whole RSS vs. RDF battle, the namespace nuttiness…

And on one of my (hosted) domains, I had to create a MIME type in the .htaccess file (show of hands – how many NT types even know what the heck a .htaccess file is? Hell, in Linux it’s now in magic.conf or httpd.conf – there is no .htaccess per se). Could I easily find the right type to set?

NOoooooooooooooooo….

I took the one from my Linux box; didn’t work. Googled a bit, found the application-type MIME setting; still didn’t work.

Oh, I have to rip out the encoding type from the XML header. (That was just a guess that worked) Then it worked (in IE; barfed in Netscape..)

Bleh.

Bottom line: The file was built fine; it validates and all that, but it behaves differently in different client-server combos.

This is never a good thing.

But the RSS feed is for aggregators, not for eyeballing in a browser window, so I should stop my whining, right?

Actually not: What about doing a site JUST with XML, with included XLS file? Will there be browser issues there? If so, this is a deal-breaker…

Ah well, for all the XML buzz that’s been generated over the last, what, four years, XML is still very much in its infancy. And in many ways, RSS is the first widely deployed XML tool. You’re seeing that little orange XML button all over the place (shouldn’t it really be RSS? Marked up in XML, but to the RSS spec[s]…).

As long as things are moving forward…

Well, after a good deal of Perl work, I think I have an RSS feed set up for this site.

The limitations are many – only applies to the current Index page (last seven days); Blogger does not have the concept of permalinks and so on – but, hey, it works.

Now I get to see if it works with this, the test entry. If it works, I set a CRON and see it I can keep it fairly current.

Note: I’ve already added it to an aggregator, and the damn thing works.

Yes, limitations. But hacking around them…

Many years from now (yes, in a galaxy far, far away or whatever…), I think a lot of us will look back on the last few years as a Golden Time on the Net.

In some ways, the “golden” is actually pyrite, but that’s often the case, isn’t it?

We are reaching the fulcrum of the Web’s use; shortly, what is now exciting/evil will either become normal or cease to exist – or, more likely, morph into something that makes it almost difficult to recall the excitement/anger the original stuff created/caused.

In many ways, we have crossed over the fulcrum – the Web, in it’s current incarnation, is a very user-friendly place. Like it or not, IE is the overwhelmingly dominant browser, so designers are designing for it in a consistent manner. (No, this is not a good thing, but it is consistent, which is good.).

The geekiness needed to navigate the Web is fading away. For the average user, even a dial-up connection is not too intimidating, and getting from here to there is becoming a no-brainer.

Other areas, however, are still firmly entrenched in geekDom, but this is starting to go away, as well. Some thoughts on what one finds/what defines the fulcrum of Net use:

So this is a fun time for the Web: Maddening (Windows updates/patches, worms), exciting (what will blogs become, if anything?), uncertain (I keep hearing about Web Services. Don’t see them. What should I learn? Java or .Net?).

And – at the same time – the Net is still much in its infancy. Well, perhaps it’s getting out of the terrible teens is more accurate, and trying to decide what it wants to do when it grows up (IPv6? Is Google God or Evil? How to solve the last mile problem?).

And for any of us with any interest or ties to Open Source Software, there is the delicious, Alice Down the Rabbit Hole drama that is SCO vs., well, everyone from the sound of it. (Read – daily – the coverage on GROKLAW – you can’t make this shit up…)

But I could be wrong. Maybe the Net is just warming up…

I could live with that.

Looking at my inbox this morning – about all spam, half of which was herbal/generic Viagra ads, the other half diet pills – I recalled the Jefferson Airplane song of oh-so-long ago:

My broadband connection was out all day yesterday (I was at job during the day, but coming home to … no Net).

Painful.

Didn’t realize how much I relied on it/was addicted to it (depending on your perspective/politics).

Back now.

Good.

I’ve finally gotten around to adding a blogroll to my pages; I’ve been procrastinating on it simply because I’ve been trying to find a hack that would enable me to not hard-code the list.

I have not been able to find that hack, dammit!

So my (incomplete) Blogroll is now out there.

Still, it doesn’t seem right to have it static….

I was just reading an (old) blog entry by Jeremy Zawodny about how Yahoo! (where he works) got Slashdotted.

OK, as the commenters pointed out, it was not Slashdotted – taken down by the deluge of requests brought on by a Slashdot post.

But that was Zaw’s point: He always reads about how sites get taken down by the Slashdot effect, and even though Yahoo! got enormous traffic due to the /. posted, it was no biggee.

For one box, running FreeBSD and Apache. (ONE box folks).

His points on why other sites go down with high traffic were just two:

1. Lack of sufficient bandwith (Obviously, not a problem for Yahoo)
2. Dynamic content

I totally agree with him, but the second point made me think, especially in light of my previous post.

For those of us with no lives (yes, we know who we are…), it’s common knowledge that one of the secrets of Yahoo’s stability and speed is it’s use of static content – the top page and other indices are rebuilt on remote boxes every [what time frame?] and pushed to the front-end Web servers.

So it’s all static content. Very fast.

Which totally made sense in the early days of the Web, with the 9600 baud (and less) modems, slower home and server boxes (WOW – a 166Mhz Pentium with 96M RAM! That must smoke!!!). Take away as much overhead – an/many OBDC request(s) – as possible and the site will be faster.

Yet this was, at the same time, the era of the static Web. While Yahoo! was clever in making as much of what would normally be a dynamic site static to improve overall performance, Yahoo! was the type of site – a search engine (more accurately for Yahoo!, an indexing site) – that was one of few that needed to be dynamic.

Back then, the only dynamic nature of sites was either via SSI or Perl CGI scripts. And they were used, generally, for things a static site could not do: Page counter, quizzes, creating pages for and storing info from form pages. (The one exception was the use of SSI for header and footer files, so site maintenance could be somewhat more manageable.)

Most sites were collections of static, hand-coded pages.

Need to change that header graphic? Search and replace all 415 pages, dude….

Then – sometime around the time of the dot.com upload – the concept of portals emerged as a business model that would make those companies/investors filthy rich.

The key of a portal was all about personalization: Ralph goes to Excite.com and gets one look/feel/content; Rebecca gets another. Same site.

Suddenly, terms like “sessions” and “database” entered the vocabulary of a lot of non-techie folk.

At the same time, hardware was getting faster, backbones were getting more robust, and connection speeds were getting up to (after the insane standards battle- K-Flex vs. [I fergit – X?]) 56K! Some folks even had ISDN, which was twice that rate. Whoo hoo!

Suddenly, regular – non Web-only – sites (think “The City of [your hometown]” Web site) startied either embracing the personalization trend to attract more customers/eyeballs (remember “attracting eyeballs”?) so they could, too, become filthy rich, and to provide a way to better manage the site.

For larger sites, true content-management tools came out (think Vignette); for smaller sites, it was a small developer/development group building a narrowly defined content management toolset.

Either way, a fundamental shift was underway: Static pages gave way to templates, which pulled – at the time of request – the appropriate data from a database to provide that page’s content (or portions thereof).

We have been moving in that direction ever since.

While the portal concept has died down (lives on, in a better positioned way, on company intranets), the concept of a database-driven Web site is the norm now (who da heck has a truly static, hand-coded site anymore? Home sites, sure, but few other types).

In fact, it’s gotten a little bit nutty lately, where you look at code for some sites (including those I’ve built….) and you see queries tossed around like rice at a wedding. A query to set this, to set that…but that’s another rant.

Basically, the Web is currently, overwhelmingly database-driven.

Which gets us back to Zaw’s second comment: Essentially, dynamic pages incur much more overhead than static pages.

Again, agreed.

Yet, with all this bandwidth and turbocharged hardware, do we really care about this anymore? And – even if we do – computers are going to get faster and more people are going to get broadband (which will get “broader” as time goes by…) and so on.

So, should we really care about this? I mean, when is the last time you really worried about file size, image loads and so on. Sure, that’s hard-wired into us now, but the things I see being done every day are things that would have been red flags not too long ago.

So should we care about static (fast) vs. dynamic (not as fast)?

I dunno.

It’s interesting, because at the same time that our hardware/connections are becoming so fast that upgrades make virtually no difference, I’m seeing a trend toward static publishing.

Rather, a Yahoo!-style publishing, where dynamic content is written out, and then served as static pages.

Case in point: Blogs.

Blogs are a typical content-managed solution: Templates are tweaked for visual representation; content is stored in a database. All content changes are made via a back-end tool that stores info in a database.

Very familiar, so far….

But here comes the twist: In most cases (let’s say, the default setting on most blog tools), the data is then written to static pages, which is what is ultimately served up by the Web server.

So while the site is dynamic in that there is databased content, the content the end user sees is not pulled from the database at run time.

Interesting.

But is it necessary?

OK, for tools like Userland Radio and the tool I’m currently using (basic Blogger), it’s necessary because the database does not exist on the Web server. Created in one place (with resident database), and then pushed to Web server as static pages.

Works well; makes it tough to integrate “run at run-time” tools, but, yes, snappy.

But – except for the tools that make it necessary to create static pages – are static pages really necessary?

Again, I dunno.