world gone mad

Although the awareness of open source appears low, the lack of Internet-based apps poses an opportunity to bolster open source penetration READ MORE

I was trying to check out the new Zabbix beta's when I ran into a couple of problems. Mostly missing dependencies but also some bugs.

The big troubles start when trying to install the frontent.
It seems like php code shipped in the last release might well work when you are upgrading but it fails when you are doing a fresh install. As I never got a Setup Wizard.

The clue is to get the most recent code from their SVN repo svn co svn://svn.zabbix.com/trunk , that actually allows you to run the setup wizard.

If you are installing the latest RPM's they are also not really requiring all the dependencies you need to run the frontend.

yum/apt-get install php-bcmath php-pear

Will get you a step further.

Another thing is the new password policy.. it seems the Admin/blank is gone.
I have no clue what the new password is but you are encouraged to set it yourselve in the mysql database

md5('new_pass') or echo -n new_pass | md5sum will help you .

Big thnx to the friendly folks over at #zabbix on irc.freenode.net for helping me out earlier.

Next steps .. debugging autodiscovery :)

A lot of people have been waiting for this a long time - so I am very happy to announce that we will publish the first Alpha that runs natively on Linux end of September. We will release generic binaries and the 5.1 source tarball including everything that is needed to build yourself.

This will not be a feature-complete beta release yet, but we are trying to get the basic functionality working so everybody can start modeling right away. We spent most of the last few months working on updating the Back-End implementation to use the GRTv4 and to prepare our platform-independent mForms. I expect the actual Front-End port to make good progress.

Let me share two screenshots I just made on Ubuntu, our Linux distro of choice. The first one shows the MySQL Model page. The Sakila demo model has been loaded.

The second screenshot shows the EER Diagram. This first Alpha will still use the software rendering option but we are looking forward to enable the OpenGL pretty soon.

Keep monitoring the blog - we will publish new screenshots as we go.

(Author's note: not necessarily actually a practical idea. But fun!)So pictured here is a histogram of a moderately large set of random integers. Each vertical line represents the total number of entries at each particular integer. Since each number is made up of multiple random factors (10 different random numbers, each between 0 and 100, added together), the distribution tends toward a bell curve.So how did I build the graph? Excel? PHP? Nope. Just a MySQL query. ...

The city drastically cut the number of parking permits given to Education Department employees for parking around schools, to 11,150 from 63,212.

The Bloomberg administration is asking elementary school principals across New York City to give standardized tests in English and math to children as young as kindergarten.

Short Version: You can find a fantastic video here about bundling customized AMIs and registering them with Amazon so that you can launch as many instances of your new AMI as you want. The video is so good that I don’t bother writing out the steps to do the bundling (it would be pretty darn long). These are some notes about launching an AMI, customizing it, and mounting an EBS volume to it (the video linked above doesn’t cover EBS). Also, check out the ElasticFox tool which is a very good GUI for doing simple EC2 operations. Nice if you’re just getting started or doing some simple tests.

There are two ways you can go about creating a custom machine image (AMI) for use with Amazon EC2: You can create an image locally by dd’ing to a file, mounting it with “-o loop” creating a filesystem on it, and bootstrapping the whole thing yourself, or you can grab an existing AMI that will serve as a “good enough” base for you to make your customizations, then bundle the customized image.

I’ll be talking about the latter option, where you identify a “good enough” image, customize it for your needs, and save that as your AMI. Unless you’re doing some kind of highly specialized installation, or are a control freak, you shouldn’t really need to start from scratch. I was just building a test image, and wanted a CentOS 5.2 base installation.

Here’s the command you can use to browse the AMIs you have access to (they’re either public, or they’re yours):

$ ec2dim -a

If that command looks funny to you, it’s likely because you’re used to seeing the really long versions of the AWS commands. Amazon also provides shorter versions of the commands. No, really - have a look! The long version of this command is:

$ ec2-describe-images -a

Too long for my taste, but it’s nice to know it’s there.

So, rather than start from scratch, I grabbed a base image that was close enough for my needs, and customized it. It’s a 5.1 base image, pretty well stripped of things that I don’t need, and a few that I do, but that’s ok. I’d rather start with less than more.

So step one is to launch an instance of the AMI I’ve chosen to be my ‘base’. Simple enough to do:

$ ec2run ami-0459bc6d -k ec2-keypair

And that’s pretty much it. It takes a couple of minutes (literally) for the machine to actually become available. You can check to see if it’s still in “pending” state or if it’s available by running ‘ec2din’. Without arguments, that’ll show you the status of any instances you have pending or runnning. Once the instance is running, you’ll be able to glean the hostname from the information provided.

An important note at this point: Don’t confuse “image” with “instance”. For the OO types in the crowd, an “image” is an object. It does nothing by itself until you instantiate it and create an “instance” of that object. For sysadmins, the “image” is like a PXE boot image, which does nothing until you boot it, thereby creating an “instance”.

The reason I used “PXE” and “object” in the above is because of the implication it makes: you can launch as many instances of an object as you want from a single object definition. You can boot as many machines as you want from a single PXE boot image. Likewise, you can launch as many Amazon EC2 instances from an image as you want.

So, in the time it took you to read those last two paragraphs, your instance is probably running. I now grab the hostname for my instance, and ssh to it using my keypair:

$ ssh -i ec2-keypair root@<hostname>

Now that I’m in, I can customize the environment, and then “bundle” it, which will create a new AMI with all of my customizations. With the instance in question, I installed a LAMP stack, and a few other sundry tools I need to perform my testing. I also ran “yum -y upgrade” which will go off and upgrade the machine to CentOS 5.2.

One thing I want to do with this instance is test out the process for creating an EBS volume. The two pieces of information I need to do this are the size of the volume I want to create, and the “zone” I want to create it in. You can figure out which zone your instance is running in using ‘ec2din’ on your workstation (not in your instance). I took that information and created my image in the same zone using the ‘ec2addvol’ command. If you don’t have that command on your workstation, then you don’t have the latest version of the Amazon command line tools. Here’s the command I ran:

$ ec2addvol -z us-east-1b -s 25

To see how it went, run ‘ec2dvol’ by itself and it’ll show you the status of all of your volumes, as well as the unique name assigned to your volume, which you’ll need in order to attach the volume to your instance. To do the ‘attachment’, you need the name of the volume, the name of the instance (use ‘ec2din’), and you need to choose a device that you’ll tell your instance to mount. Here’s what I ran (on my workstation):

$ ec2attvol -d /dev/sdx -i i-xxxxxxxx -v vol-xxxxxxxx

Now you can go back to the shell on your instance, mount the device, create a file system, create a mount point, add it to fstab, and, as they say in the UK, “Bob’s yer uncle”. By the time I wrote this post, I had already shut down my instance, but here are the commands (caveat emptor: this is from memory):

# mkfs.ext3 /dev/sdx # mkdir /vol # mount /dev/sdx /vol

If that all works ok, you can add a line to /etc/fstab so that it’ll be mounted at boot time, but I haven’t yet figured out how to attach a volume to an instance at boot time. The mount doesn’t work if you don’t attach the volume to the instance first. You’ll get a “device doesn’t exist” error if you try it. Clues hereby solicited. I assume I could probably use ‘boto’ and some Python code to get this done, but doing the same with a shell script wrapper around the Amazon tools might also be possible — but I don’t know how reliable that would be, because you’re at the mercy of Amazon and how they decide their tools should present the data (and *if* they provide the data you need for a particular operation down the road).

So now I have an EBS volume, and an instance. The volume is attached to the instance, and I can do things with it. I’m testing some database stuff, so I copied a database over to the volume, which was now mounted, so I could just ’scp mydb.tbz root@<instance>:/vol/.’

Once my database is there, I can attach it to pretty much whatever I want, which makes it nice, because I can test the same database, and the same database code, and see how the different size Amazon instances affect the performance, which gives me more performance data to work with. For production purposes, I’ll have to look more closely at the IO metrics, play with attaching multiple volumes and spreading out the IO, and I also want to test the ’snapshot’ capabilities. It’s also nice to know that if I needed to launch this in production (there are no plans to do so, but you never know), I could upgrade the database “hardware” more or less instantly

If anyone has code or tools to help automate the management of all of this stuff, please send links! If I come up with any myself, I’ll most likely post it here.

Now that I have a customized AMI with all of my packages installed and my config changes made, I need to bundle this so that I can boot as many instances of this particular configuration as I want. An important note about bundling this *particular* image is that you MUST run ‘depmod -a; modprobe loop’ before bundling, since this process basically abstracts the manual process of bundling an image, which involves mounting a file as a volume, which requires a loopback mount.

The video I used to do the bundling is here, and if you can live through the disgustingly bad burps and chirps in the (Flash version) audio, it’s an excellent tutorial for bundling custom AMIs. While the process *is* pretty straightforward, it involves a number of steps, and the video goes through all of them, and it worked perfectly the first time through.

addthis_url = 'http%3A%2F%2Fwww.protocolostomy.com%2F2008%2F08%2F27%2Fmore-adventures-in-amazon-ec2-and-ebs%2F'; addthis_title = 'More+Adventures+in+Amazon+EC2+%28and+EBS%29'; addthis_pub = 'jonesy';

I was surprised to find yesterday when using MySQL 5.1.26-rc with a client I’m recommending 5.1 to, some information not seen in the EXPLAIN plan before while reviewing SQL Statements.

Using join buffer

+----+-------------+-------+--------+---------------+--------------+---------+------------------------+-------+----------------------------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+-------------+-------+--------+---------------+--------------+---------+------------------------+-------+----------------------------------------------+ | 1 | SIMPLE | lr | ALL | NULL | NULL | NULL | NULL | 1084 | Using where; Using temporary; Using filesort | | 1 | SIMPLE | ca | ref | update_check | update_check | 4 | XXXXXXXXXXXXXXXXX | 4 | Using where; Using index | | 1 | SIMPLE | ce | ALL | NULL | NULL | NULL | NULL | 13319 | Using where; Using join buffer | | 1 | SIMPLE | co | eq_ref | PRIMARY | PRIMARY | 4 | XXXXXXXXXXXXXXXXX | 1 | Using where | +—-+————-+——-+——–+—————+————–+———+————————+——-+———————————————-+ 4 rows in set (0.00 sec) mysql> select version(); +———–+ | version() | +———–+ | 5.1.26-rc | +———–+ 1 row in set (0.00 sec)

Sergey Petrunia of the MySQL Optimizer team writes about this in Use of join buffer is now visible in EXPLAIN.

I came across the OStatic Free hosting service that provide Solaris + Glassfish (Java Container) + MySQL.

They offer “Now you can get free Web hosting on Cloud Computing environment free of charge for up to 12 months. ”

The catch “accumulate 400 Points for participating on the site“.

A rather novel approach, you get 100 points for registering, but then only 5-15 points per task. The equates to approximately at least 30 tasks you need to perform, such as answering a question, or reviewing somebodies application. That seems like a lot of work?

In this case, Free has definitely a cost and time factor to consider.

Hi all,

The schedule for PyWorks has been posted! I’m really excited about three things:

1) there are some really cool talks that I’m looking forward to attending. There are a couple of sysadmin-related talks, AppEngine, TurboGears, Django, and an area I’ve been especially slow to move into: testing (I know, shame on me). There’s lots more so be sure to check it out.

2) the conference scheduling process is over

3) I get to meet a lot of people face-to-face that I’ve worked with in the past on Python Magazine developing articles, or interacted with on IRC, etc. One thing I like about conferences surrounding open source technologies is you get to thank people face-to-face for the sweat they poured into some of the tools I use regularly. Mark Ramm, Kevin Dangoor, Michael Foord, Brandon Rhodes, and a collection of Python Magazine authors will be speaking there, and other Python Magazine folks and generally familiar faces will be in attendance.

Enjoy!

For those still unaware, PyWorks will be held in Atlanta, Nov. 12-14, 2008. It’s sponsored by MTA, the publisher of Python Magazine, as well as php|architect. In fact, the php|works conference will be held simultaneously with PyWorks, and attendees of one are free to access talks in the other at will. There will also be a “center track” that will cover some more generic topics of interest to developers without regard to the language in use. Check it out!

addthis_url = 'http%3A%2F%2Fwww.protocolostomy.com%2F2008%2F08%2F27%2Fpyworks-conference-schedule-posted%2F'; addthis_title = 'PyWorks+Conference+Schedule+Posted'; addthis_pub = 'jonesy';

Russell is a hundred-some-odd-year-old shoemaker I've been ordering from for the past 15 years. In addition to the quality of the workmanship and materials, you get the simple, timeless pleasure of a hand-crafted, made-to-order shoe/boot. Options include any number of various soles, hides (including supplying your own!), styles, insulation, toe cap, steel shank, and other custom options. For me, the Vibram sole was perfect. But others might be after oil resistance, traction, silence, longer life, etc. For instance, I'm not a hunter, but for those that are, Russell also has snakeproof boots and options for preventing thorns/cactus. That ability to customize your sole/leather/style per application is great.

There's a downloadable instruction form for sending in your measurements, which they keep on record for a decade. I am very flat-footed and they were able to accommodate the necessary additional space for orthotics. Not that unusual, but I recommended Russell to a friend with extraordinarily narrow feet (especially for how long they are). They were able to create his size no problem; and he's since ordered a half dozen different pairs over the years. One other thing I'd add is they do repair work as well, so you won't need to toss them as they get really old -- and you get the added bonus of having someone who knows the shoe doing the work. I have both the Cavalier boot and the Buckle Chukka. They're not cheap, but the Cavalier boots I mainly wear now I've had for at least 10 years, and the pair has only gotten better with age.

-- Wrye Martin

Russell Moccasin Footwear
$360 (cavalier)
Available from Russell Moccasin

Related Entries:
Rocky Side-Zip Paraboots Steger Mukluks Leather Therapy

James Kobelius writes in Network World how the need for scalable real-time business intelligence will create a convergence of technologies centered on database virtualization:
"Real-time is the most exciting new frontier in business intelligence, and virtualization will facilitate low-latency analytics more powerfully than traditional approaches. Database virtualization will enable real-time business intelligence through a policy-driven, latency-agile, distributed-caching memory grid that permeates an infrastructure at all levels.

As this new approach takes hold, it will provide a convergence architecture for diverse approaches to real-time business intelligence, such as trickle-feed extract transform load (ETL), changed-data capture (CDC), event-stream processing and data federation. Traditionally deployed as stovepipe infrastructures, these approaches will become alternative integration patterns in a virtualized information fabric for real-time business intelligence." Kobelius makes it clear that this "virtualized information fabric" is an ambitious program that will be accomplished only over a number of years, but the underlying trends are visible now: for example, the convergence of distributed caches with databases, as evidenced by Oracle's acquisition of Tangosol, and Microsoft's recently announced Project Velocity.

This envisioned system contains so many moving parts that a new paradigm is will be needed to link them together. I don't think that databases are the answer. They elegantly handle stored data, but founder when dealing with change, caching, and the kind of replication problems you encounter when implementing virtualized and distributed systems. For example, database triggers are the standard way of managing change in a database, and are still clunky fifteen years after they were introduced; and Enterprise Information Integration (EII) systems were an attempt to extend the database model to handle federated data, but only work well for a proscribed set of distribution patterns.

I wrote recently about how SQLstream can implement trickle-feed ETL and use the knowledge it gleans from the passing data to proactively manage the mondrian OLAP engine's cache. SQLstream also has adapters to implement change-data capture (CDC) and to manage data federation.

In SQLstream, the lingua franca for all of these integration patterns is SQL, whereas ironically, if you tried to achieve these things in Oracle or Microsoft SQL Server, you would end up writing procedural code: PL/SQL or Transact SQL. Therefore streaming SQL - a variant of what Kobelius calls event-stream processing where, crucially, the language for event-processing language is SQL - seems the best candidate for that unifying paradigm.

OpenSQL Camp 2008 is coming! When and Where: Charlottesville, Virginia, USA November 14, 15, and 16 2008!

Organised by Baron Schwartz & others, and attended by loads of cool and interesting people (Brian, Monty and Baron are already on the attendee list!) you'd better get ready for a dynamite weekend of learning, contributing, and having fun! I'm hoping to be there too.

Some Key facts:

• It is of, by and for the community (you).
  
• At this event, all Open Source databases are created equal. We?ll learn together and grow together.
  
• It?s a combination conference and hackathon.
  
• It?s free.
  
• It is Friday night Nov 14, 18:00 through Sun the 16th at 18:00 in Charlottesville, Virginia USA in a very cool location.
  
• The website, where all details will be posted: http://opensqlcamp.pbwiki.com/
  
• The mailing group, where details will be discussed, decided and arranged: http://groups.google.com/group/opensqlcamp
  
• Date, time and place are confirmed and will not change. You can make your travel plans now. There?s travel information on the wiki.
  
• We already have some great speakers who have offered to give great talks. These and other details will show up on the wiki as they?re finalised.
  
• Sponsors are needed. See the wiki.

Despite the name, this will be different from other Camp conferences you?ve been to. This is a combination of a planned event (with great speakers and sessions), semi-planned spontaneity (sessions to be decided by attendees the night before), and a hackfest. It?s the best elements cherry-picked from all the conferences (and un-conferences) you?ve been to.

What should you do next? Go to the website and register yourself. Then go join the mailing list. And buy your plane tickets before they get too expensive. And tell your friends, and blog about it. See you there!

Sometime over the course of new arrivals’ first year in the city, they find themselves becoming New Yorkers.

Aka, colliding MD5, but in a very cool 12-way demonstration:
We have used a Sony Playstation 3 to correctly predict the outcome of the 2008 US presidential elections. In order not to influence the voters we keep our prediction secret, but commit to it by publishing its cryptographic hash on this website. The document with the correct prediction and matching hash will be revealed after the elections.Read all about it at http://www.win.tue.nl/hashclash/Nostradamus/

(yes I share my first name with one of the authors, but you'll notice that the last name, while similar, is not identical - it's really not me. I'm not that much of a maths or crypto whiz ;-)

A couple of article have been published recently that point to a growing realisation/admission about the role that open source will play in the future of enterprise software.

In “The Commercial Bear Hug of Open Source” Dan Woods details the various methods by which open source has become increasingly commercial in recent years, while in “The Microsoft-Novell Deal and Trust in Princes” Bruce Byfield discusses the relationship between business and open source.

Neither article is perfect. Woods, in particular, appears to paint open source in the role of the glorious failure - failing to surpass traditional licensing models and being subsumed into the mainstream (a subject I’ve touched on before).

For his efforts Woods earns the wrath of Dana Blankenhorn, who points out that Woods has confused the idealism of free software and Richard Stallman with the pragmatism of the open source and Eric Raymond.

“It [Raymond's 'open source' concept] accepted the idea of commercial interests from the start,” writes Blankenhorn. “It saw new business models evolving from shared development effort.”

For that reason, Woods asking “Isn’t open source a community-based movement that was set to overtake the world of commercial software? Wasn’t the famous LAMP stack, Linux, Apache, MySQL and Perl, Python and PHP going to open a world in which software existed outside the traditional realm of property?” is something of a straw man.

However, his central point that “commercial and open source are fellow travelers” is valid, as is his point that “it is almost impossible to tell the difference between the most popular open source software and commercial alternatives”.

That brings us to Byfield’s article in Datamation. While it is ostensibly a discussion about the extension of Microsoft’s Linux deal with Novell, the article focuses heavily on the relationship between business and free and open source software (FOSS).

“What tends to get lost is this: the fact that business is friendly to FOSS does not mean that it has adopted its values,” he writes. “The free software camp’s concern with philosophical and political freedom has almost certainly not been adopted by most FOSS-friendly companies, while the open source camp’s emphasis on increased software quality is probably shared by middle-management at best. Business –gasp!– is interested in FOSS to improve the bottom line, and often no other reason.”

This is a statement that leaves Matt Asay with mixed emotions. “Commercial open source is starting to find its feet,” he writes. “There isn’t a grand contradiction between giving code away (seeding the market) and suggesting a purchase (reaping the market). In fact, the two go together perfectly. As we realize this, open source will become even more dominant. Open source and business can be bosom buddies, not enemies.”

I would maintain that the relationship is closer than that. However, from Byfield’s viewpoint there remain barriers. “Sooner or later, an open source business is going to act more like a business and less like a citizen of the FOSS community,” he warns. “The two can certainly co-exist, and both can benefit from doing so. But, forced to choose, the average FOSS-based business is going to choose business interests over FOSS every time,” he adds.

The tone of Byfield’s article is driven in part by his evident belief that business and open source are at least partially exclusive. This, like Woods’ confusing of free and open source software, is a mistake in my opinion. Business and open source are intertwined.

It is worth remembering that open source is a business tactic, not a business model. Open source is not a market in and of itself, nor is it a vertical segment of the market. Open source is a software development and/or distribution model that is enabled by a licensing tactic. It enables new revenue generation strategies.

Companies that build revenue streams around open source software do not choose between business and open source, they choose business and open source. You could call this a commercial bear hug of open source, or you could see it as an evolution of commercial business models based on the economic principle that sharing development has the capacity to lower development costs.

Either way the result is the same - the increased adoption of open source as a development and distribution model by mainstream business. Which is, after all, what the open source movement is all about.

See also:
Commercial licensing is a double-edged sword
Asking the right questions of open source
Open source: assimilate and thrive
Judging open source business models

You will need:

• CMake (at least 2.4.7)
• Bazaar (the newer the better - 1.6 was just released - at least use that)
• Gnu Bison
• Visual Studio (Express works, but I’m talking about 2005 here)
• … and all this installed on a Microsoft Windows machine.
• … and to hate yourself, you are going to be using Windows after all.

Then, get and build it:

1. Get the source:
   bzr branch lp:~mysql/mysql-server/mysql-5.1-telco-6.4-win
2. Run CMake. the CMake GUI can now be used to select compile options! You’ll have to set the path “where is the source code” to where you put the source code in step 1.
3. Hit “Configure” in CMake
4. Select the target (i.e. the version of Visual Studio you’re going to use)
5. Select the build options. HINT: WITH_NDBCLUSTER_STORAGE_ENGINE may be a useful one to enable
6. Hit Configure again
7. Hit Ok.
8. CMAKE now generates the Visual Studio project. Use this time to drink some good scotch.
9. Open Mysql.sln (which should launch Visual Studio)
10. Go Build -> Build Solution (or hit F7)

Now you can go and have much whisky as this will take a few minutes. You should now have a set of built binaries for MySQL Cluster on Windows. Scary.

The recent disclosure that the owners of Riverton Houses, a 1,228-unit apartment complex in Harlem, might default on their loan has shocked the real estate industry.

After a gas explosion killed their mother in October, four daughters lean on their father as they slowly recover.

Bucking a national trend, New York City’s poverty rate declined in the year ending July 1, 2007, the Census Bureau reported, but the gap between rich and poor grew from the previous year.