More Disk Space With LVM

Man holding two floppy disks
I’m starting to really like LVM.

See, the other day I had a problem with a virtual machine I was using, starting to run out of space. I’d only given it 10GB (well it wasn’t very demanding), and the OS auto-installer decided to divide this into ~6.5GB data and ~3GB swap. Sadly, it was a VMDK virtual disk rather than a VDI; VDIs are resizeable on the fly. I was faced the with thought of either cloning it to a larger virtual disk, or worse, rebuilding the whole image from scratch. Not having several hours to spare, I discovered that the OS had installed itself on LVM, and lo, I was saved.

First, I reduced the amount of swap used:

swapoff /dev/vg_clusterhost/lv_swap
lvreduce -L1.5G /dev/vg_clusterhost/lv_swap
lvextend -L+1.42G /dev/vg_clusterhost/lv_root
resize2fs -p /dev/vg_clusterhost/lv_root
mkswap /dev/vg_clusterhost/lv_swap
swapon /dev/vg_clusterhost/lv_swap

swapoff /dev/vg_boxhost/lv_swap

lvreduce -L1.5G /dev/vg_boxhost/lv_swap

mkswap /dev/vg_boxhost/lv_swap

swapon /dev/vg_boxhost/lv_swap

That little bundle of code turns off swap on my LVM partition, shrinks it to 1.5GB, and sets it up as a swap partition again.

Now with all that newfound free space in my volume group, I can stretch out my data volume:


lvextend -l +100%PVS /dev/vg_boxhost/lv_root

resize2fs -p /dev/vg_boxhost/lv_root

First, add 100% of the remaining free space in the physical volume to the logical volume, then stretch out the filesystem to fill the volume.

The whole thing took seconds rather than the hours I’d have needed to clone or rebuild. LVM, eh?

Image courtesy of nhoj leunamme, licensed under CC BY-NC-ND 2.0

Screen scraping and Web Real Estate

Man cleaning a window with a squeegee
As mentioned previously, I’ve started a new blog elsewhere. Still in its infancy, so there are only a handful of posts up yet.

I noticed early on that I was getting quite a bit of traffic though, even though I hadn’t pushed articles to social media; a blog with only one or two posts isn’t that attractive, so I figured I’d settle in and get the furniture arranged, so to speak, before seeing what sort of audience gravitated there. But the numbers were odd.

Continue reading

MySQL Blogging

I’ve taken up residence at https://blogs.oracle.com/jsmyth/ for all my MySQL bloggy goodness. I’ll still blog about all of the wide and varied other things here, because it’s entirely possible that the world of MySQL users out there don’t really care about Perl or how to configure Portugese Model M keyboards in Linux.

In the meantime, expect to see a bunch of different questions and answers over there that have cropped up over the last while in my MySQL course writing, questions from students and instructors, and interesting issues that pop up when challenges and conundrums face someone who isn’t a MySQL expert but has to work with it (hey, that describes about 95% of MySQL’s userbase, right?).

Remapping \ and | on US Keyboards in UK Layout

IBM Model M, US Layout

Lately I’ve started a small collection of IBM Model-M keyboards. Well, two. And a modern Unicomp Model-M that I somehow managed to murder. The problem with the original ones is that they’re not the usual layout, for me. One is a Portuguese layout, and the other is a US layout. Both have the problem that one of the keys I need isn’t available. The key in question is the one I’d ordinarily get “\” and “|” from. This key is at the bottom left of the Irish keyboard layout (and the UK layout, which is very similar), next to Z. The key does not exist on US layout keyboards. There is a key with those two symbols on it, but that key serves the purpose of the Irish “#” and “~” key, both of which are found elsewhere on the US keyboard layout. In short, we have an extra key here that US keyboards simply do not have.

Sorry about the long pre-amble. Here’s the meat: on my main computer (a laptop that travels around with me from desk to desk, where I have my various Models-M), I run both Linux and Windows. And, being a bit of a techie, I need both “\” and “|” to work for me, so I put them under the key to the left of the digit “1”, the one that usually does “`” and “¬” and “¦“. Here’s how I did it.

Short version.

In Linux:

 xmodmap -e 'keycode 49 = grave bar grave bar backslash brokenbar'

In Windows:

Download and run the Microsoft Keyboard Layout Creator, and create a new Layout.
http://www.microsoft.com/en-us/download/details.aspx?id=22339

Continue reading

Privacy and Encryption: Lose one, lose 'em all

Privacy: a photo by alancleaver_2000

Let me start with a little imagination. An analogy, if you will.

Imagine, if you will, that you are on holiday. You want to send a “wish you were here” type message to your friend back home. You have two choices: a cheapish picture postcard, or a more expensive sealed envelope. What do you choose? If you’re like the vast majority of us, you will likely choose the postcard, for both picturesque and cost reasons.

On the other hand, let’s assume you want to send your credit card number and salary details to an insurance agency. Again, do you choose the postcard, or the sealed envelope? I will not insult your intelligence by saying what the “right answer” is. I will however allow someone else to continue this analogy on my behalf, with a slight piece of satire:


the government has announced that it will soon be mandatory to use state-approved envelopes to send all mail. these new envelopes will be entirely transparent when viewed under a federally produced lightbulb, but there is no need to worry about these lamps getting out to bad people, since it is time-tested proof that all government employees are completely honest and lack all self-serving traits present in every other human being. besides, it’s for your own good and protection! and if you have something to write that you don’t want everyone to read, maybe it’s time for that all-important self-examination to reveal your underlying paranoia complex…

(quote taken from a Slashdot comment, written by someone called CrudPuppy)

How does that make you feel? If you honestly sat down and thought about that eventuality, would you be happy about the idea of someone, anyone, being able to read your most private thoughts and words? Is it fair to expect that once it’s written down, “The Government” — whoever that may be at the time — has a right to view it? The fact is, some governments are currently attempting to change their laws to curtail the individual’s right to this sort of privacy, whether you want to believe it or not. For example, the American Wiretap Bill prohibits using emails as evidence without a warrant, but this article shows that the authorities want to read parts of such communcations without a warrant. Continue reading

Fragmentation or Diversity?

'Diversity Clucks' by chrisjfry

In conversation with a client recently, we discussed the issue of fragmentation in the Java EE space, and how it impacts large organisations. Java EE (previously J2EE) is a set of specifications and standards, with no definitive IDE or application platform. This is by design, and is in contrast with .NET development, which (almost) invariably takes place in Microsoft’s Visual Studio IDE, and deployed to Microsoft’s Internet Information Server application platform runnning on Windows. Java EE systems may, on the other hand, be developed in Oracle’s JDeveloper, IBM’s Rational Application Developer for Websphere, IntelliJ, Eclipse, or NetBeans. Applications may be deployed to Tomcat, BEA Weblogic, IBM Websphere, JBoss, Glassfish, Oracle’s Application Server, and so on.

As an example, consider a large organisation with several development projects on the go at any time, each one started by a different owner/business unit, and potentially hiring individual contractors to get started on the bones of the project, from a spec they’ve created together. The contractor often chooses the framework based on their own experience, and so begins the tie-in for that particular project. As projects mature, their internal development staff take over maintenance of the project, and discover that they have to learn another framework, platform, or even IDE as they transfer between offices.

One solution employed may involve hiring the same large consultancy firm for many of their projects. This ensures consistency — a large consultancy firm will have internal development guidelines and standards — but is quite expensive, so far from ideal. Many organisations hire individuals to provide the expertise required to kickstart a project, and then add their own developers once the maintenance phase arrives. This is where the fun begins.

A modern Java EE application might use JSF or even Visual JSF. Or maybe Struts 2. Or perhaps Spring MVC. Each of these relies on a proper build process, so maybe we’ll use the IDE to build it. Netbeans. Or perhaps Eclipse. Or maybe an IDE-independent build tool, like Ant. Or perhaps Maven. Then we’ll deploy it to a suitable application platform. Glassfish, maybe. Or JBoss. It’s no longer enough just to learn the language; one must also learn the multivarious frameworks, tools, platforms to get one’s work done. One client told me that of the ten developers they had working in their Java EE team, each one had a distinct setup on their workstation. And this was on the same application, all using the same tools and the same platform. So why not just stick with one framework?

Consider Struts. Well, actually, we can’t, and that’s sorta the point. Struts (1) and Struts 2 are significantly different from each other, to the point that migration from Struts to Struts 2 is quite tricky. The developer’s website links to a three-hour tutorial, a three-part guide, and in the Migration Strategies, suggests (as its first option) running Struts 1 and Struts 2 in parallel. So, I think it’s fair to say they are sufficiently different platforms as to be considered separate.

Having said all that, I still think it’s a good thing that this diversity exists. Yup, what I called fragmentation earlier is now diversity. Look at the open source world: the Linux kernel has had thousands of developers working on it, yet is as enterprise-ready (if not more so) than many more monolithic systems. There is no single Linux distribution; Red Hat, Ubuntu, SuSE — all have carved their own very large niches in the industry, and each with a large degree of success. Sure, a project may start and then flounder as another, similar project dominates the niche. Perhaps one or two major players succeed at the expense of others, and we end up with competition in the space, with quality improving on both sides; KDE vs. Gnome, Eclipse vs. NetBeans, .NET vs. Java EE. Evolution is needed to get to that point, and that’s where diversity is a good thing.

Therein lies the challenge for large organisations. The results of such evolution are good for the industry, but the process of evolution is painful. Your modern large organisation is not the instrument of change and improvement; you are not in the business of helping the world decide which framework is the next big thing. You need to make the decision: either reap the benefits of the last generation’s evolution, and back the previous winner, or pick one of the current generation of technologies and stick with it. This is the challenge of conservatism vs. early adoption, but it’s a nettle that must be grasped.

Photo “Diversity Clucks” by chrisjfry

Scheduling Software Development

Brian's Classroom Support Calendar, StanfordEdTech

Don’t use anything fancy like Microsoft Project. The trouble with Microsoft Project is that it assumes that you want to spend a lot of time worrying about dependencies….I’ve found that with software, the dependencies are so obvious that it’s just not worth the effort to formally keep track of them.

– Joel Spolsky, Painless Software Schedules

That article has many other great insights, including:

If you are sloppy, and pick big “chunky” tasks (“implement grammar correction”), then you haven’t really thought about what you are going to do. And when you haven’t thought about what you’re going to do, you just can’t know how long it will take.

…and

Many rookie software managers think that they can “motivate” their programmers to work faster by giving them nice, “tight” (unrealistically short) schedules. I think this kind of motivation is brain-dead. When I’m behind schedule, I feel doomed and depressed and unmotivated. When I’m working ahead of schedule, I’m cheerful and productive. The schedule is not the place to play psychological games.

As the article is over ten years old, Joel has end-of-lifed it, and replaced it with this one from 2007. I still like the original!

Photo: “Brian’s Classroom Support calendar” by StanfordEdTech