I hate SkyNet

Originally posted on Wed, 11/07/2012

Yesterday I put my Terminator 2 Blu-Ray into the player, and it said “SkyNet has detected that you have accessed this disk before.”
30 seconds later: “SkyNet is loading…”
30 seconds later, a map of the Earth, with a bunch of stuff which might have been disabled menu options, or just fake menu options.  It might have said, “SkyNet cannot determine your location.”  Oh, and a “continue” option.
For the next 15 seconds: “SkyNet has detected an Internet connection…”
“Downloading high-resolution media…”

SkyNet did not know where I had stopped playing the day before.  It did not know which version of the movie I preferred.  Out of my 20 minute treadmill routine, 4 minutes were spent just getting back to the right scene.

If only Skynet were a little smarter and a little faster.

A better system of government

Originally posted on Tue, 11/06/2012

I’ve been seeing (and commenting on) a lot of suggestions about how to improve our democracy. It’s been said that democracy is the worst form of government, except for all the others. Winston Churchill is most famous for saying that (“it’s been said” and all), but we’ve over half a century with lots of advances in game theory and psychology, so perhaps it’s time to revisit this assertion.

First we must decide what the purpose of government is. It’s not strictly to enforce the will of the people, for several reasons. Even in a democracy of one person– or a dictatorship– a leader might not know what he wants or be wrong, by later judgement. With more people, there’s even less agreement. What’s more, it’s important for people to have certain rights, regardless of the will of the people.

For the purpose of argument, I’d say that the purpose of government is twofold:

  1. To ensure an equitable distribution of power. At a minimum, to keep tyrants, corrupt officials, invaders, thugs, bullies, and thieves from taking over. Taken to an extreme, this may mean heavy-handed wealth redistribution. There’s a lot of room for interpretation in between, depending on what you consider equitable. My own take is that, in each generation, every person should have a fighting chance to rise to the top of society, and nobody should become completely disenfranchised. (Note that this is aspirational: I don’t know of any government that always succeeds at either.)
  2. To facilitate cooperation, as necessary. That is, to make sure that services that can’t or shouldn’t be provided individually are taken care of, either by providing them directly or indirectly through laws, regulations, and policies. That’s really more of a means than an end, so I’ll append: …in order to sustain the best of human values, for the present and future generations.

That being the case, is democracy the best choice? Many of the biggest problems that face our world are environmental or economic, where experts have a far more nuanced (and often completely different) view than the population at large. Government is most successful when people are intimately involved in measuring outcomes, but don’t necessarily care about implementation details. Sanitation, for example. People don’t know (or care) where the trash goes, so long as they don’t have to care, but will complain immediately if it doesn’t get picked up. Or schooling. School works best when parents complain immediately when things go wrong, and taxpayers are willing to pay to make things go better. But global warming? That takes decades to measure, and few have the expertise to measure it.

Economics is similar. The biggest factors that effect the economy, in my opinion, take 10-20 years to have an effect. Education. Investments in roads and other infrastructure. Cultivating a business community that makes long range plans. And preserving free-market enterprise. (“Business friendly” policies tend to be pro-existing-business, and therefore anti-free-market.)

The biggest advantage of democracy is that it is extremely good at distributing power. Power tends to consolidate. The wealthy and powerful– whether individuals, associations, or corporations– use their influence to gain more influence. It’s a corrosive effect that seeps through the cracks in every society. And once it becomes overwhelming, the disenfranchised try to take back power.

This is not unique to democracy. If anything, democracy works to protect the bullies– by keeping them from getting so powerful that they are overthrown violently. I see society as like a car going down a road. Ideally, the driver keeps the car in the middle of the lane, but at a minimum tries to keep it out of the ditch. The driver, in a democracy, is lazy. People have better things to do than to pay attention to politics. So it’s like a car with a sleepy driver, who wakes up when the car drifts too far, gets back into the lane, and dozes off again. That’s a whole lot better than monarchy and other dictatorships, which is like a bus where the driver doesn’t have to care about staying in the lane. It occasionally crashes, people die, the bus gets repaired, and someone else becomes the new driver. (Or you have a parliament which tries to replace reckless drivers, when the parliament itself isn’t reckless.)

So the qualities that make for a good government are (1) measurable progress toward goals (call it transparency), and (2) responsiveness, which is an antidote to corruption. There are plenty of organizations that do well in this regard that aren’t democratic. Successful, customer-focused companies (such as Apple) make it easy for people to judge them, and people do. They are held to a higher standard, which they don’t always meet. Apple, for example, delights its customers with high-quality products, and is also more transparent than any other electronics company when it comes to fair labor practices in their factories. The irony is that people associate Apple with poor labor practices, because we know more about them. You can be sure that Dell and HP don’t have better practices. Similarly in government, it appears the most corrupt when you hear about every little whiff of potential corruption, which is in fact when it’s the least corrupt.

So is democracy the best? For exposing and fixing corruption, you need activism to come from all corners. That doesn’t necessarily mean everyone voting in the way we think of it. Our election cycle is too infrequent to deal with immediate problems, yet too frequent to provide long-term leadership. And yet, because power tends to collect, we can’t afford to have politicians who persist for too long, lest they become too powerful.

Here are a few things I’d like to see for good governance, whether it’s in a democracy as we know it, or not:

  • A democratic process that engages people at the appropriate level. Not asking them to make decisions about technical issues they can’t be expected to become experts in, nor asking them to choose several Soil And Water District Supervisors, and over dozen judges (as I do). But to give them an opportunity to choose a few leaders, whose qualities they can be expected to research and judge.
  • A selection process which promotes consensus builders. So you aren’t choosing between the far left and the far right. Nor are you necessarily choosing the “safest,” most center-of-the-road candidate. I think multiple-choice voting (e.g. instant run-off) would be far better than our current system in this regard. That is, the winner would have to be acceptable to a majority of voters, not simply get more votes than the alternative. But you also need a system for choosing the candidates that’s more flexible than the current party system, where a radical group can overtake the nomination process.
  • A culture of engagement, where corruption is not acceptable. Where diverse opinions are discussed, but lies and distortions are not tolerated. In some ways, the echo chamber of the Internet is a step back from the old days when most people got their news from a few authoritative sources.
  • Resilience against power grabs. Small groups that can take over a nomination process, for one. But also organizations that lobby for tax loopholes that aren’t a big deal individually, but add up to a lot of corruption over time. I’m not sure what the cure for the latter is, except perhaps to limit the complexity of certain kinds of laws. (Referenda aren’t the answer: deceptive advertising influences voters even more than lobbyists influence politicians.)

If you were to design a system from scratch based on these principals, you’d end up with something unlike democracy as we practice it today. But it’s not impossible to get there from within our own system. Like so many things, the theoretical properties (how many legislators, how many parties, electoral or direct elections, etc.) aren’t as important as how well it is implemented. And that implies constant vigilance.

Liars & Outliers

Originally posted on Tue, 10/23/2012

I’ve been a Bruce Schneier fan for years. I read his blog often enough that I don’t feel the need to read his books. But then he offered a discount on a signed edition of his latest book– with the one stipulation that I write a review of it. So here’s the review.

A lot of brilliant thinkers tend to get stuck in their own perspective. There are plenty of mathematical geniuses who can’t contemplate the implications of their ideas. Plenty of programmers who can’t understand why users don’t recognize the brilliance of their user interfaces. Bruce Schneier isn’t one of them. His rose to fame with Applied Cryptography, a book of algorithms. But because great encryption doesn’t help when it’s part of weak security, he’s written written with increasing breadth about security. After 9/11, he wrote Beyond Fear about how we individually and as a society make poor security trade-offs. Now he’s written Liars & Outliers: Enabling the Trust that Society Needs to Thrive. This takes an even wider view, as he asks: how is it that people can trust each other at all? For example, a man gets into a taxi in a foreign city. The man and the driver will never see each other again. The taxi driver could easily rob him and get away, or he could ride without paying. And yet every day all over the world, drivers and passengers can trust each other.

When I got this book, my first thought was it’s too thick. After all, Schneier has said he wants this to be read by all sorts of decision makers. I’ve heard that if you want to be read by busy people, make your book just long enough to be read in a single long plane trip. Then again, I’m not a best-selling author. But it turns out this isn’t such a long read, it’s 250 pages, plus another 100 pages of notes and references. (This book-to-notes ratio is up there with Dr. Tatiana’s Sex Advice to All Creation, which has a remarkable amount of overlap with Liars & Outliers.) It’s really two books in one: a quicker, drier, less technical read without endnotes, or a more colorful read with endnotes. I kept one bookmark in the endnotes at all times.

Liars & Outliers is an overview of trust and cooperation. It draws primarily from sociology, economics, and psychology with a big helping of evolutionary biology, game theory, and security. Philosophy and theology also show up, to add a bit of color. It’s about the different strategies for enforcing conformity in a group, and when they do and don’t work. This is practical stuff for anyone who needs to manage a large group, whether it’s an online discussion group, a corporation, or a country of taxpayers. The book takes pains to discuss this from a neutral perspective: the forces are the same whether it’s a just society versus murderers or a murderous society against saints. To this end, Schneier uses the rather awkward term “defector” to refer to rule breakers.

The book is summed up quite well in the first chapter. In fact, if you’re in a hurry you could just read the first two and the last chapter. Occasionally the book starts to read like a taxonomy, as Schneier explains the breadth of, for example, security techniques. This is an academic book trying– and usually succeeding– in being a general-audience book. While it does drag a little at times, there are plenty of popular just-for-fun nonfiction books that drag more. (I’ll admit, though I read a lot, I’m not much of a book reader.) And with the whole history of deceit and treachery to draw upon, he has plenty of colorful examples.

I feel like I’ve said a lot about the book without saying much about the contents of the book. As someone who not only reads Schneier’s blog, but reads many of the same sources he draws upon, there weren’t many ah-ha moments for me. Perhaps the biggest one is this: trust is rare in nature. It’s hard to establish and easy to break, but once established it yields huge benefits. And humans are the most cooperative and trusting species on Earth. Why? In part because we have the greatest capacity to evaluate reputations, so we know when not to trust. In part because of a sense of morality which leads us to punish rule breakers.

I hear plenty of people argue that corporations are inherently inhuman and inhumane. While Schneier doesn’t say that exactly, he spells out each of the pressures society uses to enforce conformity to social norms, and how corporations respond to only two (reputational and institutional [legal.])

The overarching message of the book is that there are different ways of establishing a trusting relationship, and they work on different scales. Neighbors directly evaluate each other. People who know each other only by reputation can go by that. Less intimate groups use morals and institutions to maintain group norms. And when all else fails, security mechanisms can make non-compliance difficult. None of these completely eliminate non-conformity, which is good because even good rules can have bad consequences. The trick is using the right tools– and the right amount of pressure– under the right circumstances. It’s depressing to consider all the ways that things go wrong– from suicide bombers to self-destructive investment banks to a lack of a global response to global warming. But at the same time, humans are amazing creatures in that we even have a cooperative arrangement as abstract as the United Nations or a coalition of corporations.

Are green traffic lights really turquoise?

Originally posted on Wed, 10/17/2012

Yesterday I posted the following query to my friends:

Okay, everyone, here’s a quick homework assignment:

While you’re commuting tonight and tomorrow morning, look at the “go” stop lights. What color are they, REALLY? Yes, I know they are supposed to be green. But are they, really? And are different ones different colors? How many do you see that match the following colors:

1. Yellow-green
2. Pure green
3. Turquoise
4. Blue

No fair looking at photos of traffic lights, either. This is about color perception with your eyes.

(BTW, I’m partially color blind.)

So here’s what that’s all about. I’m red/green color blind, and green traffic lights have a blueish tint so that people like me can tell green from red. And yet people think of green lights as being a pure green. If you do a Google image search for green traffic light you get a mix of photographs– with blue-green lights– and clip art with pure green. Clearly illustrators aren’t looking at real lights.

This makes a difference to me because I see lots of user interfaces where they use red/green or red/amber/green to display key information: from traffic maps to MacOS window widgets. They never seem to use official traffic light colors. The worst, though, is red/green LEDs on appliances such as battery chargers. There’s absolutely no other context that can be used to indicate the status, since the brightness is identical.

Now, color perception is a tricky thing. It took me years to realize that I really am color blind, because I can nearly always figure out the color from context. (If that sounds weird, consider that you usually don’t notice your disabled color perception when you enter a low-light or colored light environment.) And color itself isn’t in nature– it’s a product of human perception. Consider that an apple looks to be the same color whether you see it under the cool light of noontime sun or the yellow cast of sunset. The light rays that hit your eyes are completely different, but your brain uses the background to compensate in order to construct a stable notion of color. That’s why I specified no photos– the background makes a big difference.

So the results? One person said yucca green, which she described as bluish. (Oddly, that swatch has slightly more red than blue.) Another said there’s variation, and the bluish ones are weird. And another said mostly pure green, with some yellow-green.

Oddly, I’d put the “yellow-green” option in just as a ringer; I never expected anyone to choose it. However, if the background is a bright blue sky, green will look more yellow.

So are green lights turquoise? I don’t trust my color vision enough to say. But you can compare. Blue-green traffic light, turquoise (equal parts blue and green light.)

Here are more details, for the pedantic.

Below is a picture of the official luminance ranges for traffic signals. Modern green traffic signals are allowed to be in the range from this:          to this:         . That’s assuming you’re looking at my color swaths on a pure white background. The ITE standard specifies colors relative to the range of human perception (in CIE 1931 color space)– that fingernail shape in the graph below. Computer displays, which shine only three colored lights, display a more limited, triangular color range with red, green, and blue corners. And you can’t get any of the legal traffic light colors from a computer screen!


BTW, I got my CIE 1931 to RGB conversions here. You probably won’t find a better source, because any serious color expert would point out that you can’t accurately convert CIE 1931 to RGB.

And here’s a mind-blowing example of how language affects color perception.

Letter to great grandchildren

Originally posted on Fri, 10/12/2012

I haven’t restored my old blog entries yet, but if you want to read about my Seventh Generation letter writing project, here’s a snapshot, courtesy of the Internet Archive, which includes a few related posts. I’m relying on the Internet Archive to keep online copies of the letters I write to my descendants, in case something should happen to the paper copies.

I wrote another letter this summer, this time to my great grandkids, and I’ve now put it here. It’s scrambled, so please don’t unscramble it unless you are visiting from sometime between 2080 and 2100 or so and it’s addressed to you. Or if people in your time don’t use HTML with Javascript anymore, you may update it for the appropriate recipients.

One fun thing I’ve done this year is take pictures of the Boggle game that my wife, Jordan, was playing with my mom while I was writing the letter. I haven’t posted them yet; I haven’t figured out how to scramble them. The idea is for my great grandkids to play Boggle against my relatives.

How to get an iPhone 5

Originally published on Sat, 10/06/2012

Here’s how I’m doing it, not that I recommend this to anyone else.

  1. Pre-order online, on the first day. Get informed that you will need to wait 2-3 weeks.
  2. 3 weeks and 1 day later, get a call from Apple saying the order has been put on hold by Sprint.
  3. Call Sprint, wait on hold for 5 minutes. Tell them the name of your first pet.
  4. Get told warmly that they cannot help you– as much as she wants to– because the order was made through a third party (Apple) and Apple must call Sprint’s National Sales Support Desk. They will only talk to third party retailers, “I can’t even talk to them,” she says. Helpfully, she gives you a phone number.
  5. Call Apple, have an automated voice tell you it can help you with anything, just ask in a complete sentence. Humor the computer for a few moments before telling it you need to talk to a human being. It transfers immediately, and a woman answers on the fourth ring. Explain the situation, give her the number. She puts you on hold while she calls.
  6. A few minutes later, she says they gave her a different number to call. She also verifies that she has your wrong phone number (home number, presumably). She calls the next number, with you on hold.
  7. A few minutes later, the Apple rep introduces you to a Sprint rep, who wishes to ask a security question. In anticipation, you tell him your first cat’s name. After ascertaining the situation, the Sprint rep puts the two of you on hold while he talks to his supervisor. After a while, he comes back and says you must contact the National Sales Support Desk. Once informed that you’ve already done that, he talks to his other supervisor (more hold music) and he agrees to wait on hold while he places the call.
  8. All three of you talk to a man who inverstigates and finds no good reason for the delivery of your iPhone to be put on hold. He seems almost confident that the problem can be resolved. But first, he must ask for the Apple rep’s store ID number. She has no such number, since she is a customer service rep. The man replies, “we can only validate through a store” and gives some helpful hints as to where she might find an Apple department that he’s authorized to talk to.
  9. The Apple rep hangs up on Sprint and puts you on hold while she finds a way to communicate through proper channels. Eventually she brings you into a conference call. The other person transfers the two of you to Sprint, this time without hold music. At this point, the Apple rep explains that Apple cannot complete the delivery, so they are cancelling it, but will send you email so that when you place a new order you can respond to that email with your order number and Apple will expedite the delivery. They’ll try to get it to you as close to the original delivery date as possible. You can even order a different color or storage capacity.
  10. The latest Sprint rep assures you that there was no reason for the original hold on your order, claims it may have been a computer glitch– and offers to remove the hold. The Apple rep explains that the order has been cancelled. It will take 24 hours for the cancel to propagate. Do not order a new phone before then, or you will be renewing a contract that has just been renewed– and the order will be placed on hold as invalid.
  11. Wait 24 hours, or better yet, until Monday.
  12. Hope for the best.

That’s 1 hour, 20 minutes for the second call. Mostly on hold.

Making 3D pictures in The Gimp

For my 4Dth birthday, I’m designing a scavenger hunt which will include several 3D pictures which require red/blue 3D glasses to view. It’s easy to make them in The Gimp (or Photoshop, if you have a wad to blow on such things.) Figuring out exactly how to do it isn’t so easy. The basic idea is to merge two photos, one from the left eye’s perspective and the other from the right eye’s perspective. The final image should have only the red component of the left-eye picture and none of the red component of the right-eye picture. If you’re shooting a still life, use any camera and take one photo slightly to the left of the other. For close-ups, the eye separation should be very small: an inch at most. For a scene of a room, the eyes should be a natural distance apart, about 3 inches.

  1. For the left eye picture, add a layer above the photo filled with pure red ( RGB, a.k.a. #ff0000).
  2. Set the mode for that layer mode to “Addition.” You should see the photo in colors ranging from black to red. With your 3D glasses on, the red lens should show the picture as it appears on the screen, the blue lens should show a black picture.
  3. Under the Layer menu, select “Merge Down” to combine the two layers.
  4. For the right eye picture (a separate document), add a layer above the photo filled with pure blue-green ( RGB, a.k.a. #0000ff).
  5. Set the mode for that layer to “Addition”. With your 3D glasses on, you should see black through the left (red) eye.
  6. Copy (or “copy visible”) this picture.
  7. Paste into the other (left eye) picture. After pasting, hit the “new layer” button in the Layers dialog to turn the floating pasted image into a layer.
  8. Set the layer mode to “Multiply.” You should now have a different photo visible through each lens of your 3D glasses.

Scala emoticons

There are a lot of great things about Scala, but there are a few things where that programming language just goes overboard. Consider that there are several emoticons built into the standard library:

  • +:= (vampire pope) for adding items at the front of a sequence
  • <:< (sad witch) for comparing generic types
  • :/ (uneasy) fold right

And that’s just the ones I know about. Unfortunately, you can’t use a search engine to find any more, because none of the major search engines index emoticons. :/ (try it). Fortunately there is a well-hidden reference page.

Post a comment if you find any more.

Thoughts on recursion

I was poking through Code Complete, 2nd Edition by Steve McConnell recently, after I’d handed the book to a computer science student. The book is showing its age, as good object-oriented design is better understood than it used to be, but it remains the best book on how to program well.

One thing that struck me, though, was a particularly strong statement against recursion, on page 397 (2nd Edition):

If a programmer who worked for me used recursion to compute a factorial, I’d hire someone else.

To put this in context, McConnell had just presented a few appropriate uses for recursion (quicksort and maze solving [i.e. graph searching]), and is railing against the simple examples one typically learns in computer science classes. He sees recursion as an advanced technique, which should be avoided unless it provides a clear advantage.

McConnell says recursive algorithms are harder to read. I don’t buy that; it depends on your audience. But his biggest beef is with the stack. When a function calls itself over and over again, the stack grows (keeping track of all the variables in all those function calls) and you can run out of memory, the dreaded stack overflow.

In many computer science departments, the first language you learn is Scheme, which doesn’t support iteration, so you can’t compute a factorial (or do any other kind of looping) without recursion. Scheme handles this with tail recursion elimination, where if the last thing a function does is call itself (i.e. tail recursion), the Scheme interpreter overwrites the stack frame– essentially replacing recursion with a loop.

Since most languages don’t do tail recursion elimination, and since it can be harder to write a function as tail recursive, I mostly agree with McConnell– despite my initial reaction. However, I was thinking about what makes some algorithms amenable to recursion while others are a bad idea.

McConnell’s examples of good recursion– quicksort and graph searching– both involve branching. Most tree algorithms are recursive: one might say a tree is a recursive data structure. The recursive algorithm uses the stack to track the branching, which is cleaner than handling your own stack. (A good programmer can replace any recursive algorithm with a stack-based one.) What makes factorials different is that there’s no branching, so the stack is redundant. So we have two classes of algorithms:

  • Branching algorithms, which use recursion to track which branches have been visited.
  • Linear (non-branching) algorithms, which should be written in a tail-recursive manner to avoid stack overflows, or written with loops when the compiler/interpreter doesn’t support tail recursion elimination.

I should add that branching algorithms are also susceptible to stack overflows, but since the stack is providing a necessary service, it can’t be avoided with a non-recursive algorithm.

So here’s why I end up agreeing with McConnell: to properly write non-branching recursive algorithms, you must (1) write them in a tail-recursive manner, and (2) run them with tail call elimination. After you write a tail-recursive function, it’s easy to accidentally modify it to no longer be tail-recursive, and the difference can be subtle. That’s why Scala provides the tailrec annotation: so a function that’s supposed to be tail recursive won’t compile unless Scala can apply the tail call elimination. If you need the compiler’s help to safely apply a technique, it’s an advanced technique, and shouldn’t be done lightly.

UCU: better than MVC

I propose a replacement for the MVC (Model-View-Controller) model for software design. The problem with MVC is primarily that View and Controller both address the same concern: the user experience. After well over a decade, software developers now know better. My new model keeps the model, but all the words are changed to avoid confusion with MVC.

UCU: Utility — Connector — User Interface

  • A Utility provides a well-defined service. For example, accessing a file or storing a collection of items. It has a contract which defines what it does, and its correctness is measured against the contract. That contract is some combination of documentation, code (method signatures, interfaces, etc.), and a test suite.
  • A Connector is how components are discovered and constructed. Thus this category includes constructors, factory methods, and dependency injection. Connectors should be simple, since a program consists of a rat’s nest of interconnected components. Bad connections are hard to test, and even harder to debug, so the only recourse is to make them hard to be buggy. Constructors should do no extraneous work, except for checking for illegal arguments. When there’s an illegal connection, the program should fail immediately. For example, a constructor for a database utility which takes a database URL should check the syntax of the database URL, but it should not try to connect to the database. (If the database is inaccessible, that’s a problem with the database, not with the program trying to access it.)
  • The User Interface (UI) is the part of the program that relates to how the user perceives and interacts with the program. Its correctness is measured against the user’s experience. (Many practitioners prefer the term UX, User eXperience. In this case, because you can program an interface but not an experience, I am using UI to refer to the software.) Humans have different perspectives, and their expectations may change over time. As a result, UIs are developed using rapid prototyping techniques. Like a Broadway stage, the UI should be designed for frequent and rapid changes. UI components that get reused frequently are really utilities, thus classes tend to migrate from UI status to utility status.

For more reading, here are my earlier thoughts, before UCU gelled. And here’s a Google Tech Talk on writing clean, testable code which influenced UCU.