Will.Whim

It’s been too long since my last update, but life has been busy, especially with the Microsoft purchase. But a few notes:

1. I enjoyed singing Sacred Harp at the Michiana and Kalamazoo annual singings this weekend, and James Nelson-Gingerich gave me the *first* copy of the print version of 26th edition of the Harmonia Sacra for my work on the Harmonia Sacra website.
2. I’m looking forward to a family reunion of all my brothers (five of us!) this coming weekend.
3. I’ve been off to California a couple of times to meet about the Microsoft purchase, and I got my first “Microsoft Live!” tee-shirt
4. Summer in Michigan is a wonderful thing this year.
5. We just had our one-year anniversary of living in ‘the new house.’ So I guess it isn’t the new house, especially since we finally sold the old one.

(The usual disclaimers: my opinion only, not my current or future employers)

When Powerset began a couple of years ago, a lot of commentators called us — and still do call us — a would be Google killer. This, despite repeated comments by senior staff that this wasn’t what we were about. As a company, Google is hard to beat. Our goal was audacious, but not that audacious. Our goal was to build a better search experience: to use natural language technology to provide better search results, both by having a better understanding of web documents as well as user queries.

But natural language technology has always only been part of the mix. We have, from the beginning, seen ourselves as doing “keywords plus”; that is, we have always planned to do what the other search engines do (keyword search, link analysis, blah, blah, blah), but add on top of this signals coming from parsing and semantic understanding. For example, we’d like to do as good a job as Google (say) on queries like ‘powerset microsoft’, but do even better on queries such as ‘Who acquired Powerset?’ and ‘Which company did Microsoft just buy?’ and everything in between.

What I didn’t realize when I joined the company is how some of the same technology would create innovations in the user interface, too. Powerset’s ‘Factz’ are a nice addition to the standard search page, and our ’snippets’ are the best in the business. When I first typed in ‘stars of BSG‘ in the Powerset search box, I was floored by the beauty of the results.

So I think we met our audacious goal: a better search experience. Microsoft seems to think so; after all, they bought the company.

And here’s the thing: we were bought by Microsoft. Microsoft’s market cap is still 90 billion dollars greater than Google. If anyone is able to capitalize a little ol’ startup like Powerset to make us a big player in search, it’s Microsoft. In fact, it’s clear (to me at least) we have a new mission, which is just the old mission the pundits wrongly labeled us with at the start: As a search company, our mission is now to beat Google.

Interesting times ahead.

microsoft acquires powerset

This might not be worth a post, but anyway:

Arnold Zwicky has a post at Language Log on “home” vs. “house” in (American) English, citing (among other things) the commentators in the Merriam-Webster’s Dictionary of English Usage saying:

A final note on home ownership: MWDEU reports that “a number of commentators have remarked on the tendency to buy a home and sell a house”.

I was curious to see whether this ‘tendency’ was real, so I checked the Google 3-gram data (see announcement). I think for this tendency to be real, the following should be true:

(1) buy a home >> buy a house
(2) sell a house >> sell a home
(3) (buy a home/buy a house) >> (sell a home/sell a house)

Here are the bare facts:

buy a home : 328,584
buy a house: 235,019
sell a home: 193,088
sell a house: 25,632

and the ratios:

buy a home/buy a house: 1.40
sell a home/sell a house: 7.53
buy/sell ratio of ratios 0.19
(buy a home+sell a home)/(buy a house+sell a house): 2.00

As you can see, (1) is true, but (2) and (3) are not (2 and 3 are related, of course). According to the Google data, ‘home’ is twice is likely than ‘house’ in both these contexts, and ’sell a home/sell a house’ is much greater than the ‘buy a home/buy a house’ ratio.

It must be all those realtors (oh, excuse me, ‘REALTOR®s’).

A response to Mark’s tagging me.

Seven factz about me:

1. Will Fitzgerald didn’t use a computer until he was 23. It had cards.
2. It is difficult for Will Fitzgerald to refer to himself in the third person.
3. Despite his descriptivist linguistic training, Will Fitzgerald tries to distinguish ‘between’ and ‘among’.
4. Will Fitzgerald was known as “Bill” until he was 33.
5. Will Fitzgerald has half of a house named in his honor. He doesn’t know why.
6. Will Fitzgerald once made his ESL class erupt in laughter by referring to ‘green Jewesses’ instead of ‘green beans’. (judias verdes vs. judías verdes).
7. Will Fitzgerald (who has moved around a lot) has been a member or active participant of a Methodist church(United Methodist, Roseville), a Southern Baptist church (Calvary Baptist, Roseville), an independent fundamentalist church (First Church, Wellston), a Reformed Church in America congregation (University Reformed, East Lansing), an Evangelical Presbyterian church (Evangelical Presbyterian, Carbondale, now affiliated with the Presbyterian Church of America), a Spanish Baptist church (Iglesia Evangélica Bautista de Gracia, Barcelona), a Presbyterian USA church (North Presbyterian, Kalamazoo), a Christian Reformed church (Immanuel CRC, Kalamazoo), a Mennonite USA/Brethren bi-affiliated church (Reba Place Church, Evanston, now just Mennonite-affiliated), two independent Christian communities (Reba Place Fellowship, Evanston and Church of the Sojourners, San Francisco), a Canadian Anglican church (Christ Church Cathedral, Fredericton) and two Mennonite USA churches (Pine Grove, Battle Creek and Kalamazoo Mennonite Fellowship. He has more or less made up his mind.

That was fun, Mark. Rather than tag seven others, I present to you this fish tank. Click on the tank to feed the fish.

It’s all (well, mostly) about representation. Peter Turney:

[F]iguring out how to represent the problem is 95% of the work. By the time you have the representation right, the tool that you use to finish the remaining 5% is not terribly important.

Over at FiveThirtyEight, the following ‘homework assignment’ was given:

How many unique ways are there to acquire at least 270 electoral votes without any excess?

I figured it would be a ‘large’ number, but I was surprised at the actual total: 51,199,463,116,367 (or, fifty-one trillion and change). about 2.3% of all possible combinations (This exact number is based on the simplifying assumption of treating Maine and Nevada as giving up their electoral votes in the same way as all the other states). The answer was given by Isabel Lugo, a mathematician.

If you blindly checked all the possibilities at a rate of 1000 per second, it would take over 17,000 centuries. That’s older than John McCain.

I was wrong. And I’ve very glad.

# The evolution of a Ruby programmer

def sum(list)
  total = 0
  for i in 0..list.size-1
    total = total + list[i]
  end
  total
end

def sum(list)
  total = 0
  list.each do |item|
    total += item
  end
  total
end

def test_sum_empty
  sum([]) == 0
end

def test_sum_one
  sum([10]) == 10
end

def test_sum_several
  sum([10,10,10])==30
end

def sum(list)
  total = 0
  list.each{|i| total += i}
  total
end

def sum(list)
  list.inject(0){|a,b| a+b}
end

class Array
  def sum
    inject{|a,b| a+b}
  end
end

describe “Enumerable objects should sum themselves” do

  it ’should sum arrays of floats’ do
    [1.0, 2.0, 3.0].sum.should == 6.0
  end

  it ’should sum values in sets’ do
    require ’set’
    Set.new([1,2,3,3,3,2,1]).sum.should == 6
  end

  it ’should join arrays of arrays’ do
    [[1],[2],[3]].sum.should == [1,2,3]
  end

  it ’should concatenate arrays of strings ‘ do
    ['a','man','a','plan'].sum.should == ‘a man a plan’.gsub(/ /,”)
  end 

  it ’should work on hash tables, too — appending their key/values’ do
    {:a => 3, :b => 4}.sum.find_all{|a| a.is_a? Numeric}.sum.should == 7
  end
end

module Enumerable
  def sum(zero=false)
    zero ? inject(zero){|a,b| a+b} : inject{|a,b| a+b}
  end
end

# Greenspun’s 10th Rule of Programming version (suggested by BenD’s comment)

module Enumerable
  def reduce(by,zero=false)
    zero ? inject(zero){|a,b| a.send(by,b)} : inject{|a,b| a.send(by,b)}
  end

  def sum(zero=false)
    reduce(:+,zero)
  end
end

Many of us are familiar with the commercial in which the question is asked, “How do you spell ‘relief’?” And the answer is “R-O-L-A-I-D-S.” According to Wikipedia, this commercial has been around since the 1970’s.

I came across an earlier example from 1793, in Elder John Leland’s “The history of Jack Nips”:

Like other boys, I wished to be in fashion, and as the Presbyterians were the most fashionable, I applied myself to the study of their books, but was not a little puzzled to reconcile their writings with my boyish thoughts. I could not, for my gizzard, understand their orthography, until I was more than sixteen. They would spell thus: c-i-r, cir, c-u-m, cum, c-i, ci, s-e-d, baptism.

(The full tract can be found here, but beware the cheesy embedded sound file).

Not quite the same thing is found in Shakespeare’s Much Ado About Nothing. Ursula and Hero are discussing Beatrice, and Hero says:

I never yet saw man,
How wise, how noble, young, how rarely featured,
But she would spell him backward. ( Much Ado About Nothing, 3.1)

Wikipedia says it was first published in 1600. But I still think Leland’s turn is different enough to consider it separately. Was Leland the first person to use the spelling of a word to stand for a quality of a different concept all together?

(Ireny isn’t a word, but it should be)

I’ve been lurking on a discussion list, which will go unnamed. Mr ALLCAPS said that everyone in group X is either a Y or a Z. Mr. Irenic suggested that there might be a third category. Mr. ALLCAPS accused Mr. Irenic of being a ‘closet Y’ (with the comment ‘No Offense, just identification’). Mr Irenic replied:

Let me assure you that no offense is taken. Though I think your identification is slightly skewed, I don’t really mind whether one thinks I am a “closet Y”. Whatever I believe, I think I put it forth in the last post and it is not in the closet. What I don’t address clearly is not because it is in the closet, but probably because I simply don’t know. If you think I’m a Y, I have no control over that, nor any particular concern for it.

I really love the modesty and clarity of Mr. Irenic’s statement.

From:
Sheby County, Ky:

SHELBYVILLE, KY. Local law enforcement along with Shelby County Emergency Management are asking if anyone has any information leading to the whereabouts of Maxine Ethington. Maxine is an elderly citizen of Shelby County suffering from Alzheimer’s. She was last seen on May 25, 2008 at approximately 8:00p.m. She drives a 2000 Grey or Silver Buick Sentry, license # KY 186BLY. The vehicle may have an old plate attached with #’s KY 495-JYT. There is possible damage to the left side of her vehicle with only three hub caps.

If you have seen this woman or have any clues as to her whereabouts please contact the Shelby County Dispatch at 502-633-2323 or the Shelbyville Police Department at 502-633-2326.

America the Beautiful has interesting linguistic properties. In particular, it uses verbs in the subjunctive mood, which is quite unusual in modern English, except in fixed expressions (such as “God forbid.”). The subjunctive is usually used to describe a condition that doesn’t exist, or a condition that one wishes to exist, such as “if I were a rich man.”

The interesting thing about the subjunctive in English is that it often indistinguishable from the simple past tense or a request/command form. In “http://www.powerset.com/explore/go/God-bless-America,” you can tell it’s not the past tense (that would be “God blessed America, of course), but it’s a bit ambiguous about the command: does it mean “May God bless America” (the subjunctive) or “God, (please) bless America!” (the command form). From the context, it’s clearly addressed to God directly: before the chorus, Berlin’s lyrics say, ‘we raise our voices in a solemn prayer.”

“America the Beautiful” is not addressed to God; God is explicitly in the third person:

America! America! God shed His grace on thee,
And crown thy good with brotherhood, from sea to shining sea.

Does it mean that God has shed grace on America, or is a hope that God may shed grace on America? Only by looking at the second line can we tell: “And crown thy good…”. If it were the past tense, it would be “And crowned thy good…”, but it’s in the bare form, and so must be the subjunctive.

Interestingly, the second verse changes into the past tense (a verse rarely sung):

O beautiful, for pilgrim feet
Whose stern, impassioned stress
A thoroughfare for freedom beat
Across the wilderness!

In theory, this is ambiguous: it could mean “May pilgrim feet beat across the wilderness,” but it’s clear that when Katharine Lee Bates wrote “America the Beautiful” in the 1890’s, she wasn’t hoping for a new pilgrim exodus across the continent, but was expressing a manifest destiny already fulfilled. The second chorus returns to the subjunctive, though–it’s not “mended” and “confirmed,” but “mend” and “confirm:”

America! America! God mend thine ev’ry flaw;
Confirm thy soul in self control, thy liberty in law!

The second is somewhat similar (gratitude for “heroes proved in liberating strife” and a subjunctive hope in the chorus that “God thy gold refine, ’til all success be nobleness”–two subjunctives in a row there!

The last verse has always confused me. How could anyone in the 1890’s thing that:

Thine alabaster cities gleam
Undimmed by human tears!

But in context, it’s a subjunctive-like dream:

O beautiful, for patriot dream
That sees beyond the years,
Thine alabaster cities gleam
Undimmed by human tears!

In other words, patriots dream of a future where American cites gleam and are without tears.

Well, may God bless America, may God mend its every flaw, and continue to shed us with grace. And may our cities be clean and full of justice, and a sense of unity grow among all its people–from sea to shining sea.

Fun Michigan poem in the May 19, 2008 issue of the New Yorker by Bob Hicok, entitled “A Primer.”

You never forget
how to be from Michigan when you’re from Michigan.
It’s like riding a bike of ice and fly fishing.
The Upper Peninsula is a spare state
in case Michigan goes flat…

The entire poem: A Primer.

ACLU report:

NEW YORK - The results of an internal Justice Department investigation released today reveal that officials at the highest level of government — including the White House — received reports on the abuse of prisoners in U.S. military custody overseas as early as 2002. Congress called on the department’s Office of the Inspector General (OIG) to conduct the investigation after documents made public through an American Civil Liberties Union Freedom of Information Act (FOIA) request revealed FBI agents at Guantánamo had raised concerns about methods used by military interrogators. Today’s government report is the first to identify that then-National Security Advisor Condoleezza Rice received complaints of torture.

More later, perhaps.

Mainstream media, are you paying attention yet?

Who invented the wireless telephone?

It’s been 100 years since “American inventor and Kentucky melon farmer” Nathan B. Stubblefield received the patent for the first wireless cell phone (UK Telegraph article, via Mirabilis).

Today, the company I work for, Powerset, launched its search product for general public use at powerset.com. It’s a really cool search and browsing engine for Wikipedia, with lots of information gleaned from the Freebase project as well.

Reading about Mr. Stubblefield made me want to know about other inventors who were melon farmers. Searching for “inventors who raise melons” does, in fact, return Powerset’s republished Wikipedia page about Mr. Stubblefield, with its first sentence helpfully highlighted. And, as it turns out, lots of other inventors who raised melons, including (of course) melon researchers, with nice results about watermelon, muskmelon, and galias.

Most people agree, even Peter Norvig (Google’s head of research), that search is in its early days. Google, Yahoo!, Live, Ask, and the other search engine companies have led the way, and Powerset is adding a new set of signals, based on principles from natural language understanding and knowledge representation, to the mix. (And, note, these are additional signals; no one from Powerset has ever claimed that current search signals, such as the presence of keywords or page rank, were unimportant.) These are relatively early days for Powerset, and early days for search. Early, and exciting, days.

And that makes me wonder: Who said, “You ain’t seen nothing yet?”

Powerset’s search for Wikipedia is now live!

The Detroit Free Press is reporting on a new plan to seat Michigan delegates at the Democratic convention. The idea is to give Clinton 69 regular delegates, and Obama 59. “The proposal also would seat the state’s 29 superdelegates.”

I suspect the hidden (Clinton) agenda is to seat the superdelegates.

Since these superdelegates were responsible for the mess we’re in, it would make more sense to me to split the delegates 64/64 between Clinton and Obama, and leave the superdelegates at home.