Late last week, the New York Times dropped the not-so-surprising revelation that the US government has its nose deep into the world's largest international financial database, the Society for Worldwide Interbank Financial Telecommunication (SWIFT) database, looking for leads on terrorist activity. If you haven't been following this story, then you'll want to jump down to the last part of this post, because I'm first going to dive right into why I support this particular antiterror program.
Regular Ars readers who're familiar with my previous, critical coverage of other invasive, electronic snooping programs—criticism that goes all the way back through the Carnivore/ECHELON days and extends right up to the NSA's domestic surveillance program (formerly TIA)—might be surprised that I could support a program that taps into international financial records and looks for terrorist connections. Isn't such activity just as invasive and scary as the NSA listening in on our phone calls and reading our email? In a word, no.
Working with the market vs. working against the market
Anonymity is the enemy of commerce. This has been true four millennia, and it's even more true in our modern world of cashless commerce than it was in antiquity. Our entire modern financial system is built on the ability to verify the identity of all the parties involved in market transactions, either directly or by using a proxy like verifying that a particular credit card transaction fits the cardholder's typical purchasing behavior.
My point here is that, when the US government dips into a large financial database in an attempt to trace money as it flows between parties, they know exactly who they're spying on. The SWIFT snooping program works because the feds can start working their way through the network of transactions at a known node—a terrorist or terrorist financier. They then can look to see who that person is dealing with, and who their contacts are dealing with, and so on. This is the polar opposite of the NSA program, in which the government starts with a data flow and then tries to figure out identities of all the parties involved in the communication.
It is crucial that critics of the NSA program, and of other technologies of mass surveillance (TMS) efforts, keep these two types of programs separate. The SWIFT program makes sense, because you begin with a discrete and inherently finite collection of identities and try to trace the myriad connections between them. In contrast, the NSA program attempts to work in the opposite direction, from an overwhelming volume of connections to a very small pool of identities. If the former program works, it's because the financial industry has gotten very good at identifying all the parties to a transaction; if the latter program works, it's because we got lucky and happened to be snooping the right call at the right time.
Let me put this in market terms: there are massive, overwhelming incentives for the financial community to be able to verify the identity of each node in a network of financial transactions; as I just said, business is built on this knowledge and the trust that it engenders. This is not true for telecom networks, where the incentives are structured to reward transport capabilities—bandwidth, quality of service, access, on-time delivery, etc.—between nodes that may or may not be anonymous.
Thus the SWIFT snooping program exploits the strengths that the market has endowed financial databases with, while the NSA snooping program is fighting an uphill battle against the ever growing volume of communications data that the market demands from telecom networks.
Oversight is key
Although I think that the SWIFT snooping program is a good idea, I almost certainly wouldn't support a similar program for snooping domestic transactions. Why? Because I'm one of those "give me liberty or give me death" fanatics, which means that I have two main criteria for any kind of government program that involves spying on innocent citizens:
Operational effectiveness
Oversight
There's so much international money tied up in the integrity of the SWIFT database that I have a fairly high degree of trust that the SWIFT snooping program is subject to strict controls and international oversight. We're not talking about a domestic company that's going to roll over for the feds, and a Congress that's going to look the other way while the Executive branch does whatever it likes. There's real money at stake here, and much of it belongs to foreigners who are going to be concerned about things like corporate espionage and the US using the data to give domestic businesses an unfair edge.
A very short intro to the SWIFT database story
One of the US government's first priorities in the aftermath of 9/11 was to strike terrorism right in the pocketbook. It was in the context of their attempts to freeze terrorist assets and to trace the sources of funding for international terror organizations that the Bush administration first learned of the SWIFT database. They immediately moved to gain access to it, and at one point they allegedly wanted a copy of the entire thing for antiterror purposes.
Here's the NYT's description of the SWIFT international banking database:
Swift's database provides a rich hunting ground for government investigators. Swift is a crucial gatekeeper, providing electronic instructions on how to transfer money among 7,800 financial institutions worldwide. The cooperative is owned by more than 2,200 organizations, and virtually every major commercial bank, as well as brokerage houses, fund managers and stock exchanges, uses its services. Swift routes more than 11 million transactions each day, most of them across borders.
The cooperative's message traffic allows investigators, for example, to track money from the Saudi bank account of a suspected terrorist to a mosque in New York. Starting with tips from intelligence reports about specific targets, agents search the database in what one official described as a "24-7" operation. Customers' names, bank account numbers and other identifying information can be retrieved, the officials said.
Immediately following the NYT's story, the Republican outrage machine cranked into high gear over the revelations, with a chorus of right-wing bloggers, pundits, and even the President himself condemning the story as damaging to national security. In fact, by turning the outrage knob up to eleven, the administration and its surrogates have been able to turn the conversation about the program almost completely into one about freedom of the press vs. national security. (This is a move that the left has been wholly complicit in, by the way.)
For what it's worth, I think it's a tragedy, and probably even a threat to national security, that the NYT is our now our first line of defense against Executive overreach. That used to be the job of Congress.
According to The Hive, Microsoft is working on a new way to prevent rootkits from occurring in Windows Vista. The technology, called Microsoft Patch Guard, will stop unauthorized software from modifying the Windows Kernel. Using Patch Guard, kernel mode drivers cannot write and replace items, and the technology will altogether stop third party software from extending and changing the Kernel (which is not allowed in Windows Vista).
Currently, Patch Guard is available in the x64 versions of Windows XP. It prevents kernel mode drivers from modifying system service dispatch tables, the interrupt descriptor table (IDT), and the global descriptor table (GDT).
It seems that with the help of their lawyers from Boies, Schiller and Flexner (Boies of US v. IBM, US v. Microsoft, and Bush v. Gore fame), The SCO Group has decided to expand their legal action on copyright infringement considerably. In maneuvering that conjures up images of the RIAA, SCO has decided to file copyright infringement suits against several of the 1,500 firms that received notice earlier this year of SCO's claims. Blake Stowell, the spokesperson for SCO was obtuse about the specific targets:
The first legal target is unnamed at this point, but "it will be one of the companies that received one of the [1,500] letters we sent out back in May, although it would have to be in the Fortune 1000," Stowell said.
Clearly SCO is taking a similar tact to the RIAA in which they appear to be trying to induce settlements from firms by making an example out of a select few. SCO believes that these suits will be considerably less complicated than their suit against IBM, and will therefore reach resolution quickly. One can't help but wonder on precisely what grounds that would be, as SCO has yet to reveal anything specific which would lend them a legal leg to stand on.
Perhaps more startling though, is the new suggestion from SCO that not just Linux, but BSD as well is in violation of their copyrights and that furthermore, they are planning on pursuing litigation on that front as well. The incestuous relationship between (Unix) System V, BSD, and Linux seems to have provided SCO with the impetus to start examining the settlement that was made between BSD and Unix System Laboratories in 1994.
"As part of the settlement agreement that took place between BSD and AT&T and Novell, there were certain files that had to have the copyright attribution put back in," Stowell said. "Copyright attribution has been stripped away from certain code, and we're seeing that same situation taking place with Linux with those same exact BSD files," he said. "Those files have gone back into Linux, and the copyright attribution has been stripped away."Those BSD files must have their copyright attribution restored, Stowell said.
Additionally, "there were Unix System V files within BSD that were not supposed to be there, and those files had to be removed from BSD," he said. "And we're claiming those files that were supposed to have been removed from BSD have made their way into Linux."
Between BSD and Linux, SCO has all but claimed everything Unix-y as falling under their copyright. It's not altogether clear who precisely SCO would go after in litigation over their BSD claims, but it's a safe bet that it will be someone. Presumably they are looking at all commercial organizations involved with the packaging and distribution of the major BSD flavors; and if their above strategy regarding Linux holds, they'll go after large companies that employ BSD-based systems as well. As an Aberdeen Group analyst observes though, it's not an insignificant burden of proof to show that just because there is BSD code in Linux, said code in Linux must be a derived work of System V simply because BSD code was in System V (or vice-versa). If you aren't confused yet, SCO has more to keep your head spinning. They are also suing Novell for competing with them. Last I checked, Novell isn't choosing new software companies to sue every week by throwing darts at a listing of companies on the NASDAQ, so it's hard to say how exactly they are competing with SCO, but as usual SCO asserts that it all comes back to Linux.
SCO claims its potential legal action in connection with the Novell-SUSE merger is based on a contract clause that allegedly prevents Novell from competing against SCO's Unix products. SCO claims it acquired this noncompete contract as part of an agreement between Novell and one of SCO's previous incarnations, The Santa Cruz Operation and has announced plans to sue Novell as soon as Novell's acquisition of SUSE is complete.
SCO has the burden of proof on two counts here. They must first show that such a noncompete clause does exist in a contract, and that said contract is valid. They must then demonstrate that their own business is one of selling Unix, and that the SUSE product (and it's use by Novell) is sufficiently similar to warrant competition under the terms of the contract. Considering that SCO has yet to demonstrate that their business is anything other than one of litigation or that their claims about Linux are valid, this suit is of dubious merit at present. Novell has already taken several shots at SCO, so it's a safe bet that a battle over the SUSE acquisition will get pretty nasty.~Write-up by Andy "UWSalt" Morton
Potentially buried in today's announcement of lawsuits filed by SCO against DaimlerChrysler and AutoZone were some significant developments in the ongoing SCO vs. IBM court battle. The lawsuit had been bogged down in discovery, and back in December, the presiding judge gave SCO 30 days to produce the particular Linux code it claims is in violation of their copyrights. Becoming fed-up with the lack of progress in the case, the judge has now ordered SCO fully comply with IBM's discovery demands within 45 days. This means that SCO needs to cough it up:
SCO has 45 days to identify "all specific lines of code" they allege IBM put into Linux from AIX or Dynix; identify and provide "with specificity all lines of code in Linux that it claims rights to; provide and identify with specificity the lines of code that SCO distributed to other parties, and this is to include "where applicable the conditions of release, to whom the code was released, the date and under what circumstances such code was released."
For its part, IBM has the same amount of time to provide some of the information requested by the SCO: "the releases of AIX and Dynix consisting of 'about 232 products' as was represented by Mr. Marriott at the February 6, 2004 hearing." Essentially, the judge is rejecting SCO's argument that they need IBM to first fork over AIX and Dynix in order to answer IBM's request. Instead, the two litigants will have to work in tandem to satisfy one another's discovery requests.
At this point, the burden is on SCO to comply in order for the case to move forward. It is a lock that the rest of the Linux world will be watching to see if SCO finally divulges the exact code that Linux allegedly copied. Continued foot-dragging on the part of SCO will provide more impetus for their opponents to vigorously contest SCO's claims - do not look for AutoZone and DaimlerChrysler to settle with SCO. At any rate, in 45 days, it will become apparent if SCO is holding four of a kind or if they have gone all-in with nothing in the hole.
For those interested, a text copy of the order is available from Groklaw.
Just when you think the SCO's legal efforts can't get any more
ridiculous, the litigation-frenized Utah-based company always
manages to
come through in a pinch. The latest comes in the form of an
recently unsealed
internal e-mail
(PDF) from 2002 in which an engineer at SCO, Michael Davidson, says
that a thorough search of Linux source code revealed no
infringing code. In the missive, originally composed by Davidson
and forwarded to SCO CEO Darl McBride, the aim of the code audit is
spelled out explicitly:
The project was a result of SCO's executive management refusing
to believe that it was possible for LInux and much of the GNU
software to have come into existance without *someone* *somehwere*
having copied pieces of propretary UNIX source code to which SCO
owned the copyright. The hope was that would find a "smoking gun"
somwhere in code that was being used by Red Hat and/or the other
Linux companies that would give us some leverage.
That was the original basis for SCO's lawsuit against IBM and attempts to get
companies using Linux to purchase SCO Source licenses. Buy a license which would
cover alleged SCO-owned intellectual property in the kernel, and you don't have
to worry about the SCO legal team descending on your company like ravens on
roadkill.
At the end, the lengthy project laid an egg.
Bob worked on the project for (I think) 4 to 6 months during which time he
looked at the Linux kernel, and a large number of libraries and utilities and
compared them with several different vesrions of AT&T UNIX source code.
At the end, we had found absolutely *nothing*. ie no evidence of any
copyright infringement whatsoever.
If you can remember back to when the lawsuit was filed in March
2003, SCO originally accused IBM of misappropriating SCO's Unix
technology and thereby "contaminating" all the work they have ever
done on Linux. Since that time, SCO has changed its tune in various
court filings, but the underlying theme has always been that Linux
stole code from Unix, and SCO claims they own the copyrights to
Unix. (The Novell case
has undermined that assertion as well). Now it is apparent that
SCO and its pugnacious CEO McBride have known that there is no
infringing code since the summer of 2002.
"That e-mail probably creates a lot more questions than it
answers," SCO spokesman Blake Stowell said. "We'll be fully
prepared to address that, but we will be doing that in a court
setting if it is necessary."
Later, SCO released an e-mail from 1999 from an outside
consultant that purported to find some troubling similarities
between the Linux and Unix source code, and claimed that the 2002
does not specifically refer to when the code audit Davidson is
referring to actually took place. However, Davidson seems to be
referring back to that same 1999 study in his e-mail. Either way,
you can't trump Davidson's e-mail with something produced three
years earlier.
If there's one thing that has become crystal clear as the various pieces of
SCO litigation have been meandering slowly through the courts, it's that the
SCO's case looks like the proverbial house built on sand. And high tide is on its way.
In a recent interview, SCO Group CEO Darl McBride shares his vision for the future of SCO. Most notably, he declares that SCO is going to be shifting its focus away from litigation towards its ostensible core business, Unix. In fact, he does not anticipate filing any further lawsuits:
"I think right now we've got the claims in front of the various courts that we need in order to get our complaints heard and to get them argued and to get resolution. With respect to being more vocal or going after new targets at the customer level, we don't see the need for that. We had the need to get the basic issues on the table, but we're fine to argue the merits of what we have out there right now (in) the current litigation setting."
Given that recent court rulings have made it clear what the judges think of the "basic issues on the table," further litigation would likely be a waste of what's left of SCO's bankroll. Given that setback and the fact that BayStar (the venture capital firm which had invested US$50 million) is threatening SCO with legal action to recoup its investment, SCO is apparently losing the stomach for more legal battles.
McBride is using this week's SCO Forum 2004 to tout the company's strategy. They recently filed for a trademark on Unix Systems Laboratories, AT&T's old Unix division, which looks from here like an attempt to give them some old school Unix street cred. In addition, they are apparently looking at resurrecting Project Monterrey (a failed collaboration with IBM to port Unix to the Itanium) and possibly working on a 64-bit version for the Opteron and Xeon CPUs. Of course, if their current litigation efforts turn out the way just about everybody but SCO expects, they probably will not be around long enough see any of those plans come to fruition.
The SCO Group's lawsuit against Linux distributors IBM, Red Hat, and Novell hit a major snag yesterday as Magistrate Brooke C. Wells of the US District Court in Salt Lake City dismissed 182 of SCO's 294 claims. The dismissal is part of a 39-page ruling that comes down hard on SCO for continually refusing to provide specific details about which lines of code in SCO's products were stolen by Linux programmers.
"SCO's arguments are akin to SCO telling IBM, 'Sorry we are not going to tell you what you did wrong because you already know,'" Wells wrote in the ruling. "Given the amount of code that SCO has received in discovery, the court finds it inexcusable that SCO is, in essence, still not placing all the details on the table."
SCO's reaction to the news was predictably unrevealing. "Our legal team is reviewing the judge's ruling and will determine our next steps in the near future," said SCO spokesperson Blake Stowell.
The painfully long and drawn-out saga of the SCO lawsuit started in March 2003, when SCO sued IBM for allegedly "devaluing" their version of the Unix operating system. The lawsuit was extended to include Novell later that year. SCO claimed that both companies had taken code from Unix and put it into Linux. The company was ordered in 2004 to hand over examples of the infringing code in no less than 45 days, but SCO has managed to dither and delay without producing any hard evidence. Clearly, the judges are losing patience.
So will SCO decide to reveal their trump card after all? It seems highly unlikely. In a memo that was leaked last year, SCO's internal code audit from 2002 found no infringing code in Linux. SCO's response at the time was to reveal an 1999 e-mail from an outside consultant that allegedly found "troubling similarities" between Linux and Unix code. However, similarities do not equal stolen code. SCO has even backtracked from their initial assessment, stating that the problem may lie in "methods and concepts" rather than stolen lines of code.
SCO may have known for a while that their case was hopeless, and are simply deciding to see the case through in hopes that they may at least recover a small amount of money through technicalities. The company stated in 2004 that they wanted to concentrate more on new products than on lawsuits. This latest news may force them to do just that.
SCO's stock fell from US$4 to below US$3 on the news, and currently sits at US$3.25.
Electronics have long been recognized as a weak link when it comes to secure conversation. From bugs hidden in lampshades to phone taps to keystroke tracking software, electronics provide the easy path to monitoring and censoring communications. In no area is that so apparent, perhaps, as in text messaging, as some users around the globe are discovering the hard way.
Text messaging and the first level of censorship begins at the phone. While it's certainly possible to enter any word using the alphabetic method in which a=2, b=2-2, c=2-2-2, d=3 and so on, it isn't very convenient. This has led manufacturers to develop alternate systems like T9, which make it easier to enter common words. T9 works by using algorithms to determine what word a user is trying to enter. Punching 2-2-8 might default to "cat" for example, since that's a common word which uses the letters associated with those numbers. It might also give you "bat" however, which is another logical guess based on the letters available through those keystrokes. Usually, a provision is made for selecting words other than the algorithm's first guess.
Where things start to get hairy is when a user enters something like 3-8-2-5, which can spell either "dual" or a somewhat naughty word which you won't find in your family newspaper. (Raise your hand if you aren't looking at a phone right now. I thought so.) In that case, the manufacturer could design the phone to provide the second word as an alternate, or more likely, avoid it altogether. In a nation like the US, avoiding a word which some might find objectionable is a business decision that probably prevents some complaint letters. In other countries, it could be a government mandate, and the banned word might not be 3-8-2-5, but something like "liberty" or "Taiwan."
At first, that sounds inconvenient, yet relatively benign. After all, a user could still switch to alphabetic entry and write anything they want, right? Perhaps, but the second level of control involves monitoring and censoring the messages of users, as the Chinese government has been doing since the SARS outbreak of 2004. At that time, word about the SARS epidemic spread like wildfire despite very little coverage by the government-controlled press. Since realizing the informative power of mobile phones, Chinese authorities have monitored and filtered text messages as a matter of course.
The problem doesn't begin or end with China. Security agencies in countries as diverse as Iran and Germany have been spotted responding to text messages regarding political leaders or outlawed ideologies. Much of this communication scanning is done with the compliance of the mobile phone providers, which simply consider it the price of doing business in various countries. We've seen this before, as in the case of Google and other portals filtering search results to suit local authorities.
The good news is that censoring communication continues to remain something of an arms race. While aficionados of such evil words as "Taiwan" or 3-8-2-5 might sometimes find themselves under scrutiny by the Powers That Be, there's nothing to stop them from switching to slang which means the same thing. Taiwan might be referred to as "the neighbors," for instance, while 3-8-2-5 could be "frak."
The firewall, which uses routers supplied by Cisco, works in part by inspecting Web traffic for certain keywords that the Chinese Government wish to censor, including political ideologies and groups it finds unacceptable.
The Cambridge research group tested the firewall by firing data packets containing the word "Falun" at it, a reference to the banned Falun Gong religious group. The researchers found that it was possible to circumvent the Chinese intrusion detection systems (IDS) by ignoring the forged transmission control protocol (TCP) resets injected by the Chinese routers, which would normally force the endpoints to abandon the connection.
"The machines in China allow data packets in and out, but send a burst of resets to shut connections if they spot particular keywords," explained Richard Clayton of the University of Cambridge computer laboratory. "If you drop all the reset packets at both ends of the connection, which is relatively trivial to do, the Web page is transferred just fine."
Clayton added that this means the Chinese firewall can be used to launch denial of service attacks against specific IP addresses within China, including those of the Chinese Government itself.
The IDS uses a stateless server, which examines each data packet both going in and out of the firewall individually, unrelated to any previous request. By forging the source address of a packet containing a "sensitive" keyword, people could trigger the firewall to block access between source and destination addresses for up to an hour at a time.
If an attacker had identified the machines used by regional government offices, they could block access to Windows Update, or prevent Chinese embassies abroad from accessing specific Chinese Web content.
"Due to the design of the firewall, a single packet addressed from a high party official could block their Web access," said Clayton.
Even though this technique would only block communication between two particular points on the Internet, the researchers calculated that a lone attacker using a single dial-up connection could still generate a "reasonably effective" denial of service attack. If an attacker generated 100 triggering packets per second, and each packet caused 20 minutes of disruption, 120,000 pairs of endpoints could be prevented from communicating at any one time.
Clayton, speaking at the Sixth Workshop on Privacy Enhancing Technologies in Cambridge last week, said that the researchers had reported their findings to the Chinese Computer Emergency Response Team (CERT).
Having the public use your company name as a verb is, in one way, a marketer's Shangri-la. Many companies would kill for the name recognition and popularity of "xerox," "google," and "hoover." For the companies themselves, though, being "verbed" has its dark side. A company that does not defend its trademark risks losing it when it becomes a common figure of speech, which explains why Google is not happy about its recent inclusion in a new Merriam-Webster dictionary.
Merriam-Webster's Collegiate Dictionary now includes the word "google," which it defines as a verb (prediction: within a decade, it will also be an expletive, as in "go google yourself!"). Sensitive to the search engine's concerns (and to its legal team), the dictionary does note that the word is trademarked. Dictionaries, news outlets, and websites that do not note this risk receiving a strongly-worded letter asking them to either remove the offending use of the term or note that it is trademarked.
Right now, "to google" actually means to use the Google search engine. Google no doubt fears that the world will go the way of "hoover," which has become a generic term for "vacuum" in the UK. If "google" ever comes to mean "I searched for it on the Internet," the company's careful branding and promotion will be diluted and the name will lose value.
"Google" joins other up-and-comers in the new dictionary, sharing page space with words like "himbo" (male bimbo) and "mouse potato" (think couch potato, but with a computer). Although it has just made it into the dictionary, "google" has been on linguists' radar screens for years. In 2002, for instance, it was judged the "most useful" new word of the year by the American Dialect Society.
We've known for a little while that Intel's next-generation Itanium2 part, Montecito, is due out in July, but now a specific ship date has surfaced: July 18th. Montecito began sampling way back in 2004, and when I saw an ISSCC presentation on it in early 2005, it was supposedly just around the corner back then. So after over a year's delay, I'm sure HP—the primary remaining vendor of Itanium2 systems— is ready to start selling systems.
Digitimes has a few added details on the much delayed processor, with their sources indicating that the unit price will range from US$749 to US$3,692. That's a lot of scratch, but Montecito is a lot of processor.
The 1.72 billion transistor, dual-core beast will scream in floating point, and it may help build the momentum that the Itanium Solutions Alliance has kept going. In spite of persistent rumors of the death of the architecture, Itanium2 brings in more money every year. Of course, at the rate things are going, it'll be a long time before Intel recoups its investment in IA-64, but I guess stranger things have happened.
For a good overview of Montecito, be sure to check out Paul DeMone's Sizing up the Super Heavyweights. Sure, it was published back in 2004, but its coverage of Montecito hasn't aged nearly as much as SGI's senior executives in the past two years.
Mike Shaw, ABC's President of Advertising Sales, said this week that he would love to have the opportunity to shut down the "fast forward" button on users' DVRs. Though he did not claim that commercial-skipping TiVo owners were thieves, Shaw is clearly unhappy with how easy it is to skip his network's ads.
Shutting down the fast forward feature on DVR isn't an easy proposition, though; ABC would require support from DVR makers, who could build their devices to recognize the presence of a broadcast flag. The flag would tell the DVR whether fast forwarding should be allowed during any particular show. But what DVR maker would want to do this? Shaw told MediaDailyNews that he "would love it if the MSOs [cable companies], during the deployment of the new DVRs they're putting out there, would disable the fast-forward [button]." It doesn't require a great leap of imagination to suspect that ABC is in such talks with cable companies right now.
From the network's perspective, this is probably the best way to neutralize ad-skipping, which has taken its toll on network ad revenues over the last few years. Trying to talk a company like TiVo into disabling features that consumers want is probably a hard sell, but cable companies might be more receptive to the idea. Most cable operators also make money from local ad revenue, and would stand to gain revenue by making it harder for their customers to skip ads.
What about a customer backlash? In Shaw's view, this is unlikely, as he believes that most people enjoy their DVRs simply because they can time-shift video content like a high-tech VCR; commercial skipping is just gravy. Take away the gravy, and customers should still be happy to eat their mashed potatoes.
"I'm not so sure that the whole issue really is one of commercial avoidance," Shaw said. "It really is a matter of convenience—so you don't miss your favorite show. And quite frankly, we're just training a new generation of viewers to skip commercials because they can. I'm not sure that the driving reason to get a DVR in the first place is just to skip commercials. I don't fundamentally believe that. People can understand in order to have convenience and on-demand (options), that you can't skip commercials."
The majority of DVR users do skip commercials, though, making this a potentially risky strategy for the cable companies. So long as competitors like TiVo and HTPCs exist, unhappy customers can always go elsewhere. Should TiVo, especially, ever fold, most consumers would no doubt settle for whatever is offered by their local cable provider.
Even as ABC pushes to halt fast forwarding, customer anger at TV advertising is growing. Though Shaw notes that "we've had the exact same commercial load for three years in a row," he's been hearing more complaints from customers about these intrusions into stories they are trying to watch. While no doubt irritating to consumers, who would skip commercials if they could, there's no denying the fact that network TV shows are only free because of that advertising. Finding the right balance is proving to be tricky.
In Europe, by contrast, some television services are actually encouraging more use of DVRs. While ABC seeks ways to restrict DVR functionality, satellite provider BSkyB wants to offer more of it to its European customers. The company just announced a plan to allow remote recording of TV shows by using one's mobile phone. Users can set their home DVRs by texting to a special number or by accessing a program guide right from their handset.
Give customers what they want and they will open their wallets. That seems to be the story behind the newest figures from Nielsen SoundScan, which paint a more positive picture of the music industry than the RIAA's gloomy prognostications often suggest.
The figures from the first half of this year show that CD sales, not surprisingly, are falling. Compared to the first half of last year, they are down by more than 4 percent. More than making up for that decrease is the tremendous growth in digital downloads. Downloads of complete albums soared by 126 percent, while downloads of individual tracks grew by a mere 77 percent. As you might imagine, individual track downloads still dwarf those of complete albums, lending credence to the theory that the album as an art form is dying.
Oddly enough, the year's biggest smash comes from the soundtrack to Disney's TV movie, High School Musical, which has moved 2.6 million copies. And you wonder why marketers target children.
The study can also be used to get a sense of the independent music market. The AP calculates that indie labels control 12.79 percent of the market, while the Hollywood Reporter tots up the figures a bit differently to arrive at 19.3 percent (they include independent distributors which are operated by major labels). Overall, the music industry has actually grown in the last year, selling 0.1 percent more than it did in the first half of 2005. That's not massive growth, but neither is it the jobs-killing decline that the industry used to moan about a few years ago.
Though the general public is sometimes portrayed as a greedy mass of pirates, the new figures show that the eyepatch-wearing demographic is only one part of the equation. Giving good legal alternatives, consumers are happy to stay legit if that means that they can conveniently cherry-pick digital hits for their own collections.
It's been so long since I've seen something move out of the infamous "three to five years away" category and into actual volume production that I can't actually remember the last time it happened. But Freescale is making it happen now with magnetoresistive RAM (MRAM).
Freescale just announced that they've gone into volume production of a 4Mbit (256K x 16-bit) MRAM chip. The newly announced MR2A16A MRAM part, which has been in development for a decade and which sampled back in 2004, already has attracted a number of buyers and should start showing up in products before too long.
Back before Freescale was spun off from Motorola, we covered a couple of Motorola demonstrations of working MRAM chips. At the time, IBM, Infineon, Motorola, and the legion of twenty or so other companies that have been working on this technology were projecting that the first MRAM products would hit the market in 2004. Freescale has beat the rest of the pack, albeit two years behind schedule, and I expect to see a steady march of MRAM product announcements in the next year as the rest of the memory industry begins bringing their own products to market.
The big picture for MRAM
I went fishing about for datasheets on NAND flash, and for devices organized as 16-bit words I found access times (read cycle) in the 50ns to 90ns range. This makes the new 35ns MRAM part quite a bit faster than current NAND flash. However, it's nowhere near the 3ns to 5ns access time of DDR2 DRAM; 35ns is more on the order of old EDO SDRAM.
So while MRAM's current access time and capacity won't make it an immediate replacement for either DRAM or NAND Flash, both of these numbers will scale to make it a major long-term player in the storage market. For large categories of devices that are more sensitive to power than raw performance, MRAM will replace DRAM and Flash altogether. For the longer-term performance picture, MRAM well replace DRAM in all but some niche applications. IBM's research shows that they can get access times down into the 3ns range, and a paper from earlier this year shows that they expect to see MRAM access times between 5ns and 20ns.
As for hard disk technology, I think it's pretty clear that MRAM isn't going to become the default medium for mass storage any time soon. I imagine that many types of portable devices, from laptops to media players, will eventually use a combination of MRAM and hard disk technology. A single large MRAM pool could combine the functions of both main memory and the kinds of backing store/caching technology that's starting to make its way to market in the form of Intel's Robson. This fast memory pool would be coupled with hard disk-based mass storage.
MRAM's biggest medium-term impact will be on the Flash market. MRAM is superior to Flash memory in pretty much every way—it's faster, and unlike Flash, it can take an unlimited number of reads and writes. As the rest of the industry starts to move into volume production of MRAM, the new technology is going to start moving into niches currently occupied by Flash and begin squeezing it out in some places.
Nearly everyone has been affected by Google and its sheer ubiquity—to the point where "to google" has even become an officially recognized verb. Google gained this dominance by providing the best web searching service, which required the ability to quickly "crawl" the web, finding and indexing all the content it could get its hands on. Anyone who has set up a web server and peered at the access logs know that the Google spiders come quickly and often.
One thing that most people weren't aware of, however, is that Google is indexing more than just text and images. The search engine is also capable of indexing and searching binary files, a feature that the security firm Websense has been taking advantage of to uncover malicious and hacked web sites all over the world.
The company utilized a little-known feature of Google to search for binary strings representing Windows-based worms such as W32.Bagel and W32.Mytob. "They [Google] actually look inside the internals of an executable and index that information," said Dan Hubbard, senior director of security at Websense.
Websense plans to share the code they have developed using the Google APIs with other security researchers, but does not plan to release it to the public. Hubbard fears that would-be virus authors could use the tool to jump start their activities. "Instead of buying them on the black market [an attacker] could search for them and download them on his own," he said.
Hubbard isn't the only one concerned about the possibilities of searching for binary patterns on the web. Claudiu Spulber, of the Homemade Computer Tutorials blog, pointed out that hackers could embed common search terms into the binary, and then hope that users looking for a particular page would find a link to the program, click on it, and run the executable. The blog post includes an example of this, an illegitimate version of the shareware "Backup4all" program, but interestingly the malicious version no longer shows up on the results page. According to a spokesperson for Google, the company continues to keep an eye out for this practice.
Should Google continue to index binary files, despite the potential drawbacks? The company's position is that the more things on the Internet that are searched, the better things are for everyone, and that people shouldn't worry too much about any possible misuse. Still, the more powerful a tool becomes, the more the potential for abuse increases. This applies not only to Google, but to the Internet in general. As always, skeptical computing is the best defense.
Raise your hand: who here has ever booth-squatted for hours in a Panera, nursing that bottomless cup of coffee, using the restaurant's free WiFi connection the entire time? It's a common enough scenario that cafe and coffee shop owners, long used to offering free WiFi as a way to get customers in the door, are fighting back.
The Boston Globe has an interesting piece on these "WiFi wars." Cafe owners have found that WiFi brings customers, but also a host of problems. Some people purchase nothing at all, some buy the cheapest item on the menu, and most stay for hours at a time, tying up tables that are especially needed during the lunch rush. Others park outside and surf the Net from the comfort of their vehicles—for months on end.
Coffee shop owners are now fighting back. Customers who purchase nothing are politely asked to do so, while WiFi is sometimes shut down at peak periods. Other shops have begun charging for the service, more as a way to limit access than to make money. Panera has put up a message on the first page people see after connecting to their network; it asks patrons to limit use during peak hours and makes them click a button agreeing to the terms of use.
But the issue is not just an economic one; it has a cultural side as well. Cafe owners and traditional patrons are concerned that the shops are becoming offices. Confronted by a sea of laptops and hard-working coffee sippers, other guests may feel less able to talk, laugh, and be sociable. The forest of raised laptop screens might also keep patrons from talking with one another, and that social element has long been a part of cafe culture. It was this problem that led one Seattle coffee shop to start shutting off the WiFi on weekends last year. Not only did revenue go up, but the atmosphere in the cafe changed as well.
Coffee shops raise, in miniature, the essential political question: what sort of society do we want to create? Not surprisingly, there's a difference of opinion. Customers who use cafes to meet others and to socialize with friends are disappointed by the many laptops and by the shortage of tables. Those working on laptops find themselves wishing that the retired friends at the next table could talk about their golf game in lower tones. And each owner has a vision of her own.
Personal experience suggests that most laptop users who work at coffee shops don't do so because they are too cheap to rent an office. No, people whose work involves the solitary punching of laptop keys enjoy being around other people, and coffee shops and cafes provide a comfortable and inexpensive environment where one can feel less isolated while working. Looked at in this way, the "WiFi wars" are less over Internet access and more about the type of jobs that we do—jobs that require little human interaction but do require the constant tether of a network connection.
According to documents filed by Creative with the United States International Trade Commission in mid-May, Steve Jobs approached a Creative employee at MacWorld 2001 to talk shop about Creative's then flagship product, the Nomad Jukebox. What begun at this innocuous meeting may well have led to the creation of a digital audio empire that could have rivaled what Apple has put together on its own. On the other hand, it may have also led to stunning failure for both parties.
In the past few months, one of Apple's biggest rivals in the digital audio player space, Creative, has begun competing with Apple using legal tactics. Looking back before the birth of the iPod, Creative had a significant lead on Apple, with almost a full year's experience in developing and selling hard-disk-based players. What is almost completely unknown, however, is that Apple was willing to work with Creative to develop an Apple-licensed player as opposed to building their own from scratch.
According to Creative's legal filings (available via PACER) with the US International Trade Commission, Apple had originally sought to license Creative's IP and create the iPod on Creative's platform. The filings reveal that Apple was unsure that this scheme was going to be profitable in the long run, and proposed the radical (for Apple) idea of partnering with Creative to create a digital audio player. Creative decided against joining forces, and the rest is history.
While one might be quick to assume that a merger between the two companies would have led to a marriage punctuated with hundred dollar bills falling from the mirrored ceiling of their honeymoon suite, it may also be that a mating of these two wildly different corporations would have been destined for failure. Apple's reckless abandon coupled with Creative's bulky hardware and stodgy corporate culture would likely have resulted in clashes of personality, among other things.
As it stands, Apple scored big when Creative turned down their proposal, freeing them up to engineer the iPod as they had envisioned it, rather than working off of one of Creative's prototyped designs—or using Creative's patented IP, as the lawsuits allege.
Creative and Apple's relations were dotted by pithy remarks and under-the-belt jabs until this lawsuit business reared its head. Apple was able to take a radically different approach to marketing, growing their market while making a lot of money. As a result, the iPod has become synonymous with the portable digital audio player market. Today, Creative and Apple stand with patent lawsuits loaded, waiting for the other to flinch first—unless they can come to a settlement themselves.
Yesterday I promised a second story on memory from the EET, so here it is. We've done a number of MRAM "technology demo" stories over the years, and I guess we'll keep doing them until the technology actually hits the mass market and we can start reporting on actual MRAM-based gagdetry. Anyway, now for this year's MRAM demo story: Motorola just demoed a 1 Mbit MRAM device with copper interconnects. The new design is bigger and faster than Moto's previous 256Kbit demo chip, which I reported on last year.
By way of explaining what MRAM is and why it's such a cool thing, I'll just reproduce a news post that I did back in 2000. (It seems like I'm blockquoting myself alot lately... I would just link it up, but our archives for that year don't have embedded bookmarks that take you right to the post.)
If you've read the Ars Technica RAM Guide, then you're familiar with the problems associated with making DRAM work the way it's supposed to. Charges that leak out and have to be refreshed prevent DRAM from reaching the speed of something like SRAM. And even SRAM is volatile, meaning that its data disappears when you turn the power off. For a while, flash RAM was supposed to be the ultimate in memory technology--a nonvolatile type of RAM that's fast and doesn't require refresh circuitry. But flash RAM has its own problems, not the least of which is its extremely high cost per bit.
Enter Magnetic RAM (MRAM). MRAM uses magnetic bits, like a hard disk drive, to provide fast, cheap, nonvolatile storage that's supposed to be 30 times faster than today's DRAM. It could be used in mobile gadgets, as well as in PCs. The PC applications are particularly interesting, because it would essentially eliminate the need for a hard disk drive. This means that everything boots instantly. Of course, in the beginning, when it's still more expensive than secondary storage, it'll probably only be used to store the OS and maybe some critical applications.
A number of companies are working on MRAM projects, and this Wired article describes the work going on at IBM. IBM's MRAM project is being headed up by the same guy who invented the GMR technology that's in all modern hard drives, so it's got quite a pedigree. And as for when analysts expect MRAM to appear on the market... say it with me, now: "it's about five years away." Props to Sirius for the link.
For the past few years, IBM, Moto, and others have consistently placed the introduction of MRAM-based devices in 2004, and this latest article also gives the same date. So maybe we'll actually see this stuff within the predicted timeframe. Also, for more on the workings of MRAM, see this excellent overview of next-generation nonvolatile memory technologies from the EET.
Chaos theory is generally a misunderstood and misused part of science. Essentially chaos theory tells us that some nonlinear systems* are inherently unpredictable even when we understand how the factors involved interact. The typical example used for this is weather prediction so I will stick with that example. We have a pretty sophisticated understanding of how our atmosphere works, yet short-term weather forecasts based on this understanding are generally pretty bad. This is because although we understand the dynamics (e.g., how temperature and pressure interact) really well, we don't know the pressure, temperature, and everything else at every location on earth to infinite precision. In a nonlinear system these uncertainties can blow up to huge proportions, rendering any precise prediction** of the weather over Cape Town useless. So that takes care of nonlinear systems. Linear systems, on the other hand, are predictable... or are they?
Some new research, which has been percolating for a few years has started to question that knowledge. It all started with the mathematical discovery that the addition of suitable linear functions leads to something that is chaotic by every test available to us. Since then work has continued, with the latest examining the action of electronic filterssimilar to those used to tune your radio or TV. This latest bit of research has demonstrated both mathematically and with a physical test filter that it is not so simple after all. If you drive such a filter with a random signal then the time-reversed data has all the attributes of a system that exhibits chaotic system.
Although no one is aware of a real linear system that exhibit chaotic behavior, the simplicity of the filter system implies that it is unlikely that such systems do not exist. The fact that such systems exist at all is a bit of shock, and at the very least they may offer some insight into how noise can interact with electrical and mechanical systems in unexpected ways.
* An example of a linear process is an idealized spring, where the extension of the spring depends only on the force extending it. However, if you extend a spring too far it will become nonlinear, in which case the extension depends on the force applied and how far the spring is extended.
** However, this doesn't prevent one knowing the average behavior of the whole system, which is what global climate models do. In fact a great deal of information can be extracted from these models, as long as you don't want specific, "what will be the temperature in Baltimore at 3 pm Thursday," information.
The laser has been a boon to science, technology and business in general. I think it is a fair comment to say that there are few aspects of our lives which are not illuminated by coherent light in some way. The development of methods to control material growth has prompted many new developments in laser technology, which have been largely responsible for the increasing carrier capacity of modern communications systems. As we reported on earlier, the growth of materials with structures precise enough to control the optical properties of the medium are now a reality.
Materials that have a nice periodic structure will act like a semiconductor for light, where only light with a particular wavelength range can be transmitted by the structure. Researchers have made such a structure by "drilling" holes into the gallium arsenide part of what is essentially a laser diode. But instead of drilling the holes in a perfectly regular pattern, they periodically skipped holes. These gaps in the structure form little islands where all light can propagate, thus the surrounding structure looks like a big bunch of mirrors, all reflecting light back into the gap.
These devices can take advantage of something called the Purcell effect, which you can think of as any light emitter, be it atom or electron-hole pair, experiencing all the possible ways it can emit and "choosing" one. The choice is usually governed by how much light is floating around the emitter at the time. Thus, within the gap, there are very few choices for emission, so the light quickly builds up in the gap, leading to a bright glow and finally lasing. Outside the gap, there are no choices for emission, so the emitter doesn't emit radiation. The implications are that these lasers have a very short time period between excitation and laser action occurring, which means they can be switched on and off very quickly. In fact, the researchers showed that they can switch the laser at 100GHz, which was as fast as their detector manage.
Normally this paragraph is reserved for me to impart a few words outlining what I think is the real significance of the research just described. However, in this case I think the 100GHz modulation speed speaks for itself. Google, I think your pipes are going to get a lot fatter.
A while back we covered some research that suggests that dredging up a memory may involve reliving aspects of it. Clearly, however, the mind doesn't want to be constantly living in the past. So how does it decide what's worth re-living? Research with people suffering from retrograde amnesia suggest that both the encoding of memories and the ensuing recollection may be a two track practice, with the act of linking these tracks being a crucial step. Some recent work using an anti-anxiety drug (midazolam) that mimics the effects of amnesia provides support for this model in otherwise healthy brains.
The proposed model of memory and recollection that's being tested suggests that the brain handles episodic memory (think of this as a surveillance video of your life) separately from its handling of recognition and familiarity. The key step in creating a recollection of the past is to tie the recognition into the appropriate piece of episodic memory. This linking, termed "unitization" by the authors, is what allows a recognition of familiarity to trigger a memory. The authors tested individuals by having them study words, images, and abstract diagrams, and then testing whether they remembered items from their study in the presence and absence of midazolam. Recognition of abstract diagrams, which do not trigger recollection well, was equally bad with or without the drug. But in the presence of midazolam, words and images, which normally trigger recollection, were as difficult to recollect as the abstract diagrams.
The researchers propose that the poor recollection in the presence of midazolam reflects the study participants relying on familiarity, rather than recollection. They propose that both forming and traversing the link between recognition and memory is a demanding process, and the drug simply limits the ability of the brain to meet these demands, rather than specifically targeting this process. This also explains why abstract diagrams are so poor at triggering recollection: too many resources are spent simply on interpreting them. It also makes some sense out of why the recollection process works poorly in the elderly, who may have reduced mental resources in general.
This report helped make sense out of an article that recently appeared in the New York Times weekend magazine, which covered rare individuals who have a persistent form of déjà vu in which every experience seems familiar. In this case, it was suggested that the recognition and familiarity activity gets out of synch with the memory storage activity. By the time it fires, the memory has already been stored as part of the past, and is identified as such. The net result is a recollection machinery that is constantly sending out signals that every experience is familiar, mere instants after the experience itself.
I've noticed there's a bit of confusion about the differences between the two ways to create many-to-many relationships using Rails associations. That confusion is understandable, since has_many :through is still pretty new there isn't much written about it. has_and_belongs_to_many is the old, established player and most stuff out there assumes that's what you use for a many-to-many relationship. In fact, a lot of people don't seem to grasp that there is a difference at all!
As we all learned from watching classic movies, the best way to tell the difference between two prospective choices is to have a dance-off. You get to see everyone's moves (which of course are an accurate reflection of inner character), nobody has to die, and the hummable tune makes it a shoe-in for an Oscar nomination. Well, it's either that or do one of those boring "compare and contrast" essays they taught us about in sixth grade English class.
So who are the players we have to choose between? Let's take a quick look at them before the music starts and we get to see their moves.
Join Table: Simple Associations
Table:
create_table "dancers_movies", :id => false do |t|
t.column "dancer_id", :integer, :null => false
t.column "movie_id", :integer, :null => false
end
Models:
class Dancer < ActiveRecord::Base
has_and_belongs_to_many :movies
end
class Movie < ActiveRecord::Base
has_and_belongs_to_many :dancers
end
has_and_belongs_to_many associations are simple to set up. The join table has only foreign keys for the models being joined - no primary key or other attributes. (Other attributes were supported using push_with_attributes for a while, but that feature has been deprecated.) There is no model class for the join table.
class Appearance < ActiveRecord::Base
belongs_to :dancer
belongs_to :movie
end
class Dancer < ActiveRecord::Base
has_many :appearances, :dependent => true
has_many :movies, :through => :appearances
end
class Movie < ActiveRecord::Base
has_many :appearances, :dependent => true
has_many :dancers, :through => :appearances
end
has_many :through associations are pretty easy to set up for the simple case, but can get tricky when using other features like polymorphism. The table for the join model has a primary key and can contain attributes just like any other model.
Checking out the moves
Here's the basic feature comparison of the two options.
Association
has_and_belongs_to_many
has_many :through
AKA
habtm
through association
Structure
Join Table
Join Model
Primary Key
no
yes
Rich Association
no
yes
Proxy Collection
yes
no
Distinct Selection
yes
noyes
Self-Referential
yes
yes
Eager Loading
yes
yes
Polymorphism
no
yes
N-way Joins
no
yes
There's a lot of good stuff packed into that table, so let's break it down and see what it's all about.
Structure
has_and_belongs_to_many uses a simple join table where each row is just two foreign keys. There's no model class for the join as the join table records are never accessed directly.
has_many :through upgrades the join table to a full-fledged model. It uses a model class to represent entries in the table.
Primary Key
Join tables have no primary key. I've heard some people like to create a primary key from the pair of foreign keys, but Rails won't use that primary key for anything. I'm not sure what you'd get from creating that key, though it might give you a performance benefit depending on your database. (I'm not a DBA so I have nothing more to say about that.)
Join models have primary keys, just like every other model. This means you can access and manipulate records directly.
Rich Association
Way back before Rails 1.1, you could use push_with_attributes to store extra attributes in your habtm join table. There were all sorts of problems with doing that, including not being able to update the attributes later on. push_with_attributes has now been deprecated. If you want a rich association with extra attributes, use a join model.
Proxy Collection
One of the advantages of using habtm is that associations are proxy collections. That means you can create entries in the join table using the association's << method, just like with has_many associations. Since join model records have those extra attributes, it is more complicated to create them automatically the same way join table entries can be. Rails punts on this, so you have to create the join model entries manually. (For a full explanation, see my Why aren't join models proxy collections? article.)
Distinct Selection
Sometimes join table (or model) can have multiple references between the same records. For example, a person may contribute to a book as a writer and an illustrator. If you have multiple references, the database will happily return you all those multiple copies in response to your query. The option :uniq tells the association to filter out duplicate objects so you only get a single copy of each. This is similar to using the DISTINCT keyword in SQL, though the removal of duplicates happens in Ruby instead of the database. When this article was first written only habtm supported :uniq, but now through associations do as well.
Self-Referential
Both habtm and through associations can be self-referential. Users being friends with users is an example of a self-referential relationship. You can do that with habtm using the :foreign_key and :association_foreign_key options on the association. You can do the same thing with through associations, though it's obscure how to do it so I'll have to write up how to manage it soon.
Eager Loading
Both habtm and through associations support eager loading of associated objects with the :include option.
A habtm association can only join two models. But sometimes you need to represent an association of multiple models. For example, a booking might represent a flight, a passenger, and a seat assignment. Using a through association, you can create a join model that joins as many models as you need. The tricky part is building the queries to get at the associated objects conveniently.
And the winner is...
has_and_belongs_to_many is light on his feet and has some smooth moves. But has_many :through is versatile, even if he has to work harder and his moves are a bit rough in places.
Seriously, there's no way to pick a winner here. Like any engineering decision, choosing a join table or a join model is a matter of picking the right tool for the job (or the right dancer for the part). Now that you've seen our players go head to head, you can make a better choice about who should get that part.
Late last week, the New York Times dropped the not-so-surprising revelation that the US government has its nose deep into the world's largest international financial database, the Society for Worldwide Interbank Financial Telecommunication (SWIFT) database, looking for leads on terrorist activity. If you haven't been following this story, then you'll want to jump down to the last part of this post, because I'm first going to dive right into why I support this particular antiterror program.
Regular Ars readers who're familiar with my previous, critical coverage of other invasive, electronic snooping programs—criticism that goes all the way back through the Carnivore/ECHELON days and extends right up to the NSA's domestic surveillance program (formerly TIA)—might be surprised that I could support a program that taps into international financial records and looks for terrorist connections. Isn't such activity just as invasive and scary as the NSA listening in on our phone calls and reading our email? In a word, no.
Working with the market vs. working against the market
Anonymity is the enemy of commerce. This has been true four millennia, and it's even more true in our modern world of cashless commerce than it was in antiquity. Our entire modern financial system is built on the ability to verify the identity of all the parties involved in market transactions, either directly or by using a proxy like verifying that a particular credit card transaction fits the cardholder's typical purchasing behavior.
My point here is that, when the US government dips into a large financial database in an attempt to trace money as it flows between parties, they know exactly who they're spying on. The SWIFT snooping program works because the feds can start working their way through the network of transactions at a known node—a terrorist or terrorist financier. They then can look to see who that person is dealing with, and who their contacts are dealing with, and so on. This is the polar opposite of the NSA program, in which the government starts with a data flow and then tries to figure out identities of all the parties involved in the communication.
It is crucial that critics of the NSA program, and of other technologies of mass surveillance (TMS) efforts, keep these two types of programs separate. The SWIFT program makes sense, because you begin with a discrete and inherently finite collection of identities and try to trace the myriad connections between them. In contrast, the NSA program attempts to work in the opposite direction, from an overwhelming volume of connections to a very small pool of identities. If the former program works, it's because the financial industry has gotten very good at identifying all the parties to a transaction; if the latter program works, it's because we got lucky and happened to be snooping the right call at the right time.
Let me put this in market terms: there are massive, overwhelming incentives for the financial community to be able to verify the identity of each node in a network of financial transactions; as I just said, business is built on this knowledge and the trust that it engenders. This is not true for telecom networks, where the incentives are structured to reward transport capabilities—bandwidth, quality of service, access, on-time delivery, etc.—between nodes that may or may not be anonymous.
Thus the SWIFT snooping program exploits the strengths that the market has endowed financial databases with, while the NSA snooping program is fighting an uphill battle against the ever growing volume of communications data that the market demands from telecom networks.
Oversight is key
Although I think that the SWIFT snooping program is a good idea, I almost certainly wouldn't support a similar program for snooping domestic transactions. Why? Because I'm one of those "give me liberty or give me death" fanatics, which means that I have two main criteria for any kind of government program that involves spying on innocent citizens:
Operational effectiveness
Oversight
There's so much international money tied up in the integrity of the SWIFT database that I have a fairly high degree of trust that the SWIFT snooping program is subject to strict controls and international oversight. We're not talking about a domestic company that's going to roll over for the feds, and a Congress that's going to look the other way while the Executive branch does whatever it likes. There's real money at stake here, and much of it belongs to foreigners who are going to be concerned about things like corporate espionage and the US using the data to give domestic businesses an unfair edge.
A very short intro to the SWIFT database story
One of the US government's first priorities in the aftermath of 9/11 was to strike terrorism right in the pocketbook. It was in the context of their attempts to freeze terrorist assets and to trace the sources of funding for international terror organizations that the Bush administration first learned of the SWIFT database. They immediately moved to gain access to it, and at one point they allegedly wanted a copy of the entire thing for antiterror purposes.
Here's the NYT's description of the SWIFT international banking database:
Swift's database provides a rich hunting ground for government investigators. Swift is a crucial gatekeeper, providing electronic instructions on how to transfer money among 7,800 financial institutions worldwide. The cooperative is owned by more than 2,200 organizations, and virtually every major commercial bank, as well as brokerage houses, fund managers and stock exchanges, uses its services. Swift routes more than 11 million transactions each day, most of them across borders.
The cooperative's message traffic allows investigators, for example, to track money from the Saudi bank account of a suspected terrorist to a mosque in New York. Starting with tips from intelligence reports about specific targets, agents search the database in what one official described as a "24-7" operation. Customers' names, bank account numbers and other identifying information can be retrieved, the officials said.
Immediately following the NYT's story, the Republican outrage machine cranked into high gear over the revelations, with a chorus of right-wing bloggers, pundits, and even the President himself condemning the story as damaging to national security. In fact, by turning the outrage knob up to eleven, the administration and its surrogates have been able to turn the conversation about the program almost completely into one about freedom of the press vs. national security. (This is a move that the left has been wholly complicit in, by the way.)
For what it's worth, I think it's a tragedy, and probably even a threat to national security, that the NYT is our now our first line of defense against Executive overreach. That used to be the job of Congress.
Over the last decade and a half, the Internet and the World Wide Web have
emerged as unprecedented open platforms for speech and innovation. Now, there
is a raging debate about how to ensure that the Internet remains open,
accessible and innovative, while at the same time, encourage deployment of
new broadband network. The debate thus far, however, has proceeded on the
mistaken assumption that this is an either/or choice; that we have to choose
between a non-discriminatory, slow, insecure network or a potentially
discriminatory, high-speed, cleaner Internet tied together with other
broadband services. This paper argues that it is possible to preserve the
neutral, non-discriminatory essence of the Internet, without sacrificing
future growth of new Internet services and other broadband infrastructure.
At the heart of the debate is the question of whether Internet Service
Providers ought to be subject to a non-discrimination (aka neutrality)
requirement. The pro neutrality camp has argued that neutrality must be
legally mandated lest we lose the benefits that the Internet has enabled.
Those opposed to neutrality requirements generally view the Internet as a
good thing, too; they argue, however, that market forces will assure
continued access to the Internet on reasonably neutral terms, and that
legislating this requirement will stifle investment in new broadband
services.
That there is emerging competition[1] to offer high-speed Internet services is certainly good
news, but to suggest that competitive network services have no need of
non-discrimination rules is to fly in the face of hundreds of years of common
carriage tradition. But competition alone will not necessarily assure the
future of the Internet as an open platform, nor obviate the need for baseline
non-discrimination requirements, enforced through a light-weight complaint
process. For example, the mere fact that there is an competitive market for
some telephone services (wireline, wireless, VoIP, etc.), just as there are
competitive Internet service providers, is hardly a reason to suggest
dropping the basic Title II non-discrimination requirements of the
Communications Act. We keep these Title II requirements not because there is
a long list of infringements in recent memory, but because they are an
integral element of the basic operating requirements of the telephone
system. Indeed, the non-discrimination mandates in the Communications Act
are largely self-enforcing and pose little regulatory burden inasmuch as they
are the widely accepted mode of operation for the voice telephone network.
The same would be true for the Internet provided the enforcement or complaint
mechanism was designed to match the light-weight operating style of the
Internet.
Just as with common carriage for the telephone network, the open,
non-discriminatory nature of the Internet is a public policy goal that stands
on its own. It is the openness of the Internet that has given rise to the
extraordinary social and economic benefits of the Internet over the last
decade. Thus, openness is as important as potential increases in broadband
network capacity that some say will only materialize if Internet service
providers are allowed to depart from the neutral model on which the Internet
is based. This paper seeks to develop a path through the net neutrality
debate that will enable policy makers to give priority to protecting the
essentially neutral aspects of the Internet on which we depend, while giving
maximum flexibility for new services and infrastructure investments that can
benefit both Internet and non-Internet services in the future.
Why, then, does the so-called "Net Neutrality" debate seem to present a
choice between, on the one hand, preserving the openness and vitality of the
Internet we know today, and on the other hand, encouraging the possibility of
future growth and investment in high capacity broadband network services? On
the one hand, Internet community leaders from ecommerce companies, consumer
groups and civil liberties organizations warn that a failure to legislate net
neutrality guarantees will result in a loss of the economic, cultural and
political benefits that the Internet revolution has brought. On the other
hand, telecommunications and cable companies oppose any neutrality
regulation, arguing that non-discrimination obligation will limit their
incentives to build the type of video-capable broadband networks they believe
the market demands.
But are these two goals truly incompatible? The debate, and the very term
'Net Neutrality,' conflates a critical distinction between a focused need for
important Internet neutrality principles and the broader question about
whether or how to regulate other broadband communications networks
(such as digital cable television, or interactive services that may be
developed separately from the Internet in the future). Internet neutrality is
both a factual reality today and a necessity for the future. Whether
neutrality or non-discrimination requirements should be applied to broadband
networks more generally raises important communications policy questions that
should be considered, but need not be decided immediately.
To differentiate those essential features of the Internet that must be
protected from regulatory questions about other new broadband networks
separate from the Internet, this paper describes four essential features of
Internet Neutrality:
Non-discriminatory routing of packets
User control and choice over service levels
Ability to create and use new services and protocols without prior
approval of network operators
Non-discriminatory peering of backbone networks.
These principles taken together constitute the social contract among
Internet service providers that has been indispensable to its great openness
and success. They are equally important regardless of whether the service is
broadband or narrowband, wireless or wireline, fiber optic, copper pair or
coax. Understanding the Internet requires taking this holistic view of the
Internet as a set of business, technical and social arrangements. While
traditional telecommunications policy thinking divides the world into
'facilities' and different bandwidth levels, these are not the appropriate
categories within which we should regulate or de-regulate the Internet.
Indeed, the very foundation of the Internet is its ability to connect
efficiently a broad array of quite different networks, allowing a publisher
of information to reach a global audience without regard to which network the
recipient is on. To allow the nation's leading Internet access providers to
upend this fundamental global understanding would be to undermine the
Internet itself.
By distinguishing between InternetNeutrality and more
general NetNeutrality, it is possible to establish basic
non-discriminatory neutrality requirements that will preserve the neutral
aspects of the Internet that have brought commercial and non-commercial
benefits to hundreds of millions of people around the world. At the same
time, policy makers should carefully monitor the evolution of new broadband
networks and services. As long as those new networks operate in a manner that
does not actively interfere with or unfairly compete against Internet
services, policy makers should allow the private sector a freer hand in
designing and operating new broadband infrastructure.
I. The Neutral Internet
The Internet is neutral. This is a statement of fact about how the
Internet is designed and operated, not a matter for debate in public policy
circles. The neutrality of the Internet has made it an open platform for the
free flow of information, ideas and commerce. The challenge in the net
neutrality debate is not to try to decide whether or not the Internet should
be neutral, for it is. Rather, the challenge is to identify and preserve (or
at least not erode) the essential conditions of neutrality that have
characterized the Internet. At the same time, we should not constrain changes
in the non-essential aspects of the Internet if we are to continue to
encourage growth and innovation by access providers.
Commenting on the threat of a non-neutral Internet, Vint Cerf of Google
and Internet founder stated the worry that many have:
"In the Internet world, both ends essentially pay for access to the
Internet system, and so the providers of access get compensated by the
users at each end," said Cerf, who helped develop the InternetÃs basic
communications protocol. "My big concern is that suddenly access providers
want to step in the middle and create a toll road to limit customers"
ability to get access to services of their choice even though they have
paid for access to the network in the first place."[2]
Traditional telecommunications companies tend to have a different view of
this issue. In a statement that is widely credited with unleashing the debate
to begin with last year, Ed Whitacre, CEO of AT&T, declared:
"Now what they would like to do is use my pipes free, but I ain't going to
let them do that because we have spent this capital and we have to have a
return on it. So there's going to have to be some mechanism for these
people who use these pipes to pay for the portion they're using. Why should
they be allowed to use my pipes? The Internet can't be free in that sense,
because we and the cable companies have made an investment and for a Google
or Yahoo! or Vonage or anybody to expect to use these pipes [for] free is
nuts!"[3]
What exactly is wrong with making customers or service providers pay for
the choices they make? In fact, we all already pay for the network
services that we use in rough proportion to the cost of those services.
Today network costs are allocated between users (who pay for their own access
to the Internet) and large services (like Amazon, who pay a much higher price
for their Internet connection because they put a lot more traffic load on the
Internet). But what would be wrong with a change to this arrangement -
allowing large content providers like Google or Amazon to pay for the
privilege of having their data get to customers on an expedited basis, or
perhaps even to the virtual exclusion of their competitors?
What is wrong is that forcing a web site operator to pay twice so that a
user can have access to its content would begin to break the unique
many-to-many nature of how information is linked together on the Internet.
Once data is put on the Internet (at a Web site, for example), then the
speaker can be confident that anyone in the world can reach that data,
regardless of which Internet Service Provider they use. While not everyone
who requests that data will have the same quality of service, it is up to the
requestor to decide what service level is appropriate for his or her needs.
The provider of the content need not be involved in this decision and need
not worry about negotiating a transport arrangement with every ISP of every
potential user. The genius of the Internet is that it avoids this bottleneck
and is thus about to act as an extraordinarily open conduit for speech and
commerce. This is the heart of what makes the Internet different from other
communications networks.
The Web and other Internet services are built to take advantage of the
real time, many-to-many communications capability of the underlying Internet.
Many Internet applications we use appear to be largely just point-to-point,
such as email. In fact, although the path of any given email is
point-to-point, the various ISPs along that path are not known in advance to
the sender. It would cripple the functionality of email as we know it if, in
addition to knowing the right email address for your intended recipient, you
also had to know that recipient's ISP and make arrangements to pay in advance
for efficient transit over that ISP's network.
In order to explore the neutral nature of the Internet, I examine two
common Internet/Web usage scenarios. While some deduce network neutrality
principles from the formalism of Internet architecture (such as the
End-to-End[4] principle), I
prefer to look at these application scenarios, both of which are
representative of the Internet's potential as a platform for the free flow of
information, ideas and commerce, and also the Internet's ongoing capacity to
support innovative and unplanned uses for new applications and services. The
first scenario presents a sketch of the Internet mechanisms a major news Web
site might use today in order to deliver a large volume of text and images,
along with streaming video. Following this, I examine the operation of blogs,
an example of a leading edge Web application that hardly existed several
years ago and developed quite spontaneously, without any central planning or
coordination with existing Internet Service Providers.
Scenario A: Multimedia news delivery
This scenario[5] shows how
CNN manages the distribution of multimedia content around the world. A
problem that CNN (or any other distributor of high-volume content on the Web)
faces is that users ìfar awayî (in network terms) will experience delays in
access to content. This is because, among other things, each time an article
or video stream is requested, the web server has to send out a new response,
even if lots of users are requesting that same document. The general solution
is to 'cache' (store) the content at points around the Internet that are
closer to large groups of users. This minimizes load on the CNN web server
and also reduces the delivery time because the content is closer to the
user's location. CNN might use a commercial caching service to achieve
better performance in content delivery. While this site is not able to
provide video in real-time, it is able, using existing Internet technology,
to deliver time-delayed streaming video to a very large user base all around
the world.
Scenario A: Multimedia news delivery
The net effect of this elaborate (and expensive) arrangement is better
performance for users and the ability to reach more people for CNN. From a
user's perspective, the caching is entirely invisible. The user need not take
any action to achieve these benefits and all data appears to come from
cnn.com. From CNN's perspective, this arrangement has a real cost, but they
can decide whether and how much to spend on caching based on the benefit to
their business. The cache provider, in order to offer this service, must have
access to Internet services (connections) at a large number of ISPs around
the world. Those connections are largely indistinguishable from any other
connection that any other customer would buy, but if they were not available,
then the cache service would not be able to put together this type of
globally-operating cache. In turn, the cache provider offers caching service
to any (legal) content provider who is willing to pay the fee. There is no
discrimination based on content type or on the competitive relationship
between the content provider and the ISPs involved.
This scenario highlights one way that traffic prioritization is already
occurring on the Internet. Caching is just one of many ways to improve the
performance of video delivery. Large institutions that can afford to pay the
cache services rates can improve their services, whereas smaller
organizations may not be able to afford caching services. But the services
are available to all without discrimination.
Scenario B: Decentralized information flow through a mixed-media blog
Internet services such as the World Wide Web and instant messaging depend
even more deeply on the many-to-many communications features of the Internet.
Some of the most socially valuable and commercially popular services on the
Web take advantage of the ability to link together information from many
sources across the Web into what appears to be a single information resource
(a.k.a. a Web page).
This scenario shows how a new type of service (the blog) is built in an ad
hoc manner on top of many of the features on which previous scenarios
depend. The blog shown aggregates data from several other Web services
(Flickr and Amazon) along with the information on the blog site itself.
Scenario B: Blogs
In examining how blogs in general work, the first key observation is that
that this and all other blogs were created without any need for permission,
either from the blogger's ISP or the sites from which other data has been
aggregated. These independent sites have no prior connection arrangements-the
linkage of all the sites is made possible by the creativity and coordination
of the blog host. To the end user all the data on the page appears to be
located at the same URL, but it is actually drawn from a number of different
sites. The blog creator presumes that all content included on the blog page
will in fact be delivered to the end user, because of the long-standing and
fundamental expectation that intermediary ISPs do not block or interfere with
any legitimate content on today's Internet. The threat of a non-neutral
Internet is that content that the blogger seeks to combine (which may be
served from a variety of ISPs) may be unavailable to certain users because
the user's ISP might block or degrade access to some of the content
aggregated in the blog. It would impose an insurmountable burden on bloggers
and other Internet speakers to have to figure out in advance which readers
have access to which sets of content and which are blocked from access
because of their ISP's discriminatory policies.
The blog is just one of the current hot new Web applications, credited
with enabling a wide variety of political, cultural and economic benefits.
Social networking sites such as MySpace are another application that relies
on the creativity of the individual and linking among sites possible on
today's Internet. Indeed, these applications show how the open, neutral
platform of the Internet enables a unique style of decentralized information
flow. Though blogs are this year's leading edge application, there is no
doubt that other new types of applications and services that we have not yet
imagined will come along as well. That will only be possible, however, if we
maintain the essential neutrality features of the Internet going forward.
II. Defining and Protecting the Essential Conditions of Internet
Neutrality
The neutrality of the Internet arises out of a combination of basic
architectural features of Internet and World Wide Web standards, and business
practices on the retail and back-end of Internet service provider networks,
all in a delicate balance with the competitive market forces that tie service
providers, technology developers, and content providers together in a global,
voluntary agreement to maintain these practices and standards. This agreement
has been maintained out of an implicit but shared belief that cooperation to
keep the Internet functioning as an open, interconnected and
non-discriminatory platform serves the interests of the parties individually,
as well as collectively.
The web of technical and business arrangements that keep the Internet
operating is maintained because of the logic of today's Internet marketplace,
but it is important to note that it did not arise solely from market forces.
Much of the initial protocol design and implementation occurred in
government-funded research labs and saw its initial implementation in the
academic community. Even early steps toward commercialization were carefully
guided by government funders[6]
and were based on the cooperative spirit that grew up in the pre-commercial
Internet days. At the same time, the tremendous growth of the commercial
Internet in the 1990s that was responsible for bringing Internet access to
hundreds of millions of people around the world was and is a purely
market-driven phenomenon. So while it is indisputable that market forces have
led the growth of the Internet, we cannot credit the market with creating the
initial conditions that made the Internet the success that it is.
Some argue[7] that market
forces will keep Internet access service operating in a non-discriminatory
manner. Indeed it is hard to imagine too many people paying for Web access
when they can only reach those Websites with whom the ISP has made a special
arrangements. However, to use this prediction about future behavior of the
market as an argument against law that protects important values is
over-reaching. Indeed, if this same view were applied to the voice telephone
market, one might conclude that that non-discrimination requirement at the
heart of Title II of the Communications Act should be repealed. Following the
flawed anti-neutrality logic, the fact that there are a variety of ways to
buy voice telephone service and that no one would seriously try to sell
telephone service that only reaches a limited group of phone numbers. So in
spite of the growing competitive options for accessing the telephone network,
the Communications Act still enforces the following non-discrimination
requirement:
It shall be unlawful for any common carrier to make any unjust or
unreasonable discrimination in charges, practices, classifications,
regulations, facilities, or services for or in connection with like
communication service, directly or indirectly, by any means or device, or
to make or give any undue or unreasonable preference or advantage to any
particular person, class of persons, or locality, or to subject any
particular person, class of persons, or locality to any undue or
unreasonable prejudice or disadvantage."[8]
The regulatory structure that enforces this provision has changed
considerably. Price regulation has been dramatically scaled back and network
operators have nearly unlimited freedom to invest in new network services.
However, despite all of these changes, the underlying requirement of openness
on which the telephone system is based remains.
Given the mixed market and non-commercial provenance of the Internet, the
consolidation of the ISP market and the lack of any nondiscrimination
requirements for broadband networks,[9] it is simply impossible to predict that just because market
forces support a neutral Internet today, that this will remain the case in
the future. Hence, we must identify the essential indicia of Internet
Neutrality in order to know what operating principles of the Internet require
protection. The essential conditions of neutrality can be summarized as:
1. Content- and provider-neutral end-to-end routing service on
non-discriminatory terms.
The sine qua non of the Internet is the ability for everyone to
connect and have their traffic routed to the desired location.
Non-discriminatory routing enables Internet users to find the blogs they are
looking for, e-commerce sites to know that they can reach a world market, and
political campaigns to be confident that they can reach potential voters
(unless they live in China, Iran, or other repressive locales). Any network
claiming to offer Internet service must enable users to deliver packets to
any other part of the Internet and must accept, on behalf of its customers,
packets from any other Internet user or service.
Non-discriminatory routing means similar packets travel across the
Internet in a similar manner, but it not mean that every single packet is
treated exactly the same way. Engineering considerations result is different
routes for different packets. We would consider this reasonable
discrimination. But if those differences are used to discriminate against a
certain type of content or against a particular speaker, then the
discrimination is unreasonable. (Principle 2 below addresses approaches to
quality of service standards that can be handled in a non-discriminatory
manner.)
The two scenarios above illustrate the importance of this
non-discrimination principle. CNN counts on the fact that any user of any ISP
can, if he or she chooses, reach the CNN site. If AT&T follows the
double-charging pricing model laid out by its CEO Whitacre (CNN pays its ISP
to reach the Internet and then pays AT&T to reach its last mile) then
users might find themselves unable to access CNN news because CNN and
AT&T couldn't agree on a price for service. We are familiar with these
disputes in the cable television market where occasionally certain channels
are unavailable on certain cable systems. However, the Internet's
provider-neutral routing has never permitted such a result.
Non-discriminatory service provisioning is also necessary to enable the
growth of innovative new services such as content caching that facilitate
better access to multimedia content. These services rely on being able to
connect caching equipment to any ISP where it is needed. Today ISPs are
willing to sell cache services this access, but statements from some network
operators raise doubts about whether this will remain the case.
Non-discriminatory routing enables blogs, some of which depend upon
on-the-fly combinations of information from many different sources, to know
that any user can reach all content on the Web. If some of the
content on a blog comes from a site or ISP that didn't manage to negotiate a
reasonable price with some network operators, then that blog becomes
inaccessible to users of that ISP. Again, given threats to add extra charges
for carriage, this sort of breakdown in negotiation is likely. Today,
adherence with the non-discriminatory routing principle is universal. Without
this adherence, the Internet would lose its uniquely open and democratic
character, and risk becoming one more highly centralized and controlled media
platform.
2. User control over bandwidth and service levels.
Every user pays for the cost of connecting to the Internet, at the level
of service appropriate to their own needs, and then is able to have access to
the entire Internet, and use any legal application or device in doing
so. User control over service level choices must be preserved in order to
enable maximum participation in the Internet. Users in this context may be
individuals who mostly consume content (and only produce a little), or large
content providers that serve high volumes of content to others. Principle 1
assures that all parts of the Internet remain connected, but every
participant in the Internet may not necessarily be able to connect at the
same speed or with the same quality of service. Some cannot afford the
highest speed service, others do not have access to networks with high speed,
and still others just don't need it. But whatever the reason, a critical
aspect of the operation of the Internet is that each user is able to purchase
differing levels of service while still remaining connected. Those who read
blogs that contain links to video clips might like to see that video at a
high quality and speed, but will have to settle for poor picture quality if
they can't afford a 6MB per second service. However, they can still read the
text and see the images and lower-quality video.
The freedom to buy the bandwidth that one can afford means that hundreds
of millions of people around the world have been able to participate in the
Internet even if not at high speed. These decisions are always up to the
user, not the network operator or the content provider. So while content
providers do have mechanisms (caching services) for improving their users'
experiences, they know that today anyone can reach their services without the
content provider having to pay the user's ISP additional fees on top of that
which the user pays. The decentralization of this decision-making has been
vital to extending the reach of the Internet.
A key challenge that will face the evolving Internet is adapting to
requirements for delivery of high-quality video and other time-sensitive
data. Various quality of service standards have been proposed and it is
important that Internet Service Providers (include cable and telephone
companies) have the flexibility to experiment with different technical
standards and business arrangements for meeting these needs. As long as these
mechanisms are made available on a non-discriminatory basis, and are offered
subject to the choice and control of individual users, it should be possible
to preserve the neutral Internet and still adapt to new traffic
requirements.
3. Ability to create new protocols and applications on top of existing
Internet and Web standards without prior coordination with or approval of
network operators.
In order to enable ongoing innovation in new Internet services and
applications, ISPs must continue to support current technical standards and
avoid blocking the development of new technologies that will be created in
the future. In less than a decade of commercial operation, the Internet grew
from an environment which supported only a small number of services (email,
file transfer, terminal emulation) to a network that supports the World Wide
Web, instant messaging, voice-over-IP, streaming video, peer-to-peer file
sharing, and numerous other services. That innovation was possible because
the underlying architecture of the Internet enables the creation of new
services without the agreement and negotiation with network operators. As Tim
Berners-Lee, the inventor of the World Wide Web, wrote (in his blog):
When, seventeen years ago, I designed the Web, I did not have to ask
anyone's permission. The new application [the Web] rolled out over the
existing Internet without modifying it. I tried then, and many people still
work very hard still, to make the Web technology, in turn, a universal,
neutral, platform.[10]
The introduction of new services (such as the Web or instant messaging)
has always been viewed by ISPs as a boon, because they increase the demand
for Internet access. It is true that these new services, if they become
popular, have in some cases required ISPs to make various changes in the way
they manage their networks, but these network investments have always been
seen as worthwhile by network operators because of the new demand created for
Internet service. Similarly, the introduction of new content (such as search
services, maps, video sites, and blogs) has until now been viewed by ISPs as
a boom, because like new applications they increase the demand for Internet
access. Fundamentally, network operators opposed to neutrality no longer are
satisfied at benefiting from the increased demand for Internet access, seek
to dramatically alter the business model that has been vital to the
Internet's success.
4. Open, non-discriminatory interconnection to Internet peering
points.
The global connectivity of the Internet is maintained through a complex
set of interconnection (peering) arrangements amongst the world's largest
Internet Service Provider networks. While these arrangements, which together
form the Internet ìbackbone,î are hidden from view of most users
(individual, commercial, non-commercial), they are essential to the
Internet's operation. The essential function of these peering arrangements is
that they ensure that every ISP is able to offer access to and receive
traffic from every address on the public Internet. Without this, the global
nature of email, the World Wide Web, and other services would be crippled.
Just as Internet Service Providers must maintain neutral, non-discriminatory
relationships with users and content providers, so it is important that they
maintain open peering arrangements with other networks.
Despite some consolidation[11] in control over Internet peering points, I do not see
evidence of the need for comprehensive regulation in this area at this time.
However, because of the importance of these peering relationships, I suggest
that the openness of these vital arrangements be closely monitored.
Caveat: What Internet Neutrality Isn't
While we learn from the above principles about the fundamentals of a
neutral Internet, we also learn what is NOT essential to the Internet's
neutrality. We learn:
Service-level pricing is widespread and is no threat to neutrality,
provided each user (endpoint) on the network can choose the level of
services appropriate to its needs and provided that many-to-many,
end-to-end traffic flow remains possible.
Such service-level pricing can be applied at both ends to content
providers (for example, a small blog web site needing to purchase less
throughput than a popular large news site) and end users at the
retail level (for example, an ISP offering a low cost monthly plan for
casual web browsers, and a higher monthly plan for users that want to
engage in extensive file sharing). So long as users across the Internet
can choose appropriate levels or amounts of Internet access, the
Internet's neutrality principles are not threatened.
Caching is an important part of the Internet today and enables a
variety of adaptations for high-bandwidth applications such as streaming
video. There is no objection to network operators providing caching
services as long as they continue to offer open access to their networks
so that unaffiliated caching services are able to compete on a level
playing field.
A variety the peering arrangements has helped the Internet to grow and
adapt to new traffic patterns. This variety is no threat to
neutrality.
Based on the 4 essential principles of Internet Neutrality, I conclude
that legislation to protect such neutrality should meet the following
requirements:
Focus on the essential conditions of Internet Neutrality: The public
policy goal should be to preserve access to the neutral Internet but not
to reach into the operation of video programming services not carried
over the Internet. Therefore, Internet Neutrality rules should not be
confused with more general broadband neutrality rules.
Avoid supplanting the private-sector-driven innovation process: No
command and control regulatory process should be created that would be in
a position to approve or disapprove new protocols or business practices.
On the other hand, a clear set of legislated rules with a streamlined
complaint process adjudicated by the FCC or the FTC should be
provided.
We can be hopeful that Internet services and applications will continue to
develop in a vigorously competitive environment, without the need for any
regulatory intervention. But the neutral character of the Internet is so
important to the economic, political and cultural life of the nation that
Congress should act now to set out basic rules which will allow the Neutral
Internet to continue to flourish and grow.
III. The role of neutrality and non-discrimination requirements in new
broadband networks
I have shown that the Internet is neutral in several critical respects and
believe that this neutrality should be protected by statute. The application
of these same neutrality principles to other high capacity broadband
communications networks possibly related to but not synonymous with the
Internet, poses a more complex set of questions. I begin by describing
various new network offerings either announced or under construction by telco
and cable network operators and then consider some service scenarios that
raise neutrality concerns. In some cases I conclude that Internet neutrality
principles properly apply to these hybrid situations. In other case I believe
that careful monitoring and general reliance on existing telecommunications
regulation (both Title II and Title VI) , as well as antitrust law will
suffice unless there is evidence of actual anticompetitive harm.
A. General description of new broadband marketplace
Over the last few years, major telephone and cable TV companies have
announced plans to build new broadband network infrastructure in order
provide an array of services. Offers are said to include:
High definition television service: both channel-based and
on-demand.
High-speed internet access (at rates of 6-15MB or higher)
Voice telephone service (in some cases using a VOIP technology)
Other interactive multimedia services
These services raise a variety of questions about which regulatory
frameworks do and should apply. Indeed, one of the first rounds of the Net
Neutrality debate, the Brand-X case, centered on the regulatory
classification of cable modem services used to offer Internet access.
B. Broadband Internet Scenarios
Of the services described above, some directly implicate Internet
neutrality while others have no particular relationship with the Internet
(though may raise other significant communications policy questions). There
is also a middle category in which Internet neutrality questions arise given
the close relationship between the services themselves and the Internet. I
consider:
Network charges to carry packets from certain e-commerce Web sites
Internet-based Voice over IP service using new quality of service
techniques
Television carried over private IP networks (IPTV) not connected to the
Internet
In each case, we must ask whether Internet neutrality is harmed in any way
and if so, how the four Internet Neutrality principles from Section II
apply.
1. Discriminatory access to certain Internet services
An early instigator of the Net Neutrality debate was a statement by the
CEO of AT&T, declaring that every packet that traversed his network was
going to have to pay, even if
September 12, 2006 @ 9:42PM
- posted by Matt Mondok
A closer look at Microsoft's Volume Activation program for Vista
With the release of Windows Vista and Longhorn Server coming relatively soon, Microsoft is looking to try a new form of product activation with its volume licensing customers. Through Volume Activation (VA2), Microsoft customers will have two possible ways to activate Windows Vista and Longhorn Server.
The first method works in the same manner as the MSDN where a single key can be used to activate multiple Windows Vista and Longhorn Server operating systems. This type of key is called a Multiple Activation Key (MAK), and machines that use it can either activate directly against Microsoft's servers or have one centralized PC issue the request for activation on behalf of a group of machines. MAKs are typically found in decentralized networks, and the client machines only require a one-time activation.
The second method for activation, a Key Management Service (KMS), does not include any direct communication with Microsoft's servers. Rather, it requires systems to be activated against one centralized server which is not hosted by Microsoft but the business itself. Every 180 days client systems must activate against the KMS server, although activation will typically occur each time the client enters the corporate network. As you can see, the KMS is ideal for environments where the users are constantly connected to a company's network.
Microsoft is also allowing a 30-day grace period for systems which have yet to be activated. After 30 days, systems will enter Reduced Functionality Mode (RFM) where users can do nothing but reactivate the OS. In order to remind users of the looming activation, Microsoft will issue a notification stating, "Activate today or some features will no longer work." The warning will become visible towards the end of the grace period.
Finally, besides expiring, machines may also enter the RFM period if a MAK activated system's hardware changes or if a KMS activated system's hard drive changes.
The company's overall goal with VA2 is to help reduce piracy from volume licensing key leakage while at the same time make things easer and more scalable for legitimate Microsoft customers.
September 12, 2006 @ 11:24AM
- posted by John Timmer
Short circuiting learning
This first paragraph is going to be tough reading but, trust me, you will need to know it to understand the results. Electrical activity in nerve cells is primarily a matter of ions moving in and out of the cells. The flow of these ions is largely controlled by specialized pore proteins, called ion channels, that dot the surface of the nerve cells. The controlled opening and closing of these ion channels is what allows electrical signals to propagate along nerves. Most of that control is exerted over whether a signal gets started or not, which determines whether the cell will respond to cues from the environment or signals from its fellow nerve cells. Here, the frequency of signals can play a role in regulating future activity. Following repeated signals, a cell can modulate its response, making it more or less likely to respond to the same signal in the future. This modulation process is thought to be the key to processes such as memory, in which a signal permanently changes the way the nervous system responds.
With all that background out of the way, we can finally get to the actual results. Although the larval stage of the fruit fly Drosophila (translation: the maggot) has a very simplified nervous system, it's capable of undergoing conditioning, the same process by which Pavlov's dogs learned to associate sound with food. You can actually train a maggot to associate an aroma with food. In a recently released paper, however, a group in Germany managed to train the maggot without ever having exposed it to food. In short, they manufactured a memory.
How did they perform this trick? They took advantage of an ion channel from a single-celled organism that can be opened by exposure to blue light. Put it in a nerve cell and, in theory, the cell will fire a signal when exposed to light of the right wavelength. They confirmed that, when this channel is expressed in the nerves that control muscles, exposure to blue light would cause the maggot to twitch—the light served as a substitute for the normal signal that activated these nerve cells. But could simply activating these cells artificially trigger learning? To test this, the researchers expressed the channel in neurons that use dopamine, which is known to be associated with aversion behaviors. After several rounds of coordinated exposure to light and an aroma, the maggots learned to associate the smell with discomfort, even though they had never been exposed to any uncomfortable situations. The same thing worked for feeding: express the channel in a different set of neurons, and the maggots could be trained to associate smell with food, even though they'd never been fed.
The research group suggests that the light activated channel provides a powerful tool when combined with the ability to express it in specific populations of neurons. They hope to continue on and examine the role that different types of neurons play in behavior and learning in flies, and suggest that there might be some way to adapt the system to work in larger (and less transparent) organisms.
In theory, "place-shifting" does for location what "time-shifting" has done for scheduling. A few cables here, a few networking adjustments there, and you can use a product like the Slingbox or the software-based Orb to watch your TV (or TiVo, or DVD player) from just about anywhere you can get a network connection, be it your office, your hotel room, or the other side of the planet. Yet what makes place-shifting devices so powerful also makes them appear very dangerous to established entertainment and media companies.
Consider recent comments
from HBO's Bob Zitter that accuse the Slingbox's makers of violating HBO's own copyrights by "rebroadcasting" content across the Internet, or Major League Baseball's strong dislike for the way place-shifting affects their business model. Grumbles
withstanding, there
have been no lawsuits over place-shifting yet, but that could change in the blink of an eye. In the United States, the issues derive primarily from battles over what
constitutes fair use. In Europe, the situation is more complicated because of the way television is regulated. A recent controversy surrounding the Slingbox demonstrates
how place-shifting will ultimately bring about another round of disputes over fair use and the complex interrelationship between innovation and protecting the rights of both
consumers and content producers.
Innovate or placate?
When Sling Media launched the Slingbox, the company knew that it would draw the attention of content producers. The device features the ability to stream video sources across
the Internet (or your local TCP/IP network), and if you work in Hollywood, that alone might be enough to send you crawling up the walls.
When I spoke with Sling Media right after their product launched last year, they
made two things clear to me. First, they stressed that using the Slingbox for anything other than personal use was a violation of the EULA. Second, they said that there was
no approved way to capture the Sling Stream and save it to disk (you can't blame me for asking). I interpreted both decisions as attempts to placate rights holders. What Sling Media did not reveal to
me at the time was their
intention to also encrypt the video stream. This will become important in a moment.
The Slingbox lived in isolation for the first several months of its life, streaming video to users and making it possible for travelers to watch out-of-market sports
while on the road. Several months went by before Applian Technologies showed up on the scene offering a product they dubbed "Replay SlingCorder"—an application that conveniently
intercepts the Slingbox's video stream and records it. The application extended the usefulness of the Sling Box by allowing users to record TV that comes through the device, much
like a DVR.
Sling Media responded by asking Applian to
change the name of their product, apparently feeling as though the "Sling" in the title raised trademark implications. Within days, Sling Media also unveiled a new beta
firmware that introduced encryption. Whatever its name, Applian's recording software was shut out. For its part, Sling Media insists that encryption has been
planned since day one, and they also say that the inclusion of encryption in the newest beta firmware was a coincidence in terms of timing.
What's not a coincidence, however, is
that fact that applications can no longer (legally) record the Sling Stream. Jeremy Toeman, vice president of market development for Sling Media, told Ars Technica that the company
believes that content companies have a right to protect their content, while consumers have a right to, well, consume it. "We try to stay in that fair balance," Toeman
said, adding that "content providers want to know that their content is secure." Adding encryption to the stream is one way to prevent unauthorized use of Sling Media's product—unauthorized use that could tip the balance of rights they're trying to achieve.
Sling Media is now taking heat for their decision from some customers, despite the fact that their recent firmware significantly improved their product's performance. In a series of posts on the the Sling Community site (not affiliated with Sling
Media), Toeman explained that Applian had developed their product without contacting Sling Media, and that the whole mess really wasn't about punishing one company.
"Unfortunately, it's clear that the Applian At-Large Recorder has gotten caught in the middle of this. As a small company, we haven't had the resources to properly support
third-party developers at present, although it is our goal to eventually have an API or SDK for third-parties to use. If you consider the numerous ongoing software developments
we have at Sling (Windows, OSX, Windows Mobile, other platforms, etc), we have to pick and choose where our engineering efforts go extremely carefully, and while we have very
ambitious goals, we still have a lot of realities to deal with!"
The "realities" Toeman referred to include the legal landscape. Timeshifting, for instance, has been upheld by the courts, and devices like TiVo have made it quite popular in
recent years—much to the entertainment industry's chagrin. Place-shifting, on the other hand, has no Supreme Court decision to protect it. Any company looking to capitalize on the
promises of place-shifting must contend with the possibility of finding place-shifting listed as Public Enemy #1 of the video entertainment business, and that could spell trouble.
Any innovator working on a entertainment-based product will do everything it can to avoid catching the Napster bug, because once a product or service becomes synonymous with
copyright infringement, the only money for the manufacturer to collect will be for legal fees.
Place-shifting has more serious threats than HBO. Consider, for example, the European Union. The so-called "TV
without Frontiers directive" is supposed to be all about establishing guidelines for TV in this great Internet age. But it's becoming another "is it safe to innovate?" piece of legislation that says, in a nutshell, if it looks like TV and smells like TV, it must be regulated like TV. The result is that
place-shifting devices
could incur licensing fees for the content they "distribute," despite the fact that they are designed only to transmit a personal video stream. (By most reasonable
accounts, a real broadcaster is someone broadcasting to multiple people in multiple locations.)
A quick polish of the crystal ball can show why this is a bad idea. As the trucks tubes that power the Internet get faster and faster, and as common home
networks become more and more capable of handling multiple video streams, streaming is poised to become ubiquitous. Sony, Philips and others are already talking about televisions
which can stream content to other TVs. Apple and Sony are both trying to build businesses that promote, in part, streaming media capabilities. Microsoft wants its Media Center to
stream DVDs throughout the home, and Vista's media center capabilities are as designed to support up to five simultaneous streams to "media extenders" out of the box. On the audio
front, we're now beginning to see how the next generation of portable audio players are looking to tap into wireless streaming to open up new possibilities for sharing and
exploring. The possibilities are endless, but they'll be moot if "TV content" is magically granted a special status that makes it illegal to stream it without a license.
That's not the only bullet to dodge, however. While there is no similar legislation in the works in the United States, the murky world of fair use makes for a dodgy environment in which to innovate. One need only think of the conflict between Hollywood and software developers over the creation of DVD backups. Fair use proponents insist that making backups of DVDs is protected by law, while Hollywood's DVD protection scheme (CSS) was designed explicitly to prevent backups or copies from being made. The insane state of affairs is all the more ridiculous since it is a section of the US Copyright Law that makes it illegal to circumvent encryption for any use whatsoever (i.e., the DMCA).
For place-shifting, the situation is even more confusing because, as I mentioned above, the recordability of content and the basic portability of content is generally respected by
all rights holders. When that happens online, however, content producers get upset. Capitol Broadcasting President Jim Goodmon made his concerns rather clear. "I have a deal with
the cable system, and they have retransmission consent for the cable system in this market," he said. "They don’t have it for everyplace else. They can’t do that; there’s no way
that’s legal." Yet no one has established why streaming video is any different from, say, streaming audio when it is done for personal use. Users don't need licenses to stream
music from a home computer to another system across the Internet if it is for personal use.
Why should it be any different with video? What we do with content that we've purchased shouldn't be something for the government to restrict in the name of giving content
companies more revenue opportunities (yet this is precisely what the DMCA does). Double dipping has
unfortunately become a hot new way to squeeze money out entertainment offerings, and to be sure, there are loads of people who wouldn't mind paying a telecommunications company
extra money to watch SportsCenter on their phone. But there are just as many (if not more) folks out there who feel as though we're already paying for ESPN at home, so why pay
twice?
As proof of how sad the system is, the major mobile operators are protecting their very limited "double dipping" offerings by blocking out the completely unlimited
alternatives, such as place-shifting support on phones. When companies who don't exactly charge "minor fees"
for high-speed mobile bandwidth start locking out high-bandwidth applications just so they can sell their own limited video entertainment options, something is seriously wrong.
None of this is to say that premium content services can't or shouldn't be developed. But with potentially bad legislation brewing in the EU and a lack of clearly articulated consumer rights in the US, innovation becomes risky for all the wrong reasons.
Worse yet, the "risks" comprise a minefield of largely unknown, idiosyncratic "dangers" represented by the content owners and their ever-shifting goals. Let's face it, not every budding innovator wakes up in the morning saying, "I'd love to pit my idea against the entire legal power of Hollywood." The longer this situation remains, the longer technology loses out.
Like a creeping fog, DRM smothers more and more media in its clammy embrace, but the sun still shines down on isolated patches of the landscape. This isn't always due to the decisions of corporate executives; often it's the work of hackers who devote considerable skill to cracking the digital locks that guard everything from DVDs to e-books. Their reasons are complicated and range from the philosophical to the criminal, but their goals are the same: no more DRM.
We're going to revisit the history of the most famous DRM cracks. While the stories themselves are fascinating, one of the merits of such an exercise is to use the lessons of the past to consider the challenges of the future. Along the way, we'll address the following important questions:
Will DRM someday be unbreakable? Do content companies care if it is?
Who or what is a "Beale Screamer"?
What does the history of DRM mean for new technologies such as Blu-ray discs and HDCP links?
Can a marker violate the DMCA?
What's more important: technology, Congress, or the market?
Will a Stalin statue make a brief cameo appearance in the conclusion of this article?
We'll start our survey with one of the most-used DRM schemes in the country, Apple Computer's FairPlay.
It's all about the music
FairPlay
The FairPlay system, despite the professional effort that went into it, turned out to be surprisingly hackable, given Apple's reputation for robustness and general platform security. When Apple decided to take a bite out of the digital music market several years ago, it marshaled its internal technical resources to develop a home-grown DRM scheme dubbed FairPlay. By the middle of 2004, the encryption scheme had been cracked more times than a toppled Humpty Dumpty. Jon Johansen, the Norwegian hacker partly responsible for cracking the encryption on DVDs (see below), released the primitive QTFairUse, which attempted to bypass (rather than break) the FairPlay encryption. QTFairUse relied on Apple's software to decrypt the protected song files and then grabbed the unencrypted music from RAM. It then wrote this data to an unencrypted AAC file that turned out not be readable by most music players.
QTFairUse would not be the program to bring unencumbered iTunes downloads to the mainstream user, but it did represent one possible line of attack. Another approach was provided by playfair, a little program capable of stripping the DRM from iTunes files. Instead of grabbing the unencrypted data, playfair relied on grabbing the key FairPlay encryption uses. This key was stored on the iPod and was also easily accessible on Windows systems; once it was grabbed, songs could be decrypted and written to disk (Mac systems initially required the iPod to be attached to the computer).
This approach meant that you could only decrypt songs to which you had the rights anyway, but Apple was still unhappy about it. They modified their software to make the key much harder to grab. They also leaned on the web hosting company that playfair used; the project was also pulled from SourceForge. Showing just how hard it is to stuff the code genie back in the bottle, though, playfair development continued. The project was renamed "Hymn" and new versions are still being released, though all still have problems with certain versions of iTunes.
A third approach came from PyMusique, software originally written so that Linux users could access the iTunes Music Store. The software took advantage of the fact that iTMS transmits DRM-free songs to its customers and relies on iTunes to add that gooey layer of DRM goodness at the client end. PyMusique emulates iTunes and serves as a front end to the store, allowing users to browse and purchase music. When songs are downloaded, however, the program "neglects" to apply the FairPlay DRM. (A variant of PyMusique, called SharpMusique, has been developed and maintained by Johansen, though it has not been updated in 10 months).
The attacks on FairPlay have been enlightening because of what they illustrate about the current state of DRM. They show, for instance, that modern DRM schemes are difficult to bypass, ignore, or strip out with a few lines of code. In contrast to older "patches" of computer software (what you would generally bypass a program's authorization routine), the encryption on modern media files is pervasive. All of the software mentioned has still required Apple's decoding technology to unscramble the song files; there is no simple hack that can simply strip the files clean without help, and the ciphers are complex enough to make brute-force cracks difficult.
Apple's response has also been a reminder that cracking an encryption scheme once will no longer be enough in the networked era. Each time that its DRM has been bypassed, Apple has been able to push out updates to its customers that render the hacks useless (or at least make them more difficult to achieve). The resulting cat-and-mouse game between the company and its users will no doubt become a familiar feature of future DRM schemes, most of which are now built with the ability to update themselves even after deployment.
On the other side, it's also more difficult than ever to shut down a cracking project. Coders simply move their work from server to server until they find a spot where the long arm of Apple Legal cannot reach.
Windows Media
When it comes to music, Microsoft's own DRM system comes second only to Apple's FairPlay—though that's not saying much in the US, where Apple has an 80 market share. Still, it's a solid achievement for the Redmond, WA-based company, which beat out earlier contenders like Liquid Audio and Real. Windows Media Audio has been hacked less frequently than Apple's competing system, but as security analyst Bruce Schneier puts it, this is due to "market share, nothing more." Because relatively few music downloads are sold in the format, hackers devote less energy to cracking it.
Windows Media Audio has stood secure for several years now, but it was memorably hacked back in late 2001, when version two of the encryption was in use. A hacker using the pseudonym "Beale Screamer" posted a long message to the sci.crypt newsgroup in which he described how to circumvent the DRM protections of Windows Media Audio and provided code to do so. His program, dubbed "freeme," worked, but with limitations; users needed to have a license for the song before the decryption would work. It also was designed only for version 2 of Microsoft's DRM, the one included with Media Player 7.
Beale Screamer actually had some complimentary words for the Microsoft engineers who designed the scheme:
You guys have put together a pretty good piece of software. Really. ... My real beef is with the media publishers' use of this software, not the technology itself. However, it's easy to see where software bloat and inefficiency comes from when this code is examined: every main DLL has a separate copy of the elliptic curve and other basic crypto routines, and parameters passed back and forth between modules are encrypted giving unnecessary overhead, not to mention all the checks of the code integrity, checks for a debugger running, code encryption and decryption. Perhaps you felt this was necessary for the "security through obscurity" aspect, but I've got to tell you that this really doesn't make a bit of difference. Make lean and mean code, because the obscurity doesn't work as well as you think it does.
Mr. Screamer ("Beale" to his friends) made a point of asking users not to use freeme as a tool for violating copyrights, because such use would violate the serious point he was trying to make about DRM's dangers to fair use. Such principled messages are actually quite common among DRM crackers; whether users heed them is another story altogether.
The news created headaches for Microsoft's digital media group, but ultimately the hack was not of great concern. Microsoft was able to update their DRM, which has not been widely breached since that time.
Unlike DVDs, which had encryption built into the specification, compact discs were unencrypted and uncompressed, which made them the obvious go-to location for pristine digital copies. With the rise of broadband and the introduction of file-swapping systems like Napster in the late 1990s, record labels began to worry about the security of the venerable CD. When online music stores began selling songs in earnest, the problem was truly brought home to the labels. Why should they invest the time and money into selling DRM-laden songs that only irritated customers when those same customers could simply drive to the local store and pick up a better-quality, unencrypted version of the same music?
The labels were faced with an obvious decision: either forego DRM altogether or find some way of backporting it to CDs. Since the first option was never seriously considered, the second one prevailed, and an assortment of techniques appeared that promised to copy protect "compact discs" (copy-protected discs generally do not conform to the official Red Book CD standard maintained by Philips and therefore cannot legally use the "compact disc" logo). These fell into two groups: active and passive protection, and they worked (or failed to work) quite differently.
Passive protection introduced deliberate errors into the compact disc format. These errors were designed to exploit the subtle differences that initially existed between home CD players and computer CD drives. Solutions like the Cactus Data Shield initially promised that they could deliver near-total compatibility and excellent protection from ripping. In practice, things did not work out nearly so well.
For one thing, compatibility was an issue from the start. Despite minor differences, CD players and computer drives are similar enough that many computers could read the discs without problems, while some home CD players could not. One of the obvious virtues of the CD format is that you can bring it home from a store anywhere in the world, pop it into your player, and have it work without hassles. Copy-controlled CDs broke this long-standing compatibility for the sake of modestly increased security.
The other approach was to include active protection. This took the form of software installed on the disk that tried to install itself onto a user's computer in order to control the user's access to the drive. Such software was written only for Windows, which meant that sticking the disc into a Mac or a Linux box bypassed the code entirely. Even on Windows machines, the security provided was rudimentary at best. SunnComm's MediaMax 3 DRM system could be easily bypassed by using a black marker (a revelation that got security researcher John Halderman into hot water before SunnComm saw the light and declined to prosecute him).
Not everyone was eager to take a Sharpie to their newest albums, however, but an even simpler solution soon emerged. Most active protection could be bypassed simply by disabling Autorun in Windows (or holding down the shift key when the CD is inserted, which has the same effect).
No discussion of CD protection schemes would be complete without a mention of Sony BMG's notorious rootkit software, which was formally known as "XCP." XCP, created by UK firm First4Internet, created security vulnerabilities on a user's computer that were only made worse by the initial uninstall program. After a series of class action lawsuits, the company eventually pulled from store shelves all CDs containing the program and made restitution to affected customers.
Adobe e-books
While most attacks end simply with the company releasing an update to its software, American authorities took a harder line against Russian programmer Dmitry Sklyarov. Sklyarov had worked on password cracking techniques for his dissertation, and after leaving school was employed by Russian software firm Elcomsoft, where he was a part of the team that developed a crack for Adobe e-books. Although Elcomsoft marketed their Advanced eBook Processor as a way to make PDF files out of legally purchased e-books (something that was not allowed by the Adobe software), Adobe saw it as a piracy tool.
Back in 2001, when the Skylarov story broke, Adobe and Microsoft were both pushing hard to establish their e-book formats in the marketplace. An insecure format could potentially lead to lower profits for publishers, authors, and online bookstores, all of whom would request whatever technology was most secure. In this atmosphere, a highly public announcement of the security flaws in Adobe software was unwelcome. Skylarov attended the Las Vegas "Def Con" conference and presented a paper on e-book security (see the slides).
Adobe's system used a 40-bit key but proved difficult to crack by brute force methods (unlike CSS; see below). Skylarov estimated that a 450 Mhz Pentium III could take anywhere from 16 hours to 960 hours to break the code, but then showed how various companies (including Adobe) had made poor implementation decisions that made the system for more susceptible to hacking.
The next morning, he found himself on the receiving end of an FBI interrogation. Elcomsoft president Alexander Katalov described Skylarov's arrest this way:
On Monday morning, at around nine AM, he and another one of our employees, Andrei, were leaving the hotel for the airport. At the exit of the hotel they were approached by two men. They showed their FBI IDs. Dmitryi was immediately hand-cuffed. And they immediately took him and Andrei to different rooms. With Andrei, they simply talked—for about a half an hour, they talked with him about what's what, after which they let him go. He tried calling me several times, but the calls didn't get through. Later he called the Moscow office, where it's about nine thirty, and from there we got mail with the notification of the arrest.
Controversy over the arrest exploded across the Internet. To many, the case exemplified the problems with the Digital Millennium Copyright Act (DMCA), which made most encryption circumvention illegal. (Intriguingly, Elcomsoft's president also claimed that his company routinely sold password cracking software to the FBI and other law-enforcement agencies.) It wasn't long before even Adobe had cold feet about the entire situation, and the company withdrew its request for prosecution.
"We strongly support the DMCA and the enforcement of copyright protection of digital content," said Colleen Pouliot, Senior Vice President and General Counsel for Adobe. "However, the prosecution of this individual in this particular case is not conducive to the best interests of any of the parties involved or the industry. ElcomSoft's Advanced eBook Processor software is no longer available in the United States, and from that perspective the DMCA worked. Adobe will continue to protect its copyright interests and those of its customers."
The government later dropped charges against Skylarov, but only under the condition that he cooperate with their case against his employer. On December 17, 2002, a California jury found Elcomsoft not guilty on all charges; Skylarov was free to return to his wife and two children in Russia (read an interview with him that ran just after the trial ended). Skylarov continued his work at Elcomsoft, which no longer publishes the Advanced eBook Processor, and has authored a book on software security.
When the consortium behind the DVD standard drew up the format's specification, they included a basic DRM system called the Content Scramble System. CSS was designed to deter piracy, not by being particularly robust, but by being obscure (the DVD Copy Control Association provides no information on the subject until you take out a license from them). Quickly proving the adage that security through obscurity does not provide reliable protection, the encryption scheme was broken wide open in 1999. Exactly who cracked the code, however, remains something of a mystery.
Though Norwegian coder Jon Johansen is the name most often associated with the crack, a text file distributed with later versions of his software points out that Johansen was not the original code breaker (that honor goes to an anonymous German hacker). Johansen did make a name for himself, though, by helping to develop the DeCSS software as member of the group MoRE (Masters of Reverse Engineering).
DeCSS allowed PC users to strip their DVDs of the CSS encryption and then save the resulting files to their hard drives, and it landed Johansen in trouble with the Norwegian authorities. Despite multiple trials, the Norwegian judicial system ultimately ruled that Johansen's work was legal, in part because he was simply using it to exercise his fair use rights to watch DVDs under Linux.
Although the CSS algorithm was first broken by obtaining a key from a commercial DVD player, analysis of the cipher showed that it could be broken without trouble by brute force approaches. The cipher's key length was only 40 bits, a number dictated by US export regulations in place when the DVD specification was created. Due to the cipher's design, though, it proved much easier to crack than any 40-bit cipher should be.
DVD players are factory-built with a set of keys. When a DVD is inserted, the player runs through every key it knows until one unlocks the disc. Once this disc key is known, the player uses it to retrieve a title key from the disc. This title key actually allows the player to unscramble the disc's contents.
The decryption process might have been formidable when first drawn up, but it had begun to look weak even by 1999. Frank Stevenson, who published a good breakdown of the technology, estimated at that time that a 450Mhz Pentium III could crack the code in only 18 seconds—and that's without even having a player key in the first place. In other, words a simple brute force attack could crack the code at runtime, assuming that users were patient enough to wait up to 18 seconds. With today's technology, of course, the same crack would be trivial.
Once the code was cracked, the genie was out of the bottle. CSS descramblers proliferated, spawning a true alpha geek contest to break the encryption in the fewest lines of code; 434 bytes appears to be the current record.
Because the CSS system could not be updated once in the field, the entire system was all but broken. Attempts to patch the system (such as Macrovision's "RipGuard") met with limited success, and DVDs today remain easy to copy using a multitude of freely available tools.
In the end, CSS may have halted piracy of the most casual sort, but the general result was to make criminals out of consumers who wanted to back up their disc collections or remove the annoying but unskippable segments at the beginning of movies. Even though it's now simple to crack, most consumers don't know how to do it or do not own the proper equipment.The point is as simple one: DRM schemes do not need to be uncrackable to control people's behavior.
AACS attacks!!!
DVD encryption proved easy enough to break, but we'll soon have the chance to see how robust the next generation of video encryption schemes proves to be. AACS, the basic encryption present on both Blu-ray and HD DVD titles, is supposed to be a major step up from CSS. The scheme was put together by some of the biggest names in the industry, including Intel, Microsoft, IBM, Sony, and Disney. The spec is administered by the AACS Licensing Authority, which will attempt to control the format by licensing equipment manufacturers.
AACS relies on the well-established AES (with 128-bit keys) to safeguard the disc data. Just like DVD players, HD DVD and Blu-ray drives will come with a set of Device Keys handed out to the manufacturers by AACS LA. Unlike the CSS encryption used in DVDs, though, AACS has a built-in method for revoking sets of keys that are cracked and made public. AACS-encrypted discs will feature a Media Key Block that all players need to access in order to get the key needed to decrypt the video files on the disc. The MKB can be updated by AACS LA to prevent certain sets of Device Keys from functioning with future titles—a feature that AACS dubs "revocation."
This decision certainly makes it harder for manufacturers to design insecure devices or to purposely allow consumer workarounds (which was common for DVD region coding). Now, companies who do such things could potentially find their players unable to access new titles, something sure to infuriate buyers. As AACS drily notes, a device maker had better "treat its Device Keys as highly confidential, as defined in the license agreement."
AACS also supports a new feature called the Image Constraint Token. When set, the ICT will force video output to be degraded over analog connections. ICT has so far gone unused, though this could change at any time.
A more consumer-friendly feature is Managed Copy, which studios can use in order to let consumers make protected digital rips of their media for use on HTPCs and home media servers. These will still be encrypted and locked down, but at least people will be able to back up their HD DVD collection without breaking the DMCA.
While AACS is used by both HD disc formats, the Blu-ray Disc Association (BDA) has added some features of its own to make the format "more secure" than HD DVD. The additions are BD+ and ROM Mark; though both are designed to thwart pirates, they work quite differently.
While the generic AACS spec includes key revocation, BD+ actually allows the BDA to update the entire encryption system once players have already shipped. Should encryption be cracked, new discs will include information that will alter the players' decryption code. The BDA describes the feature this way:
The BDA also adopted "BD+", a Blu-ray Disc specific programmable renewability enhancement that gives content providers an additional means to respond to organized attacks on the security system by allowing dynamic updates of compromised code. With these enhancements, content providers have a number of methods to choose from to combat hacks on Blu-ray players. Moreover, BD+ affects only players that have been attacked, as opposed to those that are vulnerable but haven't been attacked and therefore continue to operate properly.
The other new technology, ROM Mark, affects the manufacturing of Blu-ray discs. All Blu-ray mastering equipment must be licensed by the BDA, and they will ensure that all of it carries ROM Mark technology. Whenever a legitimate disc is created, it is given a "unique and undetectable identifier." It's not undetectable to the player, though, and players can refuse to play discs without a ROM Mark. The BDA has the optimistic hope that this will keep industrial-scale piracy at bay. We'll see.
Although HD DVD and Blu-ray are brand-new, trash talk has already begun. Jon Johansen has already claimed that he will crack AACS by 2007, and has registered the deaacs.com domain. Stay tuned.
Looking back over a few of the best-known hacks of the past decade, a few key points stand out. First, DRM schemes are like houses: they have many ways in. If the front door is locked, there might be an open window. If the windows are bolted, check for a house key under the fake rock by the geraniums. If the main floor is secure, do the unexpected—slide down the chimney.
The ingenuity shown by the various hackers is a testament to the creativity of the human spirit (and to the desire for unencumbered media). The CSS algorithm can be cracked by a brute force attack in a couple of days, but it's the exception here. Most ciphers no longer succumb to brute force attacks in any reasonable amount of time, so hackers have looked for other ways to get at the content encrypted inside. Grabbing the key works well, but can be terribly complicated to do; sometimes it's just easier to let the decryption code do all the work, then grab the unencrypted stream from memory.
Technical measures aren't the only way to "hack" a DRM scheme. Our survey has highlighted the role that public pressure can play in having the locks removed and the house thrown open (consider Adobe and SunnComm, for instance). Superior encryption is worthless to a company if it ends up compromising their bottom line and brand reputation. Finally, some of the simplest attacks require nothing more than a marker or a finger. Both techniques show that a good hacker knows her technology, but she also knows when not to overengineer things.
The point here is that there are many ways to bypass DRM. Whether or not any particular method counts as a true "hack" hardly seems to matter, since they all accomplish the same goal. As Bruce Schneier tells Ars,
Does it matter if someone cracks the algorithm or the system? All the person cares about is if he can bypass the DRM system. If he can do that, the system is broken. Which particular piece broke is academically interesting, but operationally irrelevant.
The future: "lawsuit DRM"
A statue of Stalin flanked by grateful peasants used to tower over Prague. Perched above the Vltava on the Letna plain, it was the largest group statue in all of Europe at the time of its construction. It took years of manpower to build, and when complete, the beaming visage of "Papa Josef" dominated the skyline. Several short years after its completion, however, Stalin fell from grace in Communist circles and the massive monstrosity was torn down.
Many anti-DRM zealots talk about DRM technology as though convinced it will follow the same trajectory—massive spending and labor, years of careful construction, then ripped apart a few days later because no one wanted it after all. It's possible that this will be the case, but such a sanguine prediction could also be dead wrong. Though DRM's brief history has shown that formats can be cracked, few of the tools to do so have entered the popular consciousness, and the ones which have are illegal in the US. The force of law (and the risk of lawsuits) combined with the obscurity of most cracking tools means that even DRM solutions which are easily cracked can be effective at preventing casual piracy. Such technologies end up controlling only the behavior of legitimate users; those who want free copies of Dude, Where's My Car? from BitTorrent won't be deterred.
Although it doesn't stop piracy, this level of DRM may be good enough for most labels. It helps to instill a belief in the public that they can only do with their media what their media allows them to do (as opposed to exercising fair use rights, which may prove more broad than DRM restrictions). This type of thinking will enable media companies to better monetize their core products by selling them multiple times—movies on DVD, then again on Blu-ray, then once more for PSP, maybe a fourth time for your iPod, and a couple bucks for the privilege of time-shifting.
DRM need not be complicated to accomplish this. Even simple and highly-breakable encryption schemes have thrown up a legal barrier (courtesy of the DMCA) that will deter many average Americans from backing up their DVD collection (and then buying Finding Nemo a second time when the toddler sticks the disc in the toaster). HDCP, which is a form of "link encryption" that will guard the signal between HD players and televisions (or monitors), functions this way. Ed Felten, a computer science professor at Princeton, has analyzed the encryption and concluded that "the bottom line is clear. In HDCP, 'security' technologies serve not to disable pirates but to enable lawsuits. When you buy an HDCP-enabled TV or player, you are paying for this—your device will cost more and do less."
Sometimes "good enough" is good enough
DRM's not going away anytime soon, and newer techniques such as BD+ promise to make future technologies even more difficult to hack for long periods of time. Does that mean that hacking such systems will soon be pointless? Bruce Schneier says no.
"Do systems that allow you to change locks on your front door make burglary pointless?" he asks. "Of course not. The goal is to break the scheme and produce copies of the movie without DRM. Hacking DRM schemes will be fruitful as long as DRM schemes exist."
Given the time and money that developing strong DRM can take (and the fact that pirates will crack the format anyway), media companies may simply settle for more basic systems that still exert "good enough" control. After all, if "hacking" can take diverse forms, so can DRM, and the content industry's victory with the DMCA shows that effective control is not simply designed in the laboratory, but in the halls of power as well.
Since its introduction, Flash RAM has become immensely popular. It is widely used in digital music players, millions of people have USB thumbdrives on their persons, and hybrid hard drives that combine flash memory with the spinning platters are on their way. For all its usefulness, flash RAM does have its drawbacks: it's relatively slow and it has a finite (although high) number of read-write-erase cycles.
Enter PRAM. Despite sharing the same name, the new memory technology from Samsung is not to be confused with the stuff Mac owners zap when troubleshooting a balky machine. Phase-change Random Access Memory is Samsung's new memory technology that works much like flash memory, but is much faster and more durable.
So far, Samsung has built a fully-functional 512Mb chip to demonstrate the technology. It uses the same vertical diodes and three-dimensional transistor structure used in DRAM and unlike flash RAM, can write data without needing to erase data already present on the drive. Samsung says that overcoming this limitation means that PRAM can operate at speeds up to 30 times faster than flash RAM. It will also last longer—up to 10 times longer than flash RAM chips.
The chipmaker anticipates offering a full lineup of PRAM products in 2008, once it has the kinks in the manufacturing process worked out and fabrication facilities up to speed. It looks promising from a fabrication point of view, though. Samsung says that it will be less expensive to manufacture since it requires 20 percent fewer process steps than does conventional flash.
It is possible that laptops and other computers will incorporate PRAM into their systems down the road, as the battery life on a laptop with a large PRAM drive could be rather impressive. In the near-term, PRAM is more likely to be used for applications such as cell phones that use a relatively small amount of memory, but where faster reads and writes could result in a significant increase in performance.
The debacle surrounding Windows Genuine Advantage just won't go away. Late last month Microsoft found itself the target of a
complaint seeking class action status over WGA's behavior, and now we have learned that a second related suit has been filed.
Brought by Engineered Process Controls, LLC, Univex, Inc., Edward Mifsud, David DiDomizio, and Martin Sifuentes, the suit charges that WGA is essentially spyware, inasmuch as users are deceived to install it by being led to believe that it is a security update. The suit was filed in the US District Court for the Western District of Washington.
"WGA is 'spyware' that transmits data to Microsoft's central computer ("phones home") every time a PC is booted up and every 24 hour thereafter," the suit alleges. The accusations are in response to the revelation that beta versions of Windows Genuine Advantage did "phone home" once every day, but the company recently confirmed that more recent versions communicate with the company approximately once every 90 days. Microsoft maintains that the daily
checks were there because the software was in beta, not because the company intended to test daily checkups.
Daily or not, privacy advocates are outraged. The suit complains that "Microsoft does not advise users of these phone home capabilities. WGA gathers data that can easily identify individual PCs and WGA can be modified remotely to collect additional information at Microsoft initiation." The complaint continues, "software hackers can exploit WGA to not only collect data but also to modify users' computers." The suit contends that Microsoft and WGA violate Federal and Washington State law and "public policy on privacy, security, consumer deception, notice and consent."
The suit seeks compensatory damages, injunctive relief and a requirement for Microsoft to fully disclose WGA's "potential security and other risks" to the public. The suit also asks that Microsoft be made to produce a tool that can easily remove WGA from any Windows system. Should the plaintiffs prevail, Microsoft would have to scrap WGA and would be
subject to treble damages. Among the damage claims is a request seeking "either actual damages or one hundred thousand dollars per violation, whichever is greater."
For both cases, the debate is expected to center around the definition of spyware. The suit itself uses the
recently-formed Anti-Spyware Coalition's definition of spyware, which focuses on lack of consent and the "collection, use, and distribution of a user's person or other sensitive information" in constructing a definition of spyware. Microsoft has insisted that WGA is not spyware, and that it does not track user behavior or anything other than the licensed state of a Windows installation.
Microsoft sent out a press release yesterday entitled "Microsoft Continues Push to Refine and Simplify Licensing and Financing Programs for Customers and Partners" which, thanks to an overabundance of verbiage, did anything but simplify. The new licensing scheme, however, reveals some interesting information about the software giant's goals and directions.
Many of the changes involve Software Assurance, Microsoft's software licensing scheme whereby customers receive continuous updates to software in exchange for a yearly fee. The program has been criticized in the past for continuing to charge a yearly fee even if major software upgrades are delayed. Because of these and other factors, the uptake on Software Assurance has been much lower than Microsoft had hoped for, so the company has added a plethora of bonuses to try and get more companies to sign on.
Customers that purchase Windows Vista Enterprise Edition through the Software Assurance program will be able to legally run up to four copies of Windows on one computer, using virtualization software such as Microsoft Virtual PC which the company has released for free. In addition, Software Assurance customers will now be able to purchase Windows Server 2003 R2 Datacenter Edition, which was previously available only through large OEMs. Using Datacenter Edition, customers can run an unlimited number of virtualized servers at no extra cost. Companies that currently run combinations of Windows Server Standard, Enterprise, and Datacenter Editions will be able to consolidate their servers without having to pay for virtual machines or additional server licenses.
Free virtualization servers aren't the exclusive domain of Microsoft, however. After five months of beta testing, VMware announced this week that VMware Server would be a free download as well.
In addition to these virtualization options, Microsoft is throwing another bone at potential Software Assurance customers: the chance to pick up the version of Windows that nobody has ever heard of. Windows Fundamentals for Legacy PCs, the operating system formerly known as Windows for Legacy PCs, is a stripped-down version of Windows XP that is designed to work with the Microsoft Remote Desktop Connection (RDP) client or third-party clients such as Citrix. Windows Fundamentals for Legacy PCs will only be available to Software Assurance customers, although the exact shipping date of this product is unclear. Microsoft's web site originally promised that it would ship in February, then the date was changed to "late 2006," and currently the site claims that the product will ship in "June 2006," although the last update to the site was on March 30.
In total there are 18 new benefits being offered to Software Assurance customers, which Microsoft hopes will convince more companies to come on board the subscription train. Will your company buy a ticket?
The latest in what has become a series of delays in rolling out next-generation optical devices has hit Toshiba's high-definition HD DVD recorder. Toshiba said today that it will delay the launch of what would be the first high-def HD DVD recorder until the end of month in order to avoid launch-time shortages.
The RD-A1 will be available in Japan on July 27, instead of July 14, as originally planned. Unfortunately for budding HD DVD enthusiasts, Toshiba hasn't yet announced availability for the RD-A1 outside of that country.
So what will the rest of the world be missing out on? The RD-A1 sports some impressive specs. It is capable of recording 1080p video over HDMI while also including support for analog video and Japan's high-def video format. All of that high-definition video goodness is recorded onto 1TB of hard drive space, and can then be burned onto either single- or dual-layer HD DVD discs (that's 15GB or 30GB of video). It also includes an Ethernet jack for some household streaming goodness as well.
Expect to drop some serious yen for such a device: Toshiba's MSRP for the RD-A1 is ¥398,000 (currently just under US$3,450).
While we wait for the RD-A1 to hit North American shores, we can be on the lookout for a marketing blitz (PDF) from the North American HD DVD Promotion Group. The new consortium, funded by HD DVD backers Microsoft, Toshiba, HP, and Intel, has a US$150 million advertising and marketing budget, and they're ready to start spending.
Called "The Look and Sound of Perfect" (rejected titles reportedly include "What is the Sight of Two Eyes Watching" and "If A Next-Generation Optical Format Launches and Nobody Buys It, Did It Really Launch?"), the campaign emphasizes HD DVD's benefits while avoiding mentioning its competitor by name. There will even be an 18-wheeler cruising across North America with a complete HD DVD Mobile Experience inside, and the ad blitz will get heavier as the holiday season draws closer.
At this stage, marketing blitzes are arguably more important than small delays in product rollout. Both Blu-ray and HD DVD have a lot of work ahead of them to convince consumers that their DVD players need to be replaced, and that they should be replaced with one format or another. As we've said before, initial adoption is going to be painfully slow, especially with players starting at US$500 for HD DVD and US$1,000 for Blu-ray. It's going to take a lot of decked-out 18-wheelers to speed things along.
The State Department is keeping mum about it, but it appears that a number of their computers have been on the receiving end of some serious hacking attempts lately. Many of the targeted computers are associated with the department's Bureau of East Asian and Pacific Affairs. Although the sources of the attacks are difficult to pinpoint right now, one notable aspect of the attacks is the timing, which took place not long before the North Korean missile tests.
Currently, the State Department is refraining from pointing fingers in any particular direction. In fact, it isn't saying much at all. A spokesman would only admit that there were "anomalies" detected in the network traffic, and that the organization is attempting to be "prudent." What is known is that the State Department has been reconfiguring its networks, and Internet access was severely limited or eliminated at some offices in response to the assault.
Although the hacked computers themselves are considered unclassified, anonymous sources speaking to the AP admitted that some sensitive information had been accessed on the machines. Since the investigation is ongoing, the extent and exact nature of the data theft are not yet known. In addition to attacking the data, the attackers apparently installed backdoors in the systems to facilitate future reentry.
This news comes as a 40-year-old Scottish man living in England is fighting extradition to the US for hacking computers belonging to the Pentagon, NASA, and various branches of the US armed forces. Gary McKinnon faces up to 60 years in prison for his activities, which occurred over a 13-month period. His tools of the trade? An inexpensive computer and a dial-up connection. Although McKinnon was an IT professional, he claims that he could print out instructions on how to replicate his activities using nothing more than a single sheet of A4 paper.
He also purports that—during his forays into government cyberspace—he saw signs of IP addresses which indicated entry by others outside of the US military, including hackers from China, Turkey, Holland, and Germany. Although McKinnon's claims could be dismissed as the desperate raves of someone seeking to prove his value and mitigate his eventual sentence, it is reported that the Chinese government has previously attempted to penetrate networks at the Defense Department.
It is already understood that some countries—including China—consider computer hacking to be a first-level military offense. As a strategy, this makes a lot of sense, as it can be crippling to an enemy and has the potential to deliver better bang for the buck than a missile program. Given that, and a record of previousmissteps, it would do the US government well to consider just what level of computer security it considers acceptable to protect itself.
Virtualization has long been a fun toy for PC enthusiasts, and a godsend for developers who need to test their applications on a wide variety of operating systems. Now Microsoft has opened up the fun to everyone by releasing Virtual PC 2004 SP1 for Windows as a free download.
Virtual PC started life out as a commercial product sold by Connectix for the Macintosh platform that enabled Mac users to run PC applications. The company later released versions for Windows, OS/2, and Red Hat Linux. In 2003 Microsoft purchased the company and the product became known as Microsoft Virtual PC.
The download came in the form of a 18.2 MB zip file, and it installed quickly and easily on my PC without requiring any reboots, although I did have to uninstall an earlier version of Connectix Virtual PC before I could begin. When run, the application presents a small window that contains the user's various virtual operating system images. As there were no images installed, I took a fresh Ubuntu CD (well, fairly fresh, it was version 5.10) and started to put Virtual PC through its paces.
The "New Virtual Machine" wizard is fairly straightforward, although support for non-Microsoft operating systems seems somewhat lacking. From a dropdown menu I could select MSDOS, Windows 95, Windows 98, Windows ME, Windows NT, Windows 2000, Windows XP, OS/2, Windows NT Server, Windows 2000 Server, Windows 2003 Server, and "Other." Seeing as Ubuntu was probably "other," I chose that option. Virtual PC then asked for the amount of virtual RAM to allocate and the location to store the virtual hard drive.
One disappointment is that the amount of virtual RAM you can allocate to a new VM is limited by the amount of free physical RAM on your host machine. With 1 GB of RAM and several applications running, I was only able to select 128 MB for my virtual PC—any higher allocations caused the VM to fail with an out of memory error. While restricting RAM sizes to physical memory helps to increase performance of the VM, it would be nice to be able to select higher amounts of RAM and wait for the OS to swap it out.
Virtual PC uses your existing CD-ROM drives as devices in the VM, so it is easy to install a new operating system on a blank virtual hard drive by simply putting the CD in and waiting for it to autoboot. A Sound Blaster sound card and a generic network adapter are also provided as virtual devices. On my system, I had to set the network settings to Shared Networking (NAT) before Ubuntu would recognize and configure the network adapter. The virtual hard drive is auto-resizing, so it starts out as zero bytes and only grows as much as it needs to, although according to the Ubuntu install it was a 17.3 GB drive when empty. The Linux distro installed without incident, although it did take about three hours to complete the process.
Will giving away this product for free hurt other products such as VMware? Possibly, but keep in mind that in our Ars head-to-head showdown it was found to be somewhat inferior to VMware at the time it was originally released. One big advantage of VMWare is the availability of "VMware tools" that can be optionally added after the OS is installed in order to increase graphics and sound performance. VMware is also available for free in a more limited version known as VMware Player, which will run existing virtual machine images but cannot create new ones. Still, for those on a budget who either want or need to test different operating systems, having Virtual PC available for free is a great bargain.
Microsoft is currently working on Virtual PC 2007 which will support Vista and offer enhanced performance. The company plans to make it available as a free download as well. Microsoft has not made any comments about a new version of Virtual PC for the Intel-based Macintoshes, aside from saying that they are "working with Apple" on exploring the possibility of such a product. In the mean time, Mac fans are well advised to check out Parallels, a VM product that runs on Intel-based Macs.
Michael Geist, law professor at the University of Ottawa, has written a
brilliant rebuttal to the Canadian Recording Industry Association's claim that P2P piracy is costing the Canadian music
industry billions. Geist shows quite clearly how the CRIA has exaggerated their losses, exaggerated the effects seen by Canadian artists, and misplaced the blame on file-sharing.
The CRIA has routinely upped their loss estimates in order to gain headlines. Currently they argue that piracy is costing the industry C$450 million a year, to
the tune of nearly $2 billion for the years 1999 to 2004. However, if one looks at 1999 as a paradigm year, what you see is that since 1999, sales figures have decreased, but only
$432 million cumulatively. If every year since 1999 had been as good as 1999, then $432 million would be the amount "lost" in decline up through 2004. The other $1.5
billion in "losses" is simply made up: you can't document it because it is based on assumptions and politicking.
Where is this $432 million loss coming from? As far as the music industry is concerned, only "piracy" causes losses. But once again, an actual look at the numbers tells us
otherwise. The decrease in CD prices alone accounts for $50 million of this reduction in a single year, as large retail outlets exert pricing pressure on the music industry. Those
same retail giants, which sell more than half of all CDs, also don't carry a large stock, focusing instead on new releases. And wouldn't you know, there are fewer new releases today than in the past. The last nail in the coffin (were that it true) relates to the starving artist claims. The private copying levy, a kind of tax on blank media, generates millions in revenue each year, which is distributed to Canadian artists.
In all of this, it is important to note that private copying is legal in Canada, as is P2P file sharing. The CRIA is trying desperately to get this outlawed, and while the recent rebuffs of the DMCA in Canada have been impressive, the question of whether or not the right to make private copies will survive is very much up in the air.
In its quest to rid the British isles of illegal music swapping, the British Phonographic Industry (BPI) association is not finding much support from major European ISP Tiscali. A recent request to disconnect 17 users identified only by IP addresses has received a downright condescending response from Tiscali, noting that "It is not for Tiscali, as an ISP, nor the BPI, as a trade association, to effectively act as a regulator or law enforcement agency and deny individuals the right to defend themselves against the allegations made against them."
The BPI request asked for automatic action, without any due process and based on the evidence of a single screenshot showing some files available in a KaZaA account. Tiscali notes that the screenshot doesn't prove anything, and that the BPI really should get court orders before asking ISPs to take any action. "We are honour-bound by data protection," says Tiscali's Neal McCleeve, managing director of media and customer operations. "And they know we cannot release information without a court order pursued through the correct legal channels." Furthermore, a press release is hardly an appropriate channel for resolving such matters.
Despite its objections, Tiscali has contacted the user onto whom the recording industry is trying to pin the KaZaA screenshot, and asked for an explanation on the pain of account suspension if an acceptable response hasn't been received in seven days' time—assuming that the BPI has provided evidence of "a link between the user account and the IP address at the relevant time" by then.
This isn't the first time Tiscali has locked horns with the BPI, either. The Tiscali Juke Box service, which let the ISP's users stream songs and share them with other Tiscalians, is currently inactive due to BPI objections over excessive interactivity. Apparently, the record industry doesn't like users to be able to search for songs, and Tiscali deemed the service essentially worthless without search functionality, so the Juke Box has been put on ice until the issues can be resolved.
While the BPI's new approach of requesting discontinued access for file sharers does sound more reasonable than asking for money through lawsuits, it seems like the industry still has much to learn about how to use the legal system. It's not as if the RIAA is doing any better than the BPI on that front. Will the BPI put together the sustainable evidence that Tiscali is asking for? Will the identified user say anything in his or her defense? Regardless of the details, I boldly predict that this case will have absolutely no impact on worldwide, or even British, filesharing. So why can't we all just get along?
One of the problems with passwords is that they can be compromised relatively easily. While brute-force cracks are possible, it is much easier to convince users to willingly part with their passwords using social engineering. That's how phishers operate, by tricking users into entering their passwords—along with other personal information—on convincing-looking but spoofed web pages. Once they have that information, bank balances shrink while credit card balances grow.
Two-factor authentication has been touted as a solution to the problem of users giving up their passwords too easily. One group of phishers is determined to prove otherwise, as a recent attack demonstrates.
On the surface, two-factor authentication is a relatively simple solution. In order to log in to a protected site, users must enter a password as well as a second bit of information. In the case of Citibank and a handful of other financial institutions, users are given a USB dongle which displays a passphrase or string of numbers that updates every 60 seconds. It is only when the correct password is paired with a valid passphrase generated by the token that the user is granted access to their account information.
A group of phishers operating out of a Russian website attempted to trick Citibank customers in the customary manner, by directing them to a lookalike website and asking for the usual personal information. As an added bonus, the phishers also asked for the passphrase generated by the token. Once they had both pieces of the authentication information, they would presumably then transmit it onto Citibank within a 60-second time period and go about their nefarious business. It's a simple adaptation of existing methods: just add an additional field to existing forms and they are all set.
The phishing attacks demonstrates one of the weaknesses of two-factor authentication: it's still quite vulnerable to "middleman" attacks. If a malicious site is able to pose as the genuine article, collect the necessary authentication from the unsuspecting user, and act on it quickly enough, it is not much safer than traditional password-only attacks.
Some banks and other institutions have already made substantial investments in developing and deploying two-factor authentication systems. The central theme in marketing the systems to customers is added security. Microsoft had even planned to natively support it in Vista, although that ultimately met the same fate as other features originally planned for its new OS. However, as the latest bit of phishing demonstrates, it's not a cure-all. When used in conjunction with otherantiphishing tools, it can be more effective. But as long as there are gullible users, no combination of security measures will be completely foolproof.
Electronics have long been recognized as a weak link when it comes to secure conversation. From bugs hidden in lampshades to phone taps to keystroke tracking software, electronics provide the easy path to monitoring and censoring communications. In no area is that so apparent, perhaps, as in text messaging, as some users around the globe are discovering the hard way.
Text messaging and the first level of censorship begins at the phone. While it's certainly possible to enter any word using the alphabetic method in which a=2, b=2-2, c=2-2-2, d=3 and so on, it isn't very convenient. This has led manufacturers to develop alternate systems like T9, which make it easier to enter common words. T9 works by using algorithms to determine what word a user is trying to enter. Punching 2-2-8 might default to "cat" for example, since that's a common word which uses the letters associated with those numbers. It might also give you "bat" however, which is another logical guess based on the letters available through those keystrokes. Usually, a provision is made for selecting words other than the algorithm's first guess.
Where things start to get hairy is when a user enters something like 3-8-2-5, which can spell either "dual" or a somewhat naughty word which you won't find in your family newspaper. (Raise your hand if you aren't looking at a phone right now. I thought so.) In that case, the manufacturer could design the phone to provide the second word as an alternate, or more likely, avoid it altogether. In a nation like the US, avoiding a word which some might find objectionable is a business decision that probably prevents some complaint letters. In other countries, it could be a government mandate, and the banned word might not be 3-8-2-5, but something like "liberty" or "Taiwan."
At first, that sounds inconvenient, yet relatively benign. After all, a user could still switch to alphabetic entry and write anything they want, right? Perhaps, but the second level of control involves monitoring and censoring the messages of users, as the Chinese government has been doing since the SARS outbreak of 2004. At that time, word about the SARS epidemic spread like wildfire despite very little coverage by the government-controlled press. Since realizing the informative power of mobile phones, Chinese authorities have monitored and filtered text messages as a matter of course.
The problem doesn't begin or end with China. Security agencies in countries as diverse as Iran and Germany have been spotted responding to text messages regarding political leaders or outlawed ideologies. Much of this communication scanning is done with the compliance of the mobile phone providers, which simply consider it the price of doing business in various countries. We've seen this before, as in the case of Google and other portals filtering search results to suit local authorities.
The good news is that censoring communication continues to remain something of an arms race. While aficionados of such evil words as "Taiwan" or 3-8-2-5 might sometimes find themselves under scrutiny by the Powers That Be, there's nothing to stop them from switching to slang which means the same thing. Taiwan might be referred to as "the neighbors," for instance, while 3-8-2-5 could be "frak."
You can add radio to the list of media that will be facing new challenges for survival in the face of new technology. For years, WBEZ Chicago Public Radio has been a major home for jazz programming in that city, with a full one-third of its weekday schedule devoted to music, along with a bevy of music shows on the weekend. Next year, that era will be coming to an end, as the station makes the switch to an a news/talk/public affairs format. That change is in response to—among other factors—altered listening habits as WBEZ's music audience shifts to satellite radio and portable music players like the iPod.
To be sure, not everyone is happy with the planned changes, and a petition site has even been set up by listeners to protest WBEZ's new focus. The station's shift leaves the third-largest US market filled with little more than a collection of cookie-cutter music outlets owned by corporate giants like Clear Channel, Infinity, and Bonneville. One rare exception is WFMT, which plays mostly classical and is itself a subsidiary of one of the local public television stations.
The crisis faced by WBEZ is endemic of the situation being confronted by public and even commercial stations across the US. The general audience for public radio has a tendency to be affluent, college-educated white folks. That's a problem not only due to the fact that public broadcasting has a government mandate to target a wide range of listeners, but an affluent audience is more likely to be tempted by the lure of digital satellite services such as XM and Sirius, as well as the ever-popular iPod.
Switching formats to news and public affairs thus resolves two major problems. First, it allows the station to expand its base by focusing on local stories that may be of interest to those outside of its traditional reach. Second, it also fills a void left behind by the format changes of the commercial stations, many of which have moved away from news and in favor of simpler, less labor-intensive music formats.
As more listeners find their music online or switch to the focused narrowcasting of satellite radio, the market for traditional radio may be on the wane. If so, the challenges faced by WBEZ may turn out to be just the tip of a very difficult to navigate iceberg for all radio stations—commercial and public. That would be a shame, because at its best, broadcast radio provides a combination of local focus and portability that no other medium can match. While the switch to HD radio (and the medium's ability to multicast several programs at once) provides some hope for additional longevity, there's little doubt that radio—like music and movies and television—is on the verge of change.
Nothing says "we're not a faceless corporation" like a corporate blog—unless that blog is launched by Dell and features product announcements and tours of the "Enterprise Command Center."
After opening its new blog to the public last week (Name: one2one, Tag: "Direct conversations with Dell"), it didn't take long for Dell to come in for a blogosphere tongue-lashing. Jeff Jarvis complained that "Dell isn't listening. And listening, once more, is the first step in blogging." Steve Rubel made the same critique, telling Dell to "Join us. Be real. Walk the talk."
It didn't take long for the complaints to get Dell's attention. One2one's newest post, put up only this morning, is headed "Real People are Here and We're Listening." To prove it, Lionel Menchaca, Digital Media Manager at the company, went on to provide links to the blog's critics and said that Dell really, truly, actually wants to join the conversation. "We're excited to be here," Menchaca said, "and we welcome your ideas."
Dell's week-old experiment in corporate blogging illustrates the difficulties faced by companies who make the decision to engage in a public discussion of their products and their problems. There's obviously a fine line to walk here between being open to talk about corporate weaknesses and driving away potential business, but one2one shows that the blogosphere has no time for corporations who simply want to use a blog as another PR outlet. To its credit, Dell seems to want more than this for the new site.
The question is whether a corporate blog can ever be more than a marketing site. At some level, such blogs unavoidably become marketing tools—but that's not necessarily a bad thing. If a company tries to make itself look better by listening to and interacting with customers, that's the kind of marketing and PR push that we in the Orbiting HQ would like to see more often.
It usually works best when not pitched as an "official corporate blog," and Microsoft has done a decent job of this with their MSDN blogs, which actively solicit developer feedback. Well-crafted blogs can humanize an organization, but they can also provide valuable, direct feedback from customers to developers and engineers. If done right, corporate blogs can help both the customers and the company. When treated as a traditional PR vehicle, nobody wins.
We'll be using a spare PC, free open source software and an old Nokia handset to create a gateway for sending and receiving SMS text messages. If you have some basic knowledge of installing Linux (I'll be concentrating on the Debian distribution) and MySQL, then this should be a relatively trivial procedure.
Ingredients
This is what I used, you may alter according to taste, however some of my instructions are specific to Debian...
Server
You're going to need a computer to run this on, preferably one running the latest stable release of Debian (http://www.debian.org) (3.1 at the time of writing) as most of the instructions here will be Debian specific.
This machine doesn't have to be dedicated to being an SMS gateway, nor does it even have to be very powerful. A low-end pentium should suffice. It will, however, require a serial port.
Cellphone Interface
We're going to be using SMSD from the Gnokii (http://www.gnokii.org) project so you can use pretty much any GSM interface they support, I personally use a Nokia 5110 or 6110 series handset with a Nokia DAU-9P serial cable. A list of devices supported by Gnokii is available here (http://www.gnokii.org/faq.shtml#models).
You can usually find these items on EBay, the serial cables are better known as the Nokia Data Suite. If you want to try making your own cable, there are plans here (http://www.panuworld.net/nuukiaworld/hardware/cables/index.htm) and the Gnokii site has a more technical information about the cables here (http://www.gnokii.org/cables.shtml).
Beware of 3rd party no-name cables, they may not work as expected.
Getting Started
I'm going to assume you have a basic Debian installation already as installing Debian is far beyond the scope of this little document. If you've chosen to use another Linux distribution then there are probably packages available for gnokii-smsd.
Installing and configuring
Running the following should install all the required packages and dependencies we'll require for our basic SMS gateway:
aptitude install gnokii-smsd-mysql mysql-server
Once those are installed we'll login to MySQL (note: no MySQL root password has been set yet, see the security section below):
mysql -u root
Create a database (we'll call ours 'smsgw'):
create database smsgw;
Quit the MySQL client, and run the following to populate the smsgw database with the requisite tables and fields:
mysql -u root smsgw < /usr/share/doc/gnokii-smsd-mysql/sms.tables.mysql.sql
Now edit /etc/gnokiirc and change the port and model settings to match your setup.
Now connect the cellphone to the PC using the Nokia cable and run the following:
/usr/sbin/smsd -u root -d smsgw -c localhost -m mysql
This should work, however at the time of writing, there exists an issue with libmysql.so being in the wrong place. If you get an error saying:
dlopen error: /usr/lib/smsd/libmysql.so: cannot open shared object file: no such file or directory!
Creating the following symlink should fix it:
ln -s /usr/share/smsd /usr/lib/smsd
Now hopefully you are running smsd (which won't output anything to the screen)... it's time to test :)
Send a text message to the handset connected to the SMS gateway... you should see the SMS notification display on the handset's screen quickly before disappearing as smsd reads and deletes it.
Hit CTRL-C to stop smsd, log back into MySQL (mysql -u root smsgw) and run select * from inbox;, you should see something similar to the following:
+----+---------------+---------------------+----------------+-------------+-------+-----------+
| id | number | smsdate | insertdate | text | phone | processed |
+----+---------------+---------------------+----------------+-------------+-------+-----------+
| 1 | +447970123555 | 2005-11-20 16:03:15 | 20051120150343 | Testing 123 | NULL | 0 |
+----+---------------+---------------------+----------------+-------------+-------+-----------+
1 row in set (0.00 sec)
And that is pretty much it... you now have a functional SMS Gateway (I did say it was easy!). I'd suggest wrapping the smsd application in a script to ensure it runs smoothly. There are some pitfalls you should be aware of that are outlined below and please read the security considerations before deploying a home-made SMS gateway.
All that remains is to write an application to take the messages out of the database and do whatever it was you wanted to do with them, I suggest that if you intend to have multiple applications reading the messages that you write a single "SMS processor" to read the messages from the database and hand them off to individual applications.
Sending messages
Simply insert your message into the outbox table and SMSD will take care of the sending. While this is fine for one or two status messages or for things like system monitoring it should be avoided for sending bulk messages as there are far cheaper options available, see "For fun, not profit" below.
Pitfalls and oddities
The following are issues I've run into that you should probably be aware of...
Duplicates
From time to time SMSD likes to put duplicate messages in the database. You might want to write some duplicate checking into your SMS processor to mark duplicates as such or even remove them from the database.
libmysql.so
As mentioned above, there is at the time of writing an issue with libmysql.so being in the wrong place. If you get an error saying:
dlopen error: /usr/lib/smsd/libmysql.so: cannot open shared object file: no such file or directory!
Create a symlink as folows:
ln -s /usr/share/smsd /usr/lib/smsd
This will probably be fixed by the Debian maintainer for gnokii-smsd-mysql in future versions.
Security
Basic security
You'll notice that, if you followed my instructions above, SMSD runs and connects to the database as root... this is generally considered a Bad Thing. Create an SMS user on both the system and MySQL and use that instead. While you are adding an SMS user to MySQL you will probably want to create a password for the root user.
If this machine is to be connected to the Internet or any publicly accessible network, please look into firewalling it else you may find one day that someone has accessed your machine, and not only stolen any user data, but also used your SMS Gateway to send messages that you'll end up paying for.
Consider your users' privacy
It's all very well using this for your own personal use but as soon as you start offering services for other users you instantly have an obligation to those users to protect the details you gather from them. Primarily this is their phone number but in certain circumstances (such as my own SMS_To_LJ service) you may be exposed to other sensitive information.
Ask yourself, do you really need to keep every message? If the answer is yes, do you need to store more than the last few digits of their phone number? If you handle sensitive information inside of the messages, can you obfuscate it once you no longer need it?
For fun, not profit
While this project is perfect for home use or small projects it really shouldn't be relied on for business use, it has absolutely no built-in redundancy and can, at times, be quite unstable. Not only this, but sending messages using a regular consumer cellphone account can be quite expensive.
If you need a gateway with guaranteed uptime or intend to send a lot of text messages you'll probably find you are better off talking to an SMS aggregator, these are large companies with direct connections to the major cellphone networks and can offer big discounts on bulk messages.
...but I want to make money!
One of the limitations of this gateway that seriously hampers it's use in any commercial money-making application is that, due to being based on consumer equipment/services, it is completely incapable of providing premium rate SMS services.
When I was under the employ of a mobile billing company this section used to suggest that readers wishing to setup commercial SMS solutions contact an SMS aggregator but I've since gone on to start the emerging telephony company BlackFin Limited (http://www.blackfin.co.uk). Our in-house expertise covers a wide range of communications subjects including SMS and if we can't take on your project we'll certainly know someone who can. Our contact details are on our website at www.blackfin.co.uk.
Uses for an SMS gateway
I started this document as an extension to a few discussions I had with people at the MobileActive (http://www.mobileactive.org) conference about how easy it is to setup a simple SMS gateway for use in activist campaigns. The uses for this kind of gateway are pretty much left to your imagination from home automation (hook this into an X10 module and turn things on/off from a text message) and security (hook it into your alarm system) to providing access to existing web-based services... let me know if you used this document to create something interesting :)
About the author
Dan Lane is the Managing Director of BlackFin Limited, a telco based in the UK, in his spare time he messes around with disruptive communications technology and a number of other geeky projects. He can be found on the web at http://invalid.name or e-mailed at null@invalid.name (yes, really).
Dan runs the free SMS To LiveJournal (http://www.x11.net/wiki/index.php/SMS_To_LJ) gateway and was technical architect for the ARC (http://news.bbc.co.uk/1/hi/technology/4149977.stm) project.
The content on this page is provided by a Google Notebook user, and Google assumes no responsibility for this content.
