Dataesthetics: The Power and Beauty of Data Visualization

One of my areas of interest that has grown over the last couple years has been data visualization. I'm a visually-oriented learner, and I look forward to seeing any techniques, illustrations, or technologies that:

1) Allow people to assimilate information as fast as possible.

2) Deepen understanding of knowledge by visually illustrating data in new and interesting ways. There is nothing like having an intellectual epiphony after looking at a picture for a few seconds (pictures can definitely be worth a thousand words).

3) Present information in an aesthetically pleasing way. Or, in extreme examples, inspire a sense of awe!

Thanks in large part to del.icio.us, I've come across a wide assortment of sites and blogs that really illustrate some amazing work in this area. While cleaning up my knowledge management related bookmarks, I started to revisit many of the sites I've tagged in recent months. I thought I'd share some of the illustrations that really stuck out at me.

Before I dive into some of the great examples, I figured I'd address the main theme of this article: 'Dataesthetics'. Traditionally, the popular term used to describe data visualization and the aesthetics of information is called "Info-Aesthetics". Info-Aesthetics was coined by Lev Manovich, and is also the title of his semi-open source book. After some Googling on the subject, and some internal debate, I decided that the term Dataesthetics rolls off the tongue easier and sounds more appealing.

Now for the visually stunning sites. These first 2 at the top of the list collectively represent some of the most amazing data visualization projects out there. I've only listed a dozen or so of my favorite examples. But if you're inclined to explore further, I'd recommend spending a few hours going through Infosthetics and Visual Complexity.

Sites
http://infosthetics.com/ - Inspired by Manovich's definition of information aesthetics, this weblog explores the symbiotic relationship between creative design and the field of information visualization, in an emergent multidisciplinary field what could be coined as 'creative information visualization'.

http://www.visualcomplexity.com/vc/ - VisualComplexity.com intends to be a unified resource space for anyone interested in the visualization of complex networks. The project's main goal is to leverage a critical understanding of different visualization methods, across a series of disciplines, as diverse as Biology, Social Networks or the World Wide Web.

Business
LinkSViewer - interactive visual networking tool for venture capital relationships in Silicon Valley. I just blogged about this a couple weeks ago here.
Census and Maps.

Breathing Earth - Displays the carbon dioxide emission levels of every country in the world, including birth and death rates.

Social Explorer Maps - Demographic information with hundreds of interactive data maps of the United States, including historical data back to 1940.

EmotionWe Feel Fine - Harvest and graphs human feelings from a large number of weblogs.
Greenwich Emotion Map - Artist Christian Nold has been invited to collaborate with local residents from the Greenwich Peninsula to explore the area afresh and build an emotion map of the area that explores people's relationship with their local environment.

Government
Death and Taxes - A visual guide to where your federal tax dollars go to.

Global Military Spending - Beautifully illustrates how much countries spend per year on their military.

The Government Grind - How much government officials get paid.

History
Imperial History of the Middle East - Who has controlled the Middle East over the course of history?

Music
The Shape of Song - What does music look like?

MusicMap - Visual music search application.

MusicLens - Music recommendation.

LivePlasma (Music) - Music browser and recommendation.

News
NewsMap - Newsmap is an application that visually reflects the constantly changing landscape of the Google News news aggregator. A treemap visualization algorithm helps display the enormous amount of information gathered by the aggregator.

Digg Labs Stack and Swarm - A broader and deeper view of Digg.

WeekInReview - Hand drawn weekly summary of current events.

Statistics (charts and diagrams)
Understanding USA - "Understanding information is power".

NameVoyager - Explore the sea of names, letter by letter...watch trends rise and fall, and dive in deeper to see your favorite name's place in the historical tides.

Karl Hartig: Data Visualization - Great collection of charts, diagrams, and information graphics.

Del.icio.us Treemap - Del.icio.us most popular tree map.

Technology
The History of Programming Languages - plots over 50 programming languages on a multi-layered, color-coded timeline.

Java Technology Concept Map - Get an overview of the Java landscape as well as learn more about the details of its components.

Text Processing
Graphical visualization of text similarities - Amazing text visualization illustrations.

War/Conflict
Interactive Middle East Network Map - Shows both both attracting and repelling forces between various players who have interests in various regions.

The Middle East Buddy List - Who is a friend or enemy?

Strife and Power in the New Middle East - NY Times article.

Iraq War Fatalities - Iraq war coalition fatalities since Mar 2003.

Sources
http://eric-blue.com/blog/2006/10/dataesthetics
_the_power_and_be.html

Valuable Data Visualization Reference

Eric Blue has assembled a valuable list of about 30 reference links and examples of information visualization in his post, Dataesthetics: The Power and Beauty of Data Visualization. I especially liked the http://infosthetics.com/ reference. Across the board, these links provide a pretty compelling showcase of what approaches and techniques can be taken to effective data presentation.

For example, as one of the many hundreds of options available, here is Sun’s interactive Java concept map (will spawn a new window and requires Flash viewer):

Sources
http://www.mkbergman.com/?cat=2

$100 laptop may be at security forefront

CAMBRIDGE, Mass. - The $100 laptops planned for children around the world might turn out to be as revolutionary for their security measures as for their low-cost economics.
ADVERTISEMENT

The One Laptop Per Child project, a nonprofit begun at the Massachusetts Institute of Technology, aims to improve education by giving children bright-colored, hand-cranked, wireless-enabled portable computers. Governments are to buy the laptops — beginning in 2007 with up to 7 million machines in Thailand, Nigeria, Brazil and Argentina — and hand them to kids for them to own.

The machines have garnered the most attention — and some skepticism — for the design elements helping to keep their price low. Among other things, the computers will employ the free
Linux operating system, flash memory instead of a hard drive and a microprocessor that is slow by today's standards but requires minimal power.

But programmers also have been taking advantage of the start-from-scratch nature of the project to design security protocols that they hope will greatly surpass those found in mass-market computers today.

The designers are still testing their approach with outside security experts — which is widely considered wiser than keeping such matters secret. But already they believe the security setup could make it unnecessary for the laptops to have anti-virus software.

Standard computer design generally lets most any program access any file stored anywhere on the machine. That is one reason why flaws in programs can be exploited by outsiders to steal or erase private information.

By contrast, the $100 laptops will force any application to run in "a walled garden" and limit the files it can access, said Ivan Krstic, a software architect at One Laptop Per Child focused on security.

Even if the security were to fail, Krstic believes a specialized encryption technology will prevent the BIOS — the software that runs a computer when it is initially turned on — from being overwritten. That means the PC could not be rendered unable to boot up.

"It's essentially unbelievably difficult to do anything to the machine that would cause permanent hardware failure," Krstic said.

Extensive security measures are necessary because so many of the machines are expected to be built, making them a large target for mischief.

One particularly thorny potential problem is that the laptops can communicate with one another in a "mesh" network, sharing data and programming code. A computing Web site reported this week that Krstic had described that setup to the ToorCon security conference as "very scary."

But he contended to The Associated Press that the comment was taken out of context.

"We have code-sharing in the machines, which is really scary if we were not paying attention to it," he said. "But we think we have solutions to all of these problems."

One of the principal organizers of ToorCon, George Spillman, said Krstic's presentation was "very well received" because the $100-laptop designers have thought a great deal about security but "they're not arrogant enough to believe they have everything locked down."

Spillman believes at least some of the measures Krstic described are likely to be successful, though he cautioned: "There's always going to be some kind of a hole somewhere."

Walter Bender, a co-founder of MIT's Media Lab who is overseeing software and content on the $100 laptops, said children should be able to tinker with the laptops and learn how they work. To that end, these security measures can be turned off by the PCs' owners.

To protect against that leading to disaster, the laptops will automatically back up their data up on a server whenever the machines get in wireless range of the children's school. If a child loses data, the files can be restored by bringing the laptop within wireless range of the server.

On the Net:
http://laptop.org

Sources
http://news.yahoo.com/s/ap/20061006/ap_on_hi_te/hundred_
dollar_laptop;_ylt=AmBhG4VQjaFf1D1jKqOFSIi73MMF;_ylu=X3
oDMTA4ZnRnZjhkBHNlYwMxNjk1

Google in talks to buy Web video site YouTube: WSJ

SAN FRANCISCO (Reuters) - Web search leader Google Inc. is in talks to buy YouTube Inc., the world's leading Web site for video entertainment, for close to $1.6 billion, the Wall Street Journal reported on Friday, citing a person familiar with the matter.
ADVERTISEMENT

According to the newspaper, the talks are at a sensitive stage and could break off. YouTube declined comment on the report. Google representatives could not immediately be reached.

YouTube was founded in February 2005 as one of dozens of Internet video start-ups. It has exploded in popularity since last November by letting users share short video clips -- both home videos and programming copied off television.

Rumors of a Google-YouTube deal appeared Thursday on the TechCrunch blog of Web start-up powerbroker Michael Arrington, who said such talk was circulating among Silicon Valley venture capitalists after months of speculation that YouTube was an acquisition target.

"YouTube is the hottest property on the Web, one that could be worth much more than $1.6 billion if monetized properly," said RBC analyst Jordan Rohan, adding he had no information on whether YouTube would be acquired. His calculation is based on combining YouTube's audience and Google's advertising prowess.

For Google, the acquisition of YouTube would thrust the Web search leader quickly into the emerging market for video advertising, where it has only a tiny foothold compared with Yahoo Inc. (Nasdaq:YHOO - news) and various Web start-ups, Rohan said.

"Essentially Google can give less than 2 percent of its market cap and keep this platform out of the hands of everyone from Yahoo to Microsoft to Viacom," said Rohan.

"There is something to be said for that old playground game of 'keep-away,"' he said. "Defense here matters."

YouTube has asked for about $1.5 billion during talks with potential suitors in recent weeks, sources familiar with the matter told Reuters.

Google shares rose $8.69, or 2.1 percent, to close at $420.50 on the Nasdaq on Friday.

HOT PROPERTY

MTV owner Viacom, still hurting after News Corp. elbowed it aside last year to acquire top social networking site MySpace.com, recently dodged questions on whether it had courted YouTube -- another of the most popular Web destinations for young people.

"It's a very good company," Viacom Chairman Sumner Redstone said in a TV interview with Charlie Rose on Wednesday.

YouTube reports serving about 100 million videos daily and has drawn scrutiny from major media companies for copyrighted material appearing on its pages without their consent.

YouTube commands a 47 percent share of the online video search market as of September 30, compared with 22 percent for the MySpace video site and 11 percent for Google Video, according to Internet measurement firm HitWise Inc.

The site serves about 32 million visitors monthly and has about $11.5 million in venture capital financing from Sequoia Capital. A Sequoia spokesman was not available to comment.

Google has previously preempted rivals from striking deals that could threaten its dominance in key Web segments.

In December, Google agreed to pay $1 billion for a 5 percent stake in Time Warner Inc.'s AOL unit in a deal that expanded their advertising partnership.

Mark Cuban, an Internet investor and the outspoken owner of the Dallas Mavericks basketball team, told the Online News Association in Washington, D.C., that Google would be "crazy" to do a deal because of the legal challenges YouTube faces.

To control rampant copyright infringement on YouTube, Google would be forced to monitor what videos users upload to the site. "Once you have to start monitoring, the whole business changes," Cuban said of the risk to Google.

Cuban sold his pioneering Web radio company Broadcast.com for $5.7 billion to Yahoo at the dot-com era's height in 1999.

(Additional reporting by Kenneth Li, Yinka Adegoke and Michele Gershberg in New York and Emily Church in Washington D.C.)

Sources
http://news.yahoo.com/s/nm/20061006/wr_nm/
media_google_youtub_dc

Search Engines and Duplicate Content

Plagiarism, which is nothing but copied content, produces duplicate content between two websites. It is well known that search engines are adverse to duplicate or identical content and that page rankings can suffer because of plagiarism. Google has guidelines against plagiarism and warns website owners against creating multiple pages, subdomains, or domains with substantially duplicate content, as stated in its Webmaster Guidelines. Having even a substantially duplicate content, much more a totally duplicate website, will penalize a websites' page rank.

In addition to having the page rank penalized, the website containing plagiarized work or duplicate content can even be affected so much so that it won't even make it to the search results listing. This happens because web crawlers, in an effort to be more efficient, do not crawl duplicate and near-duplicate sites that they have determined in their previous crawl. Because of this, duplicate sites do not even get indexed by the web crawlers, and so are not listed in the results page. In Google, mirrored sites are delegated to the link at the bottom of the results page which says "Similar Results have been omitted..."

Higher Ranking for Plagiarists' Websites

The problem, however, lies when search engines instead of penalizing plagiarists end up penalizing the website that actually wrote the original content. Website owners of original content oftentimes find that websites who have copied their content are given a better page rank by search engines.

Why Plagiarists Get a Higher Page Rank

This problem of being ranked lower than plagiarists' site happens mainly since Google and other search engines algorithms are unable to distinguish between the original and the duplicate or copied content in their index. Although it is true that search engines in general are adverse to identical content, since there is currently no way for the search engines to know which content is original, website owners of the original content oftentimes find that their site actually gets a lower ranking than a plagiarists' site. Hence website owners of original content are the one that sometimes get penalized for plagiarism.

Also, plagiarists, who are driven by profit use other SEO techniques that drive their page rank much higher. Hence, website owners with original content but without very good optimization techniques often find themselves ranked lower than content "borrowers". This is of course a serious issue for website owners, especially business websites whose revenue gets affected by the lower page rank.

One way that search engines address the issue of duplicate content is by assigning priority for the content of the website that was indexed first. By rule of seniority, the first site that posts the content should be the site that contains the original content. This works most of the time but breaks down when plagiarists copy the content quite fast so that crawlers sometimes do index the copied site before the original one. When this happens the site with the copied content gets priority over that of the original content. To avoid this, some website owners use the Sitemaps system to submit their articles or other fresh content to Google. This ensures that the submitted content will be known to Google as the original and not the copy.

Fighting Plagiarism and its Effect on Your Page Rank

Plagiarism has been an issue long before the World Wide Web was created. The creation of the Internet has made it easier and more lucrative for plagiarists to copy content and pass it off as their own without any regard to copyright laws. As mentioned earlier it is a serious threat to business owners and at the very least is a source of irritation to bloggers whose posts or articles oftentimes get ripped off. Because of this, while most companies do their best to fight plagiarists many individuals find themselves losing heart. The fight against plagiarism seems to be a losing battle, especially to those who have no idea in countering plagiarism.

The first thing website owners can do is to find out in the first place if their site has been plagiarized. A good way to do this is by using Copyscape. Copyscape lets website owners enter their web pages' URL and then proceeds to scan other websites and notify the user for very close matches in content. Using the standard Google search also works. Simply copy a short paragraph from your webpage and paste it in Google's search box to come up with all the other websites that contain the same or similar information.

In case your content has been copied the next best step to take is to make a copy of the copied page(s) and then send an email to the website requesting them to remove the copied material or at least for a back link and proper crediting of the work, depending on your objective. In case the website owner or administrator doesn't reply at all or in a favorable manner, the next step to take is to contact the web hosting company they are using. Inform them that the site that they are currently hosting is infringing copyright and that they have a responsibility legally to regulate such sites. This could lead them to pressure the site to remove the copied content. However, this tack doesn't always work. When this happens it is time to take a more legal action and contact your lawyer. According to an advice given by Roy Sumatra, in case your web page rank is affected by the copied website do contact the search engines wherein your page rank is affected. Google is aggressive and serious in dealing with infringement issues and has guidelines for dealing with infringement of the Digital Millennium Copyright Act.

For those who want to prevent the hassle of being penalized as the copier of your own original content, as mentioned earlier, it is best to submit your content to through the Sitemaps system. By using the system, the content is submitted directly to Google and indexed by them automatically so that the content will be known as the original one and be given priority in case of plagiarism.

Sources
http://www.goarticles.com/cgi-bin/showa.cgi?C=282728

The Notion of "SOA 2.0" is Just Plain Silly

Here we go again. While the paint is still wet on this new Web 2.0 stuff, many SOA vendors and large analysts firms are calling their market SOA 2.0. It's one of the silliest things I've heard in a long while, and both the analysts and vendors who use this term should be ashamed of themselves.

I get Web 2.0 because the Web is well over 10-years-old and we've been successful in using this pervasive technology and now we're moving to newer and more exciting stuff such as AJAX and RSS thus the new version number. However, we've yet to get large-scale traction with SOA so SOA 2.0 is illogical since SOA 1.0 never existed if we're realistic.

Moreover, SOA is an architectural concept, not a software product, and to put a version number on something like that shows you don't understand the notion in the first place. SOA is a journey, not a project or product, and to try to make it such is to demean the core concept and the value it can bring. My larger concern, however, is that hype like SOA 2.0 could cause many of those moving towards SOA to become disenchanted and ignore the architectural issues, and hurt their business.

I suspect the marketing guys are at it again and that that's where this thing came from. Once again the people who buy the technology have to get involved and push back against this kind of foolishness or else you'll see it again and again. As such, I urge you to tell your vendors that SOA 2.0 is silly, and if they use the term they'll lose creditability. If enough hear that, the term will die, and other new marketing words like "SOA 3.0," "SOA Next Generation," and "SOA-nator" won't show up either.

SOA (No Version Number)

A SOA is a strategic framework of technology that allows all interesting systems, inside and outside an organization, to expose and access well-defined services and the information bound to those service that may be further abstracted to orchestration layers and composite applications for solution development. This is not a product, not a piece of software; this is an architectural concept. Am I clear?

The primary benefits of a SOA include:

- Reusing services/behaviors or the ability to leverage application behavior from application to application without a significant amount of re-coding or integration. In other words, using the same application functionality (behavior) over and over again, without having to port the code, leveraging remote application behavior as if it existed locally.
- Agility, or the ability to change business processes on top of existing services and information flows, quickly, and as needed to support a changing business.
- Monitoring, or the ability to monitor points of information and points of service in real-time, to determine the well being of an enterprise or trading community. Moreover, the ability to change processes to adjust processes for the benefit of the organization in real-time.
- Extend reach, or the ability to expose certain enterprises processes to other external entities for the purpose of inter-enterprise collaboration or shared processes. This is, in essence, next-generation supply chain integration.

The notion of a SOA isn't new at all. Attempts to share common processes, information, and services have a long history, one that began more than 10 years ago with multi-tier client/server - a set of shared services on a common server that provided the enterprise with an infrastructure for reuse and now provides for integration - and the distributed object movement. "Reusability" is a valuable objective. In the case of a SOA it's reuse of service and information bound to those services. A common set of services among enterprise applications invites reusability and, as a result, significantly reduces the need for redundant application services.
What is unique about a SOA is that it's as much a strategy as a set of technologies, and it's really more of a journey than a destination. Moreover, it's a notion that depends on specific technologies or standards such as Web Services, but really requires many different kinds of technologies and standards for a complete SOA.

SOA as a Discipline

What's clear about SOA is that while we are now beginning to see tactical successes, the large-scale benefits of leveraging this concept have yet to be understood by most organizations. Truth be told, it's going to take time before we can brag about the benefits of SOA, and perhaps the hype will have died down by then, thus some of the confusion that's around today. This confusion includes the number of WS-* standards that are around, many of which are redundant and conflicting. But that's another column or blog.

While SOA 2.0 is a silly notion, we look to evolving our thinking to a place where SOA is more "the architecture," not "an architecture." And there's a difference. What's more, we have to understand that systemic changes such as using SOA is going to take most organizations many years to implement. Unfortunately there are no shortcuts like changing version numbers.

About David Linthicum
David S. Linthicum is the president and CEO of BRIDGEWERX, and the author of several books on application integration and service-oriented architecture, and the host of the SOA Expert Podcast. You can reach Dave at David@bridgewerx.com.

Sources
http://webservices.sys-con.com/read/250501.htm

Web 2.0 Players in the Video Market

Let's review some of the Web 2.0 players in the online video market. In this review we will look at applications that are Internet-based. Some of the new entrants in the video editing arena provide downloadable tools. We are not looking at those tools in this review.





Strength: Online Editing.

Eyespot is a very interesting Web 2.0 solution. This Rich Internet Application enables you to upload and edit your videos. Most other sites expect you to perform the editing on your PC, Apple Mac or Linux machine and then upload the result. EyeSpot provides online tools to mix video and music to create a finished video. EyeSpot allows you to upload and mix video, music and photos. Here is a quick video of images I uploaded, compiled and mixed in a few minutes. EyeSpot has a lot of potential since their online tools can be integrated as a front-end to a number of other services such as blip.tv and veoh. Indeed, you can setup publishing to blogger, LiveJournal, veoh and Blip.tv in your EyeSpot profile.






Strength: Sharing

Blip.tv is a video sharing site. has already made headlines by winning a deal to provide video services to support media companies including CNN. As Marshall Kirkpatrick pointed out in TechCrunch's August 15th Blog EyeSpot has partnered with Blip.tv. The two startups offer services that are highly complimentary.





Strength: Sharing

Veoh is a video sharing site. While it is possible to view video online the real power of Veoh comes from the downloaded client for PCs and Macs. The client creates a peer-to-peer network for sharing video and allows subscribers to view high quality, full screen video. The benefit of this approach is the distribution of video becomes highly scalable. Veoh is definitely attracting interest and investment. They recently raised $12.5 million in a B venture round from investors including former Disney CEO Michael Eisner and Time Warner.





Strength: Sharing

VideoJug appears to be a video sharing site with a mission. To explain life on film. VideoJug is seeding their site with educational videos but is encouraging the Internet community to contribute instructional video. It will be interesting to see if they can differentiate themselves from the likes of YouTube and build up a critical mass of educational content.







Strength: Sharing

Flurl is a video sharing and search site. It allows media to be uploaded anonymously in a variety of formats. There is a 20MB cap on uploads that limits the size of video content. The service is advertising supported. The terms of service prevent linking to media files directly. Links are made to links that serve the content. This enables Flurl to serve advertising to viewers.





Strength: Sharing

YouTube is now one of the top ten destination sites on the Internet serving over 100 million videos each day. It has achieved this position by making it exceedingly easy to upload and share video. YouTube hosts videos converting them from multiple sources to flash movies. Video is limited to 10 minutes in length or 100MB in size.




Strength: Search

Yahoo Video is evolving as a competitor to YouTube. Yahoo members are able to upload video to Yahoo Video. Yahoo provides video search to find video content on the Internet. Video uploads are limited to 100MB and multiple formats are supported. Yahoo supports tagging, rating and sharing of videos.





Strength: Search

Google has a video search site. Google recently added the ability to upload and share video on their beta site. Uploads are limited to 100MB and the popular video file formats are supported. Google provides a desktop uploader for Windows, Mac and Linux platforms that allows files larger than 100Mb to be uploaded. During the beta phase Google offered the option to charge for video downloading. This feature is currently suspended but expect it to return. It will be another example of Google leveraging it's core capabilities and extending in to new areas. In this instance use your Google Account to manage upload of videos. Use Google analytics to check on the popularity of your video and charge for premium content using Google Checkout.




Stength: Review

MovieTally is somewhat of an anomaly but is worth mentioning as a new site in very early beta. MovieTally does not host video, nor does it search the Internet for content. MovieTally is a community-based movie collection. It uses tagging and wiki type concepts to compile movie information. I believe the site could benefit from leveraging Amazon, for example finding movies from their database if the information does not exist in MovieTally. I can see real e-commerce potential with link ups with services such as Fandango, to purchase Movie tickets, and Amazon, to buy movies.

The site is worth a visit. Register (its a quick process) and enter a couple of your favorite movies. MovieTally demonstrates many of the traits of a Web 2.0 application where network effects make the site more valuable as more people use it. I have reservations that people will really take the time to enter in Movie information, such as actor, director and plot summaries when Amazon is just a click away - but at the outset very few people expected Wikipedia to grow in to the extensive resource that it has become. I think this issue can be addressed by pulling information from Amazon using their web services toolkit.

The developer, Hayden Metsky, has certainly paid attention to building a participative architecture with cross linkages embedded throughout the site. I intend to keep an eye on the evolution of this site in the coming months. There is an opportunity for MovieTally because people are looking for a movie recommendations and ratings site that they can trust - one that reflects their perspectives and those of their friends.

What are your favorite online video resources on the Web? Let us know and join the conversation by leaving your comments below.

Sources
http://web2.wsj2.com/media_20_and_the_world_of_
online_video__never_mind_the_quali.htm

Web 1.0 vs. Web 2.0

Creating a Live.com Ecosystem

The Live.com service has tools and applications that are attractive to consumers, businesses, developers and advertisers. Each of these groups is fueled by the presence of the other groups (for more on discussion of network effects check out the earlier blog on embracing the network). Describing Windows and Office Live is a moving target but the diagram below attempts to place the various services in context in the ecosystem and identify those components that have a desktop client dependency.