September 8th, 2009

Transforming Your Company with Open Source — LinuxCon 2009

Update: 9/28/2009: Added slides

Here are the slides from my LinuxCon 2009 presentation on ways to transform your organization with open source. It builds on insights from Open Source and Interdependent IT and many other areas.

Make sure to browse through the notes for the slides as well. You can download the document from the slideshare.net page.

Here is the abstract below.

How do you get past incremental change to real transformation with open source? Most conventional advice focuses on just one dimension at a time: people, process, business or technology. To get breakthrough results, you need to view these dimensions as part of an integrated whole, and consciously address two or more of them simultaneously. Using examples from Enterprise IT experience, we’ll explore how you can tap your organization’s strengths in a variety of ways to achieve these goals. The audience for this session includes technical leaders as well as managers who want to drive change through open technologies, processes, communities, and business models. The session will discuss approaches that can be used by people in a variety of roles.

August 31st, 2009

Open Source and Interdependent IT

In The Seven Habits of Highly Effective People, Stephen Covey presents a maturity continuum that progresses from dependence, to independence, and then to interdependence.

Dependence, Independence and Interdependence

Dependence, Independence and Interdependence

Covey concisely observes: “Dependent people need others to get what they want. Independent people can get what they want through their own effort. Interdependent people combine their own efforts with the efforts of others to achieve their greater success.”

As we develop along this continuum, it’s obvious that our level of capability expands tremendously. In the workplace, dependent employees need guidance for major work tasks; independent employees can figure out what they need to know, but they might not share that knowledge; interdependent employees, in addition to their self-sufficiency, are constantly learning from and teaching their network of peers. Interdependence is highly regarded by employers, and shows up on many statements of corporate principles and values.

So how can IT organizations actually develop interdependent thinking and behavior? Here’s where open source comes in. People need experiential learning opportunities that present themselves in day to day work. In my experience with IT organizations, open source (and more broadly, open ways of working) has the power to fundamentally improve the way that people work. In the course of doing their jobs, they can readily adopt behaviors and mental models that move them higher on the maturity continuum.

If money is the only thing that can talk, the conversation becomes significantly constrained

Consider the staff of a dependent IT organization whose relationship with a proprietary vendor centers around licensing and support fees. If they want a new feature added to the product and the vendor has other priorities, there’s not much they can do. They may be able to get some attention by offering to pay even more money. But even if money talks, if it’s the only thing that can talk, the conversation becomes significantly constrained. Worse, if they carry that same mindset into internal interactions, thinking that “this is the only way that business works”, they risk reducing their own position to a zero-sum game where they can win only by competing for scarce funding. They then remain largely unaware of and unable to tap into the broader resources that they already possess, both individually and collectively.

Open source can liberate IT organizations from that dependence. For starters, they no longer have to pay a licensing fee and can run as many software instances anywhere they want, anytime they want. This is especially relevant in virtualized, cloud computing environments where most traditional licensing metrics (like the number of processors) effectively stifle exploration. They’re welcome to customize products as much as they want to, since the source code is freely available. Even if they don’t have the skillset to fix bugs or make enhancements, they can choose to contract with any company that does. In all of these cases, they become more independent than they were before. As they continue to develop, their vocabulary grows to include a fuller range of their skills and capabilities. This enables them to make more granular, intelligent tradeoffs across the spectrum of buying, integrating and building. At the independent stage, IT organizations learn to control their own destiny.

At the interdependent stage, IT organizations can bring an entire ecosystem’s capabilities to bear to solve their employer’s problems

IT Organizations get a taste for interdependence when they contribute code changes to an open source project. For example, let’s say they submit a simple patch to make a particular parameter configurable instead of being hardcoded. Once this change is accepted, instead of having to re-port this patch each time they download a new release, they can rely on it being maintained by the project developers. Developing trust with the project team can encourage them to make further contributions, with the immediate payoff of support and the longer-term reward of getting their most important features added. What really expands their horizons is when they see their work being built upon and extended further by the community, in ways that they could never have previously envisioned. They realize how much more they can accomplish through collaboration than they can on their own. At the interdependent stage, IT organizations can have holistic conversations that represent the capabilities of an entire ecosystem, bringing it to bear to solve their employer’s specific problems. And as they solve those problems, they can contribute their learnings to advance the capabilities of the ecosystem, fueling a virtuous cycle.

Not all of this happens automatically, of course, or by some fixed timetable where after N months of implementing activities X, Y and Z, IT organizations graduate to interdependence and experience all of its benefits immediately. But it does happen naturally, when people with sufficient baseline capability are immersed in an open environment, are given ample coaching and support, and can see examples of highly skilled behavior in action. Let’s look at some on-the-job experiences that illustrate this point.

When IT organizations are customers of a proprietary product and something goes wrong, it’s easy for them to develop a kneejerk reaction to call the vendor and complain. For starters, they’re paying for support. But what intensifies this is when they feel like they’ve hit a brick wall and can’t go any further because the product is closed. If this happens frequently, they may give up even before trying, become even more frustrated with the vendor, or adopt some other less effective behavior. With open source, nothing stands in the way of their own problem-solving skills. IT developers can browse through the code and see if something catches their eye. They can invoke a debugger and see what might be happening in real-time. They can run third-party analysis software on it, or undertake a detailed review themselves if they have the time and inclination. The more familiar they become with a particular project’s code and with reading source code in general, the more competent they become at solving similar problems in the future. Even if they contract for support on an open source project, they can submit more informed requests that enable the support provider to zone in on their problem more readily. They can also gauge the competence of that support organization based on their grasp of the code, and switch to a better provider if necessary.

Working with high-quality open source projects instructively leads IT organizations to a deeper understanding of and appreciation for open architectures and open standards. Code that reflects a broad understanding of the problem that it’s solving, addresses it in a clean way, and is as simple as possible, tends to be accepted. Whereas code that defines a problem too narrowly, introduces unnecessary kludges, or increases complexity without substantially increasing usefulness, tends to be rejected. This is an effective counterweight against a tendency in many IT organizations to rush out incomplete solutions that end up causing more problems over time. When contributing code, IT developers learn to think more globally, putting themselves in the shoes of others who share similar challenges albeit in vastly different environments.

Succeeding with open source builds the brands of both employee and employer, and the affinity between those brands

As they succeed with contributing to open source projects with the support of their employer, they gain the respect of fellow users/developers, which directly translates into leadership and influence. Their personal brands and their employer’s brand increase both in value and in affinity with each other. Employees get public recognition for their efforts, and simultaneously are incented to continue to be affiliated with their employer’s brand. This type of synergy is a hallmark of interdependent organizations.

An important lesson learned by IT organizations on the road to interdependence is where and when they can optimally invest their resources with open source projects. Some projects are very open to input from the practical experiences of Corporate IT. Other projects are unwilling to adopt suggestions that don’t align with their strict tenets. The best projects balance purity with pragmatism, maintaining the essence of what makes them valuable as a whole. While experimenting with various approaches, IT organizations discover which problems they want to solve on their own, and which ones they don’t. They identify where they want to be treated as a customer, where they want to interact as a peer, and where they want to play a leadership role.

Open Source is the foundation of outside innovation and exonovation

People who work in this kind of environment, day in and day out, can develop strong collaborative networks all over the world. These networks help to identify those relationships with customers, partners, and others that can be transformed from largely transactional relationships into ones of deeper mutual benefit. They develop instincts for and insights into what makes blogs, wikis, and other social software work best, and can be invaluable in shaping their employer’s social media strategy. On a broader level, interdependent IT professionals are powerful catalysts of outside innovation as described by Patricia Seybold, and exonovation [interview] by Michael Tiemann. They can operate from a visionary mental model of commerce and community.

Open Source is a complex, multidimensional subject that intersects and often integrates the fields of science, technology, business and psychology. By expressly choosing open source as a strategy for developing interdependent capability, IT organizations can provide a growth path for knowledge workers that engages them in a variety of ways and on many levels. Those who become adept at making the interdisciplinary connections can represent their organization to a number of audiences in ways that are directly compelling, and can create new streams of value. They help bring a unified sense of purpose, energy, and vision that inspires people to realize their potential.

July 13th, 2009

Exporting del.icio.us blog links to OPML

I manage my bookmarks in del.icio.us, and tag the ones that are blogs with the blog tag (in addition to subject-related tags). When I want to try out a new blog reader, I can easily export my bookmarked blogs to OPML, using the additional tags as first-level subcategories. If the RSS feed for a given blog isn’t discoverable with a <link> tag on the blog URL, I add it as a line in the notes section: RSS=http://someblog.com/rssfeed.

Here’s the Perl script, delicious_blogs_opml.pl, that exports blog links in del.icio.us to OPML format:

#! /usr/bin/perl
 
# delicious_blogs_opml.pl - Export OPML blogs feed from del.icio.us
 
use Net::Delicious;
use XML::Feed;
# use XML::OPML;
# XML::OPML doesn't work as expected
 
use strict;
 
my $debug;
$ARGV[0] eq '-d' and $debug = 1, shift;
@ARGV > 0 or die "usage: $0 [-d] delicious_username:password\n";
 
my %tags;
my ($delusername, $delpasswd) = split(/:/, $ARGV[0], 2);
my $del = Net::Delicious->new(
    {user => $delusername, pswd => $delpasswd, debug => $debug});
 
# Workaround for Net::Delicious:all_posts() not supporting a tags argument:
# $del->all_posts({tag => 'blog'}) 
$del->config('delicious_posts_all.tag' => '');
# Get all blog links from del.icio.us
my $res = $del->_execute_method("delicious.posts.all", {tag => 'blog'})
    or die "Error calling posts/all\n";
my $posts = $del->_getresults($res, 'post');
foreach my $post ($del->_buildresults('Post', $posts)) {
    my $title = $post->description;
    foreach my $tag (split(' ', $post->tags)) { 
        $tags{$tag}{$title} = $post unless $tag eq 'blog';
    }
}
 
print <<EOF;
<?xml version="1.0" encoding="UTF-8"?>
<opml version="1.0">
<head>
    <title>blogs</title>
</head>
<body>
EOF
 
foreach my $tag (sort keys %tags) {
    print qq(<outline title="$tag" text="$tag">\n);
    foreach my $blogname (sort keys %{$tags{$tag}}) {
        my $url = $tags{$tag}{$blogname}->{href};
        my ($feedurl) = XML::Feed->find_feeds($url);
        if (!$feedurl) {
            my $extended = $tags{$tag}{$blogname}->{extended};
            ($feedurl) = $extended =~ /^RSS\s*=\s*(\S+)/im;
        }
        $feedurl or die "Could not find RSS feed for $url\n";
        print qq(\t<outline title="$blogname" text="$blogname" type="rss" xmlUrl="$feedurl"/>\n);
    }
    print "</outline>\n";
}
 
print <<EOF;
</body>
</opml>
EOF
July 9th, 2009

Improving the Alltop Experience

I’ve been using Alltop for a while, and have found it to be pretty impressive. First, take a look at this creatively produced 90-second video overview:

Go to alltop.com, and then pick a topic at random. What you see, almost (I’ll come back to that later) as far as your eye can travel on your monitor, is a bunch of headlines from interesting and relevant sources for that topic. If any of those headlines catches your fancy, just move your mouse over it and an excerpt of that article pop ups up. If you like what you see, you can click on it and the full article opens up in a new browser window/tab.

The Alltop team has done a solid job at finding good sources of content, grouping them into relevant categories, and maintaining them over time. It’s great to be able to discover a pageful of quality writers on a given subject at a glance, without having to go hunting for them, one by one, in some blog directory or search engine.

Visually, Alltop is ideal for blog scanning. It packs a ton of content into a small amount of space without getting cramped. I can let my eyes roam free across the page as quickly (or as slowly) as I like, paying as much (or as little) attention as I want to the content by looking or skipping, hovering and finally clicking. This is important — it’s just as easy to find interesting stuff as it is to disregard uninteresting stuff (where interesting may of course mean different things at different times). You end up getting the most out of your time on the site.

Alltop has a customization feature, called MyAlltop, which you can use to add individual feeds from the site to a personalized Alltop page. I have one set up and will probably be expanding it over time. But there are other ways that I get more value out of Alltop, which brings me to the title of this blog post — improving the Alltop experience.

Here’s my favorite way to use Alltop. First, I bookmark all of the Alltop subject pages that I want to follow in Firefox with the subcategory alltop (I actually do this indirectly via del.icio.us tags and bookmark sync). I then click on the subcategory and select “Open All in Tabs”. I maximize my browser window, sit back, and scroll, hover, click and tab my way through the blogosphere 🙂

When I first tried this on my 1920×1200 monitor, I discovered to my chagrin that Alltop limited the display to three columns of text, regardless of the window size. There was no way I could let that get in the way of an otherwise stellar user experience! So I looked through the HTML, found what was causing this limitation, and (largely) worked around it with the Firefox Stylish extension. Stylish lets you customize the style of a web page. It does for CSS what Greasemonkey lets you do on a larger scale with HTML and JavaScript. Here is a Stylish user script that maximizes the number of columns displayed by Alltop for a 1920 pixel-wide screen:

@namespace url(http://www.w3.org/1999/xhtml);
@-moz-document domain("alltop.com")
{
    #headlines { width: 1700px !important }
    .row-clear { display: none }
}

If the ad in the upper-right hand corner gets in the way, you can hide it with .ad-unit { display: none }. Here’s a screenshot of my Alltop setup with Firefox:

My Alltop Setup (click for full size image)

My Alltop Setup (click for full size image)

I picked a well-displayed area for the screenshot, but most category pages end up with some awkward blank spaces due to the browser rendering model. One way to properly fix this would be to query the window size, and recalculate where to insert row-clear tags indicating the end of each row. I think this would be a hugely valuable enhancement for Alltop to add.

Another helpful feature would be the ability to save multiple pages in MyAlltop using predefined category names, or to create your own category pages. Then they could be opened in multiple tabs (supported by either Alltop or the browser).

I find Alltop to be an excellent addition to my toolkit of managing information overload. Sometimes, when I read through a bunch of Alltop pages, I feel like I’ve actually — dare-I-say-it — caught up with the latest thinking on a particular subject. Maybe Alltop can help you get on top of things too.

June 19th, 2009

Time Management

Here are a few books on time management that I have found to be particularly valuable:

First Things First

First Things First

I first wrote about First Things First, by Stephen Covey, in the April 1998 issue of USENIX’s ;login: magazine. I consider it to be the best conceptual exploration of the subject. Three of its most powerful messages are to:

  • Distinguish between the important-but-not-urgent and the urgent-but-not-important
  • Exercise integrity in the moment of choice
  • Reframe the idea of Time Management into one of Life Leadership

Indeed, when you reframe time management as life leadership, you’re motivated to exercise integrity in the moment of choice, and you become able to distinguish between the important-but-not-urgent and the urgent-but-not-important. This lets you spend more time on the things that are truly the most important to you.

Here’s a great video from a Stephen Covey workshop that visually demonstrates the value of putting First Things First:

 

Getting Things Done (GTD)

Getting Things Done

With GTD, David Allen provides a rich, practical toolset to help you — what else — get things done. Here are some of his key points:

  • Get your “open loops” (essentially, those incomplete things that you have agreed to do or are planning to do) out of your head and into your system, so that you don’t have to expend inefficient mental energy on that stuff. Free your mind to do what it does best.
  • When you’re feeling stuck on a project, focus on the next action that needs to be done. This is an actual, concrete, physical task that you can do that will get you closer to your goal. Identifying the next action, even if it’s seemingly minor, can often help break up logjams.
  • Develop in-context lists for where and when you are, to make optimal use of your time. For example, if you accumulate a list of calls to make and have it with you when you get into your car, you can go through a bunch of them when you’re waiting in traffic. As another example, if you just found out about a product that’s currently in beta, and probably not ready for another couple of months, have your calendar alert you 60 days from now so that your future self can follow up.

Here is a interview by Fast Company with David Allen on GTD:

 

Generally speaking, First Things First takes a top-down approach, and Getting Things Done takes a bottom-up approach. I find them to be highly valuable, complementary in nature, and I’ve successfully applied strategies and tactics from both. I skip the aspects of both systems that I find to be burdensome, and keep what I like. The result is a continually evolving, personalized approach that works best for me.

 

Type and Time Management

Type and Time Management

Speaking of personalized approaches, I think that Type and Time Management by Sharon Fitzsimmons is an excellent resource for people looking for time management tips that fit their personality. It’s also great for anyone who is coaching others. The book includes detailed interviews with several people from each of the 16 different Myers-Briggs types (I’m interviewed in the ENTP chapter); discussions of the general interactions with each personality type; a well-organized summary page for each type with specific suggestions; and a final insightful chapter on Comparisons, Contrasts, Questions and Conclusions.

March 27th, 2009

Mental Models of Commerce and Community

Commerce and community play important roles in open source ecosystems. Companies are great at building capital and targeting it towards focused goals through products and services. Communities are great at bringing together like-minded people from all walks of life and advancing their common ideals. Together, commerce and community can be an unstoppable force for turning a shared vision into reality.

That’s not to say that this dynamic duo of collaboration is always guaranteed to materialize. It takes a sufficient level of trust and mutual understanding, and a willingness to work through contentious situations with common goals in mind. I find that the mental models that people have about commerce and community substantially influence the potential value that they can realize through these interactions.

FUD Model

At the low end of trust and understanding is what I characterize as the FUD model — marked by Fear, Uncertainty, and Doubt. Here, cynicism and defensive thinking loom large. Companies are seen as soulless entities whose only reason for existence is to exploit anything and anyone to make money. Everything that a company does is viewed through this lens. For example, if a company makes an open source contribution and dares to say anything positive about it, they are subjected to an often tortured analysis that explains how their actions are really only about increasing sales with a particular demographic, blocking a competitor, casting out for free labor, or some other exclusively self-serving motive. Simpler explanations that involve corporate employees sharing a common vision with a broader community and wanting to accomplish goals beneficial both to their company and to the community are rejected outright.

Things are just as harsh in the other direction. Community members are seen mostly as naive hobbyists whose heads are in the clouds, and who couldn’t possibly be reliable because they’re not being paid to do anything. And the rest are a threat — potential or actual competitors just lying in wait to take away business or sue over any perceived infringements. The idea that a company could rely on leadership provided by the community to achieve breakthrough results is unimaginable.

Utilitarian Model

In the middle of the spectrum is the Utilitarian model. It sees past the overt cynicism of the FUD model and adopts a pragmatic mindset. It understands the conventionally-defined roles of commerce and community, and works within the limits of these boundaries. For example, enterprises adopt open source if it saves them money. Open source needs to have a business model as close as possible to that of proprietary software in order to gain customer acceptance. Open Source developers need to be managed with similar kinds of incentives and metrics to those used with proprietary software developers. A company’s success is measured by its revenue, a project’s potency by the number of lines of code.

Along with conventional wisdom, this model is often a purveyor of political correctness. For example, corporations have a burden to “give back to the open source community” as compensation for having “taken” something, independent of the observation that downloading and using open source software actually increases its ambit. Another case is where communities are pushed to develop certification schemes to assess expertise, irrespective of whether those are germane to success.

The Utilitarian model is successful at mapping the terrain and carefully working within its boundaries. The downside is that it is not inclined to challenge and expand those boundaries. People are fairly ensconced within their roles, and aren’t primed to seize creative and unconventional opportunities. Assumptions of how “corporate people” or “community people” are expected to behave can limit the recognition of teaching moments and keep people within boxes. There is also a real risk of dampening genuine passion and energy with saccharine substitutes.

Visionary Model

At the high end is the Visionary model. It sees that commerce and community each have important parts to play in realizing a shared vision. It views people holistically, as self-actualizing individuals whose journey may take them through various corporate and community roles — sometimes holding multiple roles simultaneously. These people are able to work fluidly in different contexts, tapping resources and insights from wherever is most helpful — like making use of blogs or wikis in a corporate environment, or applying project management principles with community activities. Neither calculation nor charity is an unbalancing influence on their consumption of and contribution to open source. It’s a far more natural, conversational rhythm like listening and speaking.

The Visionary model champions high levels of openness, transparency and candor as a matter of course, surpassing anything attainable by the FUD model or the Utilitarian model. There is an unshakable confidence in the power of openness to deliver innovation while developing trust. For example, a services company might share in-depth market intelligence on key customers whose adoption would put an open source project on the map in a big way. Knowing this, the community might rally to complete the development of key features, and through their contacts help the services company sign up the customers. The customers might then collaborate with each other, through the community, to make further improvements and continue the virtuous cycle.

With its broadminded perspective, the Visionary model is optimistic and resilient in the face of impediments. It’s able to protect its interests without getting trapped in the cynicism of the FUD model, and can make strategic withdrawals without surrendering to the limits of the Utilitarian model. It can make the most out of situations where an ideal solution isn’t possible.

The Visionary model holds the greatest promise for the partnership of commerce and community. It is able to navigate the shadows of conflict, and forge a strong framework for excellence.

March 5th, 2009

Managing Twitter Groups with del.icio.us tags

Update: 6/22/2009: TweetDeck now supports group-synchronization as a built in feature.

Twitterdel.icio.usI’ve recently started to use Twitter and have been impressed with the TweetDeck client. It has a great user interface and makes it very easy to follow different groups of people in multiple visual columns.

One of the first tweets I saw was from Jon Udell, always a fountain of ideas. His recent blog post Collaborative curation as a service reminded me of how flexible del.icio.us can be as a distributed store for tags. I like the curating meme that Jon is promoting, where a little bit of technical infrastructure can go a long way to enable virtually anyone to contribute.

As I started to use TweetDeck and define groups locally in the application, I pondered how it might be better if these group definitions were stored on the web. They could then be accessed from different systems where I might use TweetDeck or other twitter clients for that matter. (Of course this will all happen when Twitter implements groups natively, but in the meantime there is a gap that can be filled). Then it dawned on me — del.icio.us is an ideal place to both manage and share twitter groups!

I wrote two Perl scripts to try out this idea. The first script, twitter_delicious_sync.pl, uploads the Twitter URLs of everyone I’m following to del.icio.us and automatically tags them with ‘twitter’. (It acts as a sync process, adding/deleting links as necessary). Then I manually tag them further based on how I want to group them. So for example, I tag Tim O’Reilly with ‘technology’, and Jim Cramer with ‘finance’. I can tag people with multiple tags if I want to read them as part of multiple groups.

twitter_delicious_sync.pl

#! /usr/bin/perl
 
# twitter_delicious_sync.pl - Sync Twitter contacts to del.icio.us 
 
use Net::Twitter;
use Net::Delicious;
use Log::Dispatch::Screen;
 
use strict;
 
my ($debug, %following);
 
$ARGV[0] eq '-d' and $debug = 1, shift;
@ARGV > 1 or die "usage: $0 [-d] twitter_username:password delicious_username:password\n";
my ($twusername, $twpasswd) = split(/:/, $ARGV[0], 2);
my ($delusername, $delpasswd) = split(/:/, $ARGV[1], 2);
 
my $tw = Net::Twitter->new({apiurl => 'https://twitter.com/', 
                           apihost => 'twitter.com:443', 
                           username => $twusername, password => $twpasswd})
    or die "Couldn't connect to twitter\n";
my $friends; my $page = 1;
do {
    $friends = $tw->friends({page => $page++})
        or die("twitter: ".$tw->get_error->{error}."\n");
    foreach my $friend (@$friends) {
        $following{$friend->{screen_name}} = $friend->{name};
    }
} while @$friends;
 
my $del = Net::Delicious->new({endpoint => 'https://api.del.icio.us/v1/',
                              user => $delusername, pswd => $delpasswd})
    or die "Couldn't connect to del.icio.us\n";
# Workaround for Net::Delicious:all_posts() not supporting a tags argument:
# $del->all_posts({tag => 'twitter'}) 
$del->config('delicious_posts_all.tag' => '');
my $res = $del->_execute_method("delicious.posts.all", {tag => 'twitter'})
    or die "Error calling delicious posts/all\n";
my $posts = $del->_getresults($res, 'post');
foreach my $post ($del->_buildresults('Post', $posts)) {
    my $username = $post->href; $username =~ s#^http://twitter.com/##;
    if (exists $following{$username}) { delete $following{$username} }
    else {
        # Delete stale entries
        $del->delete_post({url => "http://twitter.com/$username"});
        $debug and print "Deleted $username\n";
    }
}
 
# Add new entries
foreach my $username (keys %following) {
    $del->add_post({url => "http://twitter.com/$username",
                    description => "Twitter / $following{$username}",
                    tags => 'twitter'});
    $debug and print "Added $username\n";
}

The second script, delicious_tweetdeck_groups.pl, creates/updates a group in TweetDeck corresponding to each tag that I have assigned to the Twitter links in del.icio.us. When I reclassify people in del.icio.us, I simply stop TweetDeck, run the script, and restart TweetDeck. [When I add new people in Twitter and tag them for the first time, I end up restarting TweetDeck twice due to an implementation issue]. In any case, it’s a pretty simple process.

delicious_tweetdeck_groups.pl

#! /usr/bin/perl
 
# delicious_tweetdeck_groups.pl - Update TweetDeck groups from del.icio.us 
 
use Net::Delicious;
use Log::Dispatch::Screen;
use DBI;
use File::Glob ':glob';
 
use strict;
 
my $debug;
$ARGV[0] eq '-d' and $debug = 1, shift;
@ARGV > 0 or die "usage: $0 [-d] delicious_username:password\n";
 
my $sqlitefile;
if ($^O =~ /linux/i) {
    $sqlitefile= bsd_glob("$ENV{HOME}/.appdata/TweetDeck*/Local Store/td_*.db");
}
elsif ($^O =~ /mswin32/i) {
    $sqlitefile = bsd_glob("$ENV{APPDATA}\\TweetDeck*\\Local Store\\td_*.db");
}
my (%groups, %gcids);
 
my ($delusername, $delpasswd) = split(/:/, $ARGV[0], 2);
my $del = Net::Delicious->new(
    {user => $delusername, pswd => $delpasswd, debug => $debug});
 
# Workaround for Net::Delicious:all_posts() not supporting a tags argument:
# $del->all_posts({tag => 'twitter'}) 
$del->config('delicious_posts_all.tag' => '');
# Get all twitter group memberships from del.icio.us
my $res = $del->_execute_method("delicious.posts.all", {tag => 'twitter'})
    or die "Error calling posts/all\n";
my $posts = $del->_getresults($res, 'post');
foreach my $post ($del->_buildresults('Post', $posts)) {
    my $username = $post->href; $username =~ s#^http://twitter.com/## or next;
    foreach my $tag (split(' ', $post->tags)) { 
        $groups{$tag}{$username} = 1 unless $tag eq 'twitter';
    }
}
 
my $dbh = DBI->connect("DBI:SQLite:dbname=$sqlitefile", '', '');
 
my $columns = $dbh->selectall_hashref('select * from columns', 'cID');
foreach my $cid (sort keys %$columns) {
    # Exclude all non-group columns
    if ($columns->{$cid}{cType} > 0 && $columns->{$cid}{cType} != 999) {
        delete $columns->{$cid}; 
        next;
    }
    # Look for corresponding groups that already exist
    my $cname = $columns->{$cid}{cName}; $cname =~ s/^Group: //;
    if (exists $groups{$cname}) {
        $gcids{$cname} = $cid;
        delete $columns->{$cid};
        next;
    }
}
 
# Assign new groups in the columns table
my @cids = sort keys %$columns;
my $sth = $dbh->prepare("update columns set cName=?, ctype=? where cID=?");
foreach my $group (sort keys %groups) {
    next if exists $gcids{$group};
    last if !defined($gcids{$group} = shift @cids);
    $sth->execute("Group: $group", 0, $gcids{$group});
}
 
# Clear any stale groups in the columns table
$sth = $dbh->prepare("update columns set cName='',ctype='' where cID=?");
foreach my $cid (@cids) { $sth->execute($cid) }
 
# Set up userid hash map
my %userids = map @$_, 
    @{$dbh->selectall_arrayref('select fScreenName,fUserID from friends')};
 
# Rebuild the groups table
$dbh->do('delete from groups');
$sth = $dbh->prepare('insert into groups (gID, gCID, gUserID) values (?,?,?)');
my $gid = 1;
foreach my $group (keys %groups) {
    foreach my $username (keys %{$groups{$group}}) {
        $sth->execute($gid++, $gcids{$group}, $userids{$username});
    }
}

Managing TweetDeck groups using del.icio.us tags opens up a broader set of possibilities. I can see how others have tagged the same people, and vice versa. If enough people were to do the same thing, we could build up a crowdsourced view of Twitter broadcasters. This is the kind of experience that made Spock so exciting in its early days, when anyone could tag everyone, with the results immediately available for all to see.

In a recent interview, Yahoo’s CEO stated that they are looking at how to “partner with sites such as YouTube, Twitter, Facebook and Skype to make the user experience as seamless as possible as they move from Yahoo to more social places”. With del.icio.us, I think Yahoo might have the capability to fill both the Twitter-group gap and the Spock-tagging gap, and advance their social software strategy in the process.