Managing Twitter Groups with del.icio.us tags
(2 votes, average: 5.00 out of 5)
Loading ...
Update: 6/22/2009: TweetDeck now supports group-synchronization as a built in feature.
I’ve recently started to use Twitter and have been impressed with the TweetDeck client. It has a great user interface and makes it very easy to follow different groups of people in multiple visual columns.
One of the first tweets I saw was from Jon Udell, always a fountain of ideas. His recent blog post Collaborative curation as a service reminded me of how flexible del.icio.us can be as a distributed store for tags. I like the curating meme that Jon is promoting, where a little bit of technical infrastructure can go a long way to enable virtually anyone to contribute.
As I started to use TweetDeck and define groups locally in the application, I pondered how it might be better if these group definitions were stored on the web. They could then be accessed from different systems where I might use TweetDeck or other twitter clients for that matter. (Of course this will all happen when Twitter implements groups natively, but in the meantime there is a gap that can be filled). Then it dawned on me — del.icio.us is an ideal place to both manage and share twitter groups!
I wrote two Perl scripts to try out this idea. The first script, twitter_delicious_sync.pl, uploads the Twitter URLs of everyone I’m following to del.icio.us and automatically tags them with ‘twitter’. (It acts as a sync process, adding/deleting links as necessary). Then I manually tag them further based on how I want to group them. So for example, I tag Tim O’Reilly with ‘technology’, and Jim Cramer with ‘finance’. I can tag people with multiple tags if I want to read them as part of multiple groups.
#! /usr/bin/perl # twitter_delicious_sync.pl - Sync Twitter contacts to del.icio.us use Net::Twitter; use Net::Delicious; use Log::Dispatch::Screen; use strict; my ($debug, %following); $ARGV[0] eq '-d' and $debug = 1, shift; @ARGV > 1 or die "usage: $0 [-d] twitter_username:password delicious_username:password\n"; my ($twusername, $twpasswd) = split(/:/, $ARGV[0], 2); my ($delusername, $delpasswd) = split(/:/, $ARGV[1], 2); my $tw = Net::Twitter->new({apiurl => 'https://twitter.com/', apihost => 'twitter.com:443', username => $twusername, password => $twpasswd}) or die "Couldn't connect to twitter\n"; my $friends; my $page = 1; do { $friends = $tw->friends({page => $page++}) or die("twitter: ".$tw->get_error->{error}."\n"); foreach my $friend (@$friends) { $following{$friend->{screen_name}} = $friend->{name}; } } while @$friends; my $del = Net::Delicious->new({endpoint => 'https://api.del.icio.us/v1/', user => $delusername, pswd => $delpasswd}) or die "Couldn't connect to del.icio.us\n"; # Workaround for Net::Delicious:all_posts() not supporting a tags argument: # $del->all_posts({tag => 'twitter'}) $del->config('delicious_posts_all.tag' => ''); my $res = $del->_execute_method("delicious.posts.all", {tag => 'twitter'}) or die "Error calling delicious posts/all\n"; my $posts = $del->_getresults($res, 'post'); foreach my $post ($del->_buildresults('Post', $posts)) { my $username = $post->href; $username =~ s#^http://twitter.com/##; if (exists $following{$username}) { delete $following{$username} } else { # Delete stale entries $del->delete_post({url => "http://twitter.com/$username"}); $debug and print "Deleted $username\n"; } } # Add new entries foreach my $username (keys %following) { $del->add_post({url => "http://twitter.com/$username", description => "Twitter / $following{$username}", tags => 'twitter'}); $debug and print "Added $username\n"; }
The second script, delicious_tweetdeck_groups.pl, creates/updates a group in TweetDeck corresponding to each tag that I have assigned to the Twitter links in del.icio.us. When I reclassify people in del.icio.us, I simply stop TweetDeck, run the script, and restart TweetDeck. [When I add new people in Twitter and tag them for the first time, I end up restarting TweetDeck twice due to an implementation issue]. In any case, it’s a pretty simple process.
#! /usr/bin/perl # delicious_tweetdeck_groups.pl - Update TweetDeck groups from del.icio.us use Net::Delicious; use Log::Dispatch::Screen; use DBI; use File::Glob ':glob'; use strict; my $debug; $ARGV[0] eq '-d' and $debug = 1, shift; @ARGV > 0 or die "usage: $0 [-d] delicious_username:password\n"; my $sqlitefile; if ($^O =~ /linux/i) { $sqlitefile= bsd_glob("$ENV{HOME}/.appdata/TweetDeck*/Local Store/td_*.db"); } elsif ($^O =~ /mswin32/i) { $sqlitefile = bsd_glob("$ENV{APPDATA}\\TweetDeck*\\Local Store\\td_*.db"); } my (%groups, %gcids); my ($delusername, $delpasswd) = split(/:/, $ARGV[0], 2); my $del = Net::Delicious->new( {user => $delusername, pswd => $delpasswd, debug => $debug}); # Workaround for Net::Delicious:all_posts() not supporting a tags argument: # $del->all_posts({tag => 'twitter'}) $del->config('delicious_posts_all.tag' => ''); # Get all twitter group memberships from del.icio.us my $res = $del->_execute_method("delicious.posts.all", {tag => 'twitter'}) or die "Error calling posts/all\n"; my $posts = $del->_getresults($res, 'post'); foreach my $post ($del->_buildresults('Post', $posts)) { my $username = $post->href; $username =~ s#^http://twitter.com/## or next; foreach my $tag (split(' ', $post->tags)) { $groups{$tag}{$username} = 1 unless $tag eq 'twitter'; } } my $dbh = DBI->connect("DBI:SQLite:dbname=$sqlitefile", '', ''); my $columns = $dbh->selectall_hashref('select * from columns', 'cID'); foreach my $cid (sort keys %$columns) { # Exclude all non-group columns if ($columns->{$cid}{cType} > 0 && $columns->{$cid}{cType} != 999) { delete $columns->{$cid}; next; } # Look for corresponding groups that already exist my $cname = $columns->{$cid}{cName}; $cname =~ s/^Group: //; if (exists $groups{$cname}) { $gcids{$cname} = $cid; delete $columns->{$cid}; next; } } # Assign new groups in the columns table my @cids = sort keys %$columns; my $sth = $dbh->prepare("update columns set cName=?, ctype=? where cID=?"); foreach my $group (sort keys %groups) { next if exists $gcids{$group}; last if !defined($gcids{$group} = shift @cids); $sth->execute("Group: $group", 0, $gcids{$group}); } # Clear any stale groups in the columns table $sth = $dbh->prepare("update columns set cName='',ctype='' where cID=?"); foreach my $cid (@cids) { $sth->execute($cid) } # Set up userid hash map my %userids = map @$_, @{$dbh->selectall_arrayref('select fScreenName,fUserID from friends')}; # Rebuild the groups table $dbh->do('delete from groups'); $sth = $dbh->prepare('insert into groups (gID, gCID, gUserID) values (?,?,?)'); my $gid = 1; foreach my $group (keys %groups) { foreach my $username (keys %{$groups{$group}}) { $sth->execute($gid++, $gcids{$group}, $userids{$username}); } }
Managing TweetDeck groups using del.icio.us tags opens up a broader set of possibilities. I can see how others have tagged the same people, and vice versa. If enough people were to do the same thing, we could build up a crowdsourced view of Twitter broadcasters. This is the kind of experience that made Spock so exciting in its early days, when anyone could tag everyone, with the results immediately available for all to see.
In a recent interview, Yahoo’s CEO stated that they are looking at how to “partner with sites such as YouTube, Twitter, Facebook and Skype to make the user experience as seamless as possible as they move from Yahoo to more social places”. With del.icio.us, I think Yahoo might have the capability to fill both the Twitter-group gap and the Spock-tagging gap, and advance their social software strategy in the process.
Category: Technology | Tags: del.icio.us, TweetDeck, twitter, Yahoo
Email | Share:
Print
Hi, this is a comment.
To delete a comment, just log in and view the post's comments. There you will have the option to edit or delete them.
Interesting. For a certain circle of folks, the Twitter handle can be a proxy for a universal identifier, and thus a useful pivot for metadata. And with automation like yours, you could come along later and change that — e.g. by adding alternate ids.
Interesting reading Kartik. How would you see your integration and crowd sourcing approach integrated with services like twist? http://twist.flaptor.com/
I think that any service that wants to associate users with areas of expertise/description could benefit. So for example, a service like Twist might be able to tell not only what topics are trending, but also what types of people are talking about those trending topics. It adds another level of metadata to the mix.
We are alike! FTF was life-changing for me. I haven’t felt a need to read Allen’s GTD — reading the 43folders.com description and the Wikipedia article were enough to get me going. I have 43 folders on my desktop and on my Palm Pre.
In spite of the optimistic view that enterprises can develop internal skills to provide their own technical support for Open Source Software packages, few actually do develop and maintain those skills. Where an enterprise lacks the internal skills to provide its own professional technical support, they need to turn to support providers just as they turned to the proprietary software developers’ support teams. The Open Source Software technical support market is consistently less expensive than comparable cost of support for proprietary products but it remains a legitimate cost that most enterprises need to budget … either in a commitment of internal resources or in a contract with an external provider. Using Open Source Software without a support plan is self-insurance with a potentially large deductible as it can be quite difficult for qualified technical support staff to suddenly get engaged and come to grips with Open Source Software integrated by the enterprise. Furthermore, enterprises need to qualify their support vendors to make sure those providers actually have team members capable of supporting mission-critical software (MySQL, Apache httpd, OpenLDAP, etc.). We agree that enterprises that become internally competent achieve the benefits you suggest but most are unwilling to make the financial commitment to sustain such a capability year on year in the face of business pressures. The technical support specialists will continue to be a valuable service pool for Open Source Software much as they are for proprietary software.
One point that I hope doesn’t get lost here is that even IT organizations that develop internal competence can contract effectively with open source technical support specialists. As I mention:
“They identify where they want to be treated as a customer, where they want to interact as a peer, and where they want to play a leadership role.”
Compared to dependent IT organizations, interdependent IT organizations can make better use of the talents of technical support specialists, since they have a deeper understanding of the value that those companies provide.
Well-put, Kartik! Marty offers a realistic, pragmatic reply. However, I’m seeing my employer dabble with the model that Kartik describes. Our Open Source & Linux Profession (OSLP) runs several development services that benefit our corporation. None of us doing this for the OSLP are IT workers, but we have had conversations with IT. Let’s say that they’re “intrigued” at this point. Could take years. Developing…
Great article Kartik. I think this is a great model. I have seen the power of open and collaborative IT development in my own experience. Unfortunately, I have also seen the difficulty in getting organizations to change their mindset and be open to this kind of approach. It can be done though, it just takes time and some level of trust building. I think the key is to just give it a shot and use small successes to pave the way forward.
[...] Open source leads to interdependent employees (as opposed to dependent or indep) http://kartiksubbarao.com/open-source-and-interdependent-it [...]
Kartik – I like the use of Covey’s model as a way of describing the effect on behaviors (or perhaps the required behaviors) of those that participate in Open Source Software development. Many of the points made here, as you suggest, apply well to the concepts of Open Innovation and may result in similar beneficial outcomes. Nice work.
Likewise, Open Innovation and Open Source share many challenges. In particular, if the decision makers sitting higher up in the food chain are not themselves interdependent, then the ability for folks in lower levels to participate in open models is significantly constrained. In my mind, a key catalyst to moving IT organizations (and others) into a more open mindset is through a top-down approach. There needs to be active encouragement, rewards, and a clear support structure (e.g. support for experimentation, failures, etc.) for folks to “participate” and embrace open principles. Otherwise, the reward system is not aligned with the model presented here and interdependence remains remains relegated to corporate statements.
Great connection to Covey! There is a growing attention in the open source field for the human factor in decision making on open source applications in enterprises. Which is good. Because it is complex to get a yes for Open Source in situations where it can add value. As an Open Innovation Ambassador I have to admit that there is lots of organisations where Open Source Application and its services suppliers network (if any) is not suitable at present.
In cases that we do have a great Open alternative to Closed Environments we have to be patient indeed. We can tell them, we can show them, they will see it, they will hear it, but in the end it is what people believe that makes the difference. Open Source has not yet come to the point that decision makers and IT directors feel backed by the currently available software, the infrastructure and services suppliers. System integration in Open Source sucks big time, if I may be so blunt. There is more. If all the worldwide available Open Source service suppliers would have to service 1% of the enterprises worldwide, we would run short on capacity quickly. It is no use blaming decision makers as long as we ourselves (the Open minded) focus only on the positive effects of Open Innovation.
But we are getting there! Let us shift our attention more towards customers concerns about Open Source. These issues go way beyond software features, staffing and hiring suppliers: e.g. Total Cost of Operation, how to sell it to the boss, smooth integration, network of suppliers, training, and … building trust.
Using open source software is becomming an industry standard, however the most basic questions remain. That is compatibility of various open source licenses, and this is a very big legal issue. Let’s take for example an IT organisation, downloaded a sourcecode with a GPL component or licensed with GNU GPL. Now to make the required innovation it requires an EPL component or licensed using Eclipse Foundation’s EPL. The innovation will not be possible since EPL and GPL are not compatible licenses, meaning that the GPL license conditions are breached and that the originator of the GPL license can sue your organisation.
It made me cry. So many dreams, so many failures, is there really a company out there that can get past the crud and collaborate successfully? If so, hire me.
[...] I Interdependent ? By brunocornec Reading the brilliant article that Kartik Subbarao wrote recently on Open Source and Interdependent IT , I was questioning myself [...]
Very interesting Kartik, as usual. And always inspiring: http://brunocornec.wordpress.com/2009/09/12/am-i-interdependent/
Well written, well developed, well presented thesis, Kartik. As some of my peers on the HP Open Source and Linux Profession leadership team (past and present) have already mentioned, this resonates with our own experience. I experienced a particular moment of recognition in reading your description of the limitations we place on ourselves by reducing all decisions to monetary ones. Many of us in business will identify with the general Catch-22 mindset that says “We can’t develop this capability for the marketplace because there’s no demand, hence, no business justification, no future revenue stream”. (The obvious Catch-22 here is there’s no market because we haven’t developed the products(s) that would helpcreate, or at least grow the market.)
I hadn’t considered the Open Source model from this point of view (interdependence) before, but like so many other ideas that weave the fabric of this development model it seems intuitively obvious in retrospect. I suppose another aspect of this I find personally appealing, if occasionally frustrating, is that it reinforces the notion that this is a marathon, not a sprint. It’s a continuum that individuals and IT organizations alike can position themselves along at any given point in time, and change only infrequently appears to happen quickly. But change is the only true constant, and it’s inevitable given sufficient time. We’re making inroads with our own IT organization that will bear fruit, and I suspect we are approaching a tipping point. I’ll keep my fingers crossed.
A good model indeed! … Looking at your Dependent-Independent-Interdependent model, it reminds me of a line… “We are taking apart each task and sending it…to whomever can do it best… and then we are reassembling all the pieces.” – from Thomas Friedman’s ‘The World is Flat’. I’m trying to related it to the “Bottom Up” approach this book talks about. Having said that, I worry how so many versions and licences of a open source software (OSS) is maintained and integrated as rightly pointed out by Ian Vernon in his comments. What’s your take on that? What should be any IT organization’s considerations (benefits and losses) before thinking about OSS as a long term strategy if not immediate? Will you agree that any big organization (already having vendor relationship) should not go for OSS in entirety but only where required as this may take some time to mature?
> Looking at your Dependent-Independent-Interdependent
> model, it reminds me of a line… “We are taking apart
> each task and sending it…to whomever can do it
> best…and then we are reassembling all the pieces.”
> – from Thomas Friedman’s “The World is Flat”. I’m
> trying to related it to the “Bottom Up” approach this
> book talks about.
Yes, I agree that Tom Friedman does an excellent job of
explaining the emerging realities and possibilities of
the flat world. In fact, this same
delegation/specialization model is very common to Unix,
Linux and many other other open source projects. Good
interfaces defined by open standards, with each component
focused on its particular job, integrated effectively
across the board.
>Will you agree that any big organization (already having
>vendor relationship) should not go for OSS in entirety
>but only where required as this may take some time to
>mature?
I don’t think that open source is an all-or-nothing
proposition, and can certainly be implemented as a phased
approach. Adopting interdependent, open ways of working
throughout the areas of people, process, technology and
business is the broader goal. That will naturally
translate into a strategy of preferring (and contributing
to) open source software, but proprietary software can be
part of the mix too, if its value is well-understood.