A Solution to the Twitter API Problem

July 1st, 2012

I am writing this article because I love Twitter, and I’ve built a business around making sense of Twitter data.

I am concerned by the recent “ominous” signs that Twitter may be heading down a path towards becoming a closed platform — a path that betrays the trust and goodwill given to them by millions of users and developers.

I want Twitter to achieve its potential, and to be as successful as possible – and I believe these goals are best-achieved by Twitter living up to its promises to be an open ecosystem.

In this article I am going to present some observations about Twitter’s policies, what might happen if they abandon them, and most importantly, what I think would be a better way for Twitter to monetize than what they appear to be doing.

My proposed solution is optimal because it will make Twitter’s advertising people happy and will catalyze tremendous revenue growth for the company — possibly enabling it to grow orders of magnitude faster and larger than it can ever be by becoming a closed platform.

Rather than closing the APIs down, there is a way for Twitter to benefit massively by making their APIs even more open, but with a built-in monetization strategy.

This solution will transform Twitter’s vast third-party app ecosystem into a massive ad network for Twitter overnight. Twitter will almost instantly become as big as ad networks like AdSense and Doubleclick, and would soon rival Facebook and possibly surpass it in advertising reach.

It’s a slam dunk for Twitter, and for the ecosystem as a whole. Everyone wins. There is no good reason NOT to do this.

Below I’ll get into the history and details.

Please make sure to read the last sections of this article where I explain this proposal.

Sign The Petition and Make Your Voice Count

If you care about Twitter remaining open, please sign this petition and help spread it.

 

Retweet This And Help Spread This Proposal

But first, I have to ask that, if you agree with the points raised below, you please help raise awareness and make sure that others hear about this article by retweeting it.

And in particular, if you agree, you should email and/or tweet this article to Twitter’s leadership so they can hear directly from you on this issue.

It’s important for Twitter to know that (a) we all love Twitter, and (b) we have a proposal for how to make Twitter better, while keeping it open, that also achieves Twitter’s goals for more ad revenue.

So if you like what I propose below, please share it with @dickc @adambain @jack and @sippey and include your thoughts too. I’m sure they would be interested to hear from you on this important issue.

 

How We Got Here: Twitter’s Many Promises to be “Open”

Once again the Web development community is up in arms about the possibility that Twitter might clamp down on the use of their APIs and data. How did we get here, again?

At this point it is still mostly FUD, due to a vague yet alarming blog post by Twitter’s product lead, indicating unspecified coming “changes” to their API terms. Hopefully Twitter will clarify this post soon.

Developers are understandably skittish and cynical due to a history of Twitter carving away API rights. And there is a widespread and growing feeling of betrayal among third-party app providers.

Dave Winer summarized the current situation well:

@davewiner: The mistake we all made with Twitter, me too, was to think a corporate API could act like an open protocol.

But, it wasn’t just that we all made a mistake. In fact, as many have pointed out, Twitter built their business by promising to be “open.” They still promise that. It says so right in their API terms:

Note the use of the words “open platform” in this excerpt. What does that actually mean? Twitter doesn’t define it, but there is plenty of precedent for how we should understand that and the intent of the agreement that it indicates.

But Twitter doesn’t only say this in their API terms, they also state it in the terms of use for users who contribute their content, the lifeblood of Twitter.

For example, consider the Twitter user agreement terms, where you grant Twitter a non-exclusive license to your Tweets:

Here Twitter makes it clear that the intent of granting them access to your intellectual property is so that they can help you make it “available to the rest of the world.”

Furthermore, consider this “Tip” in the same agreement, where Twitter states that they are creating an “open ecosystem” with “your rights in mind” — and “what’s yours is yours — you own your content”…

It is quite clear that all of us who have been contributing our intellectual property to Twitter, have been doing so under the understanding and explicit agreement that Twitter would provide an “open ecosystem” so that our data could be freely accessible (to us and to others).

I wonder to what degree these promises are legally binding, and whether Twitter can be held to them by users and third-party developers? It’s an interesting question for the legally inclined out there to analyze.

Can Twitter change the terms now, after allowing so many users and developers to proceed under totally different assumptions? Probably they can, but it’s not good corporate citizenship to do that. Furthermore, I think there is a better path for Twitter (see my proposal below).

What If Twitter Changes the API Terms to Be More Restrictive?

Given that Twitter’s API terms and user terms have long promised openness, it does raise a question: What legal rights does Twitter actually have to change those terms retroactively, this late in the game?

And what constrains may exist if they do make changes? What happens to all the content that was contributed under the old terms? What about all the apps that integrated under the old terms?

Twitter reserves the right to modify their terms at any time. However, any content that was contributed under the former license terms may actually still have to be provided under those historical terms, not the new terms. But this is a legal question.

Can Twitter retroactively change the terms under which previous content was licensed to them by users, without the express permission of those content creators?

Of course any new content contributed to Twitter after such a change could be under the new terms, but users would need to be asked to agree to the new terms before they could be held to them. Twitter would at the very least need to ask all users to sign a new terms of use agreement for their content. This would be the proper, transparent, way to proceed.

There’s one more thing to consider: even if Twitter wanted to, it would be difficult in practice to stop all third-party apps and clients from accessing their APIs and data. Restrictions of this nature are hard to define, and hard to enforce. It’s a fuzzy line — what exactly is “replicating” Twitter’s services, and what is “augmenting” them?

Furthermore, Twitter’s own Website and their own mobile apps use their APIs and it would not be hard to take advantage of that fact to scrape the API and get at their data in various ways. I think numerous workarounds would emerge — if Twitter forced the ecosystem to find them.

The Power of “We The Product”

As someone said, if you are giving your data to a service and using that service for free, then “you are the product.”

That is a fact that most people don’t realize. It’s not “We The People” anymore, it’s “We The Product” and if we all banded together, We The Product could exert some influence over those who are monetizing us. I hope that happens soon.

This whole area of what rights and responsibilities Twitter (and for that matter, Facebook and Google+ and LinkedIn and other social networks and data aggregators) have around the content that We the Product have licensed to them is an open question that should be explored further by people with more knowledge of intellectual property law.

It is possible for example that Twitter may not even have the right to restrict access to the content we granted them a non-exclusive license too — that right to restrict it was not explicitly granted to them. Yes they can restrict the use of their API, but if they hold the only copy of our intellectual property, they may have to at least provide another way to get it (for us, and anyone else).

But it’s unclear, and I hope those of you out there with more knowledge of what the law says about these questions will drill in and comment.

What About Data Portability?

One thing is certain: If Twitter is seriously considering becoming a closed platform that is a failure pattern – it will not work, in fact it will backfire. It will result in Twitter losing their position instead of gaining more marketshare.

It will also make it even more important to get your data out of Twitter, because Twitter has the only copy of your Tweets. Nobody in their right mind would want their personal lifestream locked in any proprietary silo where they can’t get it out.

Despite promises of being open, and promises that “you own your data,” in fact, anyone who has contributed any data to Twitter is currently a hostage: Twitter has your data and you can’t have it.

Although Twitter’s terms of use only grant Twitter a non-exclusive license to your data, they currently have the only copy of it (for most people) and they do not provide any way to get that data back, except through Twitter. Should they be required to?

It is interesting to note that despite previous promises to change this situation by Twitter reps, today, if you have more than 3200 Tweets in your history you cannot actually see them all, because Twitter does not show more than that number of Tweets from any timeline.

For example, here is what Twitter’s FAQs say about this now:

Unfortunately there is no way to get your old Tweets out of Twitter. There was a time when Twitter representatives publicly stated that they would address this issue “soon.” But that intention has seemingly fallen by the wayside, or become a victim of Twitter’s changing API strategy. After all, if you can’t get your data out of Twitter, you certainly can’t leave the service.

But locking people’s content into a product is not the best way to retain them. Ideally they should stay because they want to not because they have to.

In fact, platform lock-in has failed as a strategy for most big companies that tried it — and today that’s why we have things like open standards, open protocols, cloud computing, etc.

One solution, at the very least, is that Twitter should be joining the Data Liberation project, like Google did. Or at least providing a way to get your data out, like Facebook does. But they haven’t and there are no signs that they will. That’s a problem that has to be rectified.

But simply giving people their own content back is not enough.

Twitter has an opportunity to be everyone’s personal API — they could (and should) make it easy for anyone, and any app, to get your data (with your permission), through their API. It’s a huge opportunity for leadership in the future.

If they don’t do this, someone else will and Twitter will become just another silo. Silos are the problem that the Internet exists to solve. Please don’t be a silo, Twitter!

The Need for a Free, Open, Federated Short Messaging Infrastructure

Twitter is currently in control of a key piece of human communications infrastructure — the global short messaging infrastructure.

This infrastructure has become so important to humanity that regime changes now are catalyzed by it. In short this is a critical technology for our civilization today.

Is it safe that one company is in control of this infrastructure? Definitely no. No matter what company it is, it’s risky to have this critical infrastructure go through one single point of control.

First of all, centralizing something so important means that there is a single point of failure. It’s too easy for a regime or hacker to attack it and shut it off in certain areas, or for everyone.

Secondly, by mixing a critical piece of communications infrastructure with commercial interests there are serious risks to free speech — because governments for example can exert economic and legal pressure against the owner of the infrastructure. It is much safer for everyone for such an infrastructure to be federated and outside of any one party’s control.

But there’s not much we can do about this until a suitable open alternative is provided (and that can’t happen soon enough!). There really needs to be a free, open, federated short messaging infrastructure. There are in fact some contenders for this, but none have caught on yet.

Every app that integrates with Twitter (and Facebook, for that matter) should be integrating with such a network ASAP, as an insurance policy. All messages sent to Twitter and Facebook should be mirrored onto an open network now, so that they can be accessed either way. Apps could continue to use Twitter for messaging but could immediately cut over to the alternative network at any time.

What network exists that would be suitable for this, and how could we get everyone to standardize on this now? I’m curious to hear a cogent case for a solution. Why haven’t the existing contenders gotten traction?

With Great Power Comes Great Responsibility

In any case, Twitter’s role as our present-day global short messaging network is a special kind of role. It’s a privilege, and it comes with a responsibility to be a good steward.

Twitter is extremely fortunate to have the exclusive first-right to monetize this infrastructure, and I for one am happy for them to have this opportunity, provided that they behave as good stewards of the network and of our data.

If Twitter behaves as a good steward of our network and our data, they can keep that role. But if they abuse that privilege they will ultimately lose it. The world will take it away from them. The Internet will route around them.

For one thing, if Twitter becomes more closed, or abuses this stewardship role, it creates a ripe opportunity for competitors to be more open.

For example, as some have pointed out, if Twitter closes down their APIs or becomes overly restrictive, the first thing that Google+, Facebook, and LinkedIn should do is to be more open — provide one or several alternative open APIs that make Twitter less necessary.

That possibility alone should be enough to convince Twitter not to become a closed platform. It is their very openness that has brought them to power, and keeps them there.

Furthermore, now that Twitter has cut off LinkedIn, I’m sure that all third-party networks are considering whether or not to continue integrating with Twitter. Imagine what would happen if there was a mass exodus from Twitter by the big networks that connect to it. That would not be good for Twitter.

Twitter has a choice about whether this happens. It’s up to them to decide whether to push the ecosystem into that kind of a response. I don’t think they will do this. It just doesn’t make good business sense in the long run.

Depriving Users of Choice is Not a Good Business Model

Many of Twitter’s users already use, and/or pay for, third-party apps that require Twitter API access — These are apps they love and want to continue to use.

Twitter does not provide any suitable alternatives to such apps today. If Twitter simply shut off the pipes to those apps, I think they would enrage 10′s of millions of users, and lose a lot of them. It would also cut off millions of enterprise users and brands that rely on these tools. If Twitter cut off those apps, they would lose large amounts of content, users and traffic.

Here’s another interesting fact: Although Twitter states that 90% of users visit Twitter.com and Twitter’s own mobile apps, that doesn’t mean they ONLY use them.

In fact most Twitter users access Twitter in many different ways via different apps and tools in different contexts. Sometimes they use multiple tools that provide different special features they want.

Twitter would be making a mistake if they assumed that by cutting off those other apps, people would use Twitter more. Without such apps people might actually use Twitter less (because Twitter.com is just not suitable for all those other use-cases and never will be).

Some people access Twitter via the Twitter.com website, while others access it primarily via Twitter’s mobile apps. But many also use third-party mobile apps. Some access Twitter through Flipboard, or on widgets embedded in Web pages, or via third-party apps like HootsuiteRadian6, and my own company’s Bottlenose.

These third-party apps provide layers of functionality that go well-beyond merely replicating the experience provided on Twitter.com. They provide browsing, search, filtering, analytics, alerting, team collaboration and more.

They don’t compete with Twitter.com presently. Twitter.com doesn’t provide anything comparable. Preventing people from using Twitter, the way they want to, and need to, is not a good business model.

When people want your product, give it to them the way they want it. Listen to them and respond to their needs. That’s how to win them and keep them.

A Much Better Proposal for Twitter’s Revenue Model

So from all of the above it should be clear that there are many good reasons for Twitter to NOT restrict access to it’s APIs or clamp down on third-party sites and apps that integrate it.

But at the same time, we all understand and sympathize with Twitter’s legitimate for-profit needs to monetize their network. Advertising is certainly one way to do this, but it is not mutually exclusive to having an open API.

What’s the solution then? Is there anything Twitter can do to satisfy both their own needs for more eyeballs on their ads, and the needs of their enormous ecosystem of users and developers?

Yes there is. But they for some reason are not seeing it. Here it is…

Instead of trying to get all eyeballs to go to Twitter.com and Twitter’s apps, it would make much more sense to monetize all the eyeballs that are presently looking at Twitter data in other apps and sites.

This enormous network of apps and sites that consume and display Tweets today could be used to drive traffic to Twitter advertisers. It’s a huge surface area — much larger than any website or set of proprietary aps can ever achieve. And all of it could be carrying Twitter’s ads.

The problem here is that Twitter’s ad people are thinking in early 2000′s paradigms —  they are thinking about display ads on websites and apps. But they should be thinking of ads that appear INSIDE streams of content in Twitter.

We’re in a new era of the Internet. In fact, the Net has moved beyond the Web, and even beyond apps. It’s about data now. If you want to advertise, make sure your ads travel with your data, and then set your data free.

Twitter is a network, not an app. They should be monetizing the network by embedding ads in the content that flows through it, or charging to get the content without the ads. This gets the ads into every site, every UI, every app, that carries Twitter. It’s incredibly viral.

The best way to do this would be for Twitter to provide everyone with two options for how to consume their APIs:

  • OPTION A- Free API Access: Use the APIs for free, but every 25th (or nth) Tweet is an ad, from Twitter’s Ad network. Apps must carry these ads if they use the free API.

    • A large variety of apps and Websites would instantly be carrying Twitter ads.
    • This means that Twitter can control the display of its ads in third-party apps — so that is not an issue. Twitter can dictate that if you use the free API, you MUST carry their ads in the stream and you MUST display them however they want you to. That is fine. Most developers would have no problem with this (and for those that do, they can opt-out via Option B, below). This would solve all the issues at hand – Twitter can control the display and experience of their ads, and they can still benefit from a massive third-party ecosystem to spread their ads everywhere.
  • OPTION B – Premium API Access: Apps can opt-out of showing Twitter’s ads, using a premium version of the APIs, but they have to pay a very low fee, per 1000 Tweets they receive.

    • This fee should be set low enough that they can sell their own ads and still have a good profit margin. But these small payments will rapidly add up for Twitter into a huge revenue stream (it might even be larger than the revenues from ads).
    • If this option existed many apps could charge their users to upgrade to a pro version without the ads and they could pay Twitter from those subscription revenues. Or apps could sell their own ads and still pay Twitter. Twitter effectively makes money either way. Third-party apps are free to choose the business model that makes the best sense for them and their users.

The beauty of this proposal is that everyone wins. Twitter gets paid in either case — they make money from all Tweets and all third-party apps that use their API. It’s in fact far better than cutting off the API, or just running ads on Twitter’s own site and apps.

If Twitter did this, the ad people at Twitter would be happy. Furthermore, Twitter would almost instantly be the largest ad network on the Web, or at least equal to AdSense, Doubleclick and Facebook. Twitter would have ad space in thousands of apps and websites overnight. It would be a brilliant, lucrative, slam dunk of a move.

The more apps and sites that integrate with Twitter, the bigger and more profitable the company would be. And it would be unstoppable. This is really what Twitter should do. It’s the best move on the board by far.

Conclusions

Once again I want to reiterate that I personally am a huge fan of Twitter. I love what they have achieved and I love the potential that still remains in what they have built.

I don’t want to see it ruined by short-term thinking.

Billions of dollars of market value have been created by third-party companies that have built on the Twitter APIs. Numerous businesses have been formed, ventured funded, and even acquired, around these APIs. Careers and have been bet on access to these APIs.

Thousands of businesses and other organizations, including many of the Fortune 1000, now depend on sophisticated third-party apps to analyze and engage with Twitter data in ways that are not possible using Twitter’s own tools.

Tens of millions of people consume Twitter data through third-party apps that target specific needs (such as higher-end professional and enterprise apps, mobile apps for certain devices, etc.).

Twitter is extremely fortunate that all of this has happened, and they should view this as a good problem to have, rather than a bad one.

If Twitter starts shutting down their APIs or being overly restrictive they will ultimately be harming their own users and developers, and that just is not a good way to build a business. They will be selling out their own future for a short-term gain. That is a good way to destroy one’s own empire. I am pretty sure the leadership team at Twitter realizes this.

But most importantly, Twitter has a much better option — namely, the “dual API strategy” that I’ve proposed above.

Provide both a free and premium API, and require free API users to carry Twitter ads in the stream. It’s a great move and would put an end to all this FUD. Twitter would instantly have a business model that makes sense, and massive goodwill from developers. There is just no reason NOT to do this.

Please help us make Twitter’s management team aware of this proposal, just in case they have somehow missed it.

Tweet this article to them. Email it to them if you know them. Or Tweet about it to Twitter’s leadership: @dickc @adambain @jack @sippey — I’m sure they would be interested to hear from you, their valued users and developers.